cutmap 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- cutmap-0.1.0/.gitignore +31 -0
- cutmap-0.1.0/LICENSE +21 -0
- cutmap-0.1.0/PKG-INFO +343 -0
- cutmap-0.1.0/README.en.md +293 -0
- cutmap-0.1.0/pyproject.toml +56 -0
- cutmap-0.1.0/src/cutmap/__init__.py +2 -0
- cutmap-0.1.0/src/cutmap/_assets.py +149 -0
- cutmap-0.1.0/src/cutmap/broll.py +148 -0
- cutmap-0.1.0/src/cutmap/cli.py +140 -0
- cutmap-0.1.0/src/cutmap/frames.py +156 -0
- cutmap-0.1.0/src/cutmap/render.py +109 -0
- cutmap-0.1.0/src/cutmap/subclean.py +61 -0
- cutmap-0.1.0/src/cutmap/terms.txt +54 -0
- cutmap-0.1.0/src/cutmap/util.py +140 -0
cutmap-0.1.0/.gitignore
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# ── 私有覆盖模块:下载能力不随公开版发布 ──
|
|
2
|
+
src/cutmap/download.py
|
|
3
|
+
|
|
4
|
+
# Python
|
|
5
|
+
__pycache__/
|
|
6
|
+
*.py[cod]
|
|
7
|
+
*.egg-info/
|
|
8
|
+
build/
|
|
9
|
+
dist/
|
|
10
|
+
.venv/
|
|
11
|
+
venv/
|
|
12
|
+
|
|
13
|
+
# 产物与素材
|
|
14
|
+
*.mp4
|
|
15
|
+
*.srt
|
|
16
|
+
*.jpg
|
|
17
|
+
*.png
|
|
18
|
+
!examples/**/*.jpg
|
|
19
|
+
!examples/**/*.png
|
|
20
|
+
浏览.html
|
|
21
|
+
index.json
|
|
22
|
+
broll/
|
|
23
|
+
frames/
|
|
24
|
+
sheet_*.jpg
|
|
25
|
+
|
|
26
|
+
# 系统
|
|
27
|
+
.DS_Store
|
|
28
|
+
._*
|
|
29
|
+
|
|
30
|
+
# 构建产物
|
|
31
|
+
dist/
|
cutmap-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Xiangyu Kong
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
cutmap-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,343 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: cutmap
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll. Local ffmpeg + perceptual hashing, no LLM.
|
|
5
|
+
Project-URL: Homepage, https://github.com/xykong36/cutmap
|
|
6
|
+
Project-URL: Issues, https://github.com/xykong36/cutmap/issues
|
|
7
|
+
Author: Xiangyu Kong
|
|
8
|
+
License: MIT License
|
|
9
|
+
|
|
10
|
+
Copyright (c) 2026 Xiangyu Kong
|
|
11
|
+
|
|
12
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
13
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
14
|
+
in the Software without restriction, including without limitation the rights
|
|
15
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
16
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
17
|
+
furnished to do so, subject to the following conditions:
|
|
18
|
+
|
|
19
|
+
The above copyright notice and this permission notice shall be included in all
|
|
20
|
+
copies or substantial portions of the Software.
|
|
21
|
+
|
|
22
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
23
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
24
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
25
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
26
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
27
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
28
|
+
SOFTWARE.
|
|
29
|
+
License-File: LICENSE
|
|
30
|
+
Keywords: b-roll,ffmpeg,perceptual-hash,storyboard,subtitle,video,分镜
|
|
31
|
+
Classifier: Development Status :: 4 - Beta
|
|
32
|
+
Classifier: Environment :: Console
|
|
33
|
+
Classifier: Intended Audience :: End Users/Desktop
|
|
34
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
35
|
+
Classifier: Operating System :: OS Independent
|
|
36
|
+
Classifier: Programming Language :: Python :: 3
|
|
37
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
38
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
39
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
40
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
41
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
42
|
+
Classifier: Topic :: Multimedia :: Video
|
|
43
|
+
Classifier: Topic :: Multimedia :: Video :: Non-Linear Editor
|
|
44
|
+
Requires-Python: >=3.9
|
|
45
|
+
Requires-Dist: pillow>=9.0
|
|
46
|
+
Provides-Extra: dev
|
|
47
|
+
Requires-Dist: pytest>=7; extra == 'dev'
|
|
48
|
+
Requires-Dist: ruff>=0.4; extra == 'dev'
|
|
49
|
+
Description-Content-Type: text/markdown
|
|
50
|
+
|
|
51
|
+
<div align="center">
|
|
52
|
+
|
|
53
|
+
# cutmap
|
|
54
|
+
|
|
55
|
+
**Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll.**
|
|
56
|
+
|
|
57
|
+
[](README.md)
|
|
58
|
+
[](README.en.md)
|
|
59
|
+
[](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
|
|
60
|
+
[](LICENSE)
|
|
61
|
+
[](https://www.python.org)
|
|
62
|
+
[](#)
|
|
63
|
+
|
|
64
|
+
</div>
|
|
65
|
+
|
|
66
|
+
Give it a video and a subtitle file. Get back a single HTML page: the source player on
|
|
67
|
+
the left, every distinct frame on the right, each captioned with what was being said.
|
|
68
|
+
Click any frame to jump the player to that moment. Search the subtitles to locate a shot.
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
cutmap video.mp4 --srt video.srt
|
|
72
|
+
open video/浏览.html
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Everything runs locally through `ffmpeg` + `Pillow`. **No model calls, no API keys, no network.**
|
|
76
|
+
|
|
77
|
+

|
|
78
|
+
|
|
79
|
+
<sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## Why not just screenshot every N seconds
|
|
84
|
+
|
|
85
|
+
Interval sampling is both redundant and lossy. A 30-second talking-head passage yields
|
|
86
|
+
ten identical stills; a fast montage cutting five shots in three seconds yields one.
|
|
87
|
+
|
|
88
|
+
cutmap asks a different question — not *"how often should I sample"* but
|
|
89
|
+
**"which frames actually differ"**:
|
|
90
|
+
|
|
91
|
+
```
|
|
92
|
+
dense sample → perceptual hash each frame → compare against last kept frame
|
|
93
|
+
→ keep only if different enough
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Static passages collapse to a single frame. Every visual change gets one.
|
|
97
|
+
|
|
98
|
+
### Why not ffmpeg's built-in scene detection
|
|
99
|
+
|
|
100
|
+
`select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
|
|
101
|
+
|
|
102
|
+
1. **Low-contrast transitions** — white slide to white webpage. Similar overall
|
|
103
|
+
luminance structure, so the difference score never crosses the threshold.
|
|
104
|
+
2. **Intra-shot evolution** — text appearing line by line, charts animating,
|
|
105
|
+
table rows highlighting. Not a "cut" by any definition, but visually distinct.
|
|
106
|
+
|
|
107
|
+
Measured on one 16-minute video: scene detection found **104** frames,
|
|
108
|
+
perceptual dedup found **608**. One 25-second stretch that scene detection called
|
|
109
|
+
a single shot actually contained three completely different pages.
|
|
110
|
+
|
|
111
|
+
|
|
112
|
+
### A note on the algorithm's blind spot
|
|
113
|
+
|
|
114
|
+
dHash compares **relative brightness between adjacent pixels**, so it measures spatial
|
|
115
|
+
structure, not colour. Two different flat colours produce an identical (all-zero)
|
|
116
|
+
fingerprint and cannot be told apart. Footage that is largely flat-coloured or
|
|
117
|
+
near-black will therefore dedup more aggressively than you might expect.
|
|
118
|
+
|
|
119
|
+
This is also why transition cards are detected by mean luminance rather than by hash.
|
|
120
|
+
|
|
121
|
+
---
|
|
122
|
+
|
|
123
|
+
## Install
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
pip install cutmap
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Requires **ffmpeg** on your `PATH`:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
brew install ffmpeg # macOS
|
|
133
|
+
apt install ffmpeg # Debian / Ubuntu
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
---
|
|
137
|
+
|
|
138
|
+
## Usage
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
cutmap video.mp4 # looks for video.srt alongside
|
|
142
|
+
cutmap video.mp4 --srt subs.srt
|
|
143
|
+
cutmap ./material-dir/ # dir containing 源片.mp4 + 字幕.srt
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
Output lands in a directory named after the video:
|
|
147
|
+
|
|
148
|
+
```
|
|
149
|
+
video/
|
|
150
|
+
├── frames/ every distinct frame, labelled #index timecode seconds
|
|
151
|
+
├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
|
|
152
|
+
├── index.json frame timestamps + aligned subtitles (machine readable)
|
|
153
|
+
├── broll/ extracted B-roll clips + broll.json
|
|
154
|
+
└── 浏览.html ← open this
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
### Options
|
|
158
|
+
|
|
159
|
+
| Flag | Meaning |
|
|
160
|
+
|---|---|
|
|
161
|
+
| `--threshold N` | Frame density. Lower = denser. `6` dense / `10` default / `14` sparse |
|
|
162
|
+
| `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
|
|
163
|
+
| `--cols/--rows` | Contact sheet grid (default 4×4) |
|
|
164
|
+
| `--thumb-width` | Thumbnail width in px (default 480) |
|
|
165
|
+
| `--seg-max N` | Max seconds per B-roll segment (default 45) |
|
|
166
|
+
| `--clip-format` | `mp4` (default) / `gif` / `webp` |
|
|
167
|
+
| `--no-broll` | Skip B-roll extraction |
|
|
168
|
+
| `--no-frames` | Keep only contact sheets, drop individual frames |
|
|
169
|
+
| `--terms FILE` | Custom subtitle term-normalisation table |
|
|
170
|
+
|
|
171
|
+
### Picking a density
|
|
172
|
+
|
|
173
|
+
Measured on a 969-second video (2 fps sampling → 1936 raw frames):
|
|
174
|
+
|
|
175
|
+
| Threshold | Kept | Avg gap | Good for |
|
|
176
|
+
|---|---|---|---|
|
|
177
|
+
| 6 | 892 | 1.1s | Reconstructing every visual step (tutorial walkthroughs) |
|
|
178
|
+
| **10** | **597** | **1.6s** | **Default** — balances coverage and readability |
|
|
179
|
+
| 14 | 463 | 2.1s | Style study, quick scanning |
|
|
180
|
+
| 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
|
|
181
|
+
|
|
182
|
+
The curve is smooth — there is no natural breakpoint. This is an **aesthetic choice,
|
|
183
|
+
not a parameter with a computable optimum.**
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
## The browse page
|
|
188
|
+
|
|
189
|
+
One self-contained HTML file. All CSS and JS inlined; only depends on the images and
|
|
190
|
+
clips sitting next to it. Two tabs:
|
|
191
|
+
|
|
192
|
+
**Storyboard** — every distinct frame with its caption
|
|
193
|
+
**B-roll** — auto-looping clips (reads like a GIF wall)
|
|
194
|
+
|
|
195
|
+

|
|
196
|
+
|
|
197
|
+
Shared behaviour:
|
|
198
|
+
|
|
199
|
+
- Embedded source player up top — **click any timecode or frame to seek there**
|
|
200
|
+
(local file, no network)
|
|
201
|
+
- Live subtitle search, matching against the **normalised** text
|
|
202
|
+
(searching `DeepSeek` finds frames whose ASR wrote `deep sick`)
|
|
203
|
+
- Long captions collapse behind `…expand`
|
|
204
|
+
- B-roll clips play only while in viewport (IntersectionObserver) — dozens of
|
|
205
|
+
simultaneously decoding videos will freeze a browser
|
|
206
|
+
- Follows system light / dark mode
|
|
207
|
+
|
|
208
|
+
Contact sheets are written separately, for scanning a whole video in a few images:
|
|
209
|
+
|
|
210
|
+

|
|
211
|
+
|
|
212
|
+
---
|
|
213
|
+
|
|
214
|
+
## B-roll detection
|
|
215
|
+
|
|
216
|
+
Frames are sorted into three buckets by three plain rules — no model involved:
|
|
217
|
+
|
|
218
|
+
| Bucket | Rule |
|
|
219
|
+
|---|---|
|
|
220
|
+
| **Main shot** | A cluster of near-identical frames recurring across >50% of the runtime — i.e. a locked-off camera |
|
|
221
|
+
| **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
|
|
222
|
+
| **B-roll** | Everything else |
|
|
223
|
+
|
|
224
|
+
Measured on one episode: B-roll 49 segments / 731s (75.5%),
|
|
225
|
+
main shot 39 / 217s (22.4%), transitions 16 / 20s (2.1%).
|
|
226
|
+
|
|
227
|
+
Screencast-style videos with no talking head degrade gracefully — no main shot is
|
|
228
|
+
found, everything becomes B-roll, and `--seg-max` keeps it segmented.
|
|
229
|
+
|
|
230
|
+
### Clip format
|
|
231
|
+
|
|
232
|
+
Same nine-second segment:
|
|
233
|
+
|
|
234
|
+
| Format | Size | Note |
|
|
235
|
+
|---|---|---|
|
|
236
|
+
| GIF 10fps | 1.96 MB | 256 colours; visible banding on gradients |
|
|
237
|
+
| WebP | 0.39 MB | Middle ground |
|
|
238
|
+
| **MP4** | **0.08 MB** | **Default** — 24× smaller than GIF, better quality |
|
|
239
|
+
|
|
240
|
+
MP4 with `loop muted` is visually indistinguishable from a GIF in the page.
|
|
241
|
+
Reach for `--clip-format gif` only when pasting into chat apps or note tools.
|
|
242
|
+
|
|
243
|
+
---
|
|
244
|
+
|
|
245
|
+
## Subtitle term normalisation
|
|
246
|
+
|
|
247
|
+
ASR mangles proper nouns. A built-in table normalises them:
|
|
248
|
+
|
|
249
|
+
```
|
|
250
|
+
大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
|
|
251
|
+
用grock测试 → 用Grok测试
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
|
|
255
|
+
was transcribed as `deep sick`.
|
|
256
|
+
|
|
257
|
+
The default table targets AI / tech content. Bring your own:
|
|
258
|
+
|
|
259
|
+
```bash
|
|
260
|
+
cutmap video.mp4 --terms my_terms.txt
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
Format is one `regex => replacement` per line — see
|
|
264
|
+
[`src/cutmap/terms.txt`](src/cutmap/terms.txt).
|
|
265
|
+
|
|
266
|
+
**This is term normalisation, not proofreading.** It will not fix segmentation,
|
|
267
|
+
grammar, or one-off errors — those need a language model, which is out of scope here.
|
|
268
|
+
|
|
269
|
+
---
|
|
270
|
+
|
|
271
|
+
## Troubleshooting
|
|
272
|
+
|
|
273
|
+
Bugs hit while building this. Most of them **never raised an error** — they just
|
|
274
|
+
quietly produced wrong output:
|
|
275
|
+
|
|
276
|
+
**`\b` word boundaries fail in mixed CJK/Latin text**
|
|
277
|
+
CJK characters are `\w` in Python, so in `用grock测试` there is no boundary between
|
|
278
|
+
`用` and `g`. `\bgrock\b` never matches. The term table uses
|
|
279
|
+
`(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
|
|
280
|
+
|
|
281
|
+
**Flat frames can't be detected by standard deviation**
|
|
282
|
+
Black transition cards usually have burned-in white subtitles, pushing stddev to
|
|
283
|
+
23–33 — far above any "flat colour" threshold. Use mean luminance, and keep it tight
|
|
284
|
+
(`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
|
|
285
|
+
|
|
286
|
+
**Subtitle time windows must not overlap**
|
|
287
|
+
Adding a look-back to avoid empty captions makes adjacent frames claim the same cue,
|
|
288
|
+
producing duplicated sentences when concatenated. Hence two fields:
|
|
289
|
+
`subtitle` (display, may repeat) and `subtitle_own` (concatenation, strict half-open).
|
|
290
|
+
|
|
291
|
+
**ffmpeg concat resolves relative paths against the list file's directory**
|
|
292
|
+
Contact sheet file lists must contain absolute paths.
|
|
293
|
+
|
|
294
|
+
**`-v error` suppresses `showinfo` output**
|
|
295
|
+
Any ffmpeg-based statistics gathered with `-v error` silently come back as zero.
|
|
296
|
+
|
|
297
|
+
**Multi-threaded downloads corrupt files on exFAT** (if you fetch footage yourself)
|
|
298
|
+
Pre-allocating and writing at multiple offsets over exFAT-on-USB yields files with
|
|
299
|
+
the right size and wrong contents — `moov atom not found`. Download to local disk,
|
|
300
|
+
then move.
|
|
301
|
+
|
|
302
|
+
---
|
|
303
|
+
|
|
304
|
+
## Getting footage
|
|
305
|
+
|
|
306
|
+
cutmap does not download anything — it works on local files.
|
|
307
|
+
|
|
308
|
+
For Bilibili, [BBDown](https://github.com/nilaoda/BBDown) muxes subtitles into the mp4;
|
|
309
|
+
pull them out and hand both to cutmap:
|
|
310
|
+
|
|
311
|
+
```bash
|
|
312
|
+
BBDown <BV-id> --skip-ai false
|
|
313
|
+
ffmpeg -i video.mp4 -map 0:m:language:chi subs.srt
|
|
314
|
+
cutmap video.mp4 --srt subs.srt
|
|
315
|
+
```
|
|
316
|
+
|
|
317
|
+
Respect the terms of service and copyright of wherever your footage comes from.
|
|
318
|
+
Use downloaded material for personal study and research only.
|
|
319
|
+
|
|
320
|
+
---
|
|
321
|
+
|
|
322
|
+
## Credits
|
|
323
|
+
|
|
324
|
+
Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
|
|
325
|
+
|
|
326
|
+
Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**.
|
|
327
|
+
|
|
328
|
+
This project started as an attempt to study the editing in his videos — dense cutting,
|
|
329
|
+
interleaved infographics and animated demos. That is exactly the material naive
|
|
330
|
+
interval-screenshotting handles worst, and it is what forced the perceptual-dedup
|
|
331
|
+
approach in the first place. The screenshots in this README are from
|
|
332
|
+
**[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely to demonstrate this tool's
|
|
333
|
+
output; all rights remain with the original creator. If he would prefer them removed,
|
|
334
|
+
please open an issue and they will be replaced immediately.
|
|
335
|
+
|
|
336
|
+
| | |
|
|
337
|
+
|---|---|
|
|
338
|
+
| Source video | <https://www.youtube.com/watch?v=WDQjRzVcX-A> |
|
|
339
|
+
| Creator | YouTube <https://www.youtube.com/@lyi> · Bilibili <https://space.bilibili.com/4401694> |
|
|
340
|
+
|
|
341
|
+
## License
|
|
342
|
+
|
|
343
|
+
MIT
|
|
@@ -0,0 +1,293 @@
|
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
# cutmap
|
|
4
|
+
|
|
5
|
+
**Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll.**
|
|
6
|
+
|
|
7
|
+
[](README.md)
|
|
8
|
+
[](README.en.md)
|
|
9
|
+
[](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
|
|
10
|
+
[](LICENSE)
|
|
11
|
+
[](https://www.python.org)
|
|
12
|
+
[](#)
|
|
13
|
+
|
|
14
|
+
</div>
|
|
15
|
+
|
|
16
|
+
Give it a video and a subtitle file. Get back a single HTML page: the source player on
|
|
17
|
+
the left, every distinct frame on the right, each captioned with what was being said.
|
|
18
|
+
Click any frame to jump the player to that moment. Search the subtitles to locate a shot.
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
cutmap video.mp4 --srt video.srt
|
|
22
|
+
open video/浏览.html
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Everything runs locally through `ffmpeg` + `Pillow`. **No model calls, no API keys, no network.**
|
|
26
|
+
|
|
27
|
+

|
|
28
|
+
|
|
29
|
+
<sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## Why not just screenshot every N seconds
|
|
34
|
+
|
|
35
|
+
Interval sampling is both redundant and lossy. A 30-second talking-head passage yields
|
|
36
|
+
ten identical stills; a fast montage cutting five shots in three seconds yields one.
|
|
37
|
+
|
|
38
|
+
cutmap asks a different question — not *"how often should I sample"* but
|
|
39
|
+
**"which frames actually differ"**:
|
|
40
|
+
|
|
41
|
+
```
|
|
42
|
+
dense sample → perceptual hash each frame → compare against last kept frame
|
|
43
|
+
→ keep only if different enough
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Static passages collapse to a single frame. Every visual change gets one.
|
|
47
|
+
|
|
48
|
+
### Why not ffmpeg's built-in scene detection
|
|
49
|
+
|
|
50
|
+
`select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
|
|
51
|
+
|
|
52
|
+
1. **Low-contrast transitions** — white slide to white webpage. Similar overall
|
|
53
|
+
luminance structure, so the difference score never crosses the threshold.
|
|
54
|
+
2. **Intra-shot evolution** — text appearing line by line, charts animating,
|
|
55
|
+
table rows highlighting. Not a "cut" by any definition, but visually distinct.
|
|
56
|
+
|
|
57
|
+
Measured on one 16-minute video: scene detection found **104** frames,
|
|
58
|
+
perceptual dedup found **608**. One 25-second stretch that scene detection called
|
|
59
|
+
a single shot actually contained three completely different pages.
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
### A note on the algorithm's blind spot
|
|
63
|
+
|
|
64
|
+
dHash compares **relative brightness between adjacent pixels**, so it measures spatial
|
|
65
|
+
structure, not colour. Two different flat colours produce an identical (all-zero)
|
|
66
|
+
fingerprint and cannot be told apart. Footage that is largely flat-coloured or
|
|
67
|
+
near-black will therefore dedup more aggressively than you might expect.
|
|
68
|
+
|
|
69
|
+
This is also why transition cards are detected by mean luminance rather than by hash.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## Install
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
pip install cutmap
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Requires **ffmpeg** on your `PATH`:
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
brew install ffmpeg # macOS
|
|
83
|
+
apt install ffmpeg # Debian / Ubuntu
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
---
|
|
87
|
+
|
|
88
|
+
## Usage
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
cutmap video.mp4 # looks for video.srt alongside
|
|
92
|
+
cutmap video.mp4 --srt subs.srt
|
|
93
|
+
cutmap ./material-dir/ # dir containing 源片.mp4 + 字幕.srt
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Output lands in a directory named after the video:
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
video/
|
|
100
|
+
├── frames/ every distinct frame, labelled #index timecode seconds
|
|
101
|
+
├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
|
|
102
|
+
├── index.json frame timestamps + aligned subtitles (machine readable)
|
|
103
|
+
├── broll/ extracted B-roll clips + broll.json
|
|
104
|
+
└── 浏览.html ← open this
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
### Options
|
|
108
|
+
|
|
109
|
+
| Flag | Meaning |
|
|
110
|
+
|---|---|
|
|
111
|
+
| `--threshold N` | Frame density. Lower = denser. `6` dense / `10` default / `14` sparse |
|
|
112
|
+
| `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
|
|
113
|
+
| `--cols/--rows` | Contact sheet grid (default 4×4) |
|
|
114
|
+
| `--thumb-width` | Thumbnail width in px (default 480) |
|
|
115
|
+
| `--seg-max N` | Max seconds per B-roll segment (default 45) |
|
|
116
|
+
| `--clip-format` | `mp4` (default) / `gif` / `webp` |
|
|
117
|
+
| `--no-broll` | Skip B-roll extraction |
|
|
118
|
+
| `--no-frames` | Keep only contact sheets, drop individual frames |
|
|
119
|
+
| `--terms FILE` | Custom subtitle term-normalisation table |
|
|
120
|
+
|
|
121
|
+
### Picking a density
|
|
122
|
+
|
|
123
|
+
Measured on a 969-second video (2 fps sampling → 1936 raw frames):
|
|
124
|
+
|
|
125
|
+
| Threshold | Kept | Avg gap | Good for |
|
|
126
|
+
|---|---|---|---|
|
|
127
|
+
| 6 | 892 | 1.1s | Reconstructing every visual step (tutorial walkthroughs) |
|
|
128
|
+
| **10** | **597** | **1.6s** | **Default** — balances coverage and readability |
|
|
129
|
+
| 14 | 463 | 2.1s | Style study, quick scanning |
|
|
130
|
+
| 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
|
|
131
|
+
|
|
132
|
+
The curve is smooth — there is no natural breakpoint. This is an **aesthetic choice,
|
|
133
|
+
not a parameter with a computable optimum.**
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## The browse page
|
|
138
|
+
|
|
139
|
+
One self-contained HTML file. All CSS and JS inlined; only depends on the images and
|
|
140
|
+
clips sitting next to it. Two tabs:
|
|
141
|
+
|
|
142
|
+
**Storyboard** — every distinct frame with its caption
|
|
143
|
+
**B-roll** — auto-looping clips (reads like a GIF wall)
|
|
144
|
+
|
|
145
|
+

|
|
146
|
+
|
|
147
|
+
Shared behaviour:
|
|
148
|
+
|
|
149
|
+
- Embedded source player up top — **click any timecode or frame to seek there**
|
|
150
|
+
(local file, no network)
|
|
151
|
+
- Live subtitle search, matching against the **normalised** text
|
|
152
|
+
(searching `DeepSeek` finds frames whose ASR wrote `deep sick`)
|
|
153
|
+
- Long captions collapse behind `…expand`
|
|
154
|
+
- B-roll clips play only while in viewport (IntersectionObserver) — dozens of
|
|
155
|
+
simultaneously decoding videos will freeze a browser
|
|
156
|
+
- Follows system light / dark mode
|
|
157
|
+
|
|
158
|
+
Contact sheets are written separately, for scanning a whole video in a few images:
|
|
159
|
+
|
|
160
|
+

|
|
161
|
+
|
|
162
|
+
---
|
|
163
|
+
|
|
164
|
+
## B-roll detection
|
|
165
|
+
|
|
166
|
+
Frames are sorted into three buckets by three plain rules — no model involved:
|
|
167
|
+
|
|
168
|
+
| Bucket | Rule |
|
|
169
|
+
|---|---|
|
|
170
|
+
| **Main shot** | A cluster of near-identical frames recurring across >50% of the runtime — i.e. a locked-off camera |
|
|
171
|
+
| **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
|
|
172
|
+
| **B-roll** | Everything else |
|
|
173
|
+
|
|
174
|
+
Measured on one episode: B-roll 49 segments / 731s (75.5%),
|
|
175
|
+
main shot 39 / 217s (22.4%), transitions 16 / 20s (2.1%).
|
|
176
|
+
|
|
177
|
+
Screencast-style videos with no talking head degrade gracefully — no main shot is
|
|
178
|
+
found, everything becomes B-roll, and `--seg-max` keeps it segmented.
|
|
179
|
+
|
|
180
|
+
### Clip format
|
|
181
|
+
|
|
182
|
+
Same nine-second segment:
|
|
183
|
+
|
|
184
|
+
| Format | Size | Note |
|
|
185
|
+
|---|---|---|
|
|
186
|
+
| GIF 10fps | 1.96 MB | 256 colours; visible banding on gradients |
|
|
187
|
+
| WebP | 0.39 MB | Middle ground |
|
|
188
|
+
| **MP4** | **0.08 MB** | **Default** — 24× smaller than GIF, better quality |
|
|
189
|
+
|
|
190
|
+
MP4 with `loop muted` is visually indistinguishable from a GIF in the page.
|
|
191
|
+
Reach for `--clip-format gif` only when pasting into chat apps or note tools.
|
|
192
|
+
|
|
193
|
+
---
|
|
194
|
+
|
|
195
|
+
## Subtitle term normalisation
|
|
196
|
+
|
|
197
|
+
ASR mangles proper nouns. A built-in table normalises them:
|
|
198
|
+
|
|
199
|
+
```
|
|
200
|
+
大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
|
|
201
|
+
用grock测试 → 用Grok测试
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
|
|
205
|
+
was transcribed as `deep sick`.
|
|
206
|
+
|
|
207
|
+
The default table targets AI / tech content. Bring your own:
|
|
208
|
+
|
|
209
|
+
```bash
|
|
210
|
+
cutmap video.mp4 --terms my_terms.txt
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
Format is one `regex => replacement` per line — see
|
|
214
|
+
[`src/cutmap/terms.txt`](src/cutmap/terms.txt).
|
|
215
|
+
|
|
216
|
+
**This is term normalisation, not proofreading.** It will not fix segmentation,
|
|
217
|
+
grammar, or one-off errors — those need a language model, which is out of scope here.
|
|
218
|
+
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
## Troubleshooting
|
|
222
|
+
|
|
223
|
+
Bugs hit while building this. Most of them **never raised an error** — they just
|
|
224
|
+
quietly produced wrong output:
|
|
225
|
+
|
|
226
|
+
**`\b` word boundaries fail in mixed CJK/Latin text**
|
|
227
|
+
CJK characters are `\w` in Python, so in `用grock测试` there is no boundary between
|
|
228
|
+
`用` and `g`. `\bgrock\b` never matches. The term table uses
|
|
229
|
+
`(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
|
|
230
|
+
|
|
231
|
+
**Flat frames can't be detected by standard deviation**
|
|
232
|
+
Black transition cards usually have burned-in white subtitles, pushing stddev to
|
|
233
|
+
23–33 — far above any "flat colour" threshold. Use mean luminance, and keep it tight
|
|
234
|
+
(`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
|
|
235
|
+
|
|
236
|
+
**Subtitle time windows must not overlap**
|
|
237
|
+
Adding a look-back to avoid empty captions makes adjacent frames claim the same cue,
|
|
238
|
+
producing duplicated sentences when concatenated. Hence two fields:
|
|
239
|
+
`subtitle` (display, may repeat) and `subtitle_own` (concatenation, strict half-open).
|
|
240
|
+
|
|
241
|
+
**ffmpeg concat resolves relative paths against the list file's directory**
|
|
242
|
+
Contact sheet file lists must contain absolute paths.
|
|
243
|
+
|
|
244
|
+
**`-v error` suppresses `showinfo` output**
|
|
245
|
+
Any ffmpeg-based statistics gathered with `-v error` silently come back as zero.
|
|
246
|
+
|
|
247
|
+
**Multi-threaded downloads corrupt files on exFAT** (if you fetch footage yourself)
|
|
248
|
+
Pre-allocating and writing at multiple offsets over exFAT-on-USB yields files with
|
|
249
|
+
the right size and wrong contents — `moov atom not found`. Download to local disk,
|
|
250
|
+
then move.
|
|
251
|
+
|
|
252
|
+
---
|
|
253
|
+
|
|
254
|
+
## Getting footage
|
|
255
|
+
|
|
256
|
+
cutmap does not download anything — it works on local files.
|
|
257
|
+
|
|
258
|
+
For Bilibili, [BBDown](https://github.com/nilaoda/BBDown) muxes subtitles into the mp4;
|
|
259
|
+
pull them out and hand both to cutmap:
|
|
260
|
+
|
|
261
|
+
```bash
|
|
262
|
+
BBDown <BV-id> --skip-ai false
|
|
263
|
+
ffmpeg -i video.mp4 -map 0:m:language:chi subs.srt
|
|
264
|
+
cutmap video.mp4 --srt subs.srt
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
Respect the terms of service and copyright of wherever your footage comes from.
|
|
268
|
+
Use downloaded material for personal study and research only.
|
|
269
|
+
|
|
270
|
+
---
|
|
271
|
+
|
|
272
|
+
## Credits
|
|
273
|
+
|
|
274
|
+
Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
|
|
275
|
+
|
|
276
|
+
Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**.
|
|
277
|
+
|
|
278
|
+
This project started as an attempt to study the editing in his videos — dense cutting,
|
|
279
|
+
interleaved infographics and animated demos. That is exactly the material naive
|
|
280
|
+
interval-screenshotting handles worst, and it is what forced the perceptual-dedup
|
|
281
|
+
approach in the first place. The screenshots in this README are from
|
|
282
|
+
**[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely to demonstrate this tool's
|
|
283
|
+
output; all rights remain with the original creator. If he would prefer them removed,
|
|
284
|
+
please open an issue and they will be replaced immediately.
|
|
285
|
+
|
|
286
|
+
| | |
|
|
287
|
+
|---|---|
|
|
288
|
+
| Source video | <https://www.youtube.com/watch?v=WDQjRzVcX-A> |
|
|
289
|
+
| Creator | YouTube <https://www.youtube.com/@lyi> · Bilibili <https://space.bilibili.com/4401694> |
|
|
290
|
+
|
|
291
|
+
## License
|
|
292
|
+
|
|
293
|
+
MIT
|