cutmap 0.1.0__tar.gz → 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
cutmap-0.2.0/PKG-INFO ADDED
@@ -0,0 +1,367 @@
1
+ Metadata-Version: 2.4
2
+ Name: cutmap
3
+ Version: 0.2.0
4
+ Summary: Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll. Local ffmpeg + perceptual hashing, no LLM.
5
+ Project-URL: Homepage, https://github.com/xykong36/cutmap
6
+ Project-URL: Issues, https://github.com/xykong36/cutmap/issues
7
+ Author: Xiangyu Kong
8
+ License: MIT License
9
+
10
+ Copyright (c) 2026 Xiangyu Kong
11
+
12
+ Permission is hereby granted, free of charge, to any person obtaining a copy
13
+ of this software and associated documentation files (the "Software"), to deal
14
+ in the Software without restriction, including without limitation the rights
15
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
16
+ copies of the Software, and to permit persons to whom the Software is
17
+ furnished to do so, subject to the following conditions:
18
+
19
+ The above copyright notice and this permission notice shall be included in all
20
+ copies or substantial portions of the Software.
21
+
22
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
23
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
24
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
25
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
26
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
27
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
28
+ SOFTWARE.
29
+ License-File: LICENSE
30
+ Keywords: b-roll,ffmpeg,perceptual-hash,storyboard,subtitle,video,分镜
31
+ Classifier: Development Status :: 4 - Beta
32
+ Classifier: Environment :: Console
33
+ Classifier: Intended Audience :: End Users/Desktop
34
+ Classifier: License :: OSI Approved :: MIT License
35
+ Classifier: Operating System :: OS Independent
36
+ Classifier: Programming Language :: Python :: 3
37
+ Classifier: Programming Language :: Python :: 3.9
38
+ Classifier: Programming Language :: Python :: 3.10
39
+ Classifier: Programming Language :: Python :: 3.11
40
+ Classifier: Programming Language :: Python :: 3.12
41
+ Classifier: Programming Language :: Python :: 3.13
42
+ Classifier: Topic :: Multimedia :: Video
43
+ Classifier: Topic :: Multimedia :: Video :: Non-Linear Editor
44
+ Requires-Python: >=3.9
45
+ Requires-Dist: pillow>=9.0
46
+ Provides-Extra: dev
47
+ Requires-Dist: pytest>=7; extra == 'dev'
48
+ Requires-Dist: ruff>=0.4; extra == 'dev'
49
+ Description-Content-Type: text/markdown
50
+
51
+ <div align="center">
52
+
53
+ # cutmap
54
+
55
+ **Read a video like a document.**
56
+ Skim a long video in minutes to learn fast — or study it frame-by-frame to see exactly how it's cut.
57
+
58
+ [![English](https://img.shields.io/badge/lang-English-2b7489?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.md)
59
+ [![简体中文](https://img.shields.io/badge/lang-简体中文-lightgrey?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.zh-CN.md)
60
+ [![PyPI](https://img.shields.io/pypi/v/cutmap?style=flat-square&color=3775a9)](https://pypi.org/project/cutmap/)
61
+ [![CI](https://github.com/xykong36/cutmap/actions/workflows/ci.yml/badge.svg)](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
62
+ [![License: MIT](https://img.shields.io/badge/license-MIT-blue?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/LICENSE)
63
+ [![Python](https://img.shields.io/badge/python-3.9+-3776ab?style=flat-square&logo=python&logoColor=white)](https://www.python.org)
64
+ [![No LLM](https://img.shields.io/badge/LLM-not%20required-success?style=flat-square)](#)
65
+
66
+ </div>
67
+
68
+ Watching a video is slow and linear — you can't skim it the way you skim an article,
69
+ and you can't Ctrl-F a moment. cutmap fixes that. Give it a video (and subtitles if you
70
+ have them) and it builds a single HTML page you can actually **read**: every visually
71
+ distinct frame laid out in order, each captioned with what was being said. Click a frame
72
+ to jump the player there; search the subtitles to find a moment.
73
+
74
+ ![storyboard](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/storyboard.png)
75
+
76
+ <sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
77
+
78
+ Two things people use it for:
79
+
80
+ - 📖 **Learn faster.** Turn a one-hour talk or tutorial into a page you scroll in a few
81
+ minutes. See the whole structure at a glance, search the transcript, and only play the
82
+ parts you actually need.
83
+ - 🎬 **Study the edit ("拉片").** If you make videos, see exactly how someone cut theirs —
84
+ every shot change, every B-roll insert, every on-screen graphic — laid out as one storyboard.
85
+
86
+ It runs entirely on your machine. **No LLM, no API key, no network.**
87
+
88
+ ---
89
+
90
+ ## Quick start
91
+
92
+ ### The easy way — just ask your AI agent
93
+
94
+ cutmap ships with a [Claude Code](https://claude.com/claude-code) skill. Install it once,
95
+ then talk in plain language — the agent downloads the video, runs cutmap, reads the
96
+ result, and answers you:
97
+
98
+ ```bash
99
+ pip install cutmap yt-dlp # cutmap + video downloader
100
+ brew install ffmpeg # the video engine (Linux: apt install ffmpeg)
101
+
102
+ # install the bundled skill for Claude Code
103
+ git clone https://github.com/xykong36/cutmap
104
+ cp -r cutmap/skills/cutmap ~/.claude/skills/
105
+ ```
106
+
107
+ Then just say things like:
108
+
109
+ > **Download** `https://youtu.be/…` **and use cutmap to summarize what it covers**
110
+ >
111
+ > **Use cutmap on** `lecture.mp4` **and give me the section-by-section breakdown**
112
+ >
113
+ > **What's on screen around 3:41 in that video?** (process it with cutmap first)
114
+
115
+ The agent handles the download, the flags, and reading the frames for you — you never
116
+ touch the command line.
117
+
118
+ ### The manual way — command line
119
+
120
+ ```bash
121
+ pip install cutmap
122
+ brew install ffmpeg # macOS (Linux: apt install ffmpeg)
123
+
124
+ cutmap video.mp4 --srt video.srt # or just: cutmap video.mp4 (finds video.srt)
125
+ open video/browse.html # ← open this in your browser
126
+ ```
127
+
128
+ That's it. Everything you need is in the `video/` folder next to your file.
129
+
130
+ ---
131
+
132
+ ## What you get
133
+
134
+ Output lands in a directory named after the video:
135
+
136
+ ```
137
+ video/
138
+ ├── browse.html ← open this: the interactive storyboard
139
+ ├── frames/ every distinct frame, labelled #index timecode seconds
140
+ ├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
141
+ ├── index.json frame timestamps + aligned subtitles (machine readable)
142
+ └── broll/ extracted B-roll clips + broll.json
143
+ ```
144
+
145
+ ### The browse page
146
+
147
+ One self-contained HTML file — all CSS and JS inlined. Two tabs:
148
+
149
+ **Storyboard** — every distinct frame with its caption
150
+ **B-roll** — auto-looping clips (reads like a GIF wall)
151
+
152
+ ![B-roll clips auto-looping](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/broll-demo.gif)
153
+
154
+ <sub>The B-roll tab in motion: clips auto-loop like a GIF wall, but they are MP4 under the hood — 1/24 the size of actual GIFs</sub>
155
+
156
+ What you can do on it:
157
+
158
+ - **Click any frame or timecode** to seek the embedded player there (local file, no network)
159
+ - **Search the subtitles** — matches the *normalised* text, so searching `DeepSeek` also
160
+ finds frames where the auto-transcription wrote `deep sick`
161
+ - Long captions collapse behind `…expand`
162
+ - Follows your system light / dark mode
163
+
164
+ Contact sheets are also written on their own, for scanning a whole video in a few images:
165
+
166
+ ![contact sheet](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/contact-sheet.jpg)
167
+
168
+ ---
169
+
170
+ ## How it works
171
+
172
+ The interesting part is *which* frames it keeps. Sampling every N seconds is both
173
+ redundant and lossy: a 30-second talking-head passage gives you ten identical stills;
174
+ a fast montage cutting five shots in three seconds gives you one.
175
+
176
+ cutmap asks a different question — not *"how often should I sample"* but
177
+ **"which frames actually differ"**:
178
+
179
+ ```
180
+ dense sample → perceptual hash each frame → compare against last kept frame
181
+ → keep only if different enough
182
+ ```
183
+
184
+ Static passages collapse to a single frame. Every visual change gets one.
185
+
186
+ <details>
187
+ <summary><b>Why not ffmpeg's built-in scene detection?</b></summary>
188
+
189
+ <br>
190
+
191
+ `select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
192
+
193
+ 1. **Low-contrast transitions** — white slide to white webpage. Similar overall
194
+ luminance structure, so the difference score never crosses the threshold.
195
+ 2. **Intra-shot evolution** — text appearing line by line, charts animating,
196
+ table rows highlighting. Not a "cut" by any definition, but visually distinct.
197
+
198
+ Measured on one 16-minute video: scene detection found **104** frames,
199
+ perceptual dedup found **608**. One 25-second stretch that scene detection called
200
+ a single shot actually contained three completely different pages.
201
+
202
+ </details>
203
+
204
+ <details>
205
+ <summary><b>The algorithm's blind spot</b></summary>
206
+
207
+ <br>
208
+
209
+ dHash compares **relative brightness between adjacent pixels**, so it measures spatial
210
+ structure, not colour. Two different flat colours produce an identical (all-zero)
211
+ fingerprint and cannot be told apart. Footage that is largely flat-coloured or
212
+ near-black will therefore dedup more aggressively than you might expect. (This is also
213
+ why transition cards are detected by mean luminance rather than by hash.)
214
+
215
+ </details>
216
+
217
+ ---
218
+
219
+ ## Options
220
+
221
+ Most people never need these — the defaults are tuned for slides, screencasts, and
222
+ talks. Reach for them when the frame count comes out too dense or too sparse.
223
+
224
+ | Flag | Meaning |
225
+ |---|---|
226
+ | `--threshold N` | Frame density. **Lower = more frames.** `6` dense / `10` default / `14` sparse / `24` structure-only |
227
+ | `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
228
+ | `--seg-max N` | Max seconds per B-roll segment (default 45) |
229
+ | `--clip-format` | `mp4` (default) / `gif` / `webp` |
230
+ | `--no-broll` | Skip B-roll extraction (much faster) |
231
+ | `--no-frames` | Keep only contact sheets, drop individual frames |
232
+ | `--cols/--rows` | Contact sheet grid (default 4×4) |
233
+ | `--thumb-width` | Thumbnail width in px (default 480) |
234
+ | `--terms FILE` | Custom subtitle term-normalisation table |
235
+
236
+ **Rule of thumb for density:** screencasts and slides → keep the default `10`. Real
237
+ handheld / vlog / interview footage → start at `--threshold 20` (camera shake makes
238
+ every frame look "different" otherwise).
239
+
240
+ <details>
241
+ <summary><b>Density, measured</b></summary>
242
+
243
+ <br>
244
+
245
+ On a 969-second video (2 fps sampling → 1936 raw frames):
246
+
247
+ | Threshold | Kept | Avg gap | Good for |
248
+ |---|---|---|---|
249
+ | 6 | 892 | 1.1s | Reconstructing every visual step (tutorial walkthroughs) |
250
+ | **10** | **597** | **1.6s** | **Default** — balances coverage and readability |
251
+ | 14 | 463 | 2.1s | Style study, quick scanning |
252
+ | 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
253
+
254
+ The curve is smooth — there is no natural breakpoint. This is an aesthetic choice,
255
+ not a parameter with a computable optimum.
256
+
257
+ </details>
258
+
259
+ ---
260
+
261
+ ## B-roll detection
262
+
263
+ Frames are sorted into three buckets by three plain rules — no model involved:
264
+
265
+ | Bucket | Rule |
266
+ |---|---|
267
+ | **Main shot** | A cluster of near-identical frames recurring across >50% of the runtime — i.e. a locked-off camera |
268
+ | **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
269
+ | **B-roll** | Everything else |
270
+
271
+ Screencast-style videos with no talking head degrade gracefully — no main shot is
272
+ found, everything becomes B-roll, and `--seg-max` keeps it segmented.
273
+
274
+ Clips default to **MP4** (`loop muted`), which looks identical to a GIF in the page but
275
+ is ~24× smaller (0.08 MB vs 1.96 MB for a 9-second segment). Use `--clip-format gif`
276
+ only when pasting into chat apps or note tools.
277
+
278
+ ---
279
+
280
+ ## Subtitle term normalisation
281
+
282
+ Auto-transcription mangles proper nouns. A built-in table normalises them:
283
+
284
+ ```
285
+ 大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
286
+ 用grock测试 → 用Grok测试
287
+ ```
288
+
289
+ The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
290
+ was transcribed as `deep sick`. The default table targets AI / tech content; bring your
291
+ own with `--terms my_terms.txt` (one `regex => replacement` per line, see
292
+ [`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt)).
293
+
294
+ This is term normalisation, not proofreading — it will not fix grammar or one-off errors.
295
+
296
+ ---
297
+
298
+ ## Getting footage manually
299
+
300
+ If you're not using the agent skill, cutmap itself doesn't download anything — it works
301
+ on local files. For YouTube / Bilibili, grab the video and subtitles with
302
+ [yt-dlp](https://github.com/yt-dlp/yt-dlp):
303
+
304
+ ```bash
305
+ yt-dlp --write-auto-subs --sub-langs "en,zh-Hans" --convert-subs srt \
306
+ -f "bv*[height<=720]+ba/b" -o "%(id)s.%(ext)s" "<URL>"
307
+ cutmap <id>.mp4
308
+ ```
309
+
310
+ Respect the terms of service and copyright of wherever your footage comes from; use
311
+ downloaded material for personal study and research only.
312
+
313
+ ---
314
+
315
+ <details>
316
+ <summary><b>Troubleshooting — bugs I hit building this</b></summary>
317
+
318
+ <br>
319
+
320
+ Most of these **never raised an error** — they just quietly produced wrong output:
321
+
322
+ **`\b` word boundaries fail in mixed CJK/Latin text.** CJK characters are `\w` in Python,
323
+ so in `用grock测试` there is no boundary between `用` and `g`. `\bgrock\b` never matches.
324
+ The term table uses `(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
325
+
326
+ **Flat frames can't be detected by standard deviation.** Black transition cards usually
327
+ have burned-in white subtitles, pushing stddev to 23–33. Use mean luminance, kept tight
328
+ (`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
329
+
330
+ **Subtitle time windows must not overlap.** A look-back to avoid empty captions makes
331
+ adjacent frames claim the same cue. Hence two fields: `subtitle` (display, may repeat)
332
+ and `subtitle_own` (concatenation, strict half-open).
333
+
334
+ **ffmpeg concat resolves relative paths against the list file's directory** — contact
335
+ sheet file lists must contain absolute paths.
336
+
337
+ **`-v error` suppresses `showinfo` output** — any ffmpeg statistics gathered with it
338
+ silently come back as zero.
339
+
340
+ **Multi-threaded downloads corrupt files on exFAT.** Pre-allocating and writing at
341
+ multiple offsets over exFAT-on-USB yields files with the right size and wrong contents
342
+ (`moov atom not found`). Download to local disk, then move.
343
+
344
+ </details>
345
+
346
+ ---
347
+
348
+ ## Credits
349
+
350
+ Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
351
+
352
+ Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**. This project started as an
353
+ attempt to study the editing in his videos — dense cutting, interleaved infographics and
354
+ animated demos — exactly the material naive interval-screenshotting handles worst. The
355
+ screenshots in this README are from
356
+ **[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely
357
+ to demonstrate this tool's output; all rights remain with the original creator. If he
358
+ would prefer them removed, please open an issue and they will be replaced immediately.
359
+
360
+ | | |
361
+ |---|---|
362
+ | Source video | <https://www.youtube.com/watch?v=WDQjRzVcX-A> |
363
+ | Creator | YouTube <https://www.youtube.com/@lyi> · Bilibili <https://space.bilibili.com/4401694> |
364
+
365
+ ## License
366
+
367
+ MIT
cutmap-0.2.0/README.md ADDED
@@ -0,0 +1,317 @@
1
+ <div align="center">
2
+
3
+ # cutmap
4
+
5
+ **Read a video like a document.**
6
+ Skim a long video in minutes to learn fast — or study it frame-by-frame to see exactly how it's cut.
7
+
8
+ [![English](https://img.shields.io/badge/lang-English-2b7489?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.md)
9
+ [![简体中文](https://img.shields.io/badge/lang-简体中文-lightgrey?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.zh-CN.md)
10
+ [![PyPI](https://img.shields.io/pypi/v/cutmap?style=flat-square&color=3775a9)](https://pypi.org/project/cutmap/)
11
+ [![CI](https://github.com/xykong36/cutmap/actions/workflows/ci.yml/badge.svg)](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
12
+ [![License: MIT](https://img.shields.io/badge/license-MIT-blue?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/LICENSE)
13
+ [![Python](https://img.shields.io/badge/python-3.9+-3776ab?style=flat-square&logo=python&logoColor=white)](https://www.python.org)
14
+ [![No LLM](https://img.shields.io/badge/LLM-not%20required-success?style=flat-square)](#)
15
+
16
+ </div>
17
+
18
+ Watching a video is slow and linear — you can't skim it the way you skim an article,
19
+ and you can't Ctrl-F a moment. cutmap fixes that. Give it a video (and subtitles if you
20
+ have them) and it builds a single HTML page you can actually **read**: every visually
21
+ distinct frame laid out in order, each captioned with what was being said. Click a frame
22
+ to jump the player there; search the subtitles to find a moment.
23
+
24
+ ![storyboard](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/storyboard.png)
25
+
26
+ <sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
27
+
28
+ Two things people use it for:
29
+
30
+ - 📖 **Learn faster.** Turn a one-hour talk or tutorial into a page you scroll in a few
31
+ minutes. See the whole structure at a glance, search the transcript, and only play the
32
+ parts you actually need.
33
+ - 🎬 **Study the edit ("拉片").** If you make videos, see exactly how someone cut theirs —
34
+ every shot change, every B-roll insert, every on-screen graphic — laid out as one storyboard.
35
+
36
+ It runs entirely on your machine. **No LLM, no API key, no network.**
37
+
38
+ ---
39
+
40
+ ## Quick start
41
+
42
+ ### The easy way — just ask your AI agent
43
+
44
+ cutmap ships with a [Claude Code](https://claude.com/claude-code) skill. Install it once,
45
+ then talk in plain language — the agent downloads the video, runs cutmap, reads the
46
+ result, and answers you:
47
+
48
+ ```bash
49
+ pip install cutmap yt-dlp # cutmap + video downloader
50
+ brew install ffmpeg # the video engine (Linux: apt install ffmpeg)
51
+
52
+ # install the bundled skill for Claude Code
53
+ git clone https://github.com/xykong36/cutmap
54
+ cp -r cutmap/skills/cutmap ~/.claude/skills/
55
+ ```
56
+
57
+ Then just say things like:
58
+
59
+ > **Download** `https://youtu.be/…` **and use cutmap to summarize what it covers**
60
+ >
61
+ > **Use cutmap on** `lecture.mp4` **and give me the section-by-section breakdown**
62
+ >
63
+ > **What's on screen around 3:41 in that video?** (process it with cutmap first)
64
+
65
+ The agent handles the download, the flags, and reading the frames for you — you never
66
+ touch the command line.
67
+
68
+ ### The manual way — command line
69
+
70
+ ```bash
71
+ pip install cutmap
72
+ brew install ffmpeg # macOS (Linux: apt install ffmpeg)
73
+
74
+ cutmap video.mp4 --srt video.srt # or just: cutmap video.mp4 (finds video.srt)
75
+ open video/browse.html # ← open this in your browser
76
+ ```
77
+
78
+ That's it. Everything you need is in the `video/` folder next to your file.
79
+
80
+ ---
81
+
82
+ ## What you get
83
+
84
+ Output lands in a directory named after the video:
85
+
86
+ ```
87
+ video/
88
+ ├── browse.html ← open this: the interactive storyboard
89
+ ├── frames/ every distinct frame, labelled #index timecode seconds
90
+ ├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
91
+ ├── index.json frame timestamps + aligned subtitles (machine readable)
92
+ └── broll/ extracted B-roll clips + broll.json
93
+ ```
94
+
95
+ ### The browse page
96
+
97
+ One self-contained HTML file — all CSS and JS inlined. Two tabs:
98
+
99
+ **Storyboard** — every distinct frame with its caption
100
+ **B-roll** — auto-looping clips (reads like a GIF wall)
101
+
102
+ ![B-roll clips auto-looping](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/broll-demo.gif)
103
+
104
+ <sub>The B-roll tab in motion: clips auto-loop like a GIF wall, but they are MP4 under the hood — 1/24 the size of actual GIFs</sub>
105
+
106
+ What you can do on it:
107
+
108
+ - **Click any frame or timecode** to seek the embedded player there (local file, no network)
109
+ - **Search the subtitles** — matches the *normalised* text, so searching `DeepSeek` also
110
+ finds frames where the auto-transcription wrote `deep sick`
111
+ - Long captions collapse behind `…expand`
112
+ - Follows your system light / dark mode
113
+
114
+ Contact sheets are also written on their own, for scanning a whole video in a few images:
115
+
116
+ ![contact sheet](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/contact-sheet.jpg)
117
+
118
+ ---
119
+
120
+ ## How it works
121
+
122
+ The interesting part is *which* frames it keeps. Sampling every N seconds is both
123
+ redundant and lossy: a 30-second talking-head passage gives you ten identical stills;
124
+ a fast montage cutting five shots in three seconds gives you one.
125
+
126
+ cutmap asks a different question — not *"how often should I sample"* but
127
+ **"which frames actually differ"**:
128
+
129
+ ```
130
+ dense sample → perceptual hash each frame → compare against last kept frame
131
+ → keep only if different enough
132
+ ```
133
+
134
+ Static passages collapse to a single frame. Every visual change gets one.
135
+
136
+ <details>
137
+ <summary><b>Why not ffmpeg's built-in scene detection?</b></summary>
138
+
139
+ <br>
140
+
141
+ `select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
142
+
143
+ 1. **Low-contrast transitions** — white slide to white webpage. Similar overall
144
+ luminance structure, so the difference score never crosses the threshold.
145
+ 2. **Intra-shot evolution** — text appearing line by line, charts animating,
146
+ table rows highlighting. Not a "cut" by any definition, but visually distinct.
147
+
148
+ Measured on one 16-minute video: scene detection found **104** frames,
149
+ perceptual dedup found **608**. One 25-second stretch that scene detection called
150
+ a single shot actually contained three completely different pages.
151
+
152
+ </details>
153
+
154
+ <details>
155
+ <summary><b>The algorithm's blind spot</b></summary>
156
+
157
+ <br>
158
+
159
+ dHash compares **relative brightness between adjacent pixels**, so it measures spatial
160
+ structure, not colour. Two different flat colours produce an identical (all-zero)
161
+ fingerprint and cannot be told apart. Footage that is largely flat-coloured or
162
+ near-black will therefore dedup more aggressively than you might expect. (This is also
163
+ why transition cards are detected by mean luminance rather than by hash.)
164
+
165
+ </details>
166
+
167
+ ---
168
+
169
+ ## Options
170
+
171
+ Most people never need these — the defaults are tuned for slides, screencasts, and
172
+ talks. Reach for them when the frame count comes out too dense or too sparse.
173
+
174
+ | Flag | Meaning |
175
+ |---|---|
176
+ | `--threshold N` | Frame density. **Lower = more frames.** `6` dense / `10` default / `14` sparse / `24` structure-only |
177
+ | `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
178
+ | `--seg-max N` | Max seconds per B-roll segment (default 45) |
179
+ | `--clip-format` | `mp4` (default) / `gif` / `webp` |
180
+ | `--no-broll` | Skip B-roll extraction (much faster) |
181
+ | `--no-frames` | Keep only contact sheets, drop individual frames |
182
+ | `--cols/--rows` | Contact sheet grid (default 4×4) |
183
+ | `--thumb-width` | Thumbnail width in px (default 480) |
184
+ | `--terms FILE` | Custom subtitle term-normalisation table |
185
+
186
+ **Rule of thumb for density:** screencasts and slides → keep the default `10`. Real
187
+ handheld / vlog / interview footage → start at `--threshold 20` (camera shake makes
188
+ every frame look "different" otherwise).
189
+
190
+ <details>
191
+ <summary><b>Density, measured</b></summary>
192
+
193
+ <br>
194
+
195
+ On a 969-second video (2 fps sampling → 1936 raw frames):
196
+
197
+ | Threshold | Kept | Avg gap | Good for |
198
+ |---|---|---|---|
199
+ | 6 | 892 | 1.1s | Reconstructing every visual step (tutorial walkthroughs) |
200
+ | **10** | **597** | **1.6s** | **Default** — balances coverage and readability |
201
+ | 14 | 463 | 2.1s | Style study, quick scanning |
202
+ | 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
203
+
204
+ The curve is smooth — there is no natural breakpoint. This is an aesthetic choice,
205
+ not a parameter with a computable optimum.
206
+
207
+ </details>
208
+
209
+ ---
210
+
211
+ ## B-roll detection
212
+
213
+ Frames are sorted into three buckets by three plain rules — no model involved:
214
+
215
+ | Bucket | Rule |
216
+ |---|---|
217
+ | **Main shot** | A cluster of near-identical frames recurring across >50% of the runtime — i.e. a locked-off camera |
218
+ | **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
219
+ | **B-roll** | Everything else |
220
+
221
+ Screencast-style videos with no talking head degrade gracefully — no main shot is
222
+ found, everything becomes B-roll, and `--seg-max` keeps it segmented.
223
+
224
+ Clips default to **MP4** (`loop muted`), which looks identical to a GIF in the page but
225
+ is ~24× smaller (0.08 MB vs 1.96 MB for a 9-second segment). Use `--clip-format gif`
226
+ only when pasting into chat apps or note tools.
227
+
228
+ ---
229
+
230
+ ## Subtitle term normalisation
231
+
232
+ Auto-transcription mangles proper nouns. A built-in table normalises them:
233
+
234
+ ```
235
+ 大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
236
+ 用grock测试 → 用Grok测试
237
+ ```
238
+
239
+ The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
240
+ was transcribed as `deep sick`. The default table targets AI / tech content; bring your
241
+ own with `--terms my_terms.txt` (one `regex => replacement` per line, see
242
+ [`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt)).
243
+
244
+ This is term normalisation, not proofreading — it will not fix grammar or one-off errors.
245
+
246
+ ---
247
+
248
+ ## Getting footage manually
249
+
250
+ If you're not using the agent skill, cutmap itself doesn't download anything — it works
251
+ on local files. For YouTube / Bilibili, grab the video and subtitles with
252
+ [yt-dlp](https://github.com/yt-dlp/yt-dlp):
253
+
254
+ ```bash
255
+ yt-dlp --write-auto-subs --sub-langs "en,zh-Hans" --convert-subs srt \
256
+ -f "bv*[height<=720]+ba/b" -o "%(id)s.%(ext)s" "<URL>"
257
+ cutmap <id>.mp4
258
+ ```
259
+
260
+ Respect the terms of service and copyright of wherever your footage comes from; use
261
+ downloaded material for personal study and research only.
262
+
263
+ ---
264
+
265
+ <details>
266
+ <summary><b>Troubleshooting — bugs I hit building this</b></summary>
267
+
268
+ <br>
269
+
270
+ Most of these **never raised an error** — they just quietly produced wrong output:
271
+
272
+ **`\b` word boundaries fail in mixed CJK/Latin text.** CJK characters are `\w` in Python,
273
+ so in `用grock测试` there is no boundary between `用` and `g`. `\bgrock\b` never matches.
274
+ The term table uses `(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
275
+
276
+ **Flat frames can't be detected by standard deviation.** Black transition cards usually
277
+ have burned-in white subtitles, pushing stddev to 23–33. Use mean luminance, kept tight
278
+ (`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
279
+
280
+ **Subtitle time windows must not overlap.** A look-back to avoid empty captions makes
281
+ adjacent frames claim the same cue. Hence two fields: `subtitle` (display, may repeat)
282
+ and `subtitle_own` (concatenation, strict half-open).
283
+
284
+ **ffmpeg concat resolves relative paths against the list file's directory** — contact
285
+ sheet file lists must contain absolute paths.
286
+
287
+ **`-v error` suppresses `showinfo` output** — any ffmpeg statistics gathered with it
288
+ silently come back as zero.
289
+
290
+ **Multi-threaded downloads corrupt files on exFAT.** Pre-allocating and writing at
291
+ multiple offsets over exFAT-on-USB yields files with the right size and wrong contents
292
+ (`moov atom not found`). Download to local disk, then move.
293
+
294
+ </details>
295
+
296
+ ---
297
+
298
+ ## Credits
299
+
300
+ Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
301
+
302
+ Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**. This project started as an
303
+ attempt to study the editing in his videos — dense cutting, interleaved infographics and
304
+ animated demos — exactly the material naive interval-screenshotting handles worst. The
305
+ screenshots in this README are from
306
+ **[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely
307
+ to demonstrate this tool's output; all rights remain with the original creator. If he
308
+ would prefer them removed, please open an issue and they will be replaced immediately.
309
+
310
+ | | |
311
+ |---|---|
312
+ | Source video | <https://www.youtube.com/watch?v=WDQjRzVcX-A> |
313
+ | Creator | YouTube <https://www.youtube.com/@lyi> · Bilibili <https://space.bilibili.com/4401694> |
314
+
315
+ ## License
316
+
317
+ MIT