cutmap 0.1.1__tar.gz → 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: cutmap
3
- Version: 0.1.1
3
+ Version: 0.2.0
4
4
  Summary: Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll. Local ffmpeg + perceptual hashing, no LLM.
5
5
  Project-URL: Homepage, https://github.com/xykong36/cutmap
6
6
  Project-URL: Issues, https://github.com/xykong36/cutmap/issues
@@ -52,10 +52,11 @@ Description-Content-Type: text/markdown
52
52
 
53
53
  # cutmap
54
54
 
55
- **Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll.**
55
+ **Read a video like a document.**
56
+ Skim a long video in minutes to learn fast — or study it frame-by-frame to see exactly how it's cut.
56
57
 
57
- [![简体中文](https://img.shields.io/badge/lang-简体中文-lightgrey?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.md)
58
- [![English](https://img.shields.io/badge/lang-English-2b7489?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.en.md)
58
+ [![English](https://img.shields.io/badge/lang-English-2b7489?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.md)
59
+ [![简体中文](https://img.shields.io/badge/lang-简体中文-lightgrey?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.zh-CN.md)
59
60
  [![PyPI](https://img.shields.io/pypi/v/cutmap?style=flat-square&color=3775a9)](https://pypi.org/project/cutmap/)
60
61
  [![CI](https://github.com/xykong36/cutmap/actions/workflows/ci.yml/badge.svg)](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
61
62
  [![License: MIT](https://img.shields.io/badge/license-MIT-blue?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/LICENSE)
@@ -64,27 +65,113 @@ Description-Content-Type: text/markdown
64
65
 
65
66
  </div>
66
67
 
67
- Give it a video and a subtitle file. Get back a single HTML page: the source player on
68
- the left, every distinct frame on the right, each captioned with what was being said.
69
- Click any frame to jump the player to that moment. Search the subtitles to locate a shot.
68
+ Watching a video is slow and linear you can't skim it the way you skim an article,
69
+ and you can't Ctrl-F a moment. cutmap fixes that. Give it a video (and subtitles if you
70
+ have them) and it builds a single HTML page you can actually **read**: every visually
71
+ distinct frame laid out in order, each captioned with what was being said. Click a frame
72
+ to jump the player there; search the subtitles to find a moment.
73
+
74
+ ![storyboard](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/storyboard.png)
75
+
76
+ <sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
77
+
78
+ Two things people use it for:
79
+
80
+ - 📖 **Learn faster.** Turn a one-hour talk or tutorial into a page you scroll in a few
81
+ minutes. See the whole structure at a glance, search the transcript, and only play the
82
+ parts you actually need.
83
+ - 🎬 **Study the edit ("拉片").** If you make videos, see exactly how someone cut theirs —
84
+ every shot change, every B-roll insert, every on-screen graphic — laid out as one storyboard.
85
+
86
+ It runs entirely on your machine. **No LLM, no API key, no network.**
87
+
88
+ ---
89
+
90
+ ## Quick start
91
+
92
+ ### The easy way — just ask your AI agent
93
+
94
+ cutmap ships with a [Claude Code](https://claude.com/claude-code) skill. Install it once,
95
+ then talk in plain language — the agent downloads the video, runs cutmap, reads the
96
+ result, and answers you:
70
97
 
71
98
  ```bash
72
- cutmap video.mp4 --srt video.srt
73
- open video/浏览.html
99
+ pip install cutmap yt-dlp # cutmap + video downloader
100
+ brew install ffmpeg # the video engine (Linux: apt install ffmpeg)
101
+
102
+ # install the bundled skill for Claude Code
103
+ git clone https://github.com/xykong36/cutmap
104
+ cp -r cutmap/skills/cutmap ~/.claude/skills/
74
105
  ```
75
106
 
76
- Everything runs locally through `ffmpeg` + `Pillow`. **No model calls, no API keys, no network.**
107
+ Then just say things like:
77
108
 
78
- ![storyboard](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/storyboard.png)
109
+ > **Download** `https://youtu.be/…` **and use cutmap to summarize what it covers**
110
+ >
111
+ > **Use cutmap on** `lecture.mp4` **and give me the section-by-section breakdown**
112
+ >
113
+ > **What's on screen around 3:41 in that video?** (process it with cutmap first)
79
114
 
80
- <sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
115
+ The agent handles the download, the flags, and reading the frames for you — you never
116
+ touch the command line.
117
+
118
+ ### The manual way — command line
119
+
120
+ ```bash
121
+ pip install cutmap
122
+ brew install ffmpeg # macOS (Linux: apt install ffmpeg)
123
+
124
+ cutmap video.mp4 --srt video.srt # or just: cutmap video.mp4 (finds video.srt)
125
+ open video/browse.html # ← open this in your browser
126
+ ```
127
+
128
+ That's it. Everything you need is in the `video/` folder next to your file.
81
129
 
82
130
  ---
83
131
 
84
- ## Why not just screenshot every N seconds
132
+ ## What you get
85
133
 
86
- Interval sampling is both redundant and lossy. A 30-second talking-head passage yields
87
- ten identical stills; a fast montage cutting five shots in three seconds yields one.
134
+ Output lands in a directory named after the video:
135
+
136
+ ```
137
+ video/
138
+ ├── browse.html ← open this: the interactive storyboard
139
+ ├── frames/ every distinct frame, labelled #index timecode seconds
140
+ ├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
141
+ ├── index.json frame timestamps + aligned subtitles (machine readable)
142
+ └── broll/ extracted B-roll clips + broll.json
143
+ ```
144
+
145
+ ### The browse page
146
+
147
+ One self-contained HTML file — all CSS and JS inlined. Two tabs:
148
+
149
+ **Storyboard** — every distinct frame with its caption
150
+ **B-roll** — auto-looping clips (reads like a GIF wall)
151
+
152
+ ![B-roll clips auto-looping](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/broll-demo.gif)
153
+
154
+ <sub>The B-roll tab in motion: clips auto-loop like a GIF wall, but they are MP4 under the hood — 1/24 the size of actual GIFs</sub>
155
+
156
+ What you can do on it:
157
+
158
+ - **Click any frame or timecode** to seek the embedded player there (local file, no network)
159
+ - **Search the subtitles** — matches the *normalised* text, so searching `DeepSeek` also
160
+ finds frames where the auto-transcription wrote `deep sick`
161
+ - Long captions collapse behind `…expand`
162
+ - Follows your system light / dark mode
163
+
164
+ Contact sheets are also written on their own, for scanning a whole video in a few images:
165
+
166
+ ![contact sheet](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/contact-sheet.jpg)
167
+
168
+ ---
169
+
170
+ ## How it works
171
+
172
+ The interesting part is *which* frames it keeps. Sampling every N seconds is both
173
+ redundant and lossy: a 30-second talking-head passage gives you ten identical stills;
174
+ a fast montage cutting five shots in three seconds gives you one.
88
175
 
89
176
  cutmap asks a different question — not *"how often should I sample"* but
90
177
  **"which frames actually differ"**:
@@ -96,7 +183,10 @@ dense sample → perceptual hash each frame → compare against last kept fr
96
183
 
97
184
  Static passages collapse to a single frame. Every visual change gets one.
98
185
 
99
- ### Why not ffmpeg's built-in scene detection
186
+ <details>
187
+ <summary><b>Why not ffmpeg's built-in scene detection?</b></summary>
188
+
189
+ <br>
100
190
 
101
191
  `select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
102
192
 
@@ -109,69 +199,50 @@ Measured on one 16-minute video: scene detection found **104** frames,
109
199
  perceptual dedup found **608**. One 25-second stretch that scene detection called
110
200
  a single shot actually contained three completely different pages.
111
201
 
202
+ </details>
203
+
204
+ <details>
205
+ <summary><b>The algorithm's blind spot</b></summary>
112
206
 
113
- ### A note on the algorithm's blind spot
207
+ <br>
114
208
 
115
209
  dHash compares **relative brightness between adjacent pixels**, so it measures spatial
116
210
  structure, not colour. Two different flat colours produce an identical (all-zero)
117
211
  fingerprint and cannot be told apart. Footage that is largely flat-coloured or
118
- near-black will therefore dedup more aggressively than you might expect.
119
-
120
- This is also why transition cards are detected by mean luminance rather than by hash.
121
-
122
- ---
123
-
124
- ## Install
125
-
126
- ```bash
127
- pip install cutmap
128
- ```
212
+ near-black will therefore dedup more aggressively than you might expect. (This is also
213
+ why transition cards are detected by mean luminance rather than by hash.)
129
214
 
130
- Requires **ffmpeg** on your `PATH`:
131
-
132
- ```bash
133
- brew install ffmpeg # macOS
134
- apt install ffmpeg # Debian / Ubuntu
135
- ```
215
+ </details>
136
216
 
137
217
  ---
138
218
 
139
- ## Usage
140
-
141
- ```bash
142
- cutmap video.mp4 # looks for video.srt alongside
143
- cutmap video.mp4 --srt subs.srt
144
- cutmap ./material-dir/ # dir containing 源片.mp4 + 字幕.srt
145
- ```
146
-
147
- Output lands in a directory named after the video:
148
-
149
- ```
150
- video/
151
- ├── frames/ every distinct frame, labelled #index timecode seconds
152
- ├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
153
- ├── index.json frame timestamps + aligned subtitles (machine readable)
154
- ├── broll/ extracted B-roll clips + broll.json
155
- └── 浏览.html ← open this
156
- ```
219
+ ## Options
157
220
 
158
- ### Options
221
+ Most people never need these — the defaults are tuned for slides, screencasts, and
222
+ talks. Reach for them when the frame count comes out too dense or too sparse.
159
223
 
160
224
  | Flag | Meaning |
161
225
  |---|---|
162
- | `--threshold N` | Frame density. Lower = denser. `6` dense / `10` default / `14` sparse |
226
+ | `--threshold N` | Frame density. **Lower = more frames.** `6` dense / `10` default / `14` sparse / `24` structure-only |
163
227
  | `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
164
- | `--cols/--rows` | Contact sheet grid (default 4×4) |
165
- | `--thumb-width` | Thumbnail width in px (default 480) |
166
228
  | `--seg-max N` | Max seconds per B-roll segment (default 45) |
167
229
  | `--clip-format` | `mp4` (default) / `gif` / `webp` |
168
- | `--no-broll` | Skip B-roll extraction |
230
+ | `--no-broll` | Skip B-roll extraction (much faster) |
169
231
  | `--no-frames` | Keep only contact sheets, drop individual frames |
232
+ | `--cols/--rows` | Contact sheet grid (default 4×4) |
233
+ | `--thumb-width` | Thumbnail width in px (default 480) |
170
234
  | `--terms FILE` | Custom subtitle term-normalisation table |
171
235
 
172
- ### Picking a density
236
+ **Rule of thumb for density:** screencasts and slides → keep the default `10`. Real
237
+ handheld / vlog / interview footage → start at `--threshold 20` (camera shake makes
238
+ every frame look "different" otherwise).
173
239
 
174
- Measured on a 969-second video (2 fps sampling → 1936 raw frames):
240
+ <details>
241
+ <summary><b>Density, measured</b></summary>
242
+
243
+ <br>
244
+
245
+ On a 969-second video (2 fps sampling → 1936 raw frames):
175
246
 
176
247
  | Threshold | Kept | Avg gap | Good for |
177
248
  |---|---|---|---|
@@ -180,35 +251,10 @@ Measured on a 969-second video (2 fps sampling → 1936 raw frames):
180
251
  | 14 | 463 | 2.1s | Style study, quick scanning |
181
252
  | 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
182
253
 
183
- The curve is smooth — there is no natural breakpoint. This is an **aesthetic choice,
184
- not a parameter with a computable optimum.**
185
-
186
- ---
254
+ The curve is smooth — there is no natural breakpoint. This is an aesthetic choice,
255
+ not a parameter with a computable optimum.
187
256
 
188
- ## The browse page
189
-
190
- One self-contained HTML file. All CSS and JS inlined; only depends on the images and
191
- clips sitting next to it. Two tabs:
192
-
193
- **Storyboard** — every distinct frame with its caption
194
- **B-roll** — auto-looping clips (reads like a GIF wall)
195
-
196
- ![b-roll](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/broll.png)
197
-
198
- Shared behaviour:
199
-
200
- - Embedded source player up top — **click any timecode or frame to seek there**
201
- (local file, no network)
202
- - Live subtitle search, matching against the **normalised** text
203
- (searching `DeepSeek` finds frames whose ASR wrote `deep sick`)
204
- - Long captions collapse behind `…expand`
205
- - B-roll clips play only while in viewport (IntersectionObserver) — dozens of
206
- simultaneously decoding videos will freeze a browser
207
- - Follows system light / dark mode
208
-
209
- Contact sheets are written separately, for scanning a whole video in a few images:
210
-
211
- ![contact sheet](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/contact-sheet.jpg)
257
+ </details>
212
258
 
213
259
  ---
214
260
 
@@ -222,30 +268,18 @@ Frames are sorted into three buckets by three plain rules — no model involved:
222
268
  | **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
223
269
  | **B-roll** | Everything else |
224
270
 
225
- Measured on one episode: B-roll 49 segments / 731s (75.5%),
226
- main shot 39 / 217s (22.4%), transitions 16 / 20s (2.1%).
227
-
228
271
  Screencast-style videos with no talking head degrade gracefully — no main shot is
229
272
  found, everything becomes B-roll, and `--seg-max` keeps it segmented.
230
273
 
231
- ### Clip format
232
-
233
- Same nine-second segment:
234
-
235
- | Format | Size | Note |
236
- |---|---|---|
237
- | GIF 10fps | 1.96 MB | 256 colours; visible banding on gradients |
238
- | WebP | 0.39 MB | Middle ground |
239
- | **MP4** | **0.08 MB** | **Default** — 24× smaller than GIF, better quality |
240
-
241
- MP4 with `loop muted` is visually indistinguishable from a GIF in the page.
242
- Reach for `--clip-format gif` only when pasting into chat apps or note tools.
274
+ Clips default to **MP4** (`loop muted`), which looks identical to a GIF in the page but
275
+ is ~24× smaller (0.08 MB vs 1.96 MB for a 9-second segment). Use `--clip-format gif`
276
+ only when pasting into chat apps or note tools.
243
277
 
244
278
  ---
245
279
 
246
280
  ## Subtitle term normalisation
247
281
 
248
- ASR mangles proper nouns. A built-in table normalises them:
282
+ Auto-transcription mangles proper nouns. A built-in table normalises them:
249
283
 
250
284
  ```
251
285
  大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
@@ -253,70 +287,61 @@ ASR mangles proper nouns. A built-in table normalises them:
253
287
  ```
254
288
 
255
289
  The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
256
- was transcribed as `deep sick`.
290
+ was transcribed as `deep sick`. The default table targets AI / tech content; bring your
291
+ own with `--terms my_terms.txt` (one `regex => replacement` per line, see
292
+ [`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt)).
257
293
 
258
- The default table targets AI / tech content. Bring your own:
259
-
260
- ```bash
261
- cutmap video.mp4 --terms my_terms.txt
262
- ```
263
-
264
- Format is one `regex => replacement` per line — see
265
- [`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt).
266
-
267
- **This is term normalisation, not proofreading.** It will not fix segmentation,
268
- grammar, or one-off errors — those need a language model, which is out of scope here.
294
+ This is term normalisation, not proofreading it will not fix grammar or one-off errors.
269
295
 
270
296
  ---
271
297
 
272
- ## Troubleshooting
298
+ ## Getting footage manually
273
299
 
274
- Bugs hit while building this. Most of them **never raised an error**they just
275
- quietly produced wrong output:
300
+ If you're not using the agent skill, cutmap itself doesn't download anythingit works
301
+ on local files. For YouTube / Bilibili, grab the video and subtitles with
302
+ [yt-dlp](https://github.com/yt-dlp/yt-dlp):
276
303
 
277
- **`\b` word boundaries fail in mixed CJK/Latin text**
278
- CJK characters are `\w` in Python, so in `用grock测试` there is no boundary between
279
- `用` and `g`. `\bgrock\b` never matches. The term table uses
280
- `(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
304
+ ```bash
305
+ yt-dlp --write-auto-subs --sub-langs "en,zh-Hans" --convert-subs srt \
306
+ -f "bv*[height<=720]+ba/b" -o "%(id)s.%(ext)s" "<URL>"
307
+ cutmap <id>.mp4
308
+ ```
281
309
 
282
- **Flat frames can't be detected by standard deviation**
283
- Black transition cards usually have burned-in white subtitles, pushing stddev to
284
- 23–33 — far above any "flat colour" threshold. Use mean luminance, and keep it tight
285
- (`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
310
+ Respect the terms of service and copyright of wherever your footage comes from; use
311
+ downloaded material for personal study and research only.
286
312
 
287
- **Subtitle time windows must not overlap**
288
- Adding a look-back to avoid empty captions makes adjacent frames claim the same cue,
289
- producing duplicated sentences when concatenated. Hence two fields:
290
- `subtitle` (display, may repeat) and `subtitle_own` (concatenation, strict half-open).
313
+ ---
291
314
 
292
- **ffmpeg concat resolves relative paths against the list file's directory**
293
- Contact sheet file lists must contain absolute paths.
315
+ <details>
316
+ <summary><b>Troubleshooting bugs I hit building this</b></summary>
294
317
 
295
- **`-v error` suppresses `showinfo` output**
296
- Any ffmpeg-based statistics gathered with `-v error` silently come back as zero.
318
+ <br>
297
319
 
298
- **Multi-threaded downloads corrupt files on exFAT** (if you fetch footage yourself)
299
- Pre-allocating and writing at multiple offsets over exFAT-on-USB yields files with
300
- the right size and wrong contents — `moov atom not found`. Download to local disk,
301
- then move.
320
+ Most of these **never raised an error** they just quietly produced wrong output:
302
321
 
303
- ---
322
+ **`\b` word boundaries fail in mixed CJK/Latin text.** CJK characters are `\w` in Python,
323
+ so in `用grock测试` there is no boundary between `用` and `g`. `\bgrock\b` never matches.
324
+ The term table uses `(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
304
325
 
305
- ## Getting footage
326
+ **Flat frames can't be detected by standard deviation.** Black transition cards usually
327
+ have burned-in white subtitles, pushing stddev to 23–33. Use mean luminance, kept tight
328
+ (`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
306
329
 
307
- cutmap does not download anything it works on local files.
330
+ **Subtitle time windows must not overlap.** A look-back to avoid empty captions makes
331
+ adjacent frames claim the same cue. Hence two fields: `subtitle` (display, may repeat)
332
+ and `subtitle_own` (concatenation, strict half-open).
308
333
 
309
- For Bilibili, [BBDown](https://github.com/nilaoda/BBDown) muxes subtitles into the mp4;
310
- pull them out and hand both to cutmap:
334
+ **ffmpeg concat resolves relative paths against the list file's directory** — contact
335
+ sheet file lists must contain absolute paths.
311
336
 
312
- ```bash
313
- BBDown <BV-id> --skip-ai false
314
- ffmpeg -i video.mp4 -map 0:m:language:chi subs.srt
315
- cutmap video.mp4 --srt subs.srt
316
- ```
337
+ **`-v error` suppresses `showinfo` output** — any ffmpeg statistics gathered with it
338
+ silently come back as zero.
339
+
340
+ **Multi-threaded downloads corrupt files on exFAT.** Pre-allocating and writing at
341
+ multiple offsets over exFAT-on-USB yields files with the right size and wrong contents
342
+ (`moov atom not found`). Download to local disk, then move.
317
343
 
318
- Respect the terms of service and copyright of wherever your footage comes from.
319
- Use downloaded material for personal study and research only.
344
+ </details>
320
345
 
321
346
  ---
322
347
 
@@ -324,15 +349,13 @@ Use downloaded material for personal study and research only.
324
349
 
325
350
  Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
326
351
 
327
- Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**.
328
-
329
- This project started as an attempt to study the editing in his videos dense cutting,
330
- interleaved infographics and animated demos. That is exactly the material naive
331
- interval-screenshotting handles worst, and it is what forced the perceptual-dedup
332
- approach in the first place. The screenshots in this README are from
333
- **[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely to demonstrate this tool's
334
- output; all rights remain with the original creator. If he would prefer them removed,
335
- please open an issue and they will be replaced immediately.
352
+ Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**. This project started as an
353
+ attempt to study the editing in his videos — dense cutting, interleaved infographics and
354
+ animated demos exactly the material naive interval-screenshotting handles worst. The
355
+ screenshots in this README are from
356
+ **[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely
357
+ to demonstrate this tool's output; all rights remain with the original creator. If he
358
+ would prefer them removed, please open an issue and they will be replaced immediately.
336
359
 
337
360
  | | |
338
361
  |---|---|
cutmap-0.2.0/README.md ADDED
@@ -0,0 +1,317 @@
1
+ <div align="center">
2
+
3
+ # cutmap
4
+
5
+ **Read a video like a document.**
6
+ Skim a long video in minutes to learn fast — or study it frame-by-frame to see exactly how it's cut.
7
+
8
+ [![English](https://img.shields.io/badge/lang-English-2b7489?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.md)
9
+ [![简体中文](https://img.shields.io/badge/lang-简体中文-lightgrey?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.zh-CN.md)
10
+ [![PyPI](https://img.shields.io/pypi/v/cutmap?style=flat-square&color=3775a9)](https://pypi.org/project/cutmap/)
11
+ [![CI](https://github.com/xykong36/cutmap/actions/workflows/ci.yml/badge.svg)](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
12
+ [![License: MIT](https://img.shields.io/badge/license-MIT-blue?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/LICENSE)
13
+ [![Python](https://img.shields.io/badge/python-3.9+-3776ab?style=flat-square&logo=python&logoColor=white)](https://www.python.org)
14
+ [![No LLM](https://img.shields.io/badge/LLM-not%20required-success?style=flat-square)](#)
15
+
16
+ </div>
17
+
18
+ Watching a video is slow and linear — you can't skim it the way you skim an article,
19
+ and you can't Ctrl-F a moment. cutmap fixes that. Give it a video (and subtitles if you
20
+ have them) and it builds a single HTML page you can actually **read**: every visually
21
+ distinct frame laid out in order, each captioned with what was being said. Click a frame
22
+ to jump the player there; search the subtitles to find a moment.
23
+
24
+ ![storyboard](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/storyboard.png)
25
+
26
+ <sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
27
+
28
+ Two things people use it for:
29
+
30
+ - 📖 **Learn faster.** Turn a one-hour talk or tutorial into a page you scroll in a few
31
+ minutes. See the whole structure at a glance, search the transcript, and only play the
32
+ parts you actually need.
33
+ - 🎬 **Study the edit ("拉片").** If you make videos, see exactly how someone cut theirs —
34
+ every shot change, every B-roll insert, every on-screen graphic — laid out as one storyboard.
35
+
36
+ It runs entirely on your machine. **No LLM, no API key, no network.**
37
+
38
+ ---
39
+
40
+ ## Quick start
41
+
42
+ ### The easy way — just ask your AI agent
43
+
44
+ cutmap ships with a [Claude Code](https://claude.com/claude-code) skill. Install it once,
45
+ then talk in plain language — the agent downloads the video, runs cutmap, reads the
46
+ result, and answers you:
47
+
48
+ ```bash
49
+ pip install cutmap yt-dlp # cutmap + video downloader
50
+ brew install ffmpeg # the video engine (Linux: apt install ffmpeg)
51
+
52
+ # install the bundled skill for Claude Code
53
+ git clone https://github.com/xykong36/cutmap
54
+ cp -r cutmap/skills/cutmap ~/.claude/skills/
55
+ ```
56
+
57
+ Then just say things like:
58
+
59
+ > **Download** `https://youtu.be/…` **and use cutmap to summarize what it covers**
60
+ >
61
+ > **Use cutmap on** `lecture.mp4` **and give me the section-by-section breakdown**
62
+ >
63
+ > **What's on screen around 3:41 in that video?** (process it with cutmap first)
64
+
65
+ The agent handles the download, the flags, and reading the frames for you — you never
66
+ touch the command line.
67
+
68
+ ### The manual way — command line
69
+
70
+ ```bash
71
+ pip install cutmap
72
+ brew install ffmpeg # macOS (Linux: apt install ffmpeg)
73
+
74
+ cutmap video.mp4 --srt video.srt # or just: cutmap video.mp4 (finds video.srt)
75
+ open video/browse.html # ← open this in your browser
76
+ ```
77
+
78
+ That's it. Everything you need is in the `video/` folder next to your file.
79
+
80
+ ---
81
+
82
+ ## What you get
83
+
84
+ Output lands in a directory named after the video:
85
+
86
+ ```
87
+ video/
88
+ ├── browse.html ← open this: the interactive storyboard
89
+ ├── frames/ every distinct frame, labelled #index timecode seconds
90
+ ├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
91
+ ├── index.json frame timestamps + aligned subtitles (machine readable)
92
+ └── broll/ extracted B-roll clips + broll.json
93
+ ```
94
+
95
+ ### The browse page
96
+
97
+ One self-contained HTML file — all CSS and JS inlined. Two tabs:
98
+
99
+ **Storyboard** — every distinct frame with its caption
100
+ **B-roll** — auto-looping clips (reads like a GIF wall)
101
+
102
+ ![B-roll clips auto-looping](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/broll-demo.gif)
103
+
104
+ <sub>The B-roll tab in motion: clips auto-loop like a GIF wall, but they are MP4 under the hood — 1/24 the size of actual GIFs</sub>
105
+
106
+ What you can do on it:
107
+
108
+ - **Click any frame or timecode** to seek the embedded player there (local file, no network)
109
+ - **Search the subtitles** — matches the *normalised* text, so searching `DeepSeek` also
110
+ finds frames where the auto-transcription wrote `deep sick`
111
+ - Long captions collapse behind `…expand`
112
+ - Follows your system light / dark mode
113
+
114
+ Contact sheets are also written on their own, for scanning a whole video in a few images:
115
+
116
+ ![contact sheet](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/contact-sheet.jpg)
117
+
118
+ ---
119
+
120
+ ## How it works
121
+
122
+ The interesting part is *which* frames it keeps. Sampling every N seconds is both
123
+ redundant and lossy: a 30-second talking-head passage gives you ten identical stills;
124
+ a fast montage cutting five shots in three seconds gives you one.
125
+
126
+ cutmap asks a different question — not *"how often should I sample"* but
127
+ **"which frames actually differ"**:
128
+
129
+ ```
130
+ dense sample → perceptual hash each frame → compare against last kept frame
131
+ → keep only if different enough
132
+ ```
133
+
134
+ Static passages collapse to a single frame. Every visual change gets one.
135
+
136
+ <details>
137
+ <summary><b>Why not ffmpeg's built-in scene detection?</b></summary>
138
+
139
+ <br>
140
+
141
+ `select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
142
+
143
+ 1. **Low-contrast transitions** — white slide to white webpage. Similar overall
144
+ luminance structure, so the difference score never crosses the threshold.
145
+ 2. **Intra-shot evolution** — text appearing line by line, charts animating,
146
+ table rows highlighting. Not a "cut" by any definition, but visually distinct.
147
+
148
+ Measured on one 16-minute video: scene detection found **104** frames,
149
+ perceptual dedup found **608**. One 25-second stretch that scene detection called
150
+ a single shot actually contained three completely different pages.
151
+
152
+ </details>
153
+
154
+ <details>
155
+ <summary><b>The algorithm's blind spot</b></summary>
156
+
157
+ <br>
158
+
159
+ dHash compares **relative brightness between adjacent pixels**, so it measures spatial
160
+ structure, not colour. Two different flat colours produce an identical (all-zero)
161
+ fingerprint and cannot be told apart. Footage that is largely flat-coloured or
162
+ near-black will therefore dedup more aggressively than you might expect. (This is also
163
+ why transition cards are detected by mean luminance rather than by hash.)
164
+
165
+ </details>
166
+
167
+ ---
168
+
169
+ ## Options
170
+
171
+ Most people never need these — the defaults are tuned for slides, screencasts, and
172
+ talks. Reach for them when the frame count comes out too dense or too sparse.
173
+
174
+ | Flag | Meaning |
175
+ |---|---|
176
+ | `--threshold N` | Frame density. **Lower = more frames.** `6` dense / `10` default / `14` sparse / `24` structure-only |
177
+ | `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
178
+ | `--seg-max N` | Max seconds per B-roll segment (default 45) |
179
+ | `--clip-format` | `mp4` (default) / `gif` / `webp` |
180
+ | `--no-broll` | Skip B-roll extraction (much faster) |
181
+ | `--no-frames` | Keep only contact sheets, drop individual frames |
182
+ | `--cols/--rows` | Contact sheet grid (default 4×4) |
183
+ | `--thumb-width` | Thumbnail width in px (default 480) |
184
+ | `--terms FILE` | Custom subtitle term-normalisation table |
185
+
186
+ **Rule of thumb for density:** screencasts and slides → keep the default `10`. Real
187
+ handheld / vlog / interview footage → start at `--threshold 20` (camera shake makes
188
+ every frame look "different" otherwise).
189
+
190
+ <details>
191
+ <summary><b>Density, measured</b></summary>
192
+
193
+ <br>
194
+
195
+ On a 969-second video (2 fps sampling → 1936 raw frames):
196
+
197
+ | Threshold | Kept | Avg gap | Good for |
198
+ |---|---|---|---|
199
+ | 6 | 892 | 1.1s | Reconstructing every visual step (tutorial walkthroughs) |
200
+ | **10** | **597** | **1.6s** | **Default** — balances coverage and readability |
201
+ | 14 | 463 | 2.1s | Style study, quick scanning |
202
+ | 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
203
+
204
+ The curve is smooth — there is no natural breakpoint. This is an aesthetic choice,
205
+ not a parameter with a computable optimum.
206
+
207
+ </details>
208
+
209
+ ---
210
+
211
+ ## B-roll detection
212
+
213
+ Frames are sorted into three buckets by three plain rules — no model involved:
214
+
215
+ | Bucket | Rule |
216
+ |---|---|
217
+ | **Main shot** | A cluster of near-identical frames recurring across >50% of the runtime — i.e. a locked-off camera |
218
+ | **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
219
+ | **B-roll** | Everything else |
220
+
221
+ Screencast-style videos with no talking head degrade gracefully — no main shot is
222
+ found, everything becomes B-roll, and `--seg-max` keeps it segmented.
223
+
224
+ Clips default to **MP4** (`loop muted`), which looks identical to a GIF in the page but
225
+ is ~24× smaller (0.08 MB vs 1.96 MB for a 9-second segment). Use `--clip-format gif`
226
+ only when pasting into chat apps or note tools.
227
+
228
+ ---
229
+
230
+ ## Subtitle term normalisation
231
+
232
+ Auto-transcription mangles proper nouns. A built-in table normalises them:
233
+
234
+ ```
235
+ 大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
236
+ 用grock测试 → 用Grok测试
237
+ ```
238
+
239
+ The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
240
+ was transcribed as `deep sick`. The default table targets AI / tech content; bring your
241
+ own with `--terms my_terms.txt` (one `regex => replacement` per line, see
242
+ [`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt)).
243
+
244
+ This is term normalisation, not proofreading — it will not fix grammar or one-off errors.
245
+
246
+ ---
247
+
248
+ ## Getting footage manually
249
+
250
+ If you're not using the agent skill, cutmap itself doesn't download anything — it works
251
+ on local files. For YouTube / Bilibili, grab the video and subtitles with
252
+ [yt-dlp](https://github.com/yt-dlp/yt-dlp):
253
+
254
+ ```bash
255
+ yt-dlp --write-auto-subs --sub-langs "en,zh-Hans" --convert-subs srt \
256
+ -f "bv*[height<=720]+ba/b" -o "%(id)s.%(ext)s" "<URL>"
257
+ cutmap <id>.mp4
258
+ ```
259
+
260
+ Respect the terms of service and copyright of wherever your footage comes from; use
261
+ downloaded material for personal study and research only.
262
+
263
+ ---
264
+
265
+ <details>
266
+ <summary><b>Troubleshooting — bugs I hit building this</b></summary>
267
+
268
+ <br>
269
+
270
+ Most of these **never raised an error** — they just quietly produced wrong output:
271
+
272
+ **`\b` word boundaries fail in mixed CJK/Latin text.** CJK characters are `\w` in Python,
273
+ so in `用grock测试` there is no boundary between `用` and `g`. `\bgrock\b` never matches.
274
+ The term table uses `(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
275
+
276
+ **Flat frames can't be detected by standard deviation.** Black transition cards usually
277
+ have burned-in white subtitles, pushing stddev to 23–33. Use mean luminance, kept tight
278
+ (`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
279
+
280
+ **Subtitle time windows must not overlap.** A look-back to avoid empty captions makes
281
+ adjacent frames claim the same cue. Hence two fields: `subtitle` (display, may repeat)
282
+ and `subtitle_own` (concatenation, strict half-open).
283
+
284
+ **ffmpeg concat resolves relative paths against the list file's directory** — contact
285
+ sheet file lists must contain absolute paths.
286
+
287
+ **`-v error` suppresses `showinfo` output** — any ffmpeg statistics gathered with it
288
+ silently come back as zero.
289
+
290
+ **Multi-threaded downloads corrupt files on exFAT.** Pre-allocating and writing at
291
+ multiple offsets over exFAT-on-USB yields files with the right size and wrong contents
292
+ (`moov atom not found`). Download to local disk, then move.
293
+
294
+ </details>
295
+
296
+ ---
297
+
298
+ ## Credits
299
+
300
+ Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
301
+
302
+ Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**. This project started as an
303
+ attempt to study the editing in his videos — dense cutting, interleaved infographics and
304
+ animated demos — exactly the material naive interval-screenshotting handles worst. The
305
+ screenshots in this README are from
306
+ **[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely
307
+ to demonstrate this tool's output; all rights remain with the original creator. If he
308
+ would prefer them removed, please open an issue and they will be replaced immediately.
309
+
310
+ | | |
311
+ |---|---|
312
+ | Source video | <https://www.youtube.com/watch?v=WDQjRzVcX-A> |
313
+ | Creator | YouTube <https://www.youtube.com/@lyi> · Bilibili <https://space.bilibili.com/4401694> |
314
+
315
+ ## License
316
+
317
+ MIT
@@ -4,9 +4,9 @@ build-backend = "hatchling.build"
4
4
 
5
5
  [project]
6
6
  name = "cutmap"
7
- version = "0.1.1"
7
+ version = "0.2.0"
8
8
  description = "Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll. Local ffmpeg + perceptual hashing, no LLM."
9
- readme = "README.en.md" # PyPI 面向全球用户,用英文;GitHub 首页仍是中文
9
+ readme = "README.md" # 英文主版,同时用于 PyPI GitHub 首页;中文见 README.zh-CN.md
10
10
  requires-python = ">=3.9"
11
11
  license = { file = "LICENSE" }
12
12
  keywords = ["video", "storyboard", "b-roll", "ffmpeg", "subtitle", "perceptual-hash", "分镜"]
@@ -1,2 +1,2 @@
1
1
  """cutmap —— 视频分镜分析:感知去重抽画面、B-roll 自动识别、图文对照浏览"""
2
- __version__ = "0.1.1"
2
+ __version__ = "0.2.0"
@@ -26,13 +26,13 @@ def _step(n: int, total: int, msg: str) -> None:
26
26
  def resolve(target: str, srt_opt: str | None):
27
27
  """输入 → (视频路径, 字幕路径或None, 工作目录)"""
28
28
  if os.path.isdir(target):
29
- mp4 = os.path.join(target, "源片.mp4")
29
+ mp4 = os.path.join(target, "source.mp4")
30
30
  if not os.path.exists(mp4):
31
31
  cand = glob.glob(os.path.join(target, "*.mp4"))
32
32
  if not cand:
33
33
  sys.exit(f"目录里没有 mp4: {target}")
34
34
  mp4 = max(cand, key=os.path.getsize)
35
- srt = srt_opt or os.path.join(target, "字幕.srt")
35
+ srt = srt_opt or os.path.join(target, "subtitle.srt")
36
36
  return mp4, (srt if os.path.exists(srt) else None), target
37
37
 
38
38
  if os.path.isfile(target):
@@ -102,8 +102,8 @@ def build(outdir: str, terms: str | None = None, log=print) -> str:
102
102
  {broll_pane}
103
103
  <script>{JS}</script></body></html>
104
104
  """
105
- out = os.path.join(outdir, "浏览.html")
105
+ out = os.path.join(outdir, "browse.html")
106
106
  open(out, "w", encoding="utf-8").write(page)
107
- log(f" ✓ 浏览.html ({os.path.getsize(out)//1024} KB, "
107
+ log(f" ✓ browse.html ({os.path.getsize(out)//1024} KB, "
108
108
  f"分镜 {len(fcards)} + B-roll {len(bcards)})")
109
109
  return out
cutmap-0.1.1/README.en.md DELETED
@@ -1,294 +0,0 @@
1
- <div align="center">
2
-
3
- # cutmap
4
-
5
- **Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll.**
6
-
7
- [![简体中文](https://img.shields.io/badge/lang-简体中文-lightgrey?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.md)
8
- [![English](https://img.shields.io/badge/lang-English-2b7489?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/README.en.md)
9
- [![PyPI](https://img.shields.io/pypi/v/cutmap?style=flat-square&color=3775a9)](https://pypi.org/project/cutmap/)
10
- [![CI](https://github.com/xykong36/cutmap/actions/workflows/ci.yml/badge.svg)](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
11
- [![License: MIT](https://img.shields.io/badge/license-MIT-blue?style=flat-square)](https://github.com/xykong36/cutmap/blob/main/LICENSE)
12
- [![Python](https://img.shields.io/badge/python-3.9+-3776ab?style=flat-square&logo=python&logoColor=white)](https://www.python.org)
13
- [![No LLM](https://img.shields.io/badge/LLM-not%20required-success?style=flat-square)](#)
14
-
15
- </div>
16
-
17
- Give it a video and a subtitle file. Get back a single HTML page: the source player on
18
- the left, every distinct frame on the right, each captioned with what was being said.
19
- Click any frame to jump the player to that moment. Search the subtitles to locate a shot.
20
-
21
- ```bash
22
- cutmap video.mp4 --srt video.srt
23
- open video/浏览.html
24
- ```
25
-
26
- Everything runs locally through `ffmpeg` + `Pillow`. **No model calls, no API keys, no network.**
27
-
28
- ![storyboard](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/storyboard.png)
29
-
30
- <sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
31
-
32
- ---
33
-
34
- ## Why not just screenshot every N seconds
35
-
36
- Interval sampling is both redundant and lossy. A 30-second talking-head passage yields
37
- ten identical stills; a fast montage cutting five shots in three seconds yields one.
38
-
39
- cutmap asks a different question — not *"how often should I sample"* but
40
- **"which frames actually differ"**:
41
-
42
- ```
43
- dense sample → perceptual hash each frame → compare against last kept frame
44
- → keep only if different enough
45
- ```
46
-
47
- Static passages collapse to a single frame. Every visual change gets one.
48
-
49
- ### Why not ffmpeg's built-in scene detection
50
-
51
- `select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
52
-
53
- 1. **Low-contrast transitions** — white slide to white webpage. Similar overall
54
- luminance structure, so the difference score never crosses the threshold.
55
- 2. **Intra-shot evolution** — text appearing line by line, charts animating,
56
- table rows highlighting. Not a "cut" by any definition, but visually distinct.
57
-
58
- Measured on one 16-minute video: scene detection found **104** frames,
59
- perceptual dedup found **608**. One 25-second stretch that scene detection called
60
- a single shot actually contained three completely different pages.
61
-
62
-
63
- ### A note on the algorithm's blind spot
64
-
65
- dHash compares **relative brightness between adjacent pixels**, so it measures spatial
66
- structure, not colour. Two different flat colours produce an identical (all-zero)
67
- fingerprint and cannot be told apart. Footage that is largely flat-coloured or
68
- near-black will therefore dedup more aggressively than you might expect.
69
-
70
- This is also why transition cards are detected by mean luminance rather than by hash.
71
-
72
- ---
73
-
74
- ## Install
75
-
76
- ```bash
77
- pip install cutmap
78
- ```
79
-
80
- Requires **ffmpeg** on your `PATH`:
81
-
82
- ```bash
83
- brew install ffmpeg # macOS
84
- apt install ffmpeg # Debian / Ubuntu
85
- ```
86
-
87
- ---
88
-
89
- ## Usage
90
-
91
- ```bash
92
- cutmap video.mp4 # looks for video.srt alongside
93
- cutmap video.mp4 --srt subs.srt
94
- cutmap ./material-dir/ # dir containing 源片.mp4 + 字幕.srt
95
- ```
96
-
97
- Output lands in a directory named after the video:
98
-
99
- ```
100
- video/
101
- ├── frames/ every distinct frame, labelled #index timecode seconds
102
- ├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
103
- ├── index.json frame timestamps + aligned subtitles (machine readable)
104
- ├── broll/ extracted B-roll clips + broll.json
105
- └── 浏览.html ← open this
106
- ```
107
-
108
- ### Options
109
-
110
- | Flag | Meaning |
111
- |---|---|
112
- | `--threshold N` | Frame density. Lower = denser. `6` dense / `10` default / `14` sparse |
113
- | `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
114
- | `--cols/--rows` | Contact sheet grid (default 4×4) |
115
- | `--thumb-width` | Thumbnail width in px (default 480) |
116
- | `--seg-max N` | Max seconds per B-roll segment (default 45) |
117
- | `--clip-format` | `mp4` (default) / `gif` / `webp` |
118
- | `--no-broll` | Skip B-roll extraction |
119
- | `--no-frames` | Keep only contact sheets, drop individual frames |
120
- | `--terms FILE` | Custom subtitle term-normalisation table |
121
-
122
- ### Picking a density
123
-
124
- Measured on a 969-second video (2 fps sampling → 1936 raw frames):
125
-
126
- | Threshold | Kept | Avg gap | Good for |
127
- |---|---|---|---|
128
- | 6 | 892 | 1.1s | Reconstructing every visual step (tutorial walkthroughs) |
129
- | **10** | **597** | **1.6s** | **Default** — balances coverage and readability |
130
- | 14 | 463 | 2.1s | Style study, quick scanning |
131
- | 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
132
-
133
- The curve is smooth — there is no natural breakpoint. This is an **aesthetic choice,
134
- not a parameter with a computable optimum.**
135
-
136
- ---
137
-
138
- ## The browse page
139
-
140
- One self-contained HTML file. All CSS and JS inlined; only depends on the images and
141
- clips sitting next to it. Two tabs:
142
-
143
- **Storyboard** — every distinct frame with its caption
144
- **B-roll** — auto-looping clips (reads like a GIF wall)
145
-
146
- ![b-roll](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/broll.png)
147
-
148
- Shared behaviour:
149
-
150
- - Embedded source player up top — **click any timecode or frame to seek there**
151
- (local file, no network)
152
- - Live subtitle search, matching against the **normalised** text
153
- (searching `DeepSeek` finds frames whose ASR wrote `deep sick`)
154
- - Long captions collapse behind `…expand`
155
- - B-roll clips play only while in viewport (IntersectionObserver) — dozens of
156
- simultaneously decoding videos will freeze a browser
157
- - Follows system light / dark mode
158
-
159
- Contact sheets are written separately, for scanning a whole video in a few images:
160
-
161
- ![contact sheet](https://raw.githubusercontent.com/xykong36/cutmap/main/examples/contact-sheet.jpg)
162
-
163
- ---
164
-
165
- ## B-roll detection
166
-
167
- Frames are sorted into three buckets by three plain rules — no model involved:
168
-
169
- | Bucket | Rule |
170
- |---|---|
171
- | **Main shot** | A cluster of near-identical frames recurring across >50% of the runtime — i.e. a locked-off camera |
172
- | **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
173
- | **B-roll** | Everything else |
174
-
175
- Measured on one episode: B-roll 49 segments / 731s (75.5%),
176
- main shot 39 / 217s (22.4%), transitions 16 / 20s (2.1%).
177
-
178
- Screencast-style videos with no talking head degrade gracefully — no main shot is
179
- found, everything becomes B-roll, and `--seg-max` keeps it segmented.
180
-
181
- ### Clip format
182
-
183
- Same nine-second segment:
184
-
185
- | Format | Size | Note |
186
- |---|---|---|
187
- | GIF 10fps | 1.96 MB | 256 colours; visible banding on gradients |
188
- | WebP | 0.39 MB | Middle ground |
189
- | **MP4** | **0.08 MB** | **Default** — 24× smaller than GIF, better quality |
190
-
191
- MP4 with `loop muted` is visually indistinguishable from a GIF in the page.
192
- Reach for `--clip-format gif` only when pasting into chat apps or note tools.
193
-
194
- ---
195
-
196
- ## Subtitle term normalisation
197
-
198
- ASR mangles proper nouns. A built-in table normalises them:
199
-
200
- ```
201
- 大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
202
- 用grock测试 → 用Grok测试
203
- ```
204
-
205
- The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
206
- was transcribed as `deep sick`.
207
-
208
- The default table targets AI / tech content. Bring your own:
209
-
210
- ```bash
211
- cutmap video.mp4 --terms my_terms.txt
212
- ```
213
-
214
- Format is one `regex => replacement` per line — see
215
- [`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt).
216
-
217
- **This is term normalisation, not proofreading.** It will not fix segmentation,
218
- grammar, or one-off errors — those need a language model, which is out of scope here.
219
-
220
- ---
221
-
222
- ## Troubleshooting
223
-
224
- Bugs hit while building this. Most of them **never raised an error** — they just
225
- quietly produced wrong output:
226
-
227
- **`\b` word boundaries fail in mixed CJK/Latin text**
228
- CJK characters are `\w` in Python, so in `用grock测试` there is no boundary between
229
- `用` and `g`. `\bgrock\b` never matches. The term table uses
230
- `(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
231
-
232
- **Flat frames can't be detected by standard deviation**
233
- Black transition cards usually have burned-in white subtitles, pushing stddev to
234
- 23–33 — far above any "flat colour" threshold. Use mean luminance, and keep it tight
235
- (`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
236
-
237
- **Subtitle time windows must not overlap**
238
- Adding a look-back to avoid empty captions makes adjacent frames claim the same cue,
239
- producing duplicated sentences when concatenated. Hence two fields:
240
- `subtitle` (display, may repeat) and `subtitle_own` (concatenation, strict half-open).
241
-
242
- **ffmpeg concat resolves relative paths against the list file's directory**
243
- Contact sheet file lists must contain absolute paths.
244
-
245
- **`-v error` suppresses `showinfo` output**
246
- Any ffmpeg-based statistics gathered with `-v error` silently come back as zero.
247
-
248
- **Multi-threaded downloads corrupt files on exFAT** (if you fetch footage yourself)
249
- Pre-allocating and writing at multiple offsets over exFAT-on-USB yields files with
250
- the right size and wrong contents — `moov atom not found`. Download to local disk,
251
- then move.
252
-
253
- ---
254
-
255
- ## Getting footage
256
-
257
- cutmap does not download anything — it works on local files.
258
-
259
- For Bilibili, [BBDown](https://github.com/nilaoda/BBDown) muxes subtitles into the mp4;
260
- pull them out and hand both to cutmap:
261
-
262
- ```bash
263
- BBDown <BV-id> --skip-ai false
264
- ffmpeg -i video.mp4 -map 0:m:language:chi subs.srt
265
- cutmap video.mp4 --srt subs.srt
266
- ```
267
-
268
- Respect the terms of service and copyright of wherever your footage comes from.
269
- Use downloaded material for personal study and research only.
270
-
271
- ---
272
-
273
- ## Credits
274
-
275
- Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
276
-
277
- Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**.
278
-
279
- This project started as an attempt to study the editing in his videos — dense cutting,
280
- interleaved infographics and animated demos. That is exactly the material naive
281
- interval-screenshotting handles worst, and it is what forced the perceptual-dedup
282
- approach in the first place. The screenshots in this README are from
283
- **[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely to demonstrate this tool's
284
- output; all rights remain with the original creator. If he would prefer them removed,
285
- please open an issue and they will be replaced immediately.
286
-
287
- | | |
288
- |---|---|
289
- | Source video | <https://www.youtube.com/watch?v=WDQjRzVcX-A> |
290
- | Creator | YouTube <https://www.youtube.com/@lyi> · Bilibili <https://space.bilibili.com/4401694> |
291
-
292
- ## License
293
-
294
- MIT
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes