cutmap 0.1.1__tar.gz → 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {cutmap-0.1.1 → cutmap-0.2.0}/PKG-INFO +182 -159
- cutmap-0.2.0/README.md +317 -0
- {cutmap-0.1.1 → cutmap-0.2.0}/pyproject.toml +2 -2
- {cutmap-0.1.1 → cutmap-0.2.0}/src/cutmap/__init__.py +1 -1
- {cutmap-0.1.1 → cutmap-0.2.0}/src/cutmap/cli.py +2 -2
- {cutmap-0.1.1 → cutmap-0.2.0}/src/cutmap/render.py +2 -2
- cutmap-0.1.1/README.en.md +0 -294
- {cutmap-0.1.1 → cutmap-0.2.0}/.gitignore +0 -0
- {cutmap-0.1.1 → cutmap-0.2.0}/LICENSE +0 -0
- {cutmap-0.1.1 → cutmap-0.2.0}/src/cutmap/_assets.py +0 -0
- {cutmap-0.1.1 → cutmap-0.2.0}/src/cutmap/broll.py +0 -0
- {cutmap-0.1.1 → cutmap-0.2.0}/src/cutmap/frames.py +0 -0
- {cutmap-0.1.1 → cutmap-0.2.0}/src/cutmap/subclean.py +0 -0
- {cutmap-0.1.1 → cutmap-0.2.0}/src/cutmap/terms.txt +0 -0
- {cutmap-0.1.1 → cutmap-0.2.0}/src/cutmap/util.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: cutmap
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.2.0
|
|
4
4
|
Summary: Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll. Local ffmpeg + perceptual hashing, no LLM.
|
|
5
5
|
Project-URL: Homepage, https://github.com/xykong36/cutmap
|
|
6
6
|
Project-URL: Issues, https://github.com/xykong36/cutmap/issues
|
|
@@ -52,10 +52,11 @@ Description-Content-Type: text/markdown
|
|
|
52
52
|
|
|
53
53
|
# cutmap
|
|
54
54
|
|
|
55
|
-
**
|
|
55
|
+
**Read a video like a document.**
|
|
56
|
+
Skim a long video in minutes to learn fast — or study it frame-by-frame to see exactly how it's cut.
|
|
56
57
|
|
|
57
|
-
[](https://github.com/xykong36/cutmap/blob/main/README.md)
|
|
59
|
+
[](https://github.com/xykong36/cutmap/blob/main/README.zh-CN.md)
|
|
59
60
|
[](https://pypi.org/project/cutmap/)
|
|
60
61
|
[](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
|
|
61
62
|
[](https://github.com/xykong36/cutmap/blob/main/LICENSE)
|
|
@@ -64,27 +65,113 @@ Description-Content-Type: text/markdown
|
|
|
64
65
|
|
|
65
66
|
</div>
|
|
66
67
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
68
|
+
Watching a video is slow and linear — you can't skim it the way you skim an article,
|
|
69
|
+
and you can't Ctrl-F a moment. cutmap fixes that. Give it a video (and subtitles if you
|
|
70
|
+
have them) and it builds a single HTML page you can actually **read**: every visually
|
|
71
|
+
distinct frame laid out in order, each captioned with what was being said. Click a frame
|
|
72
|
+
to jump the player there; search the subtitles to find a moment.
|
|
73
|
+
|
|
74
|
+

|
|
75
|
+
|
|
76
|
+
<sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
|
|
77
|
+
|
|
78
|
+
Two things people use it for:
|
|
79
|
+
|
|
80
|
+
- 📖 **Learn faster.** Turn a one-hour talk or tutorial into a page you scroll in a few
|
|
81
|
+
minutes. See the whole structure at a glance, search the transcript, and only play the
|
|
82
|
+
parts you actually need.
|
|
83
|
+
- 🎬 **Study the edit ("拉片").** If you make videos, see exactly how someone cut theirs —
|
|
84
|
+
every shot change, every B-roll insert, every on-screen graphic — laid out as one storyboard.
|
|
85
|
+
|
|
86
|
+
It runs entirely on your machine. **No LLM, no API key, no network.**
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## Quick start
|
|
91
|
+
|
|
92
|
+
### The easy way — just ask your AI agent
|
|
93
|
+
|
|
94
|
+
cutmap ships with a [Claude Code](https://claude.com/claude-code) skill. Install it once,
|
|
95
|
+
then talk in plain language — the agent downloads the video, runs cutmap, reads the
|
|
96
|
+
result, and answers you:
|
|
70
97
|
|
|
71
98
|
```bash
|
|
72
|
-
cutmap
|
|
73
|
-
|
|
99
|
+
pip install cutmap yt-dlp # cutmap + video downloader
|
|
100
|
+
brew install ffmpeg # the video engine (Linux: apt install ffmpeg)
|
|
101
|
+
|
|
102
|
+
# install the bundled skill for Claude Code
|
|
103
|
+
git clone https://github.com/xykong36/cutmap
|
|
104
|
+
cp -r cutmap/skills/cutmap ~/.claude/skills/
|
|
74
105
|
```
|
|
75
106
|
|
|
76
|
-
|
|
107
|
+
Then just say things like:
|
|
77
108
|
|
|
78
|
-
|
|
109
|
+
> **Download** `https://youtu.be/…` **and use cutmap to summarize what it covers**
|
|
110
|
+
>
|
|
111
|
+
> **Use cutmap on** `lecture.mp4` **and give me the section-by-section breakdown**
|
|
112
|
+
>
|
|
113
|
+
> **What's on screen around 3:41 in that video?** (process it with cutmap first)
|
|
79
114
|
|
|
80
|
-
|
|
115
|
+
The agent handles the download, the flags, and reading the frames for you — you never
|
|
116
|
+
touch the command line.
|
|
117
|
+
|
|
118
|
+
### The manual way — command line
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
pip install cutmap
|
|
122
|
+
brew install ffmpeg # macOS (Linux: apt install ffmpeg)
|
|
123
|
+
|
|
124
|
+
cutmap video.mp4 --srt video.srt # or just: cutmap video.mp4 (finds video.srt)
|
|
125
|
+
open video/browse.html # ← open this in your browser
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
That's it. Everything you need is in the `video/` folder next to your file.
|
|
81
129
|
|
|
82
130
|
---
|
|
83
131
|
|
|
84
|
-
##
|
|
132
|
+
## What you get
|
|
85
133
|
|
|
86
|
-
|
|
87
|
-
|
|
134
|
+
Output lands in a directory named after the video:
|
|
135
|
+
|
|
136
|
+
```
|
|
137
|
+
video/
|
|
138
|
+
├── browse.html ← open this: the interactive storyboard
|
|
139
|
+
├── frames/ every distinct frame, labelled #index timecode seconds
|
|
140
|
+
├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
|
|
141
|
+
├── index.json frame timestamps + aligned subtitles (machine readable)
|
|
142
|
+
└── broll/ extracted B-roll clips + broll.json
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
### The browse page
|
|
146
|
+
|
|
147
|
+
One self-contained HTML file — all CSS and JS inlined. Two tabs:
|
|
148
|
+
|
|
149
|
+
**Storyboard** — every distinct frame with its caption
|
|
150
|
+
**B-roll** — auto-looping clips (reads like a GIF wall)
|
|
151
|
+
|
|
152
|
+

|
|
153
|
+
|
|
154
|
+
<sub>The B-roll tab in motion: clips auto-loop like a GIF wall, but they are MP4 under the hood — 1/24 the size of actual GIFs</sub>
|
|
155
|
+
|
|
156
|
+
What you can do on it:
|
|
157
|
+
|
|
158
|
+
- **Click any frame or timecode** to seek the embedded player there (local file, no network)
|
|
159
|
+
- **Search the subtitles** — matches the *normalised* text, so searching `DeepSeek` also
|
|
160
|
+
finds frames where the auto-transcription wrote `deep sick`
|
|
161
|
+
- Long captions collapse behind `…expand`
|
|
162
|
+
- Follows your system light / dark mode
|
|
163
|
+
|
|
164
|
+
Contact sheets are also written on their own, for scanning a whole video in a few images:
|
|
165
|
+
|
|
166
|
+

|
|
167
|
+
|
|
168
|
+
---
|
|
169
|
+
|
|
170
|
+
## How it works
|
|
171
|
+
|
|
172
|
+
The interesting part is *which* frames it keeps. Sampling every N seconds is both
|
|
173
|
+
redundant and lossy: a 30-second talking-head passage gives you ten identical stills;
|
|
174
|
+
a fast montage cutting five shots in three seconds gives you one.
|
|
88
175
|
|
|
89
176
|
cutmap asks a different question — not *"how often should I sample"* but
|
|
90
177
|
**"which frames actually differ"**:
|
|
@@ -96,7 +183,10 @@ dense sample → perceptual hash each frame → compare against last kept fr
|
|
|
96
183
|
|
|
97
184
|
Static passages collapse to a single frame. Every visual change gets one.
|
|
98
185
|
|
|
99
|
-
|
|
186
|
+
<details>
|
|
187
|
+
<summary><b>Why not ffmpeg's built-in scene detection?</b></summary>
|
|
188
|
+
|
|
189
|
+
<br>
|
|
100
190
|
|
|
101
191
|
`select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
|
|
102
192
|
|
|
@@ -109,69 +199,50 @@ Measured on one 16-minute video: scene detection found **104** frames,
|
|
|
109
199
|
perceptual dedup found **608**. One 25-second stretch that scene detection called
|
|
110
200
|
a single shot actually contained three completely different pages.
|
|
111
201
|
|
|
202
|
+
</details>
|
|
203
|
+
|
|
204
|
+
<details>
|
|
205
|
+
<summary><b>The algorithm's blind spot</b></summary>
|
|
112
206
|
|
|
113
|
-
|
|
207
|
+
<br>
|
|
114
208
|
|
|
115
209
|
dHash compares **relative brightness between adjacent pixels**, so it measures spatial
|
|
116
210
|
structure, not colour. Two different flat colours produce an identical (all-zero)
|
|
117
211
|
fingerprint and cannot be told apart. Footage that is largely flat-coloured or
|
|
118
|
-
near-black will therefore dedup more aggressively than you might expect.
|
|
119
|
-
|
|
120
|
-
This is also why transition cards are detected by mean luminance rather than by hash.
|
|
121
|
-
|
|
122
|
-
---
|
|
123
|
-
|
|
124
|
-
## Install
|
|
125
|
-
|
|
126
|
-
```bash
|
|
127
|
-
pip install cutmap
|
|
128
|
-
```
|
|
212
|
+
near-black will therefore dedup more aggressively than you might expect. (This is also
|
|
213
|
+
why transition cards are detected by mean luminance rather than by hash.)
|
|
129
214
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
```bash
|
|
133
|
-
brew install ffmpeg # macOS
|
|
134
|
-
apt install ffmpeg # Debian / Ubuntu
|
|
135
|
-
```
|
|
215
|
+
</details>
|
|
136
216
|
|
|
137
217
|
---
|
|
138
218
|
|
|
139
|
-
##
|
|
140
|
-
|
|
141
|
-
```bash
|
|
142
|
-
cutmap video.mp4 # looks for video.srt alongside
|
|
143
|
-
cutmap video.mp4 --srt subs.srt
|
|
144
|
-
cutmap ./material-dir/ # dir containing 源片.mp4 + 字幕.srt
|
|
145
|
-
```
|
|
146
|
-
|
|
147
|
-
Output lands in a directory named after the video:
|
|
148
|
-
|
|
149
|
-
```
|
|
150
|
-
video/
|
|
151
|
-
├── frames/ every distinct frame, labelled #index timecode seconds
|
|
152
|
-
├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
|
|
153
|
-
├── index.json frame timestamps + aligned subtitles (machine readable)
|
|
154
|
-
├── broll/ extracted B-roll clips + broll.json
|
|
155
|
-
└── 浏览.html ← open this
|
|
156
|
-
```
|
|
219
|
+
## Options
|
|
157
220
|
|
|
158
|
-
|
|
221
|
+
Most people never need these — the defaults are tuned for slides, screencasts, and
|
|
222
|
+
talks. Reach for them when the frame count comes out too dense or too sparse.
|
|
159
223
|
|
|
160
224
|
| Flag | Meaning |
|
|
161
225
|
|---|---|
|
|
162
|
-
| `--threshold N` | Frame density. Lower =
|
|
226
|
+
| `--threshold N` | Frame density. **Lower = more frames.** `6` dense / `10` default / `14` sparse / `24` structure-only |
|
|
163
227
|
| `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
|
|
164
|
-
| `--cols/--rows` | Contact sheet grid (default 4×4) |
|
|
165
|
-
| `--thumb-width` | Thumbnail width in px (default 480) |
|
|
166
228
|
| `--seg-max N` | Max seconds per B-roll segment (default 45) |
|
|
167
229
|
| `--clip-format` | `mp4` (default) / `gif` / `webp` |
|
|
168
|
-
| `--no-broll` | Skip B-roll extraction |
|
|
230
|
+
| `--no-broll` | Skip B-roll extraction (much faster) |
|
|
169
231
|
| `--no-frames` | Keep only contact sheets, drop individual frames |
|
|
232
|
+
| `--cols/--rows` | Contact sheet grid (default 4×4) |
|
|
233
|
+
| `--thumb-width` | Thumbnail width in px (default 480) |
|
|
170
234
|
| `--terms FILE` | Custom subtitle term-normalisation table |
|
|
171
235
|
|
|
172
|
-
|
|
236
|
+
**Rule of thumb for density:** screencasts and slides → keep the default `10`. Real
|
|
237
|
+
handheld / vlog / interview footage → start at `--threshold 20` (camera shake makes
|
|
238
|
+
every frame look "different" otherwise).
|
|
173
239
|
|
|
174
|
-
|
|
240
|
+
<details>
|
|
241
|
+
<summary><b>Density, measured</b></summary>
|
|
242
|
+
|
|
243
|
+
<br>
|
|
244
|
+
|
|
245
|
+
On a 969-second video (2 fps sampling → 1936 raw frames):
|
|
175
246
|
|
|
176
247
|
| Threshold | Kept | Avg gap | Good for |
|
|
177
248
|
|---|---|---|---|
|
|
@@ -180,35 +251,10 @@ Measured on a 969-second video (2 fps sampling → 1936 raw frames):
|
|
|
180
251
|
| 14 | 463 | 2.1s | Style study, quick scanning |
|
|
181
252
|
| 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
|
|
182
253
|
|
|
183
|
-
The curve is smooth — there is no natural breakpoint. This is an
|
|
184
|
-
not a parameter with a computable optimum
|
|
185
|
-
|
|
186
|
-
---
|
|
254
|
+
The curve is smooth — there is no natural breakpoint. This is an aesthetic choice,
|
|
255
|
+
not a parameter with a computable optimum.
|
|
187
256
|
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
One self-contained HTML file. All CSS and JS inlined; only depends on the images and
|
|
191
|
-
clips sitting next to it. Two tabs:
|
|
192
|
-
|
|
193
|
-
**Storyboard** — every distinct frame with its caption
|
|
194
|
-
**B-roll** — auto-looping clips (reads like a GIF wall)
|
|
195
|
-
|
|
196
|
-

|
|
197
|
-
|
|
198
|
-
Shared behaviour:
|
|
199
|
-
|
|
200
|
-
- Embedded source player up top — **click any timecode or frame to seek there**
|
|
201
|
-
(local file, no network)
|
|
202
|
-
- Live subtitle search, matching against the **normalised** text
|
|
203
|
-
(searching `DeepSeek` finds frames whose ASR wrote `deep sick`)
|
|
204
|
-
- Long captions collapse behind `…expand`
|
|
205
|
-
- B-roll clips play only while in viewport (IntersectionObserver) — dozens of
|
|
206
|
-
simultaneously decoding videos will freeze a browser
|
|
207
|
-
- Follows system light / dark mode
|
|
208
|
-
|
|
209
|
-
Contact sheets are written separately, for scanning a whole video in a few images:
|
|
210
|
-
|
|
211
|
-

|
|
257
|
+
</details>
|
|
212
258
|
|
|
213
259
|
---
|
|
214
260
|
|
|
@@ -222,30 +268,18 @@ Frames are sorted into three buckets by three plain rules — no model involved:
|
|
|
222
268
|
| **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
|
|
223
269
|
| **B-roll** | Everything else |
|
|
224
270
|
|
|
225
|
-
Measured on one episode: B-roll 49 segments / 731s (75.5%),
|
|
226
|
-
main shot 39 / 217s (22.4%), transitions 16 / 20s (2.1%).
|
|
227
|
-
|
|
228
271
|
Screencast-style videos with no talking head degrade gracefully — no main shot is
|
|
229
272
|
found, everything becomes B-roll, and `--seg-max` keeps it segmented.
|
|
230
273
|
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
| Format | Size | Note |
|
|
236
|
-
|---|---|---|
|
|
237
|
-
| GIF 10fps | 1.96 MB | 256 colours; visible banding on gradients |
|
|
238
|
-
| WebP | 0.39 MB | Middle ground |
|
|
239
|
-
| **MP4** | **0.08 MB** | **Default** — 24× smaller than GIF, better quality |
|
|
240
|
-
|
|
241
|
-
MP4 with `loop muted` is visually indistinguishable from a GIF in the page.
|
|
242
|
-
Reach for `--clip-format gif` only when pasting into chat apps or note tools.
|
|
274
|
+
Clips default to **MP4** (`loop muted`), which looks identical to a GIF in the page but
|
|
275
|
+
is ~24× smaller (0.08 MB vs 1.96 MB for a 9-second segment). Use `--clip-format gif`
|
|
276
|
+
only when pasting into chat apps or note tools.
|
|
243
277
|
|
|
244
278
|
---
|
|
245
279
|
|
|
246
280
|
## Subtitle term normalisation
|
|
247
281
|
|
|
248
|
-
|
|
282
|
+
Auto-transcription mangles proper nouns. A built-in table normalises them:
|
|
249
283
|
|
|
250
284
|
```
|
|
251
285
|
大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
|
|
@@ -253,70 +287,61 @@ ASR mangles proper nouns. A built-in table normalises them:
|
|
|
253
287
|
```
|
|
254
288
|
|
|
255
289
|
The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
|
|
256
|
-
was transcribed as `deep sick`.
|
|
290
|
+
was transcribed as `deep sick`. The default table targets AI / tech content; bring your
|
|
291
|
+
own with `--terms my_terms.txt` (one `regex => replacement` per line, see
|
|
292
|
+
[`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt)).
|
|
257
293
|
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
```bash
|
|
261
|
-
cutmap video.mp4 --terms my_terms.txt
|
|
262
|
-
```
|
|
263
|
-
|
|
264
|
-
Format is one `regex => replacement` per line — see
|
|
265
|
-
[`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt).
|
|
266
|
-
|
|
267
|
-
**This is term normalisation, not proofreading.** It will not fix segmentation,
|
|
268
|
-
grammar, or one-off errors — those need a language model, which is out of scope here.
|
|
294
|
+
This is term normalisation, not proofreading — it will not fix grammar or one-off errors.
|
|
269
295
|
|
|
270
296
|
---
|
|
271
297
|
|
|
272
|
-
##
|
|
298
|
+
## Getting footage manually
|
|
273
299
|
|
|
274
|
-
|
|
275
|
-
|
|
300
|
+
If you're not using the agent skill, cutmap itself doesn't download anything — it works
|
|
301
|
+
on local files. For YouTube / Bilibili, grab the video and subtitles with
|
|
302
|
+
[yt-dlp](https://github.com/yt-dlp/yt-dlp):
|
|
276
303
|
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
304
|
+
```bash
|
|
305
|
+
yt-dlp --write-auto-subs --sub-langs "en,zh-Hans" --convert-subs srt \
|
|
306
|
+
-f "bv*[height<=720]+ba/b" -o "%(id)s.%(ext)s" "<URL>"
|
|
307
|
+
cutmap <id>.mp4
|
|
308
|
+
```
|
|
281
309
|
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
23–33 — far above any "flat colour" threshold. Use mean luminance, and keep it tight
|
|
285
|
-
(`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
|
|
310
|
+
Respect the terms of service and copyright of wherever your footage comes from; use
|
|
311
|
+
downloaded material for personal study and research only.
|
|
286
312
|
|
|
287
|
-
|
|
288
|
-
Adding a look-back to avoid empty captions makes adjacent frames claim the same cue,
|
|
289
|
-
producing duplicated sentences when concatenated. Hence two fields:
|
|
290
|
-
`subtitle` (display, may repeat) and `subtitle_own` (concatenation, strict half-open).
|
|
313
|
+
---
|
|
291
314
|
|
|
292
|
-
|
|
293
|
-
|
|
315
|
+
<details>
|
|
316
|
+
<summary><b>Troubleshooting — bugs I hit building this</b></summary>
|
|
294
317
|
|
|
295
|
-
|
|
296
|
-
Any ffmpeg-based statistics gathered with `-v error` silently come back as zero.
|
|
318
|
+
<br>
|
|
297
319
|
|
|
298
|
-
|
|
299
|
-
Pre-allocating and writing at multiple offsets over exFAT-on-USB yields files with
|
|
300
|
-
the right size and wrong contents — `moov atom not found`. Download to local disk,
|
|
301
|
-
then move.
|
|
320
|
+
Most of these **never raised an error** — they just quietly produced wrong output:
|
|
302
321
|
|
|
303
|
-
|
|
322
|
+
**`\b` word boundaries fail in mixed CJK/Latin text.** CJK characters are `\w` in Python,
|
|
323
|
+
so in `用grock测试` there is no boundary between `用` and `g`. `\bgrock\b` never matches.
|
|
324
|
+
The term table uses `(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
|
|
304
325
|
|
|
305
|
-
|
|
326
|
+
**Flat frames can't be detected by standard deviation.** Black transition cards usually
|
|
327
|
+
have burned-in white subtitles, pushing stddev to 23–33. Use mean luminance, kept tight
|
|
328
|
+
(`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
|
|
306
329
|
|
|
307
|
-
|
|
330
|
+
**Subtitle time windows must not overlap.** A look-back to avoid empty captions makes
|
|
331
|
+
adjacent frames claim the same cue. Hence two fields: `subtitle` (display, may repeat)
|
|
332
|
+
and `subtitle_own` (concatenation, strict half-open).
|
|
308
333
|
|
|
309
|
-
|
|
310
|
-
|
|
334
|
+
**ffmpeg concat resolves relative paths against the list file's directory** — contact
|
|
335
|
+
sheet file lists must contain absolute paths.
|
|
311
336
|
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
337
|
+
**`-v error` suppresses `showinfo` output** — any ffmpeg statistics gathered with it
|
|
338
|
+
silently come back as zero.
|
|
339
|
+
|
|
340
|
+
**Multi-threaded downloads corrupt files on exFAT.** Pre-allocating and writing at
|
|
341
|
+
multiple offsets over exFAT-on-USB yields files with the right size and wrong contents
|
|
342
|
+
(`moov atom not found`). Download to local disk, then move.
|
|
317
343
|
|
|
318
|
-
|
|
319
|
-
Use downloaded material for personal study and research only.
|
|
344
|
+
</details>
|
|
320
345
|
|
|
321
346
|
---
|
|
322
347
|
|
|
@@ -324,15 +349,13 @@ Use downloaded material for personal study and research only.
|
|
|
324
349
|
|
|
325
350
|
Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
|
|
326
351
|
|
|
327
|
-
Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**.
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
output; all rights remain with the original creator. If he would prefer them removed,
|
|
335
|
-
please open an issue and they will be replaced immediately.
|
|
352
|
+
Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**. This project started as an
|
|
353
|
+
attempt to study the editing in his videos — dense cutting, interleaved infographics and
|
|
354
|
+
animated demos — exactly the material naive interval-screenshotting handles worst. The
|
|
355
|
+
screenshots in this README are from
|
|
356
|
+
**[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely
|
|
357
|
+
to demonstrate this tool's output; all rights remain with the original creator. If he
|
|
358
|
+
would prefer them removed, please open an issue and they will be replaced immediately.
|
|
336
359
|
|
|
337
360
|
| | |
|
|
338
361
|
|---|---|
|
cutmap-0.2.0/README.md
ADDED
|
@@ -0,0 +1,317 @@
|
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
# cutmap
|
|
4
|
+
|
|
5
|
+
**Read a video like a document.**
|
|
6
|
+
Skim a long video in minutes to learn fast — or study it frame-by-frame to see exactly how it's cut.
|
|
7
|
+
|
|
8
|
+
[](https://github.com/xykong36/cutmap/blob/main/README.md)
|
|
9
|
+
[](https://github.com/xykong36/cutmap/blob/main/README.zh-CN.md)
|
|
10
|
+
[](https://pypi.org/project/cutmap/)
|
|
11
|
+
[](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
|
|
12
|
+
[](https://github.com/xykong36/cutmap/blob/main/LICENSE)
|
|
13
|
+
[](https://www.python.org)
|
|
14
|
+
[](#)
|
|
15
|
+
|
|
16
|
+
</div>
|
|
17
|
+
|
|
18
|
+
Watching a video is slow and linear — you can't skim it the way you skim an article,
|
|
19
|
+
and you can't Ctrl-F a moment. cutmap fixes that. Give it a video (and subtitles if you
|
|
20
|
+
have them) and it builds a single HTML page you can actually **read**: every visually
|
|
21
|
+
distinct frame laid out in order, each captioned with what was being said. Click a frame
|
|
22
|
+
to jump the player there; search the subtitles to find a moment.
|
|
23
|
+
|
|
24
|
+

|
|
25
|
+
|
|
26
|
+
<sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
|
|
27
|
+
|
|
28
|
+
Two things people use it for:
|
|
29
|
+
|
|
30
|
+
- 📖 **Learn faster.** Turn a one-hour talk or tutorial into a page you scroll in a few
|
|
31
|
+
minutes. See the whole structure at a glance, search the transcript, and only play the
|
|
32
|
+
parts you actually need.
|
|
33
|
+
- 🎬 **Study the edit ("拉片").** If you make videos, see exactly how someone cut theirs —
|
|
34
|
+
every shot change, every B-roll insert, every on-screen graphic — laid out as one storyboard.
|
|
35
|
+
|
|
36
|
+
It runs entirely on your machine. **No LLM, no API key, no network.**
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Quick start
|
|
41
|
+
|
|
42
|
+
### The easy way — just ask your AI agent
|
|
43
|
+
|
|
44
|
+
cutmap ships with a [Claude Code](https://claude.com/claude-code) skill. Install it once,
|
|
45
|
+
then talk in plain language — the agent downloads the video, runs cutmap, reads the
|
|
46
|
+
result, and answers you:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
pip install cutmap yt-dlp # cutmap + video downloader
|
|
50
|
+
brew install ffmpeg # the video engine (Linux: apt install ffmpeg)
|
|
51
|
+
|
|
52
|
+
# install the bundled skill for Claude Code
|
|
53
|
+
git clone https://github.com/xykong36/cutmap
|
|
54
|
+
cp -r cutmap/skills/cutmap ~/.claude/skills/
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Then just say things like:
|
|
58
|
+
|
|
59
|
+
> **Download** `https://youtu.be/…` **and use cutmap to summarize what it covers**
|
|
60
|
+
>
|
|
61
|
+
> **Use cutmap on** `lecture.mp4` **and give me the section-by-section breakdown**
|
|
62
|
+
>
|
|
63
|
+
> **What's on screen around 3:41 in that video?** (process it with cutmap first)
|
|
64
|
+
|
|
65
|
+
The agent handles the download, the flags, and reading the frames for you — you never
|
|
66
|
+
touch the command line.
|
|
67
|
+
|
|
68
|
+
### The manual way — command line
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
pip install cutmap
|
|
72
|
+
brew install ffmpeg # macOS (Linux: apt install ffmpeg)
|
|
73
|
+
|
|
74
|
+
cutmap video.mp4 --srt video.srt # or just: cutmap video.mp4 (finds video.srt)
|
|
75
|
+
open video/browse.html # ← open this in your browser
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
That's it. Everything you need is in the `video/` folder next to your file.
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
|
|
82
|
+
## What you get
|
|
83
|
+
|
|
84
|
+
Output lands in a directory named after the video:
|
|
85
|
+
|
|
86
|
+
```
|
|
87
|
+
video/
|
|
88
|
+
├── browse.html ← open this: the interactive storyboard
|
|
89
|
+
├── frames/ every distinct frame, labelled #index timecode seconds
|
|
90
|
+
├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
|
|
91
|
+
├── index.json frame timestamps + aligned subtitles (machine readable)
|
|
92
|
+
└── broll/ extracted B-roll clips + broll.json
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
### The browse page
|
|
96
|
+
|
|
97
|
+
One self-contained HTML file — all CSS and JS inlined. Two tabs:
|
|
98
|
+
|
|
99
|
+
**Storyboard** — every distinct frame with its caption
|
|
100
|
+
**B-roll** — auto-looping clips (reads like a GIF wall)
|
|
101
|
+
|
|
102
|
+

|
|
103
|
+
|
|
104
|
+
<sub>The B-roll tab in motion: clips auto-loop like a GIF wall, but they are MP4 under the hood — 1/24 the size of actual GIFs</sub>
|
|
105
|
+
|
|
106
|
+
What you can do on it:
|
|
107
|
+
|
|
108
|
+
- **Click any frame or timecode** to seek the embedded player there (local file, no network)
|
|
109
|
+
- **Search the subtitles** — matches the *normalised* text, so searching `DeepSeek` also
|
|
110
|
+
finds frames where the auto-transcription wrote `deep sick`
|
|
111
|
+
- Long captions collapse behind `…expand`
|
|
112
|
+
- Follows your system light / dark mode
|
|
113
|
+
|
|
114
|
+
Contact sheets are also written on their own, for scanning a whole video in a few images:
|
|
115
|
+
|
|
116
|
+

|
|
117
|
+
|
|
118
|
+
---
|
|
119
|
+
|
|
120
|
+
## How it works
|
|
121
|
+
|
|
122
|
+
The interesting part is *which* frames it keeps. Sampling every N seconds is both
|
|
123
|
+
redundant and lossy: a 30-second talking-head passage gives you ten identical stills;
|
|
124
|
+
a fast montage cutting five shots in three seconds gives you one.
|
|
125
|
+
|
|
126
|
+
cutmap asks a different question — not *"how often should I sample"* but
|
|
127
|
+
**"which frames actually differ"**:
|
|
128
|
+
|
|
129
|
+
```
|
|
130
|
+
dense sample → perceptual hash each frame → compare against last kept frame
|
|
131
|
+
→ keep only if different enough
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
Static passages collapse to a single frame. Every visual change gets one.
|
|
135
|
+
|
|
136
|
+
<details>
|
|
137
|
+
<summary><b>Why not ffmpeg's built-in scene detection?</b></summary>
|
|
138
|
+
|
|
139
|
+
<br>
|
|
140
|
+
|
|
141
|
+
`select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
|
|
142
|
+
|
|
143
|
+
1. **Low-contrast transitions** — white slide to white webpage. Similar overall
|
|
144
|
+
luminance structure, so the difference score never crosses the threshold.
|
|
145
|
+
2. **Intra-shot evolution** — text appearing line by line, charts animating,
|
|
146
|
+
table rows highlighting. Not a "cut" by any definition, but visually distinct.
|
|
147
|
+
|
|
148
|
+
Measured on one 16-minute video: scene detection found **104** frames,
|
|
149
|
+
perceptual dedup found **608**. One 25-second stretch that scene detection called
|
|
150
|
+
a single shot actually contained three completely different pages.
|
|
151
|
+
|
|
152
|
+
</details>
|
|
153
|
+
|
|
154
|
+
<details>
|
|
155
|
+
<summary><b>The algorithm's blind spot</b></summary>
|
|
156
|
+
|
|
157
|
+
<br>
|
|
158
|
+
|
|
159
|
+
dHash compares **relative brightness between adjacent pixels**, so it measures spatial
|
|
160
|
+
structure, not colour. Two different flat colours produce an identical (all-zero)
|
|
161
|
+
fingerprint and cannot be told apart. Footage that is largely flat-coloured or
|
|
162
|
+
near-black will therefore dedup more aggressively than you might expect. (This is also
|
|
163
|
+
why transition cards are detected by mean luminance rather than by hash.)
|
|
164
|
+
|
|
165
|
+
</details>
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## Options
|
|
170
|
+
|
|
171
|
+
Most people never need these — the defaults are tuned for slides, screencasts, and
|
|
172
|
+
talks. Reach for them when the frame count comes out too dense or too sparse.
|
|
173
|
+
|
|
174
|
+
| Flag | Meaning |
|
|
175
|
+
|---|---|
|
|
176
|
+
| `--threshold N` | Frame density. **Lower = more frames.** `6` dense / `10` default / `14` sparse / `24` structure-only |
|
|
177
|
+
| `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
|
|
178
|
+
| `--seg-max N` | Max seconds per B-roll segment (default 45) |
|
|
179
|
+
| `--clip-format` | `mp4` (default) / `gif` / `webp` |
|
|
180
|
+
| `--no-broll` | Skip B-roll extraction (much faster) |
|
|
181
|
+
| `--no-frames` | Keep only contact sheets, drop individual frames |
|
|
182
|
+
| `--cols/--rows` | Contact sheet grid (default 4×4) |
|
|
183
|
+
| `--thumb-width` | Thumbnail width in px (default 480) |
|
|
184
|
+
| `--terms FILE` | Custom subtitle term-normalisation table |
|
|
185
|
+
|
|
186
|
+
**Rule of thumb for density:** screencasts and slides → keep the default `10`. Real
|
|
187
|
+
handheld / vlog / interview footage → start at `--threshold 20` (camera shake makes
|
|
188
|
+
every frame look "different" otherwise).
|
|
189
|
+
|
|
190
|
+
<details>
|
|
191
|
+
<summary><b>Density, measured</b></summary>
|
|
192
|
+
|
|
193
|
+
<br>
|
|
194
|
+
|
|
195
|
+
On a 969-second video (2 fps sampling → 1936 raw frames):
|
|
196
|
+
|
|
197
|
+
| Threshold | Kept | Avg gap | Good for |
|
|
198
|
+
|---|---|---|---|
|
|
199
|
+
| 6 | 892 | 1.1s | Reconstructing every visual step (tutorial walkthroughs) |
|
|
200
|
+
| **10** | **597** | **1.6s** | **Default** — balances coverage and readability |
|
|
201
|
+
| 14 | 463 | 2.1s | Style study, quick scanning |
|
|
202
|
+
| 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
|
|
203
|
+
|
|
204
|
+
The curve is smooth — there is no natural breakpoint. This is an aesthetic choice,
|
|
205
|
+
not a parameter with a computable optimum.
|
|
206
|
+
|
|
207
|
+
</details>
|
|
208
|
+
|
|
209
|
+
---
|
|
210
|
+
|
|
211
|
+
## B-roll detection
|
|
212
|
+
|
|
213
|
+
Frames are sorted into three buckets by three plain rules — no model involved:
|
|
214
|
+
|
|
215
|
+
| Bucket | Rule |
|
|
216
|
+
|---|---|
|
|
217
|
+
| **Main shot** | A cluster of near-identical frames recurring across >50% of the runtime — i.e. a locked-off camera |
|
|
218
|
+
| **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
|
|
219
|
+
| **B-roll** | Everything else |
|
|
220
|
+
|
|
221
|
+
Screencast-style videos with no talking head degrade gracefully — no main shot is
|
|
222
|
+
found, everything becomes B-roll, and `--seg-max` keeps it segmented.
|
|
223
|
+
|
|
224
|
+
Clips default to **MP4** (`loop muted`), which looks identical to a GIF in the page but
|
|
225
|
+
is ~24× smaller (0.08 MB vs 1.96 MB for a 9-second segment). Use `--clip-format gif`
|
|
226
|
+
only when pasting into chat apps or note tools.
|
|
227
|
+
|
|
228
|
+
---
|
|
229
|
+
|
|
230
|
+
## Subtitle term normalisation
|
|
231
|
+
|
|
232
|
+
Auto-transcription mangles proper nouns. A built-in table normalises them:
|
|
233
|
+
|
|
234
|
+
```
|
|
235
|
+
大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
|
|
236
|
+
用grock测试 → 用Grok测试
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
|
|
240
|
+
was transcribed as `deep sick`. The default table targets AI / tech content; bring your
|
|
241
|
+
own with `--terms my_terms.txt` (one `regex => replacement` per line, see
|
|
242
|
+
[`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt)).
|
|
243
|
+
|
|
244
|
+
This is term normalisation, not proofreading — it will not fix grammar or one-off errors.
|
|
245
|
+
|
|
246
|
+
---
|
|
247
|
+
|
|
248
|
+
## Getting footage manually
|
|
249
|
+
|
|
250
|
+
If you're not using the agent skill, cutmap itself doesn't download anything — it works
|
|
251
|
+
on local files. For YouTube / Bilibili, grab the video and subtitles with
|
|
252
|
+
[yt-dlp](https://github.com/yt-dlp/yt-dlp):
|
|
253
|
+
|
|
254
|
+
```bash
|
|
255
|
+
yt-dlp --write-auto-subs --sub-langs "en,zh-Hans" --convert-subs srt \
|
|
256
|
+
-f "bv*[height<=720]+ba/b" -o "%(id)s.%(ext)s" "<URL>"
|
|
257
|
+
cutmap <id>.mp4
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
Respect the terms of service and copyright of wherever your footage comes from; use
|
|
261
|
+
downloaded material for personal study and research only.
|
|
262
|
+
|
|
263
|
+
---
|
|
264
|
+
|
|
265
|
+
<details>
|
|
266
|
+
<summary><b>Troubleshooting — bugs I hit building this</b></summary>
|
|
267
|
+
|
|
268
|
+
<br>
|
|
269
|
+
|
|
270
|
+
Most of these **never raised an error** — they just quietly produced wrong output:
|
|
271
|
+
|
|
272
|
+
**`\b` word boundaries fail in mixed CJK/Latin text.** CJK characters are `\w` in Python,
|
|
273
|
+
so in `用grock测试` there is no boundary between `用` and `g`. `\bgrock\b` never matches.
|
|
274
|
+
The term table uses `(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
|
|
275
|
+
|
|
276
|
+
**Flat frames can't be detected by standard deviation.** Black transition cards usually
|
|
277
|
+
have burned-in white subtitles, pushing stddev to 23–33. Use mean luminance, kept tight
|
|
278
|
+
(`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
|
|
279
|
+
|
|
280
|
+
**Subtitle time windows must not overlap.** A look-back to avoid empty captions makes
|
|
281
|
+
adjacent frames claim the same cue. Hence two fields: `subtitle` (display, may repeat)
|
|
282
|
+
and `subtitle_own` (concatenation, strict half-open).
|
|
283
|
+
|
|
284
|
+
**ffmpeg concat resolves relative paths against the list file's directory** — contact
|
|
285
|
+
sheet file lists must contain absolute paths.
|
|
286
|
+
|
|
287
|
+
**`-v error` suppresses `showinfo` output** — any ffmpeg statistics gathered with it
|
|
288
|
+
silently come back as zero.
|
|
289
|
+
|
|
290
|
+
**Multi-threaded downloads corrupt files on exFAT.** Pre-allocating and writing at
|
|
291
|
+
multiple offsets over exFAT-on-USB yields files with the right size and wrong contents
|
|
292
|
+
(`moov atom not found`). Download to local disk, then move.
|
|
293
|
+
|
|
294
|
+
</details>
|
|
295
|
+
|
|
296
|
+
---
|
|
297
|
+
|
|
298
|
+
## Credits
|
|
299
|
+
|
|
300
|
+
Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
|
|
301
|
+
|
|
302
|
+
Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**. This project started as an
|
|
303
|
+
attempt to study the editing in his videos — dense cutting, interleaved infographics and
|
|
304
|
+
animated demos — exactly the material naive interval-screenshotting handles worst. The
|
|
305
|
+
screenshots in this README are from
|
|
306
|
+
**[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely
|
|
307
|
+
to demonstrate this tool's output; all rights remain with the original creator. If he
|
|
308
|
+
would prefer them removed, please open an issue and they will be replaced immediately.
|
|
309
|
+
|
|
310
|
+
| | |
|
|
311
|
+
|---|---|
|
|
312
|
+
| Source video | <https://www.youtube.com/watch?v=WDQjRzVcX-A> |
|
|
313
|
+
| Creator | YouTube <https://www.youtube.com/@lyi> · Bilibili <https://space.bilibili.com/4401694> |
|
|
314
|
+
|
|
315
|
+
## License
|
|
316
|
+
|
|
317
|
+
MIT
|
|
@@ -4,9 +4,9 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "cutmap"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.2.0"
|
|
8
8
|
description = "Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll. Local ffmpeg + perceptual hashing, no LLM."
|
|
9
|
-
readme = "README.
|
|
9
|
+
readme = "README.md" # 英文主版,同时用于 PyPI 与 GitHub 首页;中文见 README.zh-CN.md
|
|
10
10
|
requires-python = ">=3.9"
|
|
11
11
|
license = { file = "LICENSE" }
|
|
12
12
|
keywords = ["video", "storyboard", "b-roll", "ffmpeg", "subtitle", "perceptual-hash", "分镜"]
|
|
@@ -1,2 +1,2 @@
|
|
|
1
1
|
"""cutmap —— 视频分镜分析:感知去重抽画面、B-roll 自动识别、图文对照浏览"""
|
|
2
|
-
__version__ = "0.
|
|
2
|
+
__version__ = "0.2.0"
|
|
@@ -26,13 +26,13 @@ def _step(n: int, total: int, msg: str) -> None:
|
|
|
26
26
|
def resolve(target: str, srt_opt: str | None):
|
|
27
27
|
"""输入 → (视频路径, 字幕路径或None, 工作目录)"""
|
|
28
28
|
if os.path.isdir(target):
|
|
29
|
-
mp4 = os.path.join(target, "
|
|
29
|
+
mp4 = os.path.join(target, "source.mp4")
|
|
30
30
|
if not os.path.exists(mp4):
|
|
31
31
|
cand = glob.glob(os.path.join(target, "*.mp4"))
|
|
32
32
|
if not cand:
|
|
33
33
|
sys.exit(f"目录里没有 mp4: {target}")
|
|
34
34
|
mp4 = max(cand, key=os.path.getsize)
|
|
35
|
-
srt = srt_opt or os.path.join(target, "
|
|
35
|
+
srt = srt_opt or os.path.join(target, "subtitle.srt")
|
|
36
36
|
return mp4, (srt if os.path.exists(srt) else None), target
|
|
37
37
|
|
|
38
38
|
if os.path.isfile(target):
|
|
@@ -102,8 +102,8 @@ def build(outdir: str, terms: str | None = None, log=print) -> str:
|
|
|
102
102
|
{broll_pane}
|
|
103
103
|
<script>{JS}</script></body></html>
|
|
104
104
|
"""
|
|
105
|
-
out = os.path.join(outdir, "
|
|
105
|
+
out = os.path.join(outdir, "browse.html")
|
|
106
106
|
open(out, "w", encoding="utf-8").write(page)
|
|
107
|
-
log(f" ✓
|
|
107
|
+
log(f" ✓ browse.html ({os.path.getsize(out)//1024} KB, "
|
|
108
108
|
f"分镜 {len(fcards)} + B-roll {len(bcards)})")
|
|
109
109
|
return out
|
cutmap-0.1.1/README.en.md
DELETED
|
@@ -1,294 +0,0 @@
|
|
|
1
|
-
<div align="center">
|
|
2
|
-
|
|
3
|
-
# cutmap
|
|
4
|
-
|
|
5
|
-
**Turn any video into a browsable, searchable storyboard — and auto-extract its B-roll.**
|
|
6
|
-
|
|
7
|
-
[](https://github.com/xykong36/cutmap/blob/main/README.md)
|
|
8
|
-
[](https://github.com/xykong36/cutmap/blob/main/README.en.md)
|
|
9
|
-
[](https://pypi.org/project/cutmap/)
|
|
10
|
-
[](https://github.com/xykong36/cutmap/actions/workflows/ci.yml)
|
|
11
|
-
[](https://github.com/xykong36/cutmap/blob/main/LICENSE)
|
|
12
|
-
[](https://www.python.org)
|
|
13
|
-
[](#)
|
|
14
|
-
|
|
15
|
-
</div>
|
|
16
|
-
|
|
17
|
-
Give it a video and a subtitle file. Get back a single HTML page: the source player on
|
|
18
|
-
the left, every distinct frame on the right, each captioned with what was being said.
|
|
19
|
-
Click any frame to jump the player to that moment. Search the subtitles to locate a shot.
|
|
20
|
-
|
|
21
|
-
```bash
|
|
22
|
-
cutmap video.mp4 --srt video.srt
|
|
23
|
-
open video/浏览.html
|
|
24
|
-
```
|
|
25
|
-
|
|
26
|
-
Everything runs locally through `ffmpeg` + `Pillow`. **No model calls, no API keys, no network.**
|
|
27
|
-
|
|
28
|
-

|
|
29
|
-
|
|
30
|
-
<sub>Demo footage: [@林亦LYi](https://www.youtube.com/@lyi) — [《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A), all rights reserved by the creator</sub>
|
|
31
|
-
|
|
32
|
-
---
|
|
33
|
-
|
|
34
|
-
## Why not just screenshot every N seconds
|
|
35
|
-
|
|
36
|
-
Interval sampling is both redundant and lossy. A 30-second talking-head passage yields
|
|
37
|
-
ten identical stills; a fast montage cutting five shots in three seconds yields one.
|
|
38
|
-
|
|
39
|
-
cutmap asks a different question — not *"how often should I sample"* but
|
|
40
|
-
**"which frames actually differ"**:
|
|
41
|
-
|
|
42
|
-
```
|
|
43
|
-
dense sample → perceptual hash each frame → compare against last kept frame
|
|
44
|
-
→ keep only if different enough
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
Static passages collapse to a single frame. Every visual change gets one.
|
|
48
|
-
|
|
49
|
-
### Why not ffmpeg's built-in scene detection
|
|
50
|
-
|
|
51
|
-
`select='gt(scene,0.3)'` only catches **hard cuts**. Two things slip past it:
|
|
52
|
-
|
|
53
|
-
1. **Low-contrast transitions** — white slide to white webpage. Similar overall
|
|
54
|
-
luminance structure, so the difference score never crosses the threshold.
|
|
55
|
-
2. **Intra-shot evolution** — text appearing line by line, charts animating,
|
|
56
|
-
table rows highlighting. Not a "cut" by any definition, but visually distinct.
|
|
57
|
-
|
|
58
|
-
Measured on one 16-minute video: scene detection found **104** frames,
|
|
59
|
-
perceptual dedup found **608**. One 25-second stretch that scene detection called
|
|
60
|
-
a single shot actually contained three completely different pages.
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
### A note on the algorithm's blind spot
|
|
64
|
-
|
|
65
|
-
dHash compares **relative brightness between adjacent pixels**, so it measures spatial
|
|
66
|
-
structure, not colour. Two different flat colours produce an identical (all-zero)
|
|
67
|
-
fingerprint and cannot be told apart. Footage that is largely flat-coloured or
|
|
68
|
-
near-black will therefore dedup more aggressively than you might expect.
|
|
69
|
-
|
|
70
|
-
This is also why transition cards are detected by mean luminance rather than by hash.
|
|
71
|
-
|
|
72
|
-
---
|
|
73
|
-
|
|
74
|
-
## Install
|
|
75
|
-
|
|
76
|
-
```bash
|
|
77
|
-
pip install cutmap
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
Requires **ffmpeg** on your `PATH`:
|
|
81
|
-
|
|
82
|
-
```bash
|
|
83
|
-
brew install ffmpeg # macOS
|
|
84
|
-
apt install ffmpeg # Debian / Ubuntu
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
---
|
|
88
|
-
|
|
89
|
-
## Usage
|
|
90
|
-
|
|
91
|
-
```bash
|
|
92
|
-
cutmap video.mp4 # looks for video.srt alongside
|
|
93
|
-
cutmap video.mp4 --srt subs.srt
|
|
94
|
-
cutmap ./material-dir/ # dir containing 源片.mp4 + 字幕.srt
|
|
95
|
-
```
|
|
96
|
-
|
|
97
|
-
Output lands in a directory named after the video:
|
|
98
|
-
|
|
99
|
-
```
|
|
100
|
-
video/
|
|
101
|
-
├── frames/ every distinct frame, labelled #index timecode seconds
|
|
102
|
-
├── sheet_01..NN.jpg 4×4 contact sheets for scanning at a glance
|
|
103
|
-
├── index.json frame timestamps + aligned subtitles (machine readable)
|
|
104
|
-
├── broll/ extracted B-roll clips + broll.json
|
|
105
|
-
└── 浏览.html ← open this
|
|
106
|
-
```
|
|
107
|
-
|
|
108
|
-
### Options
|
|
109
|
-
|
|
110
|
-
| Flag | Meaning |
|
|
111
|
-
|---|---|
|
|
112
|
-
| `--threshold N` | Frame density. Lower = denser. `6` dense / `10` default / `14` sparse |
|
|
113
|
-
| `--fps N` | Dense sampling rate (default 2 — finest resolvable gap is 0.5s) |
|
|
114
|
-
| `--cols/--rows` | Contact sheet grid (default 4×4) |
|
|
115
|
-
| `--thumb-width` | Thumbnail width in px (default 480) |
|
|
116
|
-
| `--seg-max N` | Max seconds per B-roll segment (default 45) |
|
|
117
|
-
| `--clip-format` | `mp4` (default) / `gif` / `webp` |
|
|
118
|
-
| `--no-broll` | Skip B-roll extraction |
|
|
119
|
-
| `--no-frames` | Keep only contact sheets, drop individual frames |
|
|
120
|
-
| `--terms FILE` | Custom subtitle term-normalisation table |
|
|
121
|
-
|
|
122
|
-
### Picking a density
|
|
123
|
-
|
|
124
|
-
Measured on a 969-second video (2 fps sampling → 1936 raw frames):
|
|
125
|
-
|
|
126
|
-
| Threshold | Kept | Avg gap | Good for |
|
|
127
|
-
|---|---|---|---|
|
|
128
|
-
| 6 | 892 | 1.1s | Reconstructing every visual step (tutorial walkthroughs) |
|
|
129
|
-
| **10** | **597** | **1.6s** | **Default** — balances coverage and readability |
|
|
130
|
-
| 14 | 463 | 2.1s | Style study, quick scanning |
|
|
131
|
-
| 24 | 241 | 4.0s | Structure only, close to a traditional shot list |
|
|
132
|
-
|
|
133
|
-
The curve is smooth — there is no natural breakpoint. This is an **aesthetic choice,
|
|
134
|
-
not a parameter with a computable optimum.**
|
|
135
|
-
|
|
136
|
-
---
|
|
137
|
-
|
|
138
|
-
## The browse page
|
|
139
|
-
|
|
140
|
-
One self-contained HTML file. All CSS and JS inlined; only depends on the images and
|
|
141
|
-
clips sitting next to it. Two tabs:
|
|
142
|
-
|
|
143
|
-
**Storyboard** — every distinct frame with its caption
|
|
144
|
-
**B-roll** — auto-looping clips (reads like a GIF wall)
|
|
145
|
-
|
|
146
|
-

|
|
147
|
-
|
|
148
|
-
Shared behaviour:
|
|
149
|
-
|
|
150
|
-
- Embedded source player up top — **click any timecode or frame to seek there**
|
|
151
|
-
(local file, no network)
|
|
152
|
-
- Live subtitle search, matching against the **normalised** text
|
|
153
|
-
(searching `DeepSeek` finds frames whose ASR wrote `deep sick`)
|
|
154
|
-
- Long captions collapse behind `…expand`
|
|
155
|
-
- B-roll clips play only while in viewport (IntersectionObserver) — dozens of
|
|
156
|
-
simultaneously decoding videos will freeze a browser
|
|
157
|
-
- Follows system light / dark mode
|
|
158
|
-
|
|
159
|
-
Contact sheets are written separately, for scanning a whole video in a few images:
|
|
160
|
-
|
|
161
|
-

|
|
162
|
-
|
|
163
|
-
---
|
|
164
|
-
|
|
165
|
-
## B-roll detection
|
|
166
|
-
|
|
167
|
-
Frames are sorted into three buckets by three plain rules — no model involved:
|
|
168
|
-
|
|
169
|
-
| Bucket | Rule |
|
|
170
|
-
|---|---|
|
|
171
|
-
| **Main shot** | A cluster of near-identical frames recurring across >50% of the runtime — i.e. a locked-off camera |
|
|
172
|
-
| **Transition card** | Mean luminance below 8 (black) or above 245 (white) |
|
|
173
|
-
| **B-roll** | Everything else |
|
|
174
|
-
|
|
175
|
-
Measured on one episode: B-roll 49 segments / 731s (75.5%),
|
|
176
|
-
main shot 39 / 217s (22.4%), transitions 16 / 20s (2.1%).
|
|
177
|
-
|
|
178
|
-
Screencast-style videos with no talking head degrade gracefully — no main shot is
|
|
179
|
-
found, everything becomes B-roll, and `--seg-max` keeps it segmented.
|
|
180
|
-
|
|
181
|
-
### Clip format
|
|
182
|
-
|
|
183
|
-
Same nine-second segment:
|
|
184
|
-
|
|
185
|
-
| Format | Size | Note |
|
|
186
|
-
|---|---|---|
|
|
187
|
-
| GIF 10fps | 1.96 MB | 256 colours; visible banding on gradients |
|
|
188
|
-
| WebP | 0.39 MB | Middle ground |
|
|
189
|
-
| **MP4** | **0.08 MB** | **Default** — 24× smaller than GIF, better quality |
|
|
190
|
-
|
|
191
|
-
MP4 with `loop muted` is visually indistinguishable from a GIF in the page.
|
|
192
|
-
Reach for `--clip-format gif` only when pasting into chat apps or note tools.
|
|
193
|
-
|
|
194
|
-
---
|
|
195
|
-
|
|
196
|
-
## Subtitle term normalisation
|
|
197
|
-
|
|
198
|
-
ASR mangles proper nouns. A built-in table normalises them:
|
|
199
|
-
|
|
200
|
-
```
|
|
201
|
-
大家对deep sick新模型的期待值 → 大家对DeepSeek新模型的期待值
|
|
202
|
-
用grock测试 → 用Grok测试
|
|
203
|
-
```
|
|
204
|
-
|
|
205
|
-
The real payoff is **search**: querying `DeepSeek` finds every frame whose subtitle
|
|
206
|
-
was transcribed as `deep sick`.
|
|
207
|
-
|
|
208
|
-
The default table targets AI / tech content. Bring your own:
|
|
209
|
-
|
|
210
|
-
```bash
|
|
211
|
-
cutmap video.mp4 --terms my_terms.txt
|
|
212
|
-
```
|
|
213
|
-
|
|
214
|
-
Format is one `regex => replacement` per line — see
|
|
215
|
-
[`src/cutmap/terms.txt`](https://github.com/xykong36/cutmap/blob/main/src/cutmap/terms.txt).
|
|
216
|
-
|
|
217
|
-
**This is term normalisation, not proofreading.** It will not fix segmentation,
|
|
218
|
-
grammar, or one-off errors — those need a language model, which is out of scope here.
|
|
219
|
-
|
|
220
|
-
---
|
|
221
|
-
|
|
222
|
-
## Troubleshooting
|
|
223
|
-
|
|
224
|
-
Bugs hit while building this. Most of them **never raised an error** — they just
|
|
225
|
-
quietly produced wrong output:
|
|
226
|
-
|
|
227
|
-
**`\b` word boundaries fail in mixed CJK/Latin text**
|
|
228
|
-
CJK characters are `\w` in Python, so in `用grock测试` there is no boundary between
|
|
229
|
-
`用` and `g`. `\bgrock\b` never matches. The term table uses
|
|
230
|
-
`(?<![A-Za-z0-9])…(?![A-Za-z0-9])` instead.
|
|
231
|
-
|
|
232
|
-
**Flat frames can't be detected by standard deviation**
|
|
233
|
-
Black transition cards usually have burned-in white subtitles, pushing stddev to
|
|
234
|
-
23–33 — far above any "flat colour" threshold. Use mean luminance, and keep it tight
|
|
235
|
-
(`<8`): loosening to `<25` misclassifies merely-dark footage as transitions.
|
|
236
|
-
|
|
237
|
-
**Subtitle time windows must not overlap**
|
|
238
|
-
Adding a look-back to avoid empty captions makes adjacent frames claim the same cue,
|
|
239
|
-
producing duplicated sentences when concatenated. Hence two fields:
|
|
240
|
-
`subtitle` (display, may repeat) and `subtitle_own` (concatenation, strict half-open).
|
|
241
|
-
|
|
242
|
-
**ffmpeg concat resolves relative paths against the list file's directory**
|
|
243
|
-
Contact sheet file lists must contain absolute paths.
|
|
244
|
-
|
|
245
|
-
**`-v error` suppresses `showinfo` output**
|
|
246
|
-
Any ffmpeg-based statistics gathered with `-v error` silently come back as zero.
|
|
247
|
-
|
|
248
|
-
**Multi-threaded downloads corrupt files on exFAT** (if you fetch footage yourself)
|
|
249
|
-
Pre-allocating and writing at multiple offsets over exFAT-on-USB yields files with
|
|
250
|
-
the right size and wrong contents — `moov atom not found`. Download to local disk,
|
|
251
|
-
then move.
|
|
252
|
-
|
|
253
|
-
---
|
|
254
|
-
|
|
255
|
-
## Getting footage
|
|
256
|
-
|
|
257
|
-
cutmap does not download anything — it works on local files.
|
|
258
|
-
|
|
259
|
-
For Bilibili, [BBDown](https://github.com/nilaoda/BBDown) muxes subtitles into the mp4;
|
|
260
|
-
pull them out and hand both to cutmap:
|
|
261
|
-
|
|
262
|
-
```bash
|
|
263
|
-
BBDown <BV-id> --skip-ai false
|
|
264
|
-
ffmpeg -i video.mp4 -map 0:m:language:chi subs.srt
|
|
265
|
-
cutmap video.mp4 --srt subs.srt
|
|
266
|
-
```
|
|
267
|
-
|
|
268
|
-
Respect the terms of service and copyright of wherever your footage comes from.
|
|
269
|
-
Use downloaded material for personal study and research only.
|
|
270
|
-
|
|
271
|
-
---
|
|
272
|
-
|
|
273
|
-
## Credits
|
|
274
|
-
|
|
275
|
-
Built on [ffmpeg](https://ffmpeg.org) and [Pillow](https://python-pillow.org).
|
|
276
|
-
|
|
277
|
-
Special thanks to **[@林亦LYi](https://www.youtube.com/@lyi)**.
|
|
278
|
-
|
|
279
|
-
This project started as an attempt to study the editing in his videos — dense cutting,
|
|
280
|
-
interleaved infographics and animated demos. That is exactly the material naive
|
|
281
|
-
interval-screenshotting handles worst, and it is what forced the perceptual-dedup
|
|
282
|
-
approach in the first place. The screenshots in this README are from
|
|
283
|
-
**[《一个视频搞懂 DeepSeek V4!》](https://www.youtube.com/watch?v=WDQjRzVcX-A)**, used solely to demonstrate this tool's
|
|
284
|
-
output; all rights remain with the original creator. If he would prefer them removed,
|
|
285
|
-
please open an issue and they will be replaced immediately.
|
|
286
|
-
|
|
287
|
-
| | |
|
|
288
|
-
|---|---|
|
|
289
|
-
| Source video | <https://www.youtube.com/watch?v=WDQjRzVcX-A> |
|
|
290
|
-
| Creator | YouTube <https://www.youtube.com/@lyi> · Bilibili <https://space.bilibili.com/4401694> |
|
|
291
|
-
|
|
292
|
-
## License
|
|
293
|
-
|
|
294
|
-
MIT
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|