talkthrough-mcp 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 korovin-aa97
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,584 @@
1
+ Metadata-Version: 2.4
2
+ Name: talkthrough-mcp
3
+ Version: 0.1.0
4
+ Summary: Local-first MCP server that turns narrated screen recordings into agent-ready structured data: timestamped transcript, scene keyframes, OCR text, and wall-clock anchoring.
5
+ Keywords: mcp,mcp server,model-context-protocol,screen-recording,video,transcription,whisper,ocr,feedback
6
+ Author: korovin-aa97
7
+ Author-email: korovin-aa97 <korovin.aa97@gmail.com>
8
+ License-Expression: MIT
9
+ License-File: LICENSE
10
+ Classifier: Development Status :: 4 - Beta
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: Operating System :: MacOS
13
+ Classifier: Operating System :: POSIX :: Linux
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.11
16
+ Classifier: Programming Language :: Python :: 3.12
17
+ Classifier: Programming Language :: Python :: 3.13
18
+ Classifier: Topic :: Multimedia :: Video
19
+ Classifier: Topic :: Software Development :: Quality Assurance
20
+ Classifier: Typing :: Typed
21
+ Requires-Dist: faster-whisper>=1.2.1
22
+ Requires-Dist: mcp>=1.28.1
23
+ Requires-Dist: pillow>=12.3.0
24
+ Requires-Dist: rapidocr>=3.9.1
25
+ Requires-Dist: static-ffmpeg>=3.0
26
+ Requires-Python: >=3.11, <3.14
27
+ Project-URL: Homepage, https://github.com/korovin-aa97/talkthrough-mcp
28
+ Project-URL: Repository, https://github.com/korovin-aa97/talkthrough-mcp
29
+ Project-URL: Issues, https://github.com/korovin-aa97/talkthrough-mcp/issues
30
+ Project-URL: Changelog, https://github.com/korovin-aa97/talkthrough-mcp/blob/main/CHANGELOG.md
31
+ Description-Content-Type: text/markdown
32
+
33
+ # talkthrough-mcp
34
+
35
+ <!-- mcp-name: io.github.korovin-aa97/talkthrough-mcp -->
36
+
37
+ [![ci](https://github.com/korovin-aa97/talkthrough-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/korovin-aa97/talkthrough-mcp/actions/workflows/ci.yml)
38
+ [![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
39
+ [![python](https://img.shields.io/badge/python-3.11%E2%80%933.13-blue.svg)](pyproject.toml)
40
+ <!-- uncomment at PyPI publish: [![PyPI](https://img.shields.io/pypi/v/talkthrough-mcp.svg)](https://pypi.org/project/talkthrough-mcp/) -->
41
+
42
+ [Quickstart](#quickstart) · [Tools](#tools) · [FAQ](#faq) ·
43
+ [Troubleshooting](docs/TROUBLESHOOTING.md) · [Changelog](CHANGELOG.md) ·
44
+ [Contributing](CONTRIBUTING.md)
45
+
46
+ **Feedback ingestion for AI agents.** Record your screen and talk; your agent
47
+ does the rest — files the bugs, writes the spec, builds the backlog.
48
+
49
+ ![talkthrough demo: process a narrated recording, then query it lazily](assets/demo.gif)
50
+
51
+ `talkthrough-mcp` is a local-first MCP server that turns a narrated screen
52
+ recording (or any video/audio file) into agent-ready structured data:
53
+ timestamped transcript segments, scene-change keyframes, OCR'd on-screen text,
54
+ and wall-clock anchoring. Everything is served through lazy retrieval tools, so
55
+ a 30-minute recording never floods the model context — the agent pulls exactly
56
+ the transcript slice, moment bundle, or frame it needs.
57
+
58
+ There is no LLM inside the server and no cloud anywhere in the path: ffmpeg,
59
+ faster-whisper, and RapidOCR run on your machine, and the calling agent brings
60
+ the intelligence. What makes it different from screen-recorder SaaS and
61
+ video-analyzer MCPs: it works on arbitrary local files, it ships the agent
62
+ workflows (server prompts + example agents), and it anchors every timestamp to
63
+ **wall-clock time** — so "the moment I said the checkout hung" maps straight to
64
+ the right window of your server logs.
65
+
66
+ ## Quickstart
67
+
68
+ One command, no system dependencies: ffmpeg falls back to a bundled build,
69
+ OCR is pip-only, and whisper models download themselves on first use.
70
+
71
+ <!-- gen:install — generated by scripts/gen_integrations.py; do not hand-edit -->
72
+
73
+ [![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=talkthrough&config=eyJjb21tYW5kIjoidXZ4IiwiYXJncyI6WyJ0YWxrdGhyb3VnaC1tY3AiXX0%3D)
74
+ [![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_Server-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://vscode.dev/redirect/mcp/install?name=talkthrough&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22talkthrough-mcp%22%5D%2C%22type%22%3A%22stdio%22%7D)
75
+ [![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install_Server-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=talkthrough&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22talkthrough-mcp%22%5D%2C%22type%22%3A%22stdio%22%7D&quality=insiders)
76
+ [![Add to LM Studio](https://files.lmstudio.ai/deeplink/mcp-install-light.svg)](https://lmstudio.ai/install-mcp?name=talkthrough&config=eyJjb21tYW5kIjoidXZ4IiwiYXJncyI6WyJ0YWxrdGhyb3VnaC1tY3AiXX0%3D)
77
+ [![Add to Kiro](https://kiro.dev/images/add-to-kiro.svg)](https://kiro.dev/launch/mcp/add?name=talkthrough&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22talkthrough-mcp%22%5D%7D)
78
+ [![Install in Goose](https://block.github.io/goose/img/extension-install-dark.svg)](https://block.github.io/goose/extension?cmd=uvx&arg=talkthrough-mcp&id=talkthrough&name=Talkthrough&description=Turn%20narrated%20screen%20recordings%20into%20agent-ready%20data)
79
+
80
+ ### Claude Code
81
+
82
+ ```bash
83
+ claude mcp add -s user talkthrough -- uvx talkthrough-mcp
84
+ ```
85
+
86
+ Or install the full plugin (server + the five workflow commands + the triage
87
+ agent + an agent skill):
88
+
89
+ ```
90
+ /plugin marketplace add korovin-aa97/talkthrough-mcp
91
+ /plugin install talkthrough@talkthrough
92
+ ```
93
+
94
+ ### Every other MCP client
95
+
96
+ <details>
97
+ <summary><b>Claude Desktop</b></summary>
98
+
99
+ `claude_desktop_config.json`:
100
+
101
+ ```json
102
+ {
103
+ "mcpServers": {
104
+ "talkthrough": {
105
+ "command": "uvx",
106
+ "args": [
107
+ "talkthrough-mcp"
108
+ ]
109
+ }
110
+ }
111
+ }
112
+ ```
113
+
114
+ More: [`integrations/claude-desktop/`](integrations/claude-desktop/)
115
+
116
+ </details>
117
+
118
+ <details>
119
+ <summary><b>Cursor</b></summary>
120
+
121
+ `~/.cursor/mcp.json (or project .cursor/mcp.json)`:
122
+
123
+ ```json
124
+ {
125
+ "mcpServers": {
126
+ "talkthrough": {
127
+ "command": "uvx",
128
+ "args": [
129
+ "talkthrough-mcp"
130
+ ]
131
+ }
132
+ }
133
+ }
134
+ ```
135
+
136
+ More: [`integrations/cursor/`](integrations/cursor/)
137
+
138
+ </details>
139
+
140
+ <details>
141
+ <summary><b>OpenAI Codex CLI</b></summary>
142
+
143
+ `~/.codex/config.toml (or project-scoped .codex/config.toml in trusted projects)`:
144
+
145
+ ```toml
146
+ [mcp_servers.talkthrough]
147
+ command = "uvx"
148
+ args = ["talkthrough-mcp"]
149
+ ```
150
+
151
+ More: [`integrations/codex/`](integrations/codex/)
152
+
153
+ </details>
154
+
155
+ <details>
156
+ <summary><b>Gemini CLI</b></summary>
157
+
158
+ `~/.gemini/settings.json`:
159
+
160
+ ```json
161
+ {
162
+ "mcpServers": {
163
+ "talkthrough": {
164
+ "command": "uvx",
165
+ "args": [
166
+ "talkthrough-mcp"
167
+ ]
168
+ }
169
+ }
170
+ }
171
+ ```
172
+
173
+ More: [`integrations/gemini-cli/`](integrations/gemini-cli/)
174
+
175
+ </details>
176
+
177
+ <details>
178
+ <summary><b>Cline / Roo Code</b></summary>
179
+
180
+ `cline_mcp_settings.json (via MCP Servers UI)`:
181
+
182
+ ```json
183
+ {
184
+ "mcpServers": {
185
+ "talkthrough": {
186
+ "command": "uvx",
187
+ "args": [
188
+ "talkthrough-mcp"
189
+ ]
190
+ }
191
+ }
192
+ }
193
+ ```
194
+
195
+ More: [`integrations/cline/`](integrations/cline/)
196
+
197
+ </details>
198
+
199
+ <details>
200
+ <summary><b>OpenClaw</b></summary>
201
+
202
+ `~/.openclaw/openclaw.json`:
203
+
204
+ ```json
205
+ {
206
+ "mcp": {
207
+ "servers": {
208
+ "talkthrough": {
209
+ "command": "uvx",
210
+ "args": [
211
+ "talkthrough-mcp"
212
+ ]
213
+ }
214
+ }
215
+ }
216
+ }
217
+ ```
218
+
219
+ More: [`integrations/openclaw/`](integrations/openclaw/)
220
+
221
+ </details>
222
+
223
+ <details>
224
+ <summary><b>OpenCode</b></summary>
225
+
226
+ `opencode.json (project) or ~/.config/opencode/opencode.json`:
227
+
228
+ ```json
229
+ {
230
+ "mcp": {
231
+ "talkthrough": {
232
+ "type": "local",
233
+ "command": [
234
+ "uvx",
235
+ "talkthrough-mcp"
236
+ ],
237
+ "enabled": true
238
+ }
239
+ }
240
+ }
241
+ ```
242
+
243
+ More: [`integrations/opencode/`](integrations/opencode/)
244
+
245
+ </details>
246
+
247
+ <details>
248
+ <summary><b>Goose</b></summary>
249
+
250
+ `~/.config/goose/config.yaml`:
251
+
252
+ ```yaml
253
+ extensions:
254
+ talkthrough:
255
+ enabled: true
256
+ type: stdio
257
+ cmd: uvx
258
+ args: ["talkthrough-mcp"]
259
+ ```
260
+
261
+ More: [`integrations/goose/`](integrations/goose/)
262
+
263
+ </details>
264
+
265
+ <details>
266
+ <summary><b>GitHub Copilot CLI</b></summary>
267
+
268
+ `~/.copilot/mcp-config.json`:
269
+
270
+ ```json
271
+ {
272
+ "mcpServers": {
273
+ "talkthrough": {
274
+ "command": "uvx",
275
+ "args": [
276
+ "talkthrough-mcp"
277
+ ]
278
+ }
279
+ }
280
+ }
281
+ ```
282
+
283
+ More: [`integrations/copilot-cli/`](integrations/copilot-cli/)
284
+
285
+ </details>
286
+
287
+ <details>
288
+ <summary><b>Windsurf</b></summary>
289
+
290
+ `~/.codeium/windsurf/mcp_config.json`:
291
+
292
+ ```json
293
+ {
294
+ "mcpServers": {
295
+ "talkthrough": {
296
+ "command": "uvx",
297
+ "args": [
298
+ "talkthrough-mcp"
299
+ ]
300
+ }
301
+ }
302
+ }
303
+ ```
304
+
305
+ More: [`integrations/windsurf/`](integrations/windsurf/)
306
+
307
+ </details>
308
+
309
+ <details>
310
+ <summary><b>Zed</b></summary>
311
+
312
+ `settings.json (Zed)`:
313
+
314
+ ```json
315
+ {
316
+ "context_servers": {
317
+ "talkthrough": {
318
+ "source": "custom",
319
+ "command": {
320
+ "path": "uvx",
321
+ "args": [
322
+ "talkthrough-mcp"
323
+ ]
324
+ }
325
+ }
326
+ }
327
+ }
328
+ ```
329
+
330
+ More: [`integrations/zed/`](integrations/zed/)
331
+
332
+ </details>
333
+
334
+ Any other MCP stdio client uses the same server command: `uvx talkthrough-mcp`.
335
+ Per-engine folders with exactly these snippets plus verification steps live
336
+ in [`integrations/`](integrations/); agents can self-install via
337
+ [`llms-install.md`](llms-install.md).
338
+
339
+ <!-- /gen:install -->
340
+
341
+ ### Local checkout (development)
342
+
343
+ ```bash
344
+ git clone https://github.com/korovin-aa97/talkthrough-mcp
345
+ claude mcp add talkthrough -- uv run --directory /path/to/talkthrough-mcp talkthrough-mcp
346
+ ```
347
+
348
+ Then, in your agent:
349
+
350
+ > Process `~/Desktop/recording.mov` and triage it — or just invoke the
351
+ > `triage-recording` server prompt.
352
+
353
+ ## Tools
354
+
355
+ | Tool | What it does |
356
+ |---|---|
357
+ | `process_media(path, recorded_at?, vocabulary?, language?, model?, force?)` | Ingest a video/audio file: local STT, keyframes, OCR, wall-clock. Returns a compact summary. Idempotent by content hash — re-calls are instant. |
358
+ | `get_transcript(job_id, start_ms?, end_ms?, format?)` | Paginated transcript as `segments`, `text`, or `srt`; truncation returns `next_start_ms`. |
359
+ | `get_frames(job_id, at_ms? \| start_ms?+end_ms?, max_frames?, include_duplicates?)` | Keyframe images nearest a timestamp or evenly thinned across a range (unique frames by default, max 6/call). |
360
+ | `get_moment(job_id, start_ms, end_ms)` | The "one remark" bundle: transcript slice + up to 3 frames + their OCR text + wall-clock range. |
361
+ | `search(job_id, query)` | Substring search over the transcript AND on-screen OCR text; hits carry `t_ms`/`t_wall` and frame refs. |
362
+ | `extract_frame(job_id, at_ms, crop?)` | Exact-timestamp full-resolution re-extract from the source video (optional crop) when keyframes miss the instant. |
363
+ | `list_jobs()` | Recent processed recordings with durations, wall-clock starts, and counts. |
364
+
365
+ Every tool description ships 10+ usage examples, so agents pick the right tool
366
+ without extra prompting.
367
+
368
+ ## Server prompts (slash commands in MCP clients)
369
+
370
+ | Prompt | Workflow |
371
+ |---|---|
372
+ | `triage-recording` | Narrated screencast → precise findings JSON (bug/feature/question routing, frame evidence) |
373
+ | `spec-from-workshop` | Recorded workshop → structured spec with quoted decisions and open questions |
374
+ | `backlog-from-demo` | Product demo → prioritized backlog with timestamped evidence |
375
+ | `meeting-actions` | Meeting audio → action items, decisions, open questions |
376
+ | `correlate-with-logs` | Recording remarks ↔ system logs via wall-clock windows |
377
+
378
+ The same prompts live as plain files in [`examples/prompts/`](examples/prompts/)
379
+ if your client doesn't surface MCP prompts. The findings contract used by
380
+ `triage-recording` is [`examples/output-contract.schema.json`](examples/output-contract.schema.json).
381
+
382
+ ## Works as a skill too (no MCP required)
383
+
384
+ The same workflow ships as a cross-engine [Agent Skill](https://agentskills.io)
385
+ at [`.agents/skills/talkthrough/`](.agents/skills/talkthrough/) — Claude Code,
386
+ Codex CLI (`$talkthrough`), Cursor, Copilot, Gemini CLI, Goose and other
387
+ SKILL.md-compatible tools read it. Agents without MCP wiring can drive the
388
+ [CLI](#cli) directly: `talkthrough-mcp process recording.mov --json` prints the
389
+ same summary the MCP tool returns, and the job store is shared either way.
390
+
391
+ ## Wall-clock anchoring
392
+
393
+ Every timestamped result carries both `t_ms` (video-relative) and `t_wall`
394
+ (ISO 8601 real time) once the recording start is known. Resolution ladder:
395
+
396
+ 1. `recorded_at` parameter (agent/user override) → confidence `exact`
397
+ 2. QuickTime `com.apple.quicktime.creationdate` tag, carries the local
398
+ timezone (QuickTime Player recordings; ⌘⇧5 wrote it before macOS 26) → `high`
399
+ 3. Container `creation_time` tag (UTC) → `medium` — macOS 26+ ⌘⇧5/ReplayKit
400
+ screen recordings land here (no `creationdate` tag anymore); pass
401
+ `recorded_at=` when local-tz `t_wall` matters
402
+ 4. File mtime minus duration (recorders finalize files at recording END) → `low`
403
+ 5. Nothing → tools still work with relative `t_ms` only
404
+
405
+ Why it matters: "the upload spinner froze *here*" becomes a ±30 s grep window
406
+ in your server logs.
407
+
408
+ ## Privacy
409
+
410
+ Everything runs locally: your recordings never leave your machine, speech is
411
+ transcribed by a local whisper model, OCR is local ONNX inference, and there is
412
+ no telemetry. The only network access is one-time tool/model downloads (ffmpeg
413
+ build, whisper model, OCR models).
414
+
415
+ ## Languages
416
+
417
+ Narration in any of Whisper's ~99 languages works: the language is
418
+ auto-detected per recording, and the summary reports both `language` and
419
+ `language_probability` so agents can tell a confident detection from a shaky
420
+ one (silence or music at the start can fool the detector — pin it with
421
+ `language="ru"` and `force=true` when that happens).
422
+
423
+ Pick the model for your languages — per call (`model=` parameter, agents do
424
+ this themselves when a transcript comes back garbled) or as the server
425
+ default (`TALKTHROUGH_WHISPER_MODEL`):
426
+
427
+ | Model | Size | Best for |
428
+ |---|---|---|
429
+ | `small` (default) | 464 MB | English and major-language narration on CPU |
430
+ | `large-v3-turbo` | ~1.5 GB | **recommended for non-English** — near-large quality at near-small speed |
431
+ | `medium` | ~1.5 GB | conservative alternative to turbo |
432
+ | `tiny` / `base` | 75–145 MB | quick drafts, CI |
433
+ | `*.en` variants | — | English-only, slightly faster/better for EN |
434
+
435
+ Tips that work in every language: pass product names via
436
+ `vocabulary="Term1, Term2"` (biases the decoder so jargon survives), and note
437
+ that the workflow prompts instruct agents to write digests in the
438
+ **narrator's language** while keeping quotes verbatim — the server never
439
+ translates (exact quotes are evidence; translation is the agent's job).
440
+
441
+ On-screen text (OCR) defaults to RapidOCR's Latin + Chinese models. For other
442
+ scripts set `TALKTHROUGH_OCR_LANG` to your language — `ru`/`uk` (→ the
443
+ `eslav` pack), `ja`, `ko`, `ar`, `hi`, `el`, `th`, or any RapidOCR pack name
444
+ like `cyrillic` — and reprocess with `force=true`; the matching recognition
445
+ model downloads once. Spoken-language support is unaffected either way.
446
+
447
+ ## Configuration
448
+
449
+ | Env var | Default | Meaning |
450
+ |---|---|---|
451
+ | `TALKTHROUGH_WHISPER_MODEL` | `small` | default whisper model (`tiny`/`base`/`small`/`medium`/`large-v3`/`large-v3-turbo`); the `model` tool param overrides per call |
452
+ | `TALKTHROUGH_OCR` | `on` | set `off` to skip OCR |
453
+ | `TALKTHROUGH_OCR_LANG` | Latin+Chinese | recognition script for on-screen text: a language code (`ru`, `ja`, `ko`, `ar`, `hi`, …) or a RapidOCR pack name (`eslav`, `cyrillic`, `latin`, …); the model downloads once |
454
+ | `TALKTHROUGH_OCR_PARAMS` | — | advanced: JSON object of raw RapidOCR params merged over the derived ones, e.g. `{"Rec.lang_type": "cyrillic"}` |
455
+ | `TALKTHROUGH_MAX_SECONDS` | `7200` | max media duration |
456
+ | `TALKTHROUGH_MAX_FRAMES` | `600` | keyframe cap per job |
457
+ | `TALKTHROUGH_HOME` | `~/.talkthrough` | job store root |
458
+
459
+ ## CLI
460
+
461
+ The pipeline is also a CLI — useful for pre-processing long recordings outside
462
+ an agent session (the store is content-addressed, so the agent then queries the
463
+ same job instantly):
464
+
465
+ ```bash
466
+ talkthrough-mcp process ~/Videos/long-session.mov # prints the summary
467
+ talkthrough-mcp process demo.mov --json # machine-readable
468
+ talkthrough-mcp gc --keep-days 30 # clean the job store
469
+ talkthrough-mcp serve # stdio MCP server (default)
470
+ ```
471
+
472
+ First run notes: missing system ffmpeg triggers a one-time `static-ffmpeg`
473
+ download; the first transcription downloads the whisper model (~460 MB for
474
+ `small`); both are cached. After that, expect roughly 3× faster than real time
475
+ on an Apple-Silicon CPU with the default model, OCR included (a 2-minute clip
476
+ processes in ~40 s) — and instant re-runs on the same file. Progress streams
477
+ as MCP progress notifications, and the CLI prints stage lines. More:
478
+ [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md).
479
+
480
+ ## Windows (best-effort)
481
+
482
+ CI runs lint, the unit suite, and a full CLI smoke on `windows-latest`
483
+ (static-ffmpeg Windows build, whisper `tiny` transcription, OCR, and the
484
+ instant idempotent re-run). Notes: the per-job lock is POSIX `fcntl` and
485
+ degrades to a no-op on Windows — fine for a single-user machine; quote paths
486
+ with spaces (`uv run talkthrough-mcp process "C:\Videos\Screen Recording.mp4"`).
487
+ Windows is not a release gate — if something breaks, please open an issue.
488
+
489
+ ## Supported inputs
490
+
491
+ Video: `.mov` `.mp4` `.webm` `.mkv` — audio-only: `.m4a` `.mp3` `.wav` `.ogg`
492
+ `.flac` (transcript tools only; frame tools explain why they're unavailable).
493
+ Local files only.
494
+
495
+ ## Limitations
496
+
497
+ Honest edges, so you can decide fast:
498
+
499
+ - **One speaker stream.** No diarization yet — "who said it" isn't tracked
500
+ ([#4](https://github.com/korovin-aa97/talkthrough-mcp/issues/4)).
501
+ - **Local files only.** No URL/YouTube ingestion
502
+ ([#5](https://github.com/korovin-aa97/talkthrough-mcp/issues/5)) — download
503
+ first.
504
+ - **Keyframes + transcript, not motion analysis.** A glitch *between* scene
505
+ changes can be invisible in the frame set; `extract_frame` re-checks any
506
+ instant, but frame-by-frame motion reasoning is your multimodal model's job.
507
+ - **STT quality tracks the model you pick.** The default `small` favors speed;
508
+ non-English narration wants `model="large-v3-turbo"` (see
509
+ [Languages](#languages)).
510
+ - **OCR reads crisp UI text well;** tiny or low-contrast print is best-effort.
511
+ - **Wall-clock confidence depends on recorder metadata** — worst case pass
512
+ `recorded_at=` (see the ladder above).
513
+ - **Windows is best-effort** (see above).
514
+
515
+ ## How it compares
516
+
517
+ | | talkthrough | cloud recorder SaaS | meeting notetakers | typical video-analyzer MCPs |
518
+ |---|---|---|---|---|
519
+ | Runs fully locally | ✅ | ❌ | ❌ | varies |
520
+ | Any local video/audio file | ✅ | browser/app captures | meetings only | ✅ |
521
+ | Wall-clock anchoring (log correlation) | ✅ | ❌ | ❌ | ❌ |
522
+ | Ships agent workflows (prompts, skill, findings contract) | ✅ | ❌ | ❌ | ❌ |
523
+ | OCR of on-screen text, searchable | ✅ | some | ❌ | rare |
524
+
525
+ ## FAQ
526
+
527
+ **Why not just upload the video to a multimodal model (e.g. Gemini)?**
528
+ For a short, non-sensitive clip — do that. The trade-offs appear with length
529
+ and sensitivity: an hour of screen recording costs on the order of a million
530
+ tokens *per question*, the file leaves your machine, and you still can't map a
531
+ remark to `14:32:07 UTC` to grep your server logs. talkthrough indexes once,
532
+ locally, then answers any number of follow-ups from the index.
533
+
534
+ **Why not screenpipe?**
535
+ Different job. screenpipe is an always-on recorder of *your* machine going
536
+ forward (commercial license). It can't open the `.mov` a teammate or customer
537
+ just sent you. talkthrough analyzes any file it's handed — the two compose
538
+ fine.
539
+
540
+ **There are agent skills that "watch" videos. Why a server with an index?**
541
+ Watch-style skills push a budgeted frame dump into the context window (and go
542
+ sparse on long videos), often call cloud STT for the audio, and keep nothing.
543
+ talkthrough builds a persistent local index — transcript + OCR, full-text
544
+ searchable — retrieves exact frames lazily, anchors everything to wall-clock
545
+ time, and answers the next question without reprocessing.
546
+
547
+ **I use Jam for bug reports — do I need this?**
548
+ Keep Jam for browser bugs: console+network captured at record time is great
549
+ evidence. talkthrough covers what a browser extension can't — desktop apps,
550
+ mobile screencasts, ops incidents, meetings, any file — with no account, and
551
+ correlates with *server-side* logs via wall-clock time.
552
+
553
+ **Can't I just script ffmpeg + whisper myself?**
554
+ Yes — that's exactly this pipeline. What you'd be rebuilding: scene-change
555
+ detection with perceptual dedup, OCR, transcript+OCR search, the wall-clock
556
+ ladder, MCP tools with embedded usage examples, five workflow prompts, and a
557
+ findings contract. One `uvx` command instead of an afternoon of glue.
558
+
559
+ **Is it really local? What leaves my machine?**
560
+ Nothing at runtime. The network is used only for one-time downloads (ffmpeg
561
+ build, whisper/OCR models). No telemetry. See [Privacy](#privacy) — and
562
+ [SECURITY.md](SECURITY.md) treats a violation of this promise as a
563
+ vulnerability.
564
+
565
+ ## For agents & tooling
566
+
567
+ Machine-readable entry points, so AI agents can install and use this server
568
+ without a human reading docs:
569
+
570
+ - [`llms-install.md`](llms-install.md) — step-by-step install instructions for agents
571
+ - [`llms.txt`](llms.txt) — index of the documentation
572
+ - [`.agents/skills/talkthrough/SKILL.md`](.agents/skills/talkthrough/SKILL.md) — an [Agent Skill](https://agentskills.io) teaching the tool workflow; discovered automatically inside a checkout by Codex CLI (`$talkthrough`) and readable by Claude Code, Cursor, Copilot, Gemini CLI and other SKILL.md-compatible tools
573
+ - [`AGENTS.md`](AGENTS.md) — instructions for coding agents contributing to this repo
574
+ - [`server.json`](server.json) — MCP registry manifest
575
+ - [`integrations/`](integrations/) — per-engine adapters, all generated from one source of truth and drift-tested (incl. the Claude Code plugin under [`integrations/claude-code/`](integrations/claude-code/))
576
+
577
+ ## Roadmap (not in v1)
578
+
579
+ URL/YouTube ingestion · speaker diarization · cloud STT · embeddings/semantic
580
+ search · hosted/remote mode · `.mcpb` bundle · whisper.cpp backend
581
+
582
+ ## License
583
+
584
+ MIT