davinci-resolve-mcp 2.41.0 → 2.43.0

package/CHANGELOG.md

@@ -2,6 +2,60 @@
 
 Release history for the DaVinci Resolve MCP Server. The latest release is summarized in the root README; older entries live here to keep the README focused.
 
+## What's New in v2.43.0
+
+Embeddings + similarity search — Phase C of the analysis + edit-engine
+program. "Find clips/shots like this," locally, with no vendor token cost.
+
+- **Added** `src/utils/embeddings.py` and schema v10 (`embeddings` table:
+  one float32 vector per entity/kind/model, with content hashes so re-runs
+  only re-embed what changed). Brute-force cosine — numpy when present.
+- **Added** `media_analysis` actions `build_embeddings` (idempotent; clip
+  summaries, shot descriptions + deep field groups, transcript segments,
+  sampled frames with per-shot mean vectors) and `find_similar` (query by
+  free text, clip, or shot; `kind="text"|"visual"`; free-text visual queries
+  go through the CLIP text encoder).
+- **Added** backend detection, never installation (the whisper pattern):
+  text = ollama `nomic-embed-text` (serving probe) or sentence-transformers;
+  visual = open_clip ViT-B-32. Capabilities and the Diagnostics → Tools page
+  list availability with install guidance.
+- **Added** a `Semantic` toggle on the panel's Review search (shown only
+  when a text backend is detected) backed by `/api/search/semantic`, which
+  returns rows in the existing search-card shape.
+- **Validation**: full offline suite (1095 tests; 14 new, backends mocked).
+  Live on the 2026-05-17 sample root: 54 text vectors in 1.2s via ollama
+  with correct top hits for three editorial queries; 27 CLIP vectors with
+  "cracked broken windshield glass" ranking the shattered-windshield frame
+  first; panel endpoint verified and screenshots regenerated.
+
+## What's New in v2.42.0
+
+Deep shot-level vision tier — Phase B of the analysis + edit-engine program.
+Opt-in, estimate-first per-shot field filling for the Visual / Content /
+Production / Editorial / Cuttability groups the shot pages already render.
+
+- **Added** `src/utils/deep_vision.py` and three `media_analysis` actions:
+  `deepen` (estimate → confirm_token → deferred host-vision payload per
+  clip/shot), `commit_shot_vision` (writes `vision_deep_v1` provenance rows,
+  updates the canonical blob, re-exports analysis.json in lockstep), and
+  `vision_pending_sweep` (lists clips stuck in `pending_host_analysis`;
+  `reoffer=true` returns the stored payload, `expire=true` stamps them).
+- **Added** deep depth to the analyze flow: `depth="deep"` extends the
+  deferred payload with the per-shot schema and requires `confirm_deep=true`
+  after a token-cost estimate. Caps pre-call refusal applies on both paths.
+- **Added** panel affordances: `Deepen analysis` (clip view) and `Deepen this
+  shot` (shot view) copy ready-made chat prompts, per the chat-first UX.
+  Shots with no sampled frames on disk get 1–2 frames re-extracted via
+  ffmpeg, downscaled per caps (source media stays read-only).
+- **Fixed** a provenance bug in the analysis store: a source re-deriving an
+  unchanged value no longer re-attributes the row (a deep pass would
+  otherwise claim every untouched field as `vision_deep_v1`).
+- **Validation**: full offline suite (1081 tests; 15 new), and a real
+  end-to-end deep pass on the 2026-05-17 sample clip — estimate → confirm →
+  frames read by the host chat → commit → rows/blob/export parity → fields
+  visible in the panel shot view. Control-panel guide screenshots
+  regenerated from the live panel.
+
 ## What's New in v2.41.0
 
 DB-canonical clip analysis (C1) — Phase A of the analysis + edit-engine

package/README.md

@@ -1,6 +1,6 @@
 # DaVinci Resolve MCP Server
 
-[![Version](https://img.shields.io/badge/version-2.41.0-blue.svg)](https://github.com/samuelgursky/davinci-resolve-mcp/releases)
+[![Version](https://img.shields.io/badge/version-2.43.0-blue.svg)](https://github.com/samuelgursky/davinci-resolve-mcp/releases)
 [![npm](https://img.shields.io/npm/v/davinci-resolve-mcp.svg?label=npm&color=CB3837)](https://www.npmjs.com/package/davinci-resolve-mcp)
 [![API Coverage](https://img.shields.io/badge/API%20Coverage-100%25-brightgreen.svg)](docs/reference/api-coverage.md)
 [![Tools](https://img.shields.io/badge/MCP%20Tools-33%20(341%20full)-blue.svg)](#server-modes)

package/docs/SKILL.md

@@ -494,8 +494,45 @@ Key actions: `capabilities`, `install_guidance`, `resolve_output_root`, `plan`,
 `cancel_batch_job`, `resume_batch_job`, `review_timeline_markers`,
 `cleanup_artifacts`, `db_status`, `db_ingest`, `get_panel_state`,
 `set_panel_state`, `session_start_context`, `update_clip_field`,
-`update_shot_field`, `get_field_history`, `revert_field`, and
-`list_corrections`.
+`update_shot_field`, `get_field_history`, `revert_field`,
+`list_corrections`, `deepen`, `commit_shot_vision`, `vision_pending_sweep`,
+`build_embeddings`, and `find_similar`.
+
+**Embeddings + similarity (v2.43.0+).** Local-compute semantic search; no
+vendor tokens, so nothing here touches the caps ledger. Backends are
+detected, never installed (capabilities lists them with install guidance):
+text = ollama serving `nomic-embed-text` or sentence-transformers; visual =
+open_clip (ViT-B-32, needs torch).
+- `build_embeddings(kinds=["text","visual"]?, clip_id?)` — idempotent;
+  embeds clip summaries, shot descriptions (+ deep field groups), transcript
+  segments, and sampled frames (per-shot visual vector = mean of its
+  frames'). Only re-embeds entities whose content changed.
+- `find_similar(text=… | clip_id=… | clip_id+shot_index, kind="text"|"visual",
+  entity_types?, limit?)` — brute-force cosine over the project's vectors.
+  Free-text visual queries use the CLIP text encoder ("cracked windshield"
+  finds the frame). Results carry scores plus clip/shot/segment context.
+The panel search box gains a `Semantic` toggle when a text backend is
+detected. Vectors live in the per-project DB (schema v10).
+
+**Deep shot-level vision tier (v2.42.0+).** Opt-in, estimate-first. Two
+entry points share one per-shot schema (Visual / Content / Production /
+Editorial / Cuttability / description / confidence):
+- `depth="deep"` on any analyze action extends the deferred host-vision
+  payload with `deep_shot_schema`; each `shot_descriptions` entry must carry
+  the field groups. The first deep run returns `confirmation_required` with
+  a token-cost estimate — re-call with `confirm_deep=true`. Caps still apply.
+- `deepen(clip_id|clip_dir, shot_index?|shot_indices?)` runs the pass
+  post-hoc on an already-analyzed clip. First call returns the estimate +
+  `confirm_token`; re-call with the token to get the deferred payload, read
+  its `frame_paths`, and commit via
+  `commit_shot_vision(clip_id, shots=[{shot_index, ...groups...}],
+  vision_token)`. Deep fields land as `vision_deep_v1` provenance rows;
+  human corrections always survive. Shots with no sampled frames on disk get
+  1–2 frames re-extracted via ffmpeg (read-only on source media).
+`vision_pending_sweep(expire?, max_age_days?, reoffer?)` lists clips stuck
+in `pending_host_analysis`; `reoffer=true` returns each clip's stored
+deferred payload to finish the run, `expire=true` stamps them
+`expired_host_analysis` so pendings never linger silently.
 
 **DB-canonical analysis store (v2.41.0+).** The per-project SQLite DB
 (`_soul/timeline_brain.sqlite`, schema v9+) is the source of truth for clip

package/docs/guides/control-panel.md

@@ -78,7 +78,11 @@ evidence base (analyzed / superseded / vision-pending / warnings) with
 humanized source-trust and analysis-layer chips. Cards show a representative
 thumbnail, duration, shot count, status, and summary one-liner; grid and list
 layouts are available, and full-text search covers clips, summaries, tags, and
-transcripts once the search index is built.
+transcripts once the search index is built. When a local text-embedding
+backend is detected (ollama with `nomic-embed-text`, or
+sentence-transformers), a `Semantic` toggle appears next to the search box —
+it searches by meaning instead of exact words, ranking clips, shots, and
+transcript lines by similarity to your query.
 
 ### Clip detail
 
@@ -86,7 +90,10 @@ transcripts once the search index is built.
 
 The clip view shows the full summary, tags, star rating, editorial notes, and
 a contact sheet of every detected shot. From here you can open the clip in
-Resolve, jump to the transcript, or click into any shot.
+Resolve, jump to the transcript, or click into any shot. `Deepen analysis`
+copies a ready-made chat prompt that asks your MCP session to run the opt-in
+deep shot pass (cost estimate first, then per-shot Visual / Content /
+Editorial / Cuttability fields).
 
 ### Transcript
 
@@ -107,7 +114,9 @@ start", "After cut", "Flash frame", "Motion peak"). Subjective fields are
 editable inline; edits are kept with the clip's analysis and merged on top of
 future re-analysis so human notes survive fresh vision runs. `Open in Resolve`
 jumps straight to the clip in the source viewer with the shot's mark in/out
-set.
+set. The field groups (Visual, Content, Production, Editorial, Cuttability)
+are filled by the opt-in deep vision pass — `Deepen this shot` copies a chat
+prompt that runs it for just this shot, estimate first.
 
 ### Media → History
 

package/install.py

@@ -35,7 +35,7 @@ from src.utils.update_check import (
 
 # ─── Version ──────────────────────────────────────────────────────────────────
 
-VERSION = "2.41.0"
+VERSION = "2.43.0"
 # Only hard floor: mcp[cli] requires Python 3.10+. There is no upper bound —
 # Resolve's scripting bridge loads into newer interpreters on recent builds
 # (Python 3.14 verified against Resolve Studio 20.3.2). Older Resolve builds

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "davinci-resolve-mcp",
-  "version": "2.41.0",
+  "version": "2.43.0",
   "description": "NPM bootstrapper for the DaVinci Resolve MCP Server.",
   "license": "MIT",
   "author": "Samuel Gursky <samgursky@gmail.com>",

package/src/analysis_dashboard.py

@@ -4065,6 +4065,9 @@ HTML = r"""<!doctype html>
         </div>
         <div class="review-bin-filters">
           <input id="reviewSearchInput" type="search" placeholder="Search clips, summaries, tags, transcripts…" autocomplete="off">
+          <label id="reviewSemanticToggle" class="review-semantic-toggle" style="display:none" title="Search by meaning using local text embeddings instead of exact words.">
+            <input id="reviewSemanticCheckbox" type="checkbox"> Semantic
+          </label>
           <select id="reviewBinFilter" aria-label="Filter by bin">
             <option value="">All bins</option>
           </select>
@@ -6201,6 +6204,13 @@ HTML = r"""<!doctype html>
       syncPreferencesPanel();
       applyPreferencesToControls(prefs);
       renderVersionBadge();
+      // Semantic search rides on a local text-embedding backend; only show
+      // the toggle when one is detected (ollama nomic-embed-text or
+      // sentence-transformers).
+      if (state.boot?.capabilities?.embeddings?.text?.available) {
+        const toggle = $('reviewSemanticToggle');
+        if (toggle) toggle.style.display = '';
+      }
       // Paint the previous inventory immediately (stale) so a slow first
       // /api/resolve/media — common with network source media — doesn't leave the
       // panel blank. The live fetch below replaces it and clears the stale flag.
@@ -8258,7 +8268,11 @@ HTML = r"""<!doctype html>
         renderReviewBin();
         return;
       }
-      const payload = await api(`/api/index/query?q=${encodeURIComponent(query)}&limit=40`)
+      const semantic = !!$('reviewSemanticCheckbox')?.checked;
+      const endpoint = semantic
+        ? `/api/search/semantic?q=${encodeURIComponent(query)}&limit=40`
+        : `/api/index/query?q=${encodeURIComponent(query)}&limit=40`;
+      const payload = await api(endpoint)
         .catch(err => ({ success: false, error: String(err) }));
       state.review.searchResults = payload;
       renderReviewBin();
@@ -8390,7 +8404,7 @@ HTML = r"""<!doctype html>
         <span class="meta">${formatDuration(card.duration_seconds)} · ${shots.length} shots</span>
         ${cls.primary_use ? `<span class="review-chip">${escapeHtml(cls.primary_use)}</span>` : ''}
         ${cls.select_potential ? `<span class="review-chip ${selectChipClass(cls.select_potential)}">${escapeHtml(cls.select_potential)}</span>` : ''}
-        <div class="actions"><button class="secondary" id="reviewClipTranscriptBtn">Transcript</button><button class="secondary" id="reviewClipOpenInResolveBtn">Open in Resolve</button></div>
+        <div class="actions"><button class="secondary" id="reviewClipTranscriptBtn">Transcript</button><button class="secondary" id="reviewClipOpenInResolveBtn">Open in Resolve</button><button class="secondary" data-copy-chat-prompt="${escapeHtml(`Deepen the analysis of clip “${card.clip_name || state.review.currentClipId}”: call media_analysis(action="deepen", params={"clip_id": "${card.clip_id || state.review.currentClipId}"}), show me the cost estimate, and after I confirm, read the frames and commit the per-shot fields via commit_shot_vision.`)}">Deepen analysis</button></div>
       `;
       $('reviewClipSummary').textContent = data.clip_summary || data.clip_summary_oneliner || '';
       $('reviewClipTags').innerHTML = tags.map(t => `<span class="review-chip">${escapeHtml(t)}</span>`).join('');
@@ -9440,6 +9454,7 @@ HTML = r"""<!doctype html>
         <span class="meta">${escapeHtml(shot.description || '')}</span>
         <div class="actions">
           <button class="secondary" id="reviewShotOpenInResolveBtn">Open in Resolve</button>
+          <button class="secondary" data-copy-chat-prompt="${escapeHtml(`Deepen shot ${shotIndex} of clip “${(state.review.currentClipData && state.review.currentClipData.card && state.review.currentClipData.card.clip_name) || clipId}”: call media_analysis(action="deepen", params={"clip_id": "${clipId}", "shot_index": ${shotIndex}}), show me the cost estimate, and after I confirm, read the frames and commit the per-shot fields via commit_shot_vision.`)}">Deepen this shot</button>
           <button id="reviewShotEditToggleBtn" ${editing ? '' : 'class="secondary"'}>${editToggleLabel}</button>
         </div>
       `;
@@ -11082,6 +11097,11 @@ HTML = r"""<!doctype html>
         runReviewSearch(q).catch(alertError);
       }, 250);
     });
+    // Semantic toggle re-runs the current query through the embeddings index.
+    $('reviewSemanticCheckbox')?.addEventListener('change', () => {
+      const q = $('reviewSearchInput').value.trim();
+      if (q) runReviewSearch(q).catch(alertError);
+    });
     // Bin dropdown.
     $('reviewBinFilter').addEventListener('change', event => {
       state.review.binFilter = event.target.value || '';
@@ -12469,6 +12489,57 @@ def _v2_load_analysis_db_first(project_root: str, clip_dir: str) -> Optional[Dic
     return _v2_load_analysis(clip_dir)
 
 
+def _v2_semantic_search(project_root: str, q: str, *, limit: int = 20) -> Dict[str, Any]:
+    """Semantic search over the embeddings index, shaped like /api/index/query
+    rows so the existing search-card renderer works unchanged."""
+    text = (q or "").strip()
+    if not text:
+        return {"success": True, "results": []}
+    try:
+        from src.utils import embeddings, timeline_brain_db as tbd
+
+        found = embeddings.find_similar(project_root, text=text, kind="text", limit=limit)
+        if not found.get("success"):
+            return found
+        conn = tbd.connect(project_root)
+        resolve_ids: Dict[str, Optional[str]] = {}
+
+        def resolve_clip_id(clip_uuid: Optional[str]) -> Optional[str]:
+            if not clip_uuid:
+                return None
+            if clip_uuid not in resolve_ids:
+                row = conn.execute(
+                    "SELECT resolve_clip_id, clip_dir FROM clips WHERE clip_uuid = ?",
+                    (clip_uuid,),
+                ).fetchone()
+                resolve_ids[clip_uuid] = (row["resolve_clip_id"] or row["clip_dir"]) if row else None
+            return resolve_ids[clip_uuid]
+
+        rows: List[Dict[str, Any]] = []
+        for hit in found.get("results") or []:
+            entity_type = hit.get("entity_type")
+            clip_uuid = hit.get("clip_uuid") or (hit.get("entity_uuid") if entity_type == "clip" else None)
+            row: Dict[str, Any] = {
+                "result_type": "transcript" if entity_type == "segment" else "semantic",
+                "score": hit.get("score"),
+                "clip_id": resolve_clip_id(clip_uuid),
+                "clip_name": hit.get("clip_name"),
+            }
+            if entity_type == "shot":
+                row["start_seconds"] = hit.get("time_seconds_start")
+                row["summary"] = hit.get("description")
+            elif entity_type == "segment":
+                row["start_seconds"] = hit.get("start_seconds")
+                row["summary"] = hit.get("text")
+            else:
+                row["summary"] = hit.get("summary")
+            rows.append(row)
+        rows = _v2_enrich_search_results(project_root, rows)
+        return {"success": True, "query": text, "model": found.get("model"), "results": rows}
+    except Exception as exc:  # noqa: BLE001 — search must fail soft in the panel
+        return {"success": False, "error": f"{type(exc).__name__}: {exc}"}
+
+
 def _v2_clip_duration(report: Dict[str, Any]) -> Optional[float]:
     marker_plan = report.get("clip_analysis_markers") if isinstance(report.get("clip_analysis_markers"), dict) else {}
     duration = marker_plan.get("duration_seconds")
@@ -14315,6 +14386,14 @@ class Handler(BaseHTTPRequestHandler):
                 payload["results"] = _v2_enrich_search_results(self.state.project_root, payload["results"])
             self._json(payload)
             return
+        if path == "/api/search/semantic":
+            q = (query.get("q") or [""])[0]
+            try:
+                limit = int((query.get("limit") or ["20"])[0])
+            except (TypeError, ValueError):
+                limit = 20
+            self._json(_v2_semantic_search(self.state.project_root, q, limit=limit))
+            return
         # ─── C6 timeline-history surface ───────────────────────────────
         if path == "/api/timeline_versions":
             self._json(list_timelines_with_versions(self.state.project_root))

package/src/granular/common.py

@@ -80,7 +80,7 @@ if not logging.getLogger().handlers:
         handlers=[logging.StreamHandler()],
     )
 
-VERSION = "2.41.0"
+VERSION = "2.43.0"
 logger = logging.getLogger("davinci-resolve-mcp")
 logger.info(f"Starting DaVinci Resolve MCP Server v{VERSION}")
 logger.info(f"Detected platform: {get_platform()}")

package/src/server.py

@@ -11,7 +11,7 @@ Usage:
     python src/server.py --full       # Start the 341-tool granular server instead
 """
 
-VERSION = "2.41.0"
+VERSION = "2.43.0"
 
 import base64
 import os
@@ -15020,6 +15020,13 @@ async def media_analysis(action: str, params: Optional[Dict[str, Any]] = None, c
         # C1 — DB-canonical analysis store.
         "db_status",
         "db_ingest",
+        # Phase B — deep shot-level vision tier.
+        "deepen",
+        "commit_shot_vision",
+        "vision_pending_sweep",
+        # Phase C — embeddings + similarity.
+        "build_embeddings",
+        "find_similar",
     }:
         root = resolve_media_analysis_output_root(
             project_name=project_name,
@@ -15079,6 +15086,70 @@ async def media_analysis(action: str, params: Optional[Dict[str, Any]] = None, c
         if action == "db_ingest":
             from src.utils import analysis_store
             return analysis_store.ingest_project(project_root)
+        # Phase B — deep shot-level vision tier.
+        if action == "deepen":
+            from src.utils import deep_vision
+            clip_ref = p.get("clip_id") or p.get("clipId") or p.get("clip_dir") or p.get("clipDir") or p.get("file_path") or p.get("filePath")
+            if not clip_ref:
+                return _err("deepen requires clip_id, clip_dir, or file_path")
+            raw_indices = p.get("shot_indices") or p.get("shotIndices")
+            if raw_indices is None and p.get("shot_index") is not None:
+                raw_indices = [p.get("shot_index")]
+            return deep_vision.deepen_clip(
+                project_root,
+                clip_ref=clip_ref,
+                shot_indices=[int(i) for i in raw_indices] if raw_indices else None,
+                confirm_token=p.get("confirm_token") or p.get("confirmToken"),
+                job_id=p.get("job_id") or p.get("jobId"),
+            )
+        if action == "commit_shot_vision":
+            from src.utils import deep_vision
+            clip_ref = p.get("clip_id") or p.get("clipId") or p.get("clip_dir") or p.get("clipDir") or p.get("file_path") or p.get("filePath")
+            if not clip_ref:
+                return _err("commit_shot_vision requires clip_id, clip_dir, or file_path")
+            return deep_vision.commit_shot_vision(
+                project_root,
+                shots=p.get("shots"),
+                vision_token=p.get("vision_token") or p.get("visionToken"),
+                clip_ref=clip_ref,
+                author=p.get("author") or "host_chat",
+            )
+        if action == "vision_pending_sweep":
+            from src.utils import deep_vision
+            return deep_vision.vision_pending_sweep(
+                project_root,
+                expire=_media_analysis_bool(p.get("expire"), False),
+                max_age_days=p.get("max_age_days") or p.get("maxAgeDays"),
+                reoffer=_media_analysis_bool(p.get("reoffer"), False),
+            )
+        # Phase C — embeddings + similarity (local compute; no token caps).
+        if action == "build_embeddings":
+            from src.utils import embeddings
+            kinds = p.get("kinds") or p.get("kind") or ["text"]
+            if isinstance(kinds, str):
+                kinds = [kinds]
+            return embeddings.build_embeddings(
+                project_root,
+                kinds=[str(k).strip().lower() for k in kinds],
+                clip_ref=p.get("clip_id") or p.get("clipId") or p.get("clip_dir") or p.get("clipDir"),
+                include_segments=_media_analysis_bool(p.get("include_segments", p.get("includeSegments")), True),
+                max_frames_per_clip=int(p.get("max_frames_per_clip") or p.get("maxFramesPerClip") or 16),
+            )
+        if action == "find_similar":
+            from src.utils import embeddings
+            entity_types = p.get("entity_types") or p.get("entityTypes")
+            if isinstance(entity_types, str):
+                entity_types = [entity_types]
+            return embeddings.find_similar(
+                project_root,
+                text=p.get("text") or p.get("query"),
+                clip_ref=p.get("clip_id") or p.get("clipId") or p.get("clip_dir") or p.get("clipDir"),
+                shot_index=p.get("shot_index") if p.get("shot_index") is not None else p.get("shotIndex"),
+                shot_uuid=p.get("shot_uuid") or p.get("shotUuid"),
+                kind=p.get("kind") or "text",
+                entity_types=entity_types,
+                limit=int(p.get("limit") or 10),
+            )
         if action in {"build_index", "rebuild_index"}:
             return build_analysis_index(project_root, index_path=p.get("index_path") or p.get("indexPath"))
         if action == "index_status":
@@ -15490,6 +15561,11 @@ async def media_analysis(action: str, params: Optional[Dict[str, Any]] = None, c
         "list_corrections",
         "db_status",
         "db_ingest",
+        "deepen",
+        "commit_shot_vision",
+        "vision_pending_sweep",
+        "build_embeddings",
+        "find_similar",
         "get_caps",
         "set_caps_preset",
         "get_usage",

package/src/utils/analysis_store.py

@@ -253,7 +253,10 @@ def _write_subjective(
         value_json = _dumps(value)
         existing = current.get(field_path)
         if existing is not None:
-            if str(existing["value_json"]) == value_json and str(existing["source"]) == source:
+            # Identical value → keep the existing row and its provenance. A
+            # different source re-deriving the same value must not re-attribute
+            # it (a deep pass would otherwise claim every unchanged field).
+            if str(existing["value_json"]) == value_json:
                 continue
             if (
                 str(existing["source"]) == HUMAN_SOURCE