davinci-resolve-mcp 2.42.0 → 2.44.0

package/CHANGELOG.md

@@ -2,6 +2,59 @@
 
 Release history for the DaVinci Resolve MCP Server. The latest release is summarized in the root README; older entries live here to keep the README focused.
 
+## What's New in v2.44.0
+
+Cross-clip entities + bin briefing v2 — Phase D of the analysis +
+edit-engine program. Recurring people/places/props found by clustering the
+visual embeddings, confirmed with one vision call per cluster.
+
+- **Added** schema v11 (`entities` + `entity_appearances`) and
+  `src/utils/entities.py`: union-find clustering over the v10 CLIP frame
+  vectors (cosine threshold, no new deps), representative-frame selection,
+  ghost pruning across re-runs (labeled entities persist), and a
+  detection-state stash so `entity_index` always resolves against the exact
+  ordering the payload was issued with.
+- **Added** `media_analysis` actions: `detect_entities` (clusters + deferred
+  one-frame-per-cluster confirmation payload, caps pre-checked),
+  `commit_entities` (kind/label/description with conservative-label rules;
+  `merge_with` collapses duplicate clusters), `list_entities`,
+  `prepare_bin_briefing` (entities + per-clip summaries, text-only), and
+  `commit_bin_summary` (host-synthesized briefing written above the v2.0
+  aggregate in `memory/bin_summary.md`).
+- **Added** a "Recurring across this bin" card on the panel's Review page
+  (`/api/entities`), shown once labeled entities exist.
+- **Validation**: full offline suite (1105 tests; 10 new). Live on the real
+  sample root: 3 clusters detected from 16 CLIP vectors; host-chat
+  confirmation labeled the shattered-windshield POV and the white rental
+  sedan and merged the sedan's two clusters; bin briefing synthesized and
+  committed; panel card verified and screenshots regenerated.
+
+## What's New in v2.43.0
+
+Embeddings + similarity search — Phase C of the analysis + edit-engine
+program. "Find clips/shots like this," locally, with no vendor token cost.
+
+- **Added** `src/utils/embeddings.py` and schema v10 (`embeddings` table:
+  one float32 vector per entity/kind/model, with content hashes so re-runs
+  only re-embed what changed). Brute-force cosine — numpy when present.
+- **Added** `media_analysis` actions `build_embeddings` (idempotent; clip
+  summaries, shot descriptions + deep field groups, transcript segments,
+  sampled frames with per-shot mean vectors) and `find_similar` (query by
+  free text, clip, or shot; `kind="text"|"visual"`; free-text visual queries
+  go through the CLIP text encoder).
+- **Added** backend detection, never installation (the whisper pattern):
+  text = ollama `nomic-embed-text` (serving probe) or sentence-transformers;
+  visual = open_clip ViT-B-32. Capabilities and the Diagnostics → Tools page
+  list availability with install guidance.
+- **Added** a `Semantic` toggle on the panel's Review search (shown only
+  when a text backend is detected) backed by `/api/search/semantic`, which
+  returns rows in the existing search-card shape.
+- **Validation**: full offline suite (1095 tests; 14 new, backends mocked).
+  Live on the 2026-05-17 sample root: 54 text vectors in 1.2s via ollama
+  with correct top hits for three editorial queries; 27 CLIP vectors with
+  "cracked broken windshield glass" ranking the shattered-windshield frame
+  first; panel endpoint verified and screenshots regenerated.
+
 ## What's New in v2.42.0
 
 Deep shot-level vision tier — Phase B of the analysis + edit-engine program.

package/README.md

@@ -1,6 +1,6 @@
 # DaVinci Resolve MCP Server
 
-[![Version](https://img.shields.io/badge/version-2.42.0-blue.svg)](https://github.com/samuelgursky/davinci-resolve-mcp/releases)
+[![Version](https://img.shields.io/badge/version-2.44.0-blue.svg)](https://github.com/samuelgursky/davinci-resolve-mcp/releases)
 [![npm](https://img.shields.io/npm/v/davinci-resolve-mcp.svg?label=npm&color=CB3837)](https://www.npmjs.com/package/davinci-resolve-mcp)
 [![API Coverage](https://img.shields.io/badge/API%20Coverage-100%25-brightgreen.svg)](docs/reference/api-coverage.md)
 [![Tools](https://img.shields.io/badge/MCP%20Tools-33%20(341%20full)-blue.svg)](#server-modes)

package/docs/SKILL.md

@@ -495,8 +495,44 @@ Key actions: `capabilities`, `install_guidance`, `resolve_output_root`, `plan`,
 `cleanup_artifacts`, `db_status`, `db_ingest`, `get_panel_state`,
 `set_panel_state`, `session_start_context`, `update_clip_field`,
 `update_shot_field`, `get_field_history`, `revert_field`,
-`list_corrections`, `deepen`, `commit_shot_vision`, and
-`vision_pending_sweep`.
+`list_corrections`, `deepen`, `commit_shot_vision`, `vision_pending_sweep`,
+`build_embeddings`, `find_similar`, `detect_entities`, `commit_entities`,
+`list_entities`, `prepare_bin_briefing`, and `commit_bin_summary`.
+
+**Cross-clip entities + bin briefing v2 (v2.44.0+).** Recurring
+people/places/props across a project's media, found cheaply and confirmed
+with ONE vision call per cluster:
+- `detect_entities(threshold?, min_cluster_size?)` clusters the v10 CLIP
+  frame vectors (build visual embeddings first), writes provisional entity
+  rows + appearances, and returns a deferred payload with one
+  representative frame per cluster (caps pre-checked, estimate inlined).
+  The host chat reads those frames and calls
+  `commit_entities(entities=[{entity_index, kind, label, description,
+  confidence, merge_with?}], vision_token)` — conservative labels only
+  (describe what's visible; never guess names). `merge_with` collapses
+  clusters that show the same entity.
+- `list_entities` returns labeled entities with per-clip/shot appearances;
+  the panel's Review page shows a "Recurring across this bin" card.
+- `prepare_bin_briefing` returns entities + per-clip summaries (text-only,
+  no vision cost); the host writes a colleague-style markdown briefing and
+  calls `commit_bin_summary(briefing, briefing_token)`, which lands in
+  `memory/bin_summary.md` above the v2.0 aggregate.
+
+**Embeddings + similarity (v2.43.0+).** Local-compute semantic search; no
+vendor tokens, so nothing here touches the caps ledger. Backends are
+detected, never installed (capabilities lists them with install guidance):
+text = ollama serving `nomic-embed-text` or sentence-transformers; visual =
+open_clip (ViT-B-32, needs torch).
+- `build_embeddings(kinds=["text","visual"]?, clip_id?)` — idempotent;
+  embeds clip summaries, shot descriptions (+ deep field groups), transcript
+  segments, and sampled frames (per-shot visual vector = mean of its
+  frames'). Only re-embeds entities whose content changed.
+- `find_similar(text=… | clip_id=… | clip_id+shot_index, kind="text"|"visual",
+  entity_types?, limit?)` — brute-force cosine over the project's vectors.
+  Free-text visual queries use the CLIP text encoder ("cracked windshield"
+  finds the frame). Results carry scores plus clip/shot/segment context.
+The panel search box gains a `Semantic` toggle when a text backend is
+detected. Vectors live in the per-project DB (schema v10).
 
 **Deep shot-level vision tier (v2.42.0+).** Opt-in, estimate-first. Two
 entry points share one per-shot schema (Visual / Content / Production /

package/docs/guides/control-panel.md

@@ -78,7 +78,14 @@ evidence base (analyzed / superseded / vision-pending / warnings) with
 humanized source-trust and analysis-layer chips. Cards show a representative
 thumbnail, duration, shot count, status, and summary one-liner; grid and list
 layouts are available, and full-text search covers clips, summaries, tags, and
-transcripts once the search index is built.
+transcripts once the search index is built. When a local text-embedding
+backend is detected (ollama with `nomic-embed-text`, or
+sentence-transformers), a `Semantic` toggle appears next to the search box —
+it searches by meaning instead of exact words, ranking clips, shots, and
+transcript lines by similarity to your query. Once cross-clip entity
+detection has run (`detect_entities` + a one-frame-per-cluster confirmation
+in chat), a `Recurring across this bin` card lists the labeled people,
+places, and objects with their shot counts.
 
 ### Clip detail
 

package/install.py

@@ -35,7 +35,7 @@ from src.utils.update_check import (
 
 # ─── Version ──────────────────────────────────────────────────────────────────
 
-VERSION = "2.42.0"
+VERSION = "2.44.0"
 # Only hard floor: mcp[cli] requires Python 3.10+. There is no upper bound —
 # Resolve's scripting bridge loads into newer interpreters on recent builds
 # (Python 3.14 verified against Resolve Studio 20.3.2). Older Resolve builds

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "davinci-resolve-mcp",
-  "version": "2.42.0",
+  "version": "2.44.0",
   "description": "NPM bootstrapper for the DaVinci Resolve MCP Server.",
   "license": "MIT",
   "author": "Samuel Gursky <samgursky@gmail.com>",

package/src/analysis_dashboard.py

@@ -2114,6 +2114,30 @@ HTML = r"""<!doctype html>
       margin-bottom: var(--space-3);
       flex-wrap: wrap;
     }
+    .review-semantic-toggle {
+      display: inline-flex;
+      gap: 6px;
+      align-items: center;
+      color: var(--text-secondary);
+      font-size: var(--ops-text-label);
+      white-space: nowrap;
+    }
+    .review-entities {
+      margin-bottom: var(--space-3);
+      padding: var(--space-2) var(--space-3);
+      border: 1px solid var(--border-default);
+      border-radius: var(--radius-md);
+    }
+    .review-entities-title {
+      color: var(--text-secondary);
+      font-size: var(--ops-text-label);
+      margin-bottom: 6px;
+    }
+    .review-entities-chips {
+      display: flex;
+      gap: var(--space-2);
+      flex-wrap: wrap;
+    }
     .review-bin-filters input[type="search"] {
       flex: 1 1 280px;
       min-width: 200px;
@@ -4065,6 +4089,9 @@ HTML = r"""<!doctype html>
         </div>
         <div class="review-bin-filters">
           <input id="reviewSearchInput" type="search" placeholder="Search clips, summaries, tags, transcripts…" autocomplete="off">
+          <label id="reviewSemanticToggle" class="review-semantic-toggle" style="display:none" title="Search by meaning using local text embeddings instead of exact words.">
+            <input id="reviewSemanticCheckbox" type="checkbox"> Semantic
+          </label>
           <select id="reviewBinFilter" aria-label="Filter by bin">
             <option value="">All bins</option>
           </select>
@@ -4074,6 +4101,7 @@ HTML = r"""<!doctype html>
           </div>
         </div>
         <div id="reviewBinSummary" class="review-bin-summary">Loading analyzed clips…</div>
+        <div id="reviewEntitiesCard" class="review-entities" style="display:none"></div>
         <div id="reviewBinGrid" class="review-grid"></div>
         <div id="reviewSearchResults" class="review-search-results" style="display:none"></div>
       </div>
@@ -6201,6 +6229,13 @@ HTML = r"""<!doctype html>
       syncPreferencesPanel();
       applyPreferencesToControls(prefs);
       renderVersionBadge();
+      // Semantic search rides on a local text-embedding backend; only show
+      // the toggle when one is detected (ollama nomic-embed-text or
+      // sentence-transformers).
+      if (state.boot?.capabilities?.embeddings?.text?.available) {
+        const toggle = $('reviewSemanticToggle');
+        if (toggle) toggle.style.display = '';
+      }
       // Paint the previous inventory immediately (stale) so a slow first
       // /api/resolve/media — common with network source media — doesn't leave the
       // panel blank. The live fetch below replaces it and clears the stale flag.
@@ -8083,6 +8118,22 @@ HTML = r"""<!doctype html>
       renderReviewBin();
       // Coverage runs in parallel — failures here must not block the clip grid.
       refreshReadinessCard().catch(() => {});
+      refreshEntitiesCard().catch(() => {});
+    }
+
+    // Recurring people/places/props detected across the bin (Phase D).
+    // Hidden until at least one labeled entity exists.
+    async function refreshEntitiesCard() {
+      const card = $('reviewEntitiesCard');
+      if (!card) return;
+      const data = await api('/api/entities').catch(() => null);
+      const labeled = (data?.entities || []).filter(e => e.label);
+      if (!labeled.length) { card.style.display = 'none'; return; }
+      card.style.display = '';
+      card.innerHTML = `<div class="review-entities-title">Recurring across this bin</div>
+        <div class="review-entities-chips">${labeled.map(e =>
+          `<span class="review-chip" title="${escapeHtml(e.description || '')}">${escapeHtml(e.label)} · ${e.kind || 'unknown'} · ${e.shot_count || e.cluster_size || 0} shots</span>`
+        ).join('')}</div>`;
     }
 
     async function refreshReadinessCard() {
@@ -8258,7 +8309,11 @@ HTML = r"""<!doctype html>
         renderReviewBin();
         return;
       }
-      const payload = await api(`/api/index/query?q=${encodeURIComponent(query)}&limit=40`)
+      const semantic = !!$('reviewSemanticCheckbox')?.checked;
+      const endpoint = semantic
+        ? `/api/search/semantic?q=${encodeURIComponent(query)}&limit=40`
+        : `/api/index/query?q=${encodeURIComponent(query)}&limit=40`;
+      const payload = await api(endpoint)
         .catch(err => ({ success: false, error: String(err) }));
       state.review.searchResults = payload;
       renderReviewBin();
@@ -11083,6 +11138,11 @@ HTML = r"""<!doctype html>
         runReviewSearch(q).catch(alertError);
       }, 250);
     });
+    // Semantic toggle re-runs the current query through the embeddings index.
+    $('reviewSemanticCheckbox')?.addEventListener('change', () => {
+      const q = $('reviewSearchInput').value.trim();
+      if (q) runReviewSearch(q).catch(alertError);
+    });
     // Bin dropdown.
     $('reviewBinFilter').addEventListener('change', event => {
       state.review.binFilter = event.target.value || '';
@@ -12470,6 +12530,57 @@ def _v2_load_analysis_db_first(project_root: str, clip_dir: str) -> Optional[Dic
     return _v2_load_analysis(clip_dir)
 
 
+def _v2_semantic_search(project_root: str, q: str, *, limit: int = 20) -> Dict[str, Any]:
+    """Semantic search over the embeddings index, shaped like /api/index/query
+    rows so the existing search-card renderer works unchanged."""
+    text = (q or "").strip()
+    if not text:
+        return {"success": True, "results": []}
+    try:
+        from src.utils import embeddings, timeline_brain_db as tbd
+
+        found = embeddings.find_similar(project_root, text=text, kind="text", limit=limit)
+        if not found.get("success"):
+            return found
+        conn = tbd.connect(project_root)
+        resolve_ids: Dict[str, Optional[str]] = {}
+
+        def resolve_clip_id(clip_uuid: Optional[str]) -> Optional[str]:
+            if not clip_uuid:
+                return None
+            if clip_uuid not in resolve_ids:
+                row = conn.execute(
+                    "SELECT resolve_clip_id, clip_dir FROM clips WHERE clip_uuid = ?",
+                    (clip_uuid,),
+                ).fetchone()
+                resolve_ids[clip_uuid] = (row["resolve_clip_id"] or row["clip_dir"]) if row else None
+            return resolve_ids[clip_uuid]
+
+        rows: List[Dict[str, Any]] = []
+        for hit in found.get("results") or []:
+            entity_type = hit.get("entity_type")
+            clip_uuid = hit.get("clip_uuid") or (hit.get("entity_uuid") if entity_type == "clip" else None)
+            row: Dict[str, Any] = {
+                "result_type": "transcript" if entity_type == "segment" else "semantic",
+                "score": hit.get("score"),
+                "clip_id": resolve_clip_id(clip_uuid),
+                "clip_name": hit.get("clip_name"),
+            }
+            if entity_type == "shot":
+                row["start_seconds"] = hit.get("time_seconds_start")
+                row["summary"] = hit.get("description")
+            elif entity_type == "segment":
+                row["start_seconds"] = hit.get("start_seconds")
+                row["summary"] = hit.get("text")
+            else:
+                row["summary"] = hit.get("summary")
+            rows.append(row)
+        rows = _v2_enrich_search_results(project_root, rows)
+        return {"success": True, "query": text, "model": found.get("model"), "results": rows}
+    except Exception as exc:  # noqa: BLE001 — search must fail soft in the panel
+        return {"success": False, "error": f"{type(exc).__name__}: {exc}"}
+
+
 def _v2_clip_duration(report: Dict[str, Any]) -> Optional[float]:
     marker_plan = report.get("clip_analysis_markers") if isinstance(report.get("clip_analysis_markers"), dict) else {}
     duration = marker_plan.get("duration_seconds")
@@ -14316,6 +14427,22 @@ class Handler(BaseHTTPRequestHandler):
                 payload["results"] = _v2_enrich_search_results(self.state.project_root, payload["results"])
             self._json(payload)
             return
+        if path == "/api/entities":
+            try:
+                from src.utils import entities as _entities
+
+                self._json(_entities.list_entities(self.state.project_root))
+            except Exception as exc:  # noqa: BLE001 — panel reads fail soft
+                self._json({"success": False, "error": f"{type(exc).__name__}: {exc}"})
+            return
+        if path == "/api/search/semantic":
+            q = (query.get("q") or [""])[0]
+            try:
+                limit = int((query.get("limit") or ["20"])[0])
+            except (TypeError, ValueError):
+                limit = 20
+            self._json(_v2_semantic_search(self.state.project_root, q, limit=limit))
+            return
         # ─── C6 timeline-history surface ───────────────────────────────
         if path == "/api/timeline_versions":
             self._json(list_timelines_with_versions(self.state.project_root))

package/src/granular/common.py

@@ -80,7 +80,7 @@ if not logging.getLogger().handlers:
         handlers=[logging.StreamHandler()],
     )
 
-VERSION = "2.42.0"
+VERSION = "2.44.0"
 logger = logging.getLogger("davinci-resolve-mcp")
 logger.info(f"Starting DaVinci Resolve MCP Server v{VERSION}")
 logger.info(f"Detected platform: {get_platform()}")

package/src/server.py

@@ -11,7 +11,7 @@ Usage:
     python src/server.py --full       # Start the 341-tool granular server instead
 """
 
-VERSION = "2.42.0"
+VERSION = "2.44.0"
 
 import base64
 import os
@@ -15024,6 +15024,15 @@ async def media_analysis(action: str, params: Optional[Dict[str, Any]] = None, c
         "deepen",
         "commit_shot_vision",
         "vision_pending_sweep",
+        # Phase C — embeddings + similarity.
+        "build_embeddings",
+        "find_similar",
+        # Phase D — cross-clip entities + bin briefing v2.
+        "detect_entities",
+        "commit_entities",
+        "list_entities",
+        "prepare_bin_briefing",
+        "commit_bin_summary",
     }:
         root = resolve_media_analysis_output_root(
             project_name=project_name,
@@ -15119,6 +15128,66 @@ async def media_analysis(action: str, params: Optional[Dict[str, Any]] = None, c
                 max_age_days=p.get("max_age_days") or p.get("maxAgeDays"),
                 reoffer=_media_analysis_bool(p.get("reoffer"), False),
             )
+        # Phase C — embeddings + similarity (local compute; no token caps).
+        if action == "build_embeddings":
+            from src.utils import embeddings
+            kinds = p.get("kinds") or p.get("kind") or ["text"]
+            if isinstance(kinds, str):
+                kinds = [kinds]
+            return embeddings.build_embeddings(
+                project_root,
+                kinds=[str(k).strip().lower() for k in kinds],
+                clip_ref=p.get("clip_id") or p.get("clipId") or p.get("clip_dir") or p.get("clipDir"),
+                include_segments=_media_analysis_bool(p.get("include_segments", p.get("includeSegments")), True),
+                max_frames_per_clip=int(p.get("max_frames_per_clip") or p.get("maxFramesPerClip") or 16),
+            )
+        if action == "find_similar":
+            from src.utils import embeddings
+            entity_types = p.get("entity_types") or p.get("entityTypes")
+            if isinstance(entity_types, str):
+                entity_types = [entity_types]
+            return embeddings.find_similar(
+                project_root,
+                text=p.get("text") or p.get("query"),
+                clip_ref=p.get("clip_id") or p.get("clipId") or p.get("clip_dir") or p.get("clipDir"),
+                shot_index=p.get("shot_index") if p.get("shot_index") is not None else p.get("shotIndex"),
+                shot_uuid=p.get("shot_uuid") or p.get("shotUuid"),
+                kind=p.get("kind") or "text",
+                entity_types=entity_types,
+                limit=int(p.get("limit") or 10),
+            )
+        # Phase D — cross-clip entities + bin briefing v2.
+        if action == "detect_entities":
+            from src.utils import entities
+            return entities.detect_entities(
+                project_root,
+                threshold=float(p.get("threshold") or entities.DEFAULT_CLUSTER_THRESHOLD),
+                min_cluster_size=int(p.get("min_cluster_size") or p.get("minClusterSize") or entities.DEFAULT_MIN_CLUSTER_SIZE),
+                max_clusters=int(p.get("max_clusters") or p.get("maxClusters") or 24),
+                job_id=p.get("job_id") or p.get("jobId"),
+            )
+        if action == "commit_entities":
+            from src.utils import entities
+            return entities.commit_entities(
+                project_root,
+                entities_payload=p.get("entities"),
+                vision_token=p.get("vision_token") or p.get("visionToken"),
+                author=p.get("author") or "host_chat",
+            )
+        if action == "list_entities":
+            from src.utils import entities
+            return entities.list_entities(project_root, kind=p.get("kind"))
+        if action == "prepare_bin_briefing":
+            from src.utils import entities
+            return entities.prepare_bin_briefing(project_root)
+        if action == "commit_bin_summary":
+            from src.utils import entities
+            return entities.commit_bin_summary(
+                project_root,
+                briefing=p.get("briefing") or p.get("summary"),
+                briefing_token=p.get("briefing_token") or p.get("briefingToken"),
+                author=p.get("author") or "host_chat",
+            )
         if action in {"build_index", "rebuild_index"}:
             return build_analysis_index(project_root, index_path=p.get("index_path") or p.get("indexPath"))
         if action == "index_status":
@@ -15533,6 +15602,13 @@ async def media_analysis(action: str, params: Optional[Dict[str, Any]] = None, c
         "deepen",
         "commit_shot_vision",
         "vision_pending_sweep",
+        "build_embeddings",
+        "find_similar",
+        "detect_entities",
+        "commit_entities",
+        "list_entities",
+        "prepare_bin_briefing",
+        "commit_bin_summary",
         "get_caps",
         "set_caps_preset",
         "get_usage",