@autoaiai/aicut 0.1.7 → 0.1.11

package/skills/videodb/reference/legacy/search.md

@@ -0,0 +1,249 @@
+# Search & Indexing Guide (v1)
+
+> **LEGACY — do not use for new code.** This page documents the v1 indexing and search API. Every method here **still works in SDK 0.5.0 and is not deprecated** — nothing raises `DeprecationWarning`. The server may attach migration notices to `response.warnings`, surfaced as Python `UserWarning`. Read this only when maintaining existing v1 code.
+>
+> New code: [../indexing.md](../indexing.md) → [../search.md](../search.md). To port existing code: [../migration.md](../migration.md).
+>
+> **Routing warning.** In SDK 0.5.0 `video.search()` and `collection.search()` default to **v2**. The calls on this page reach the legacy engine only when you pass a legacy keyword (`search_type=`, `index_type=`, `scene_index_id=`, `result_threshold=`, `namespace=`, and similar) or a positional argument. **`score_threshold=` and `filter=` alone do NOT route to legacy** — they are forwarded to v2. When you mean the legacy engine, call `video.legacy_search(...)` explicitly.
+
+Search allows you to find specific moments inside videos using natural language queries, exact keywords, or visual scene descriptions.
+
+## Contents
+
+- Prerequisites
+- Indexing
+- Search Types
+- Working with Results
+- Cross-Collection Search
+- Search + Compile
+- Tips
+
+## Prerequisites
+
+Videos **must be indexed** before they can be searched. Indexing is a one-time operation per video per index type.
+
+## Indexing
+
+### Spoken Word Index
+
+Index the transcribed speech content of a video for semantic and keyword search:
+
+```python
+video = coll.get_video(video_id)
+
+# force=True makes indexing idempotent — skips if already indexed
+video.index_spoken_words(force=True)
+```
+
+This transcribes the audio track and builds a searchable index over the spoken content. Required for semantic search and keyword search.
+
+**Parameters:**
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `language_code` | `str\|None` | `None` | Language code of the video |
+| `segmentation_type` | `SegmentationType` | `SegmentationType.sentence` | Segmentation type (`sentence` or `llm`) |
+| `force` | `bool` | `False` | Set to `True` to skip if already indexed (avoids "already exists" error) |
+| `callback_url` | `str\|None` | `None` | Webhook URL for async notification |
+
+### Scene Index
+
+Index visual content by generating AI descriptions of scenes. Like spoken word indexing, this raises an error if a scene index already exists. Extract the existing `scene_index_id` from the error message.
+
+```python
+import re
+from videodb import SceneExtractionType
+
+try:
+    scene_index_id = video.index_scenes(
+        extraction_type=SceneExtractionType.shot_based,
+        prompt="Describe the visual content, objects, actions, and setting in this scene.",
+    )
+except Exception as e:
+    match = re.search(r"id\s+([a-f0-9]+)", str(e))
+    if match:
+        scene_index_id = match.group(1)
+    else:
+        raise
+```
+
+**Extraction types:**
+
+| Type | Description | Best For |
+|------|-------------|----------|
+| `SceneExtractionType.shot_based` | Splits on visual shot boundaries | General purpose, action content |
+| `SceneExtractionType.time_based` | Splits at fixed intervals | Uniform sampling, long static content |
+| `SceneExtractionType.transcript` | Splits based on transcript segments | Speech-driven scene boundaries |
+
+**Parameters for `time_based`:**
+
+```python
+video.index_scenes(
+    extraction_type=SceneExtractionType.time_based,
+    extraction_config={"time": 5, "select_frames": ["first", "last"]},
+    prompt="Describe what is happening in this scene.",
+)
+```
+
+To read back all scene descriptions (without searching), extract frame images, or manage scene indexes and collections, see [index.md](index.md).
+
+## Search Types
+
+### Semantic Search
+
+Natural language queries matched against spoken content:
+
+```python
+from videodb import SearchType
+
+results = video.search(
+    query="explaining the benefits of machine learning",
+    search_type=SearchType.semantic,
+)
+```
+
+Returns ranked segments where the spoken content semantically matches the query.
+
+### Keyword Search
+
+Exact term matching in transcribed speech:
+
+```python
+results = video.search(
+    query="artificial intelligence",
+    search_type=SearchType.keyword,
+)
+```
+
+Returns segments containing the exact keyword or phrase.
+
+### Scene Search
+
+Visual content queries matched against indexed scene descriptions. Requires a prior `index_scenes()` call.
+
+`index_scenes()` returns a `scene_index_id`. Pass it to `video.search()` to target a specific scene index (especially important when a video has multiple scene indexes):
+
+```python
+from videodb import SearchType, IndexType
+from videodb.exceptions import InvalidRequestError
+
+# Search using semantic search against the scene index.
+# Use score_threshold to filter low-relevance noise (recommended: 0.3+).
+try:
+    results = video.search(
+        query="person writing on a whiteboard",
+        search_type=SearchType.semantic,
+        index_type=IndexType.scene,
+        scene_index_id=scene_index_id,
+        score_threshold=0.3,
+    )
+    shots = results.get_shots()
+except InvalidRequestError as e:
+    if "No results found" in str(e):
+        shots = []
+    else:
+        raise
+```
+
+**Important notes:**
+
+- Use `SearchType.semantic` with `index_type=IndexType.scene` — this is the most reliable combination and works on all plans.
+- `SearchType.scene` exists but may not be available on all plans (e.g. Free tier). Prefer `SearchType.semantic` with `IndexType.scene`.
+- Passing `index_type=` or `scene_index_id=` is also what forces `search()` to route to the legacy engine. Without one of them the call goes to v2.
+- The `scene_index_id` parameter is optional. If omitted, the search runs against all scene indexes on the video. Pass it to target a specific index.
+- You can create multiple scene indexes per video (with different prompts or extraction types) and search them independently using `scene_index_id`.
+
+### Scene Search with Metadata Filtering
+
+When indexing scenes with custom metadata, you can combine semantic search with metadata filters:
+
+```python
+from videodb import SearchType, IndexType
+
+results = video.search(
+    query="a skillful chasing scene",
+    search_type=SearchType.semantic,
+    index_type=IndexType.scene,
+    scene_index_id=scene_index_id,
+    filter=[{"camera_view": "road_ahead"}, {"action_type": "chasing"}],
+)
+```
+
+See the [scene_level_metadata_indexing cookbook](https://github.com/video-db/videodb-cookbook/blob/main/quickstart/scene_level_metadata_indexing.ipynb) for a full example of custom metadata indexing and filtered search.
+
+## Working with Results
+
+### Get Shots
+
+Access individual result segments:
+
+```python
+results = video.search("your query")
+
+for shot in results.get_shots():
+    print(f"Video: {shot.video_id}")
+    print(f"Start: {shot.start:.2f}s")
+    print(f"End: {shot.end:.2f}s")
+    print(f"Text: {shot.text}")
+    print("---")
+```
+
+### Play Compiled Results
+
+Stream all matching segments as a single compiled video:
+
+```python
+results = video.search("your query")
+stream_url = results.compile()
+results.play()  # opens compiled stream in browser
+```
+
+### Extract Clips
+
+Download or stream specific result segments:
+
+```python
+for shot in results.get_shots():
+    stream_url = shot.generate_stream()
+    print(f"Clip: {stream_url}")
+```
+
+## Cross-Collection Search
+
+Search across all videos in a collection:
+
+```python
+coll = conn.get_collection()
+
+# Search across all videos in the collection
+results = coll.search(
+    query="product demo",
+    search_type=SearchType.semantic,
+)
+
+for shot in results.get_shots():
+    print(f"Video: {shot.video_id} [{shot.start:.1f}s - {shot.end:.1f}s]")
+```
+
+> **Note:** Collection-level search only supports `SearchType.semantic`. Using `SearchType.keyword` or `SearchType.scene` with `coll.search()` will raise `NotImplementedError`. For keyword or scene search, use `video.search()` on individual videos instead.
+
+## Search + Compile
+
+Index, search, and compile matching segments into a single playable stream:
+
+```python
+video.index_spoken_words(force=True)
+results = video.search(query="your query", search_type=SearchType.semantic)
+stream_url = results.compile()
+print(stream_url)
+```
+
+## Tips
+
+- **Index once, search many times**: Indexing is the expensive operation. Once indexed, searches are fast.
+- **Combine index types**: Index both spoken words and scenes to enable all search types on the same video.
+- **Refine queries**: Semantic search works best with descriptive, natural language phrases rather than single keywords.
+- **Use keyword search for precision**: When you need exact term matches, keyword search avoids semantic drift.
+- **Handle "No results found"**: legacy search raises `InvalidRequestError` when no results match. Always wrap legacy search calls in try/except and treat `"No results found"` as an empty result set. v2 search returns an empty response instead of raising.
+- **Filter scene search noise**: Semantic scene search can return low-relevance results for vague queries. Use `score_threshold=0.3` (or higher) to filter noise — but only alongside a legacy keyword such as `index_type=`. On its own, `score_threshold=` does not route to legacy, so `video.search(query, score_threshold=0.3)` runs against v2 indexes. Use `video.legacy_search(query, score_threshold=0.3)` when you mean this page's engine.
+- **Idempotent indexing**: Use `index_spoken_words(force=True)` to safely re-index. `index_scenes()` has no `force` parameter — wrap it in try/except and extract the existing `scene_index_id` from the error message with `re.search(r"id\s+([a-f0-9]+)", str(e))`.

package/skills/videodb/reference/migration.md

@@ -0,0 +1,113 @@
+# Migration: v1 → v2 Indexing
+
+SDK 0.5.0 introduced the **understand → index → retrieve** pipeline. This page maps the old API onto the new one and lists what changed behaviour.
+
+**Nothing was removed.** Every v1 method still exists and works in 0.5.0, and none of them raise `DeprecationWarning`. You are not forced to migrate. New code should use v2 because it can do things v1 cannot — structured filtering, aggregation, grounded answers, and multiple named indexes per video.
+
+For the v2 pipeline, see [indexing.md](indexing.md) and [search.md](search.md). For the v1 API as it stands, see [legacy/index.md](legacy/index.md) and [legacy/search.md](legacy/search.md).
+
+## Contents
+
+- Breaking changes in 0.5.0 (return type, routing, removed symbols)
+- Method mapping (v1 call to v2 equivalent)
+- Concept mapping
+- What has no v2 equivalent
+- Porting recipe
+
+## Breaking changes in 0.5.0
+
+These affect code that calls `search()` today, whether or not you migrate anything else.
+
+| Change | Symptom | Fix |
+|--------|---------|-----|
+| `search()` returns `SearchResponse`, not `SearchResult` | `AttributeError: 'SearchResponse' object has no attribute 'stream_url'` — also `player_url`, `collection_id` | Use `results.compile()` for the URL. `get_shots()`, iteration, `len()`, `play()` still work |
+| `search()` picks its engine by inspecting keywords | Silent behaviour change | `score_threshold=` and `filter=` do **not** route to legacy. See the routing note below |
+| Mixing v1 and v2 keywords in one `search()` call | `ValueError` | Split into two calls, or use `legacy_search()` |
+| `index_name` / `index_names` / `index_ids` passed to `search()` | `ValueError` | `search()` selects indexes automatically. Use `semantic_search()` to target them |
+| Server migration notices | `UserWarning` at runtime, once per code per process | Read `response.warnings` |
+
+### The routing sharp edge
+
+`video.search(query, score_threshold=0.3)` was a v1 pattern. In 0.5.0 `score_threshold` is in neither the legacy-trigger set nor the v2 keyword set, so the call routes to **v2** and searches v2 indexes — which may not exist yet.
+
+```python
+# v1 intent, now runs against v2:
+video.search(query, score_threshold=0.3)
+
+# v2:
+video.semantic_search(query, score_threshold=0.3)
+
+# v1, explicitly:
+video.legacy_search(query, score_threshold=0.3)
+```
+
+The same applies to `filter=`. Anything else from the legacy list — `search_type`, `index_type`, `scene_index_id`, `result_threshold`, and so on — does still route to legacy.
+
+---
+
+## Method mapping
+
+| v1 (still works) | v2 |
+|------------------|-----|
+| `video.index_spoken_words(force=True)` | `video.understand(analyzers=[{"type": "spoken_words"}])` → `video.index(source=analyzer)` |
+| `video.index_scenes(extraction_type, prompt)` → `scene_index_id` | `video.understand(analyzers=[{"type": "vlm", "config": {"prompt": ...}}])` → `video.index(source=analyzer)` |
+| `video.index_visuals(prompt, batch_config)` | a `vlm` analyzer, with run-level `segmentation` and per-analyzer `sampling` |
+| `video.index_audio(prompt, batch_config)` | a `spoken_words` analyzer, plus a `vlm` analyzer with `inputs=["transcript"]` when you want reasoning over it |
+| `SceneExtractionType.shot_based` | `segmentation={"type": "shot", "threshold": 30}` |
+| `SceneExtractionType.time_based`, `extraction_config={"time": 5}` | `segmentation={"type": "time", "seconds": 5}` |
+| `extraction_config={"frame_count": 2, "select_frames": [...]}` | `sampling={"strategy": "uniform", "frame_count": 8}` or `{"strategy": "interval", "every": 1}` |
+| `model_name="pro"`, `model_config` | analyzer `config={"model": "pro"}` |
+| `metadata={...}` on `index_scenes()` | record `metadata`, plus the field in `fields["filter"]` — metadata is filterable now |
+| `video.get_scene_index(id)` → `list[dict]` | `index.records(limit=, cursor=)` → `RecordPage`, or `analyzer.get_output()` |
+| `video.list_scene_index()` | `video.list_indexes()` and `video.list_understandings()` |
+| `video.delete_scene_index(id)` | `video.delete_index(index_id)` or `index.delete()` |
+| `video.search(q, search_type=SearchType.semantic)` | `video.search(q)` — note the return type change |
+| `video.search(q, index_type=IndexType.scene, scene_index_id=x)` | `video.semantic_search(q, index_names=["scene"])` |
+| `video.search(q, score_threshold=0.3)` | `video.semantic_search(q, score_threshold=0.3)` — the v1 form now runs v2 |
+| `video.search(q, result_threshold=10)` | `top_k=10` |
+| `video.search(q, filter=[...])` | `video.query(index_name=..., filter=[...])`, or `semantic_search(filter=...)` |
+| `coll.search(q)` — semantic only | `coll.search`, `coll.ask`, `coll.semantic_search`, `coll.query`, `coll.aggregate` |
+| an LLM loop over `get_scene_index()` output | `video.ask(question, include_sources=True)` |
+| counting scene records by hand | `video.aggregate(index_name=..., group_by=..., metric="count")` |
+| `rtstream.index_visuals(prompt, batch_config)` | `rtstream.understand(...)` → `rtstream.index(source=understanding.outputs["scene"])` |
+| `RTStreamSceneIndex.get_scenes()` | `RTStreamIndex.get_records()` |
+| `scene_index.create_alert(...)` | `rtstream_index.create_alert(...)` — same signature |
+| the `re.search(r"id\s+([a-f0-9]+)", str(e))` "index already exists" workaround | **gone.** `video.index()` is not force-gated. Use `video.get_index(name=...)` to fetch an existing index |
+
+That last row is the one to internalise. v1 had no way to ask "does this index already exist", so the documented workaround was to call `index_scenes()`, let it fail, and scrape the ID out of the error string. v2 has `get_index(name=...)`. Delete the regex.
+
+---
+
+## Concept mapping
+
+| v1 | v2 |
+|----|-----|
+| `scene_index_id` — an opaque string | `Index` — an object with status, fields, and records |
+| One scene index per video, effectively | Many named indexes per video, each with its own capabilities |
+| "Descriptions" — one blob of prose per scene | Typed artifact fields, optionally schema-controlled |
+| No schema | `index.field_schema`: per-field type, groups, and supported operators |
+| Indexing and searching are one step | Understanding, indexing, and retrieval are three, so one artifact can back several indexes |
+| Search or nothing | `search`, `ask`, `semantic_search`, `query`, `aggregate` |
+
+---
+
+## What has no v2 equivalent
+
+Keep using these:
+
+- **`video.extract_scenes()` / `get_scene_collection()`** — the only way to get viewable frame image URLs. Scene collections are unchanged and still current.
+- **`video.index_spoken_words()` for subtitles and captions.** `video.add_subtitle()` and `CaptionAsset(src="auto")` read the v1 spoken-word index; the SDK warns if you skip it. A v2 `spoken_words` artifact does not substitute here.
+
+---
+
+## Porting recipe
+
+1. **Find the v1 calls.** Grep for `index_scenes`, `index_spoken_words`, `index_visuals`, `index_audio`, `get_scene_index`, `scene_index_id`.
+2. **Collapse the indexing calls into one `understand()`.** Each v1 call becomes one analyzer in the list. `index_spoken_words` → `{"type": "spoken_words"}`; `index_scenes`/`index_visuals` → `{"type": "vlm"}`.
+3. **Move extraction config up to run level.** `extraction_type` and `extraction_config["time"]`/`["threshold"]` become `segmentation`. `frame_count`/`select_frames` become per-analyzer `sampling`.
+4. **Move the prompt and model into analyzer `config`.** Add a `schema` if you want fields you can filter and count on later.
+5. **Delete the try/except regex block.** Replace it with `understanding.wait_until_complete()`, then `video.index(source=analyzer)` per successful analyzer, then `index.wait_until_complete()`.
+6. **Rewrite the searches.** `search(index_type=..., scene_index_id=...)` → `semantic_search(index_names=[...])`. `result_threshold` → `top_k`. Anything reading `.stream_url` off a search result → `.compile()`.
+7. **Verify.** `video.list_indexes()` should show your new indexes with `status == "ready"`; `index.field_schema` shows what you can filter and sort on.
+
+Keep the v1 indexes until the v2 ones are ready — they coexist without conflict.