vidai-rec 0.2.0__tar.gz

vidai_rec-0.2.0/PKG-INFO

@@ -0,0 +1,138 @@
+Metadata-Version: 2.4
+Name: vidai-rec
+Version: 0.2.0
+Summary: Record, understand and improve your videos by talking to Claude (under development)
+Author-email: AlsammanAlsamman <sammanmahmoud@gmail.com>
+Maintainer-email: AlsammanAlsamman <sammanmahmoud@gmail.com>
+Project-URL: Homepage, https://github.com/AlsammanAlsamman/VidAI
+Project-URL: Issues, https://github.com/AlsammanAlsamman/VidAI/issues
+Keywords: video,recording,editing,claude,youtube,speech-to-text,ffmpeg
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: End Users/Desktop
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Multimedia :: Video :: Capture
+Classifier: Topic :: Multimedia :: Video :: Non-Linear Editor
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: numpy
+Requires-Dist: pillow
+Requires-Dist: pydantic>=2
+Requires-Dist: imageio-ffmpeg
+Requires-Dist: opencv-python-headless
+Requires-Dist: customtkinter
+Requires-Dist: faster-whisper
+Requires-Dist: rapidocr-onnxruntime
+Provides-Extra: record
+Requires-Dist: obsws-python; extra == "record"
+Requires-Dist: pynput; extra == "record"
+Provides-Extra: mcp
+Requires-Dist: mcp>=2; extra == "mcp"
+Provides-Extra: arabic
+Requires-Dist: arabic-reshaper; extra == "arabic"
+Requires-Dist: python-bidi; extra == "arabic"
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Provides-Extra: all
+Requires-Dist: obsws-python; extra == "all"
+Requires-Dist: pynput; extra == "all"
+Requires-Dist: mcp>=2; extra == "all"
+Requires-Dist: arabic-reshaper; extra == "all"
+Requires-Dist: python-bidi; extra == "all"
+Requires-Dist: pytest; extra == "all"
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/AlsammanAlsamman/VidAI/master/assets/icon.png" alt="VidAI" width="260">
+</p>
+
+<h1 align="center">VidAI</h1>
+
+<p align="center"><b>Record, understand and improve your videos — by talking to Claude.</b></p>
+
+<p align="center">
+  <img src="https://img.shields.io/badge/status-under%20development-ff3b5c" alt="status: under development">
+  <a href="https://pypi.org/project/vidai-rec/"><img src="https://img.shields.io/pypi/v/vidai-rec?color=8b6cff&label=pypi%20vidai-rec" alt="PyPI vidai-rec"></a>
+  <img src="https://img.shields.io/badge/python-3.10%2B-34d399" alt="python 3.10+">
+  <img src="https://img.shields.io/badge/platform-Linux-ffbf00" alt="Linux">
+  <img src="https://img.shields.io/badge/tests-50%20passing-34d399" alt="50 tests passing">
+</p>
+
+> [!WARNING]
+> **VidAI is under active development.** It works end to end today (record → live effects → edit → YouTube
+> export), but features, commands and file formats may still change.
+
+VidAI is a **video plugin for Claude Code**. Say `vidai`, answer a few questions, and record. While you record,
+VidAI measures small, useful statistics ("anchors") and applies live effects; you can talk to it
+(*"VidAI, zoom in"*, *"VidAI, add a title that says Hello"*). Afterwards Claude uses the anchors to edit the
+video and exports a YouTube-ready file. Everything runs locally and for free.
+
+---
+
+## How it works
+
+<p align="center">
+  <a href="https://raw.githubusercontent.com/AlsammanAlsamman/VidAI/master/assets/flowchart/vidai_flow.svg"><img src="https://raw.githubusercontent.com/AlsammanAlsamman/VidAI/master/assets/flowchart/vidai_flow.png" alt="VidAI flow: brief, anchor setup, record, live, anchors, understand, edit plan, render" width="100%"></a>
+</p>
+
+<sub>Generated by <code>assets/flowchart/flowchart.js</code> (pure Node → SVG → PNG via headless Chrome): <code>node assets/flowchart/flowchart.js</code></sub>
+
+---
+
+## What it can do now
+
+| | |
+|---|---|
+| 🎙️ **Brief** | Claude asks what the video is about, the language, style, length and sections, then decides what to track |
+| 🎥 **Record** | Its own recorder: camera, screen, or screen + camera, with mic · live preview · countdown · crash-safe files |
+| 📊 **Live anchors** | Pauses, speech (speech-to-text), text on screen (OCR), motion, scene changes, active window, markers |
+| 🗣️ **Voice** | *“VidAI record / stop / mark / mistake / new section … / important / zoom in / zoom out / captions on”* — anything else goes to Claude (English for now) |
+| ✨ **Live effects** | Text, logos, arrows and boxes, zoom, blur, live captions — plus new effects Claude writes on the spot (e.g. an object that follows your hand) |
+| 🤔 **Thinking icon** | A little animated VidAI eye bounces in the corner while Claude works on your request |
+| 🧠 **Instant models** | Teach it something during the recording (*“VidAI label yes”*), correct it (*“VidAI wrong”*); it learns immediately |
+| ✂️ **Editing** | Remove pauses and failed takes, add titles, arrows, zooms, subtitles, chapters, audio clean-up — without touching the original |
+| ⚡ **Fast** | Renders in parallel chunks split inside silences; hot loops in C |
+| 📺 **YouTube export** | MP4 + subtitles (`.srt`) + a description with chapters |
+
+---
+
+## Quick start
+
+```bash
+pip install vidai-rec
+```
+
+Working on VidAI itself (tests, Claude Code skill and MCP config included):
+
+```bash
+git clone https://github.com/AlsammanAlsamman/VidAI && cd VidAI
+python3 -m venv .venv && .venv/bin/pip install -e ".[all]"
+.venv/bin/python -m pytest -q        # optional: run the tests
+```
+
+Then open Claude Code inside the folder and say **vidai**:
+
+```bash
+claude
+> vidai
+```
+
+Claude interviews you, opens the **VidAI Recorder**, and edits the video when you are done.
+Prefer the terminal? `vidai studio` · `vidai analyze video.mkv` · `vidai plan video.mkv remove_gaps` · `vidai render video.mkv`
+
+**Requirements:** Linux (X11), a webcam and mic (PipeWire/PulseAudio), `gcc` for the C speed-ups (optional).
+ffmpeg is bundled; the speech model downloads on first use.
+
+---
+
+## Coming next
+
+- Speech-based editing: remove filler words, keep the best take automatically
+- Built-in tracking: hands, face, objects — effects that follow them
+- Arabic voice commands and on-screen text
+- More layouts, transitions and music
+
+---
+
+<p align="center">
+  Made by <b>AlsammanAlsamman</b> · <a href="mailto:sammanmahmoud@gmail.com">sammanmahmoud@gmail.com</a>
+</p>

vidai_rec-0.2.0/README.md

@@ -0,0 +1,95 @@
+<p align="center">
+  <img src="https://raw.githubusercontent.com/AlsammanAlsamman/VidAI/master/assets/icon.png" alt="VidAI" width="260">
+</p>
+
+<h1 align="center">VidAI</h1>
+
+<p align="center"><b>Record, understand and improve your videos — by talking to Claude.</b></p>
+
+<p align="center">
+  <img src="https://img.shields.io/badge/status-under%20development-ff3b5c" alt="status: under development">
+  <a href="https://pypi.org/project/vidai-rec/"><img src="https://img.shields.io/pypi/v/vidai-rec?color=8b6cff&label=pypi%20vidai-rec" alt="PyPI vidai-rec"></a>
+  <img src="https://img.shields.io/badge/python-3.10%2B-34d399" alt="python 3.10+">
+  <img src="https://img.shields.io/badge/platform-Linux-ffbf00" alt="Linux">
+  <img src="https://img.shields.io/badge/tests-50%20passing-34d399" alt="50 tests passing">
+</p>
+
+> [!WARNING]
+> **VidAI is under active development.** It works end to end today (record → live effects → edit → YouTube
+> export), but features, commands and file formats may still change.
+
+VidAI is a **video plugin for Claude Code**. Say `vidai`, answer a few questions, and record. While you record,
+VidAI measures small, useful statistics ("anchors") and applies live effects; you can talk to it
+(*"VidAI, zoom in"*, *"VidAI, add a title that says Hello"*). Afterwards Claude uses the anchors to edit the
+video and exports a YouTube-ready file. Everything runs locally and for free.
+
+---
+
+## How it works
+
+<p align="center">
+  <a href="https://raw.githubusercontent.com/AlsammanAlsamman/VidAI/master/assets/flowchart/vidai_flow.svg"><img src="https://raw.githubusercontent.com/AlsammanAlsamman/VidAI/master/assets/flowchart/vidai_flow.png" alt="VidAI flow: brief, anchor setup, record, live, anchors, understand, edit plan, render" width="100%"></a>
+</p>
+
+<sub>Generated by <code>assets/flowchart/flowchart.js</code> (pure Node → SVG → PNG via headless Chrome): <code>node assets/flowchart/flowchart.js</code></sub>
+
+---
+
+## What it can do now
+
+| | |
+|---|---|
+| 🎙️ **Brief** | Claude asks what the video is about, the language, style, length and sections, then decides what to track |
+| 🎥 **Record** | Its own recorder: camera, screen, or screen + camera, with mic · live preview · countdown · crash-safe files |
+| 📊 **Live anchors** | Pauses, speech (speech-to-text), text on screen (OCR), motion, scene changes, active window, markers |
+| 🗣️ **Voice** | *“VidAI record / stop / mark / mistake / new section … / important / zoom in / zoom out / captions on”* — anything else goes to Claude (English for now) |
+| ✨ **Live effects** | Text, logos, arrows and boxes, zoom, blur, live captions — plus new effects Claude writes on the spot (e.g. an object that follows your hand) |
+| 🤔 **Thinking icon** | A little animated VidAI eye bounces in the corner while Claude works on your request |
+| 🧠 **Instant models** | Teach it something during the recording (*“VidAI label yes”*), correct it (*“VidAI wrong”*); it learns immediately |
+| ✂️ **Editing** | Remove pauses and failed takes, add titles, arrows, zooms, subtitles, chapters, audio clean-up — without touching the original |
+| ⚡ **Fast** | Renders in parallel chunks split inside silences; hot loops in C |
+| 📺 **YouTube export** | MP4 + subtitles (`.srt`) + a description with chapters |
+
+---
+
+## Quick start
+
+```bash
+pip install vidai-rec
+```
+
+Working on VidAI itself (tests, Claude Code skill and MCP config included):
+
+```bash
+git clone https://github.com/AlsammanAlsamman/VidAI && cd VidAI
+python3 -m venv .venv && .venv/bin/pip install -e ".[all]"
+.venv/bin/python -m pytest -q        # optional: run the tests
+```
+
+Then open Claude Code inside the folder and say **vidai**:
+
+```bash
+claude
+> vidai
+```
+
+Claude interviews you, opens the **VidAI Recorder**, and edits the video when you are done.
+Prefer the terminal? `vidai studio` · `vidai analyze video.mkv` · `vidai plan video.mkv remove_gaps` · `vidai render video.mkv`
+
+**Requirements:** Linux (X11), a webcam and mic (PipeWire/PulseAudio), `gcc` for the C speed-ups (optional).
+ffmpeg is bundled; the speech model downloads on first use.
+
+---
+
+## Coming next
+
+- Speech-based editing: remove filler words, keep the best take automatically
+- Built-in tracking: hands, face, objects — effects that follow them
+- Arabic voice commands and on-screen text
+- More layouts, transitions and music
+
+---
+
+<p align="center">
+  Made by <b>AlsammanAlsamman</b> · <a href="mailto:sammanmahmoud@gmail.com">sammanmahmoud@gmail.com</a>
+</p>

vidai_rec-0.2.0/pyproject.toml

@@ -0,0 +1,55 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "vidai-rec"
+version = "0.2.0"
+description = "Record, understand and improve your videos by talking to Claude (under development)"
+readme = "README.md"
+requires-python = ">=3.10"
+authors = [{ name = "AlsammanAlsamman", email = "sammanmahmoud@gmail.com" }]
+maintainers = [{ name = "AlsammanAlsamman", email = "sammanmahmoud@gmail.com" }]
+keywords = ["video", "recording", "editing", "claude", "youtube", "speech-to-text", "ffmpeg"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: End Users/Desktop",
+  "Operating System :: POSIX :: Linux",
+  "Programming Language :: Python :: 3",
+  "Topic :: Multimedia :: Video :: Capture",
+  "Topic :: Multimedia :: Video :: Non-Linear Editor",
+]
+dependencies = [
+  "numpy",
+  "pillow",
+  "pydantic>=2",
+  "imageio-ffmpeg",
+  "opencv-python-headless",
+  "customtkinter",
+  "faster-whisper",
+  "rapidocr-onnxruntime",
+]
+
+[project.optional-dependencies]
+record = ["obsws-python", "pynput"]
+mcp = ["mcp>=2"]
+arabic = ["arabic-reshaper", "python-bidi"]
+dev = ["pytest"]
+all = ["obsws-python", "pynput", "mcp>=2", "arabic-reshaper", "python-bidi", "pytest"]
+
+[project.urls]
+Homepage = "https://github.com/AlsammanAlsamman/VidAI"
+Issues = "https://github.com/AlsammanAlsamman/VidAI/issues"
+
+[project.scripts]
+vidai = "vidai.cli:main"
+
+[tool.setuptools.packages.find]
+include = ["vidai*"]
+
+[tool.setuptools.package-data]
+vidai = ["assets/*.png", "native/*.c"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+markers = ["slow: runs Whisper on real speech (downloads a model on first run)"]

vidai_rec-0.2.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

vidai_rec-0.2.0/tests/test_analyze.py

@@ -0,0 +1,29 @@
+import pytest
+
+from vidai.analyze import analyze
+from vidai.testing import DURATION, SCENE_CUT, SILENCES
+
+
+def test_analyze_finds_silences_and_scene_cut(video):
+    a = analyze(video, workers=3)
+    assert a.duration == pytest.approx(DURATION, abs=0.1)
+    sil = [(s.start, s.end) for s in a.segments_of("silence")]
+    assert len(sil) == len(SILENCES)
+    for (s, e), (es, ee) in zip(sil, SILENCES):
+        assert s == pytest.approx(es, abs=0.2) and e == pytest.approx(ee, abs=0.2)
+    cuts = [e.t for e in a.events_of("scene_change")]
+    assert len(cuts) == 1 and cuts[0] == pytest.approx(SCENE_CUT, abs=0.6)
+    assert len(a.series["audio_level"].values) == pytest.approx(DURATION * 10, abs=2)
+
+
+def test_parallel_and_serial_analysis_agree(video):
+    a1 = analyze(video, workers=1)
+    a3 = analyze(video, workers=3, anchors=None)
+    assert [(s.start, s.end) for s in a1.segments_of("silence")] == [(s.start, s.end) for s in a3.segments_of("silence")]
+    assert [e.t for e in a1.events_of("scene_change")] == [e.t for e in a3.events_of("scene_change")]
+
+
+def test_only_selected_stats(video):
+    a = analyze(video, stats=["silence"])
+    assert "audio_level" not in a.series and "motion" not in a.series
+    assert a.segments_of("silence")

vidai_rec-0.2.0/tests/test_anchors.py

@@ -0,0 +1,45 @@
+from vidai import AnchorFile, Brief, Event, Segment, select_stats
+
+
+def test_select_stats_depends_on_brief():
+    screen = select_stats(Brief(style="screencast"))
+    cam = select_stats(Brief(style="talking_head"))
+    assert {"silence", "markers", "scene_change", "input_activity"} <= set(screen)
+    assert "motion" in cam and "input_activity" not in cam
+    assert all(isinstance(r, str) and r for r in screen.values())
+
+
+def _anchors():
+    a = AnchorFile(video="x.mp4", duration=60.0)
+    a.add_segments([Segment(start=18.0, end=21.0, kind="silence"), Segment(start=41.0, end=42.0, kind="silence")])
+    a.add_events([Event(t=30.5, kind="scene_change")])
+    return a
+
+
+def test_split_points_snap_to_silence():
+    a = _anchors()
+    assert a.split_points(3) == [19.5, 41.5]
+    chunks = a.chunks(3)
+    assert chunks[0][0] == 0 and chunks[-1][1] == 60.0
+    assert all(b > a_ for a_, b in chunks)
+
+
+def test_split_points_fall_back_to_scene_change_then_target():
+    a = AnchorFile(video="x.mp4", duration=60.0)
+    a.add_events([Event(t=31.0, kind="scene_change")])
+    assert a.split_points(2) == [31.0]
+    assert AnchorFile(video="x", duration=60.0).split_points(2) == [30.0]
+    assert AnchorFile(video="x", duration=6.0).split_points(3) == []
+
+
+def test_roundtrip_and_queries(tmp_path):
+    a = _anchors()
+    a.brief = Brief(title="t", language="ar+en")
+    p = a.save(tmp_path / "x.mp4.anchors.json")
+    b = AnchorFile.load(p)
+    assert b.model_dump() == a.model_dump()
+    assert b.brief.languages == ["ar", "en"]
+    assert len(b.segments_of("silence", 20, 50)) == 2
+    at = b.at(30.0)
+    assert at["events"][0]["kind"] == "scene_change"
+    assert b.summary()["segments"]["silence"]["count"] == 2

vidai_rec-0.2.0/tests/test_edit.py

@@ -0,0 +1,48 @@
+import pytest
+
+from vidai import AnchorFile, Chapter, Cut, EditPlan, Event, Segment, Shape, Text
+from vidai.analyze import analyze
+
+
+def test_keep_ranges_and_time_mapping():
+    p = EditPlan(source="x.mp4", duration=20.0)
+    p.add(Cut(start=5, end=8), Cut(start=7, end=9), {"op": "cut", "start": 15, "end": 25})
+    assert p.keep_ranges() == [(0.0, 5.0), (9.0, 15.0)]
+    assert p.output_duration() == 11.0
+    assert p.map_time(4.0) == 4.0 and p.map_time(10.0) == 6.0
+    assert p.map_time(6.0) is None and p.map_time_after(6.0) == 5.0
+
+
+def test_plan_json_roundtrip(tmp_path):
+    p = EditPlan(source="x.mp4", duration=10.0)
+    p.add(Text(start=0, end=1, text="hi", position=(0.2, 0.3)), Shape(start=1, end=2, shape="circle"),
+          Chapter(t=0, title="Intro"), {"op": "zoom", "start": 1, "end": 2}, {"op": "audio", "filter": "denoise"},
+          {"op": "model", "name": "m"}, {"op": "subtitle", "start": 0, "end": 1, "text": "x"})
+    q = EditPlan.load(p.save(tmp_path / "p.json"))
+    assert q.model_dump() == p.model_dump()
+
+
+def test_remove_gaps_uses_anchors(video):
+    a = analyze(video)
+    p = EditPlan(source=str(video), duration=a.duration)
+    assert p.remove_gaps(a, min_gap=0.8, keep=0.3) == 3
+    # 3 gaps of 3, 2, 3 s, each keeps 0.3 s
+    assert p.output_duration() == pytest.approx(30 - (2.7 + 1.7 + 2.7), abs=0.5)
+
+
+def test_cut_mistakes():
+    a = AnchorFile(video="x", duration=30.0)
+    a.add_segments([Segment(start=4, end=5, kind="silence"), Segment(start=10, end=12, kind="silence")])
+    a.add_events([Event(t=9.8, kind="markers", data={"type": "mistake"})])
+    p = EditPlan(source="x", duration=30.0)
+    assert p.cut_mistakes(a) == 1
+    c = p.of(Cut)[0]
+    assert c.start == 5.0 and c.end == 11.0
+
+
+def test_chapters_from_anchors():
+    a = AnchorFile(video="x", duration=100.0)
+    a.add_events([Event(t=5, kind="scene_change"), Event(t=30, kind="scene_change"), Event(t=60, kind="scene_change")])
+    p = EditPlan(source="x", duration=100.0)
+    assert p.chapters_from_anchors(a, ["Intro", "Setup", "Demo"]) == 3
+    assert [c.t for c in p.of(Chapter)] == [0.0, 30, 60]

vidai_rec-0.2.0/tests/test_lab.py

@@ -0,0 +1,55 @@
+import numpy as np
+import pytest
+
+from vidai import lab
+from vidai.lab.examples import ColorMatch, LogisticFrameClassifier
+
+
+def _color_data(seed=0):
+    rng = np.random.default_rng(seed)
+    X = rng.integers(0, 256, (4000, 3)).astype(np.uint8)
+    xf = X.astype(np.float32)
+    Y = np.clip(0.8 * xf[:, [2, 1, 0]] + 30 + 0.3 * (xf ** 2) / 255, 0, 255).astype(np.uint8)  # swap + curve
+    return (X[:3000], Y[:3000]), (X[3000:], Y[3000:])
+
+
+def test_train_until_suitable_iterates_until_target():
+    tr, va = _color_data()
+
+    def adjust(hp, metrics, r):
+        return {**hp, "degree": 2}  # linear is not enough -> add curve terms
+
+    rep = lab.train_until_suitable(ColorMatch, tr, va, "mae", 3.0, higher_is_better=False,
+                                   hparams={"degree": 1}, adjust=adjust, save_as="color_fix")
+    assert rep.suitable and len(rep.rounds) == 2
+    assert rep.rounds[0]["metrics"]["mae"] > 3.0 >= rep.rounds[1]["metrics"]["mae"]
+    m, spec = lab.load_model("color_fix")
+    assert spec.hparams["degree"] == 2
+    assert np.abs(m.predict(va[0]).astype(int) - va[1].astype(int)).mean() <= 3.0
+    assert [x["name"] for x in lab.list_models()] == ["color_fix"]
+
+
+def test_not_saved_when_not_suitable():
+    tr, va = _color_data()
+    rep = lab.train_until_suitable(ColorMatch, tr, va, "mae", 0.01, higher_is_better=False, save_as="nope",
+                                   max_rounds=3)
+    assert not rep.suitable and rep.saved_to is None and len(rep.rounds) == 1  # no adjust -> stop
+    with pytest.raises(KeyError):
+        lab.load_model("nope")
+
+
+def test_classifier_and_code_file_model(tmp_path):
+    code = tmp_path / "bright.py"
+    code.write_text(
+        "from vidai.lab.examples import LogisticFrameClassifier\n"
+        "class Bright(LogisticFrameClassifier):\n    pass\n")
+    rng = np.random.default_rng(1)
+    X = rng.random((400, 16)).astype(np.float32)
+    y = (X.mean(1) > 0.5).astype(int)
+    cls = lab._import_class(f"{code}:Bright")
+    rep = lab.train_until_suitable(cls, (X[:300], y[:300]), (X[300:], y[300:]), "accuracy", 0.9,
+                                   save_as="bright", code_file=code)
+    assert rep.suitable
+    m, spec = lab.load_model("bright")
+    assert spec.class_path == "model.py:Bright"
+    assert (m.predict(X[300:]) == y[300:]).mean() >= 0.9