dreamcontext 0.5.4 → 0.7.0

package/dist/skill-packs/video-watching/SKILL.md

@@ -0,0 +1,192 @@
+---
+name: video-watching
+description: >-
+  Watch a video the way a human would — turn it into a time-mapped transcript
+  with the on-screen visuals (slides, UI, diagrams, text) described inline, then
+  reason about it. Triggers: "watch this video", "şu videoyu izle", "videoyu
+  analiz et", "transkript çıkar", "video transcript", "bu videoda ne var", or any
+  time the user hands over a video file/link. Produces a curated
+  <video>.transcript.md NEXT TO the source video, then continues per use-case
+  (marketing teardown, FTE analysis, knowledge capture, …).
+alwaysApply: false
+ruleType: "Tool Workflow"
+version: "1.0"
+---
+
+# Video Watching
+
+Turn a video into something an AI can fully reason about: a **time-mapped
+transcript** with **what's on screen described inline at the same timestamps**.
+The output is one self-contained file dropped **next to the source video**, so any
+model (our cloud, this session, anything) can read the whole video without the
+binary.
+
+The clever part: **transcript first, then YOU decide which frames to look at.**
+You don't blindly OCR every frame. You read the transcript, see where the speaker
+references something visual ("as you can see here", "this screen", "the chart"),
+look up that moment in `frames.json`, and open only those frames.
+
+## The loop (do these in order)
+
+### 0. Get the video
+Ask for the path or link if not given. Local files work today. Remote links
+(YouTube etc.) need `yt-dlp` (`brew install yt-dlp`) — the engine auto-downloads
+if it's installed.
+
+### 1. Run the engine (mechanical — one command)
+```bash
+.claude/skills/video-watching/scripts/transcribe.sh "/abs/path/to/video.mov"
+#   language auto-detects — no need to pass it. Override only if auto mislabels a
+#   short/ambiguous clip: ./transcribe.sh "/abs/path/clip.mp4" tr
+```
+
+**Pick the mode for the kind of video — this matters most for app/UI recordings:**
+- **Talking-head / lecture / ad creative** → the default is right. Scene-detect + a
+  10s gap-fill catches the visuals.
+- **App screen-recording / onboarding funnel / UI walkthrough** → add `--mode ui`.
+  App screens linger 3–5s and change by **text only** (a questionnaire step, a
+  paywall) — they don't move enough to trip scene-detect, so the 10s default
+  silently drops most of them. `--mode ui` samples every ~2.5s so each screen lands.
+  If you only need the screens (no narration), add `--frames-only` to skip whisper:
+  ```bash
+  ./scripts/transcribe.sh "/abs/path/onboarding.mp4" --mode ui --frames-only --contact-sheet
+  ```
+
+This writes everything into `<video_dir>/<slug>.media/`:
+- `transcript.srt` / `.json` / `.txt` — timestamped transcript (large-v3-turbo) — *skipped with `--frames-only`*
+- `frames/anchor_first.jpg` / `anchor_last.jpg` — first + last frame, always captured
+  (short cut-heavy creatives carry the hook and CTA here; scene-detect misses both)
+- `frames/frame_*.jpg` — frames selected in **one time-based pass**: a scene change
+  fired (slide/UI transition) **or** `MAX_GAP` seconds elapsed since the last frame,
+  whichever comes first. The gap rule is wall-clock based (`prev_selected_t`), so it
+  works on variable-frame-rate screen recordings where frame-number sampling breaks,
+  and it guarantees every static stretch (app demos, slides, CTAs) gets a frame.
+- `frames/contact_sheet.jpg` — tiled montage of all frames, *only with `--contact-sheet`*
+- `frames.json` — **the index you read**: `[{file, t, at, type}]`, sorted by time,
+  near-duplicate timestamps collapsed (anchors always kept). `type` is `scene` (a
+  picture change fired), `gap` (a periodic fill at the `MAX_GAP` cadence), or `anchor`.
+
+The engine prints a **coverage check** at the end: `longest unsampled gap = Xs`. If it
+warns the gap is >2× `MAX_GAP`, frames are likely missing — re-run denser (`--max-gap`
+lower, or `--mode ui`) **before** any expensive deep-analysis pass.
+
+`audio.wav` is auto-deleted after transcription (it's a ~1.9MB/min whisper-only
+intermediate). The whole `*.media/` dir is gitignored — it stays next to the video
+as scratch but never enters the repo. Only `<slug>.transcript.md` is the keeper;
+delete a `.media/` folder anytime to reclaim disk (re-running regenerates it).
+
+### 2. Read the transcript
+Read `transcript.srt`. It carries the spoken content with `[mm:ss]` timing. Form a
+first-pass understanding of structure and topic.
+
+### 3. Decide which frames you NEED — then view them
+Read `frames.json`. For each moment where understanding depends on the visual —
+the speaker points at something, a slide/UI/diagram/number is on screen, or the
+transcript is ambiguous without the picture — pick the frame whose `at` is closest
+to that moment and **open it with the Read tool** (it renders images). Don't open
+all of them; open the ones that carry information. Talking-head stretches with
+nothing on screen need no frame.
+
+> Heuristic: the two `anchor` frames (first/last) almost always matter — the hook
+> and the CTA. Every `scene` frame is a candidate (the picture changed for a
+> reason). `gap` frames cover static stretches scene-detect skipped — often the
+> most informative part (an app demo or onboarding screen that doesn't "move"), so
+> check them. In `--mode ui` runs most frames are `gap` — that's expected and you
+> generally want to look at all of them, one per screen.
+
+**Need a frame the index doesn't have? Grab it on demand.** Scene-detect fires on
+motion, not on meaning — on fast-cut video the most informative moment often sits
+between the indexed frames. When the VO points at something ("look at this
+screen", a number, a result) and no indexed frame lands there, fetch that exact
+second yourself:
+```bash
+ffmpeg -y -loglevel error -ss <seconds> -i "<video>" -frames:v 1 -q:v 3 \
+  "<slug>.media/frames/grab_<mmss>.jpg"
+```
+This is the whole point of transcript-first: the frame set is yours to extend, not
+a fixed dump you're stuck with.
+
+### 4. Write the curated transcript NEXT TO the video
+Write `<video_dir>/<slug>.transcript.md` — the deliverable. One timeline, voice and
+visuals interleaved, every line timestamped:
+
+```markdown
+# <Video Title>
+source: <path or URL> · duration: <mm:ss> · transcribed: large-v3-turbo
+
+## Summary
+2–4 sentences: what this video is and what it's for.
+
+## Timeline
+- **[00:00]** <what's said>
+- **[00:06]** 🖼 <what's on screen — slide title, UI state, chart, on-screen text>
+- **[00:12]** <what's said> … 🖼 <visual if relevant>
+...
+
+## Visual notes
+- **[00:06]** Slide: "Funnel → Billing → Gateway → PSP" (4 layers)
+- ...
+```
+Rules: never invent content not in transcript or frames; mark anything uncertain;
+keep the source path in the header so the file stands alone.
+
+### 5. Hand off to the use-case
+Once the `.transcript.md` exists you know the video cold. Tell the user it's ready
+and ask what they want from it — the answer is domain-specific. Triage to the
+relevant dreamcontext skill (per the skill-triage rule) and load it:
+- **Marketing video** → teardown, hook/CTA analysis, "we could do X" (→ `growth` / `meta-marketing` skills)
+- **First-time-experience / app demo** → reconstruct the FTE flow, friction points (→ `design` / `onboarding-design`)
+- **Knowledge / training** → if it should persist for the project, hand the transcript to `dreamcontext knowledge` so it becomes durable context
+Don't guess the use-case — let the user direct it.
+
+> **UI teardown (no transcript needed).** When the goal is purely the app flow —
+> e.g. tearing down a competitor's onboarding funnel — run `--frames-only --mode ui`
+> and skip the `.transcript.md` entirely. The deliverable becomes a **curated screen
+> list** (one entry per onboarding step, in order, from `frames.json`), which feeds a
+> board (`excalidraw`) or an `onboarding-design` analysis directly. Use `--contact-sheet`
+> to eyeball coverage first, and trust the coverage warning — under-sampling here is
+> exactly what makes a teardown wrongly report "screen X wasn't shown".
+
+## What this skill is NOT
+- Not a knowledge-base writer or ingestion pipeline. This skill produces the
+  transcript artifact; persisting it (chunking, embedding, ingest into a project's
+  long-term memory) is a separate, downstream concern — keep that boundary clean.
+- Not an auto-summarizer that skips the frames. The visual pass is the point —
+  that's what separates "watched the video" from "read the subtitles".
+
+## Setup (once per machine)
+```bash
+brew install whisper-cpp ffmpeg          # yt-dlp too, only for remote links
+# model: the engine prefers large-v3-turbo, falls back to large-v3 then medium.
+# pull turbo if missing:
+#   curl -L -o ~/.cache/whisper.cpp/models/ggml-large-v3-turbo.bin \
+#     https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin
+```
+`build_frame_index.py` is stdlib-only — no venv needed. Language **auto-detects**
+(whisper); only pass a code (`tr`, `en`) to force it on a short/ambiguous clip.
+
+## Files
+```
+.claude/skills/video-watching/
+├── SKILL.md                    ← you are here
+└── scripts/
+    ├── transcribe.sh           ← video → transcript + frames + frames.json (the engine)
+    └── build_frame_index.py    ← frames/*.jpg → frames.json (pts_time index + coverage check)
+```
+
+## Flags & tuning
+Flags (each also settable as an env var, e.g. `MODE=ui`):
+- `--mode ui|lecture` — sampling preset. `ui` = `MAX_GAP` 2.5s (app screens); `lecture`
+  (default) = 10s (talking-head). Explicit `--max-gap`/`--scene-threshold` always win.
+- `--frames-only` — skip whisper; extract frames only (UI/UX teardowns).
+- `--max-gap N` — guarantee a frame at least every N seconds.
+- `--scene-threshold N` — scene-change sensitivity, lower = more frames.
+- `--contact-sheet` — also emit `frames/contact_sheet.jpg` (coverage at a glance).
+- `--lang CODE` — force the transcript language (default: auto-detect).
+
+Common adjustments:
+- Onboarding/UI recording under-sampled? `--mode ui` (or push further: `--max-gap 1.5`).
+- Too few frames on a slide-heavy talk? `--scene-threshold 0.1`.
+- Long static lecture over-sampled? `--max-gap 20`.
+- Capturing too many near-identical frames? `DEDUPE_SEC=1.0 ./transcribe.sh …`.
+- Force a model: `WHISPER_MODEL=/abs/ggml-large-v3.bin ./transcribe.sh …`.

package/dist/skill-packs/video-watching/scripts/build_frame_index.py

@@ -0,0 +1,146 @@
+#!/usr/bin/env python3
+"""Build frames.json: map each extracted frame to its pts_time on the video timeline.
+
+transcribe.sh selects frames in ONE time-based pass (scene change OR every MAX_GAP),
+dumping each kept frame's pts_time to frame_times.txt. This pairs every timestamp with
+its zero-padded frame file in selection order, adds the first/last anchor frames, drops
+near-duplicate timestamps, labels each frame (scene vs gap vs anchor), and emits
+frames.json sorted by time. It also prints a COVERAGE check — the largest unsampled
+stretch — so a human can catch under-sampling before an expensive deep-analysis pass.
+
+Type labels are reconstructed from inter-frame spacing (no second decode): a frame that
+landed sooner than MAX_GAP after the previous one was triggered by a scene change
+('scene'); one that landed at the MAX_GAP cadence is a periodic fill ('gap'). They are
+hints for which frames to prioritize, not a hard contract.
+
+Stdlib only — no venv needed.
+Usage: build_frame_index.py <OUT_DIR> [DURATION_SEC]
+Env:   DEDUPE_SEC=0.4  collapse non-anchor frames closer than this to the kept one.
+       MAX_GAP=10      the gap target the engine sampled at (drives type + coverage warn).
+"""
+import json
+import os
+import re
+import sys
+from pathlib import Path
+
+# metadata=print header line, e.g.: "frame:0    pts:321024  pts_time:12.852500"
+FRAME_RE = re.compile(r"frame:(\d+)\b.*?pts_time:([\d.]+)", re.DOTALL)
+DEDUPE_SEC = float(os.environ.get("DEDUPE_SEC", "0.4"))
+MAX_GAP = float(os.environ.get("MAX_GAP", "10"))
+
+
+def parse_times(meta_file: Path) -> dict[int, float]:
+    """frame index (0-based, as ffmpeg numbers kept frames) -> pts_time seconds."""
+    if not meta_file.exists():
+        return {}
+    text = meta_file.read_text(errors="replace")
+    return {int(n): round(float(t), 2) for n, t in FRAME_RE.findall(text)}
+
+
+def collect(out_dir: Path, prefix: str, meta_name: str) -> list[dict]:
+    times = parse_times(out_dir / "frames" / meta_name)
+    frames = sorted((out_dir / "frames").glob(f"{prefix}_*.jpg"))
+    # transcribe.sh names files 1-based (%04d starts at 0001); metadata frame: is 0-based.
+    return [{"file": f"frames/{f.name}", "t": times.get(i), "type": "key"}
+            for i, f in enumerate(frames)]
+
+
+def anchors(out_dir: Path, duration: float) -> list[dict]:
+    rows = []
+    if (out_dir / "frames" / "anchor_first.jpg").exists():
+        rows.append({"file": "frames/anchor_first.jpg", "t": 0.0, "type": "anchor"})
+    if (out_dir / "frames" / "anchor_last.jpg").exists():
+        t = round(duration, 2) if duration > 0 else None
+        rows.append({"file": "frames/anchor_last.jpg", "t": t, "type": "anchor"})
+    return rows
+
+
+def fmt(t):
+    if t is None:
+        return "??:??"
+    m, s = divmod(int(t), 60)
+    return f"{m:02d}:{s:02d}"
+
+
+def dedupe(rows: list[dict]) -> list[dict]:
+    """Keep all anchors; drop a non-anchor frame within DEDUPE_SEC of the last kept."""
+    kept: list[dict] = []
+    last_t = None
+    for r in rows:
+        is_anchor = r["type"] == "anchor"
+        if (not is_anchor and last_t is not None and r["t"] is not None
+                and abs(r["t"] - last_t) < DEDUPE_SEC):
+            continue
+        kept.append(r)
+        if r["t"] is not None:
+            last_t = r["t"]
+    return kept
+
+
+def label_types(rows: list[dict]) -> None:
+    """Reconstruct scene/gap from spacing. Frames spaced >= ~MAX_GAP are periodic
+    fills ('gap'); closer ones were pulled in early by a scene change ('scene')."""
+    near = MAX_GAP * 0.9
+    prev_t = 0.0
+    for r in rows:
+        if r["type"] == "anchor":
+            if r["t"] is not None:
+                prev_t = r["t"]
+            continue
+        if r["t"] is None:
+            r["type"] = "scene"
+            continue
+        r["type"] = "gap" if (r["t"] - prev_t) >= near else "scene"
+        prev_t = r["t"]
+
+
+def coverage(rows: list[dict], duration: float) -> tuple[float, float]:
+    """Largest unsampled stretch (s) and where it starts, over [0, duration]."""
+    ts = sorted(r["t"] for r in rows if r["t"] is not None)
+    if not ts:
+        return (duration, 0.0)
+    # Bound the right edge with the real duration; if ffprobe couldn't read it (0), fall
+    # back to the last sampled time so the tail gap (last frame -> end) still surfaces
+    # rather than being silently dropped.
+    right = round(duration, 2) if duration > 0 else ts[-1]
+    bounds = [0.0] + ts + [right]
+    worst, at = 0.0, 0.0
+    for a, b in zip(bounds, bounds[1:]):
+        if b - a > worst:
+            worst, at = b - a, a
+    return (worst, at)
+
+
+def main() -> int:
+    if len(sys.argv) < 2:
+        print("usage: build_frame_index.py <OUT_DIR> [DURATION_SEC]", file=sys.stderr)
+        return 2
+    out_dir = Path(sys.argv[1])
+    duration = float(sys.argv[2]) if len(sys.argv) > 2 else 0.0
+
+    rows = collect(out_dir, "frame", "frame_times.txt")
+    rows += anchors(out_dir, duration)
+    # Sort by time (unknown timestamps sink to the end); on a tie, anchors sort FIRST so
+    # the eq(n,0) seed frame at t=0 (and any selected frame coincident with anchor_last)
+    # dedupes away against the identical anchor instead of producing a duplicate entry.
+    rows.sort(key=lambda r: (r["t"] is None, r["t"] or 0.0, r["type"] != "anchor"))
+    rows = dedupe(rows)
+    label_types(rows)
+    for r in rows:
+        r["at"] = fmt(r["t"])
+
+    (out_dir / "frames.json").write_text(json.dumps(rows, ensure_ascii=False, indent=2))
+
+    n_anchor = sum(r["type"] == "anchor" for r in rows)
+    worst, at = coverage(rows, duration)
+    print(f"   frames.json: {len(rows)} frames indexed ({n_anchor} anchors, dedupe<{DEDUPE_SEC}s)")
+    print(f"   coverage: longest unsampled gap = {worst:.1f}s at {fmt(at)} (target MAX_GAP={MAX_GAP:g}s)")
+    if duration > 0 and worst > MAX_GAP * 2:
+        print(f"   ⚠ coverage gap is >2x MAX_GAP — frames may be missing around {fmt(at)}. "
+              f"Re-run denser: --max-gap {MAX_GAP / 2:g} (or --mode ui for app recordings).")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

package/dist/skill-packs/video-watching/scripts/transcribe.sh

@@ -0,0 +1,229 @@
+#!/usr/bin/env bash
+# Video -> time-mapped transcript + significant frames (with per-frame timestamps).
+# whisper.cpp (large-v3-turbo by default) for the transcript, ffmpeg for frames.
+#
+# This is the MECHANICAL step. It produces artifacts; it does NOT understand the
+# video. Claude reads the outputs (transcript + frames.json), decides which frames
+# matter, views them, and writes the curated <video>.transcript.md (see SKILL.md).
+#
+# Usage: ./transcribe.sh "/abs/path/to/video.mov" [lang] [flags]
+#   lang: whisper language code. DEFAULT 'auto' — whisper detects it, no need to pass.
+#         Force only if auto mislabels a short/ambiguous clip (e.g. 'tr', 'en').
+#
+# Flags (also settable as env vars):
+#   --frames-only          skip whisper entirely; extract frames only. For pure
+#                          UI/UX teardowns where no transcript is needed.   (FRAMES_ONLY=1)
+#   --mode ui|lecture      sampling preset. 'ui' = MAX_GAP 2.5s (app screens linger
+#                          3-5s and change by text only, below scene-detect). 'lecture'
+#                          (default) = MAX_GAP 10s for talking-head video.    (MODE=ui)
+#   --max-gap N            guarantee a frame at least every N seconds. Overrides the
+#                          mode preset.                                       (MAX_GAP=N)
+#   --scene-threshold N    ffmpeg scene-change sensitivity, lower = more frames. (SCENE_THRESHOLD=N)
+#   --contact-sheet        also emit frames/contact_sheet.jpg — a tiled montage of every
+#                          frame, so coverage is verifiable in one glance.   (CONTACT_SHEET=1)
+#   --lang CODE            same as the positional lang arg.
+#
+# Env overrides:
+#   WHISPER_MODEL=/abs/path/ggml-*.bin   force a specific model
+#   OUT_DIR=/abs/path                    where artifacts go (default: <video_dir>/<slug>.media)
+#                                        NOTE: avoid ':' in OUT_DIR — it is special inside the
+#                                        ffmpeg filter graph and would truncate the metadata path.
+#   DEDUPE_SEC=0.4                       collapse non-anchor frames closer than this
+#
+# Outputs (in OUT_DIR):
+#   transcript.srt       timestamped segments (human-friendly)   [skipped with --frames-only]
+#   transcript.json/.txt/.vtt                                    [skipped with --frames-only]
+#   frames/frame_*.jpg   selected frames (scene change OR every MAX_GAP, whichever first)
+#   frames/anchor_first.jpg / anchor_last.jpg  always-captured first + last frame
+#   frames/contact_sheet.jpg  tiled montage of all frames        [--contact-sheet only]
+#   frames.json          [{file, t, at, type}] — each frame's pts_time, sorted (Claude reads this)
+#   source-meta.txt      ffprobe dump  (audio.wav is created then deleted after transcription)
+set -euo pipefail
+
+# --- arg + flag parsing ---------------------------------------------------------
+# Positional: first non-flag = VIDEO, second non-flag = LANG_CODE (back-compatible).
+VIDEO=""
+LANG_CODE=""
+FRAMES_ONLY="${FRAMES_ONLY:-0}"
+MODE="${MODE:-lecture}"
+CONTACT_SHEET="${CONTACT_SHEET:-0}"
+# MAX_GAP / SCENE_THRESHOLD: track whether explicitly set so a CLI/env value beats the
+# mode preset. Empty here means "fall back to the mode preset" computed below.
+MAX_GAP="${MAX_GAP:-}"
+SCENE_THRESHOLD="${SCENE_THRESHOLD:-}"
+
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --frames-only)     FRAMES_ONLY=1 ;;
+    --contact-sheet)   CONTACT_SHEET=1 ;;
+    --mode)            MODE="${2:?--mode needs a value}"; shift ;;
+    --mode=*)          MODE="${1#*=}" ;;
+    --max-gap)         MAX_GAP="${2:?--max-gap needs a value}"; shift ;;
+    --max-gap=*)       MAX_GAP="${1#*=}" ;;
+    --scene-threshold) SCENE_THRESHOLD="${2:?--scene-threshold needs a value}"; shift ;;
+    --scene-threshold=*) SCENE_THRESHOLD="${1#*=}" ;;
+    --lang)            LANG_CODE="${2:?--lang needs a value}"; shift ;;
+    --lang=*)          LANG_CODE="${1#*=}" ;;
+    --*)               echo "unknown flag: $1" >&2; exit 2 ;;
+    *)                 if [ -z "$VIDEO" ]; then VIDEO="$1"
+                       elif [ -z "$LANG_CODE" ]; then LANG_CODE="$1"
+                       else echo "unexpected arg: $1" >&2; exit 2; fi ;;
+  esac
+  shift
+done
+
+[ -n "$VIDEO" ] || { echo "Usage: transcribe.sh <video> [lang] [--frames-only] [--mode ui] [--max-gap N]" >&2; exit 1; }
+LANG_CODE="${LANG_CODE:-auto}"   # whisper auto-detects the spoken language unless overridden
+
+# Mode presets fill in only what the caller left unset (explicit CLI/env always wins).
+case "$MODE" in
+  ui)      MODE_MAX_GAP=2.5 ;;
+  lecture) MODE_MAX_GAP=10 ;;
+  *)       echo "unknown --mode '$MODE' (use ui|lecture)" >&2; exit 2 ;;
+esac
+MAX_GAP="${MAX_GAP:-$MODE_MAX_GAP}"
+SCENE_THRESHOLD="${SCENE_THRESHOLD:-0.15}"
+
+# Guard the two numeric knobs before they reach the ffmpeg filter / Python: a non-number
+# would crash build_frame_index (float()), and MAX_GAP=0 makes gte(t-prev,0) select EVERY
+# frame (disk-fill footgun). Fail loudly instead.
+is_num='^[0-9]+([.][0-9]+)?$'
+[[ "$MAX_GAP" =~ $is_num ]] || { echo "--max-gap must be a number (got '$MAX_GAP')" >&2; exit 2; }
+awk "BEGIN{exit !($MAX_GAP > 0)}" || { echo "--max-gap must be > 0 (got '$MAX_GAP')" >&2; exit 2; }
+[[ "$SCENE_THRESHOLD" =~ $is_num ]] || { echo "--scene-threshold must be a number (got '$SCENE_THRESHOLD')" >&2; exit 2; }
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# --- remote link? download with yt-dlp (local files skip this) ------------------
+if [[ "$VIDEO" =~ ^https?:// ]]; then
+  command -v yt-dlp >/dev/null || { echo "remote link needs yt-dlp: brew install yt-dlp" >&2; exit 1; }
+  DL_DIR="${OUT_DIR:-$PWD}/_download"; mkdir -p "$DL_DIR"
+  echo "==> downloading remote video with yt-dlp"
+  yt-dlp -o "$DL_DIR/%(title)s.%(ext)s" "$VIDEO"
+  VIDEO="$(ls -t "$DL_DIR"/* | head -1)"
+  echo "==> downloaded: $VIDEO"
+fi
+[ -f "$VIDEO" ] || { echo "video not found: $VIDEO" >&2; exit 1; }
+
+command -v ffmpeg >/dev/null || { echo "ffmpeg not found (brew install ffmpeg)" >&2; exit 1; }
+
+# --- model: prefer turbo, then large-v3, then medium (override with WHISPER_MODEL)
+# Only needed for transcription — skip the whole resolution when --frames-only.
+pick_model() {
+  if [ -n "${WHISPER_MODEL:-}" ]; then printf '%s' "$WHISPER_MODEL"; return; fi
+  local p
+  for p in \
+    "$HOME/.cache/whisper.cpp/models/ggml-large-v3-turbo.bin" \
+    "$HOME/.cache/openwhispr/whisper-models/ggml-large-v3-turbo.bin" \
+    "$HOME/.cache/whisper.cpp/models/ggml-large-v3.bin" \
+    "$HOME/.cache/whisper.cpp/models/ggml-medium.bin" \
+    "$HOME/.cache/openwhispr/whisper-models/ggml-medium.bin"; do
+    [ -f "$p" ] && { printf '%s' "$p"; return; }
+  done
+  printf ''
+}
+if [ "$FRAMES_ONLY" != "1" ]; then
+  MODEL="$(pick_model)"
+  [ -n "$MODEL" ] && [ -f "$MODEL" ] || {
+    echo "no whisper model found. Install one, e.g.:" >&2
+    echo "  curl -L -o ~/.cache/whisper.cpp/models/ggml-large-v3-turbo.bin \\" >&2
+    echo "    https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin" >&2
+    echo "  or set WHISPER_MODEL=/abs/path/ggml-*.bin" >&2
+    echo "  (or pass --frames-only to skip transcription entirely)" >&2
+    exit 1
+  }
+  WHISPER_BIN="$(command -v whisper-cli || command -v whisper-cpp || command -v main || true)"
+  [ -n "$WHISPER_BIN" ] || { echo "whisper.cpp binary not found (brew install whisper-cpp). Or pass --frames-only." >&2; exit 1; }
+fi
+
+# --- where artifacts go: next to the video by default ---------------------------
+SRC_DIR="$(cd "$(dirname "$VIDEO")" && pwd)"
+SLUG="$(basename "$VIDEO")"; SLUG="${SLUG%.*}"
+SLUG="$(echo "$SLUG" | tr ' ' '-' | tr -cd '[:alnum:]._-' | sed 's/--*/-/g; s/^-//; s/-$//')"
+OUT="${OUT_DIR:-$SRC_DIR/$SLUG.media}"
+mkdir -p "$OUT/frames"
+# even-dim scaling keeps the mjpeg encoder happy on odd-width sources
+SCALE="scale=trunc(iw/2)*2:trunc(ih/2)*2"
+
+echo "==> [$SLUG] probing (mode=$MODE, max_gap=${MAX_GAP}s, scene>$SCENE_THRESHOLD, frames_only=$FRAMES_ONLY)"
+ffprobe -v error -show_entries format=duration,size:stream=codec_type,codec_name,width,height \
+  -of default=noprint_wrappers=1 "$VIDEO" | tee "$OUT/source-meta.txt"
+DURATION="$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$VIDEO" 2>/dev/null | head -1)"
+DURATION="${DURATION:-0}"
+
+if [ "$FRAMES_ONLY" = "1" ]; then
+  echo "==> [$SLUG] --frames-only: skipping audio + transcription"
+else
+  echo "==> [$SLUG] extracting 16kHz mono audio"
+  ffmpeg -y -loglevel error -i "$VIDEO" -ar 16000 -ac 1 -c:a pcm_s16le "$OUT/audio.wav"
+
+  echo "==> [$SLUG] transcribing with $(basename "$MODEL") (lang=$LANG_CODE)"
+  "$WHISPER_BIN" -m "$MODEL" -f "$OUT/audio.wav" -l "$LANG_CODE" \
+    --output-txt --output-srt --output-vtt --output-json -of "$OUT/transcript" -pp
+  # audio.wav is a pure whisper intermediate (~1.9MB/min) — drop it once the transcript exists.
+  rm -f "$OUT/audio.wav"
+fi
+
+echo "==> [$SLUG] extracting frames (scene change OR every ${MAX_GAP}s, whichever fires first)"
+# ONE decode pass, time-based and frame-rate-independent:
+#   eq(n,0)                       always seed the first frame (also primes prev_selected_t)
+#   gt(scene,$SCENE_THRESHOLD)    a scene change (slide/UI transition) fired
+#   gte(t-prev_selected_t,MAX_GAP) MAX_GAP elapsed since the last KEPT frame — guarantees
+#                                 coverage of static stretches scene-detect misses (app
+#                                 screens that change by text only). prev_selected_t is
+#                                 wall-clock seconds, so this is robust on variable-frame-rate
+#                                 screen recordings where frame-number sampling (mod(n,N)) breaks.
+# pix_fmt yuvj420p avoids mjpeg "non full-range YUV" failures on screen recordings;
+# metadata=print dumps each kept frame's pts_time so frames map back to the timeline.
+ffmpeg -y -loglevel error -i "$VIDEO" \
+  -vf "select='eq(n,0)+gt(scene,$SCENE_THRESHOLD)+gte(t-prev_selected_t,$MAX_GAP)',metadata=print:file=$OUT/frames/frame_times.txt,scale=trunc(iw/2)*2:trunc(ih/2)*2" \
+  -fps_mode vfr -pix_fmt yuvj420p -q:v 3 "$OUT/frames/frame_%04d.jpg" || true
+
+# The || true above keeps a partial result usable, but zero frames means the decode
+# produced nothing (no decodable video stream, unsupported codec, write failure). That
+# must fail loudly — silently leaving an empty frames.json is the exact under-sampling
+# trap issue #15 is about. Count safely (|| true so a no-match never trips set -e).
+NSEL="$(find "$OUT/frames" -name 'frame_*.jpg' 2>/dev/null | wc -l | tr -d ' ')"
+if [ "$NSEL" -eq 0 ]; then
+  echo "ERROR: ffmpeg extracted 0 frames from '$VIDEO'." >&2
+  echo "  The file may have no decodable video stream (audio-only?), an unsupported codec," >&2
+  echo "  or the output dir isn't writable. No frames.json written." >&2
+  exit 1
+fi
+
+echo "==> [$SLUG] capturing first + last anchor frames"
+# Short, cut-heavy creatives carry the most information in the opening hook and the
+# closing CTA — but scene-detect rarely fires at t=0 or t=end. Always grab both.
+ffmpeg -y -loglevel error -i "$VIDEO" -vf "$SCALE" -frames:v 1 -q:v 3 \
+  "$OUT/frames/anchor_first.jpg" || true
+ffmpeg -y -loglevel error -sseof -0.5 -i "$VIDEO" -vf "$SCALE" -update 1 -frames:v 1 -q:v 3 \
+  "$OUT/frames/anchor_last.jpg" || true
+
+echo "==> [$SLUG] building frames.json (timestamp index, dedup near-dups, coverage check)"
+MAX_GAP="$MAX_GAP" python3 "$SCRIPT_DIR/build_frame_index.py" "$OUT" "$DURATION"
+
+# --- optional contact sheet: one glance to verify coverage before deep analysis --
+if [ "$CONTACT_SHEET" = "1" ]; then
+  echo "==> [$SLUG] building contact sheet (coverage montage)"
+  NF="$(ls -1 "$OUT/frames"/frame_*.jpg 2>/dev/null | wc -l | tr -d ' ')"
+  if [ "$NF" -gt 0 ]; then
+    COLS="$(awk -v n="$NF" 'BEGIN{c=int(sqrt(n)); if(c*c<n)c++; print c}')"
+    ROWS="$(awk -v n="$NF" -v c="$COLS" 'BEGIN{r=int(n/c); if(r*c<n)r++; print r}')"
+    ffmpeg -y -loglevel error -pattern_type glob -i "$OUT/frames/frame_*.jpg" \
+      -vf "scale=240:-1,tile=${COLS}x${ROWS}:padding=4:margin=4" -frames:v 1 -q:v 4 \
+      "$OUT/frames/contact_sheet.jpg" \
+      && echo "    contact sheet: $OUT/frames/contact_sheet.jpg (${COLS}x${ROWS})" \
+      || echo "    contact sheet skipped (montage failed)"
+  fi
+fi
+
+# find -not -name keeps the count set -e-safe (grep -v exits 1 on no-match, tripping pipefail).
+NFRAMES="$(find "$OUT/frames" -name '*.jpg' ! -name 'contact_sheet.jpg' 2>/dev/null | wc -l | tr -d ' ')"
+echo "==> [$SLUG] DONE"
+[ "$FRAMES_ONLY" = "1" ] || echo "    transcript: $OUT/transcript.srt"
+echo "    frames:     $NFRAMES  (index: $OUT/frames.json)"
+if [ "$FRAMES_ONLY" = "1" ]; then
+  echo "    next: Claude reads frames.json, views key frames, writes the curated screen list / transcript.md"
+else
+  echo "    next: Claude reads transcript + frames.json, views key frames, writes the curated transcript.md"
+fi

package/dist/templates/init/data-structures/default.md

@@ -3,33 +3,35 @@ name: {{PRODUCT_NAME}}
 description: Data structures for {{PRODUCT_NAME}}
 type: data-structures
 product: {{PRODUCT_NAME}}
+tags:
+  - data-structures
+  - database
+  - schema
 updated: {{DATE}}
 ---
-
-# Data Structures — {{PRODUCT_NAME}}
-
-Document this product's database schema, key models, and API contracts here. This file is the source of truth for data shapes.
-
-## Conventions
-
-- One `data-structures/<product>.md` file per product in a monorepo. `default.md` is the single-product fallback.
-- Sleep-state applies the **single-observation gate** to this file: a single session that adds, removes, or changes a schema MUST be reflected here in the same sleep cycle. No pattern repetition required.
-- Tasks MAY include `product: <name>` in frontmatter to route data-structure observations to the matching product file.
-
-## Schema
-
-<!-- Example:
-
 ```sql
+-- Data Structures — {{PRODUCT_NAME}}
+-- Updated: {{DATE}}
+--
+-- Document all tables, models, and key JSON shapes here.
+-- One observation gate: a schema change in a session MUST be reflected here
+-- in the same sleep cycle. No pattern repetition required.
+--
+-- Convention:
+--   - Single-product projects use default.md.
+--   - Multi-product monorepos use <product>.md per product.
+--   - The body should always be a ```sql fenced block for dashboard highlighting.
+--   - Use SQL comments (-- ...) for documentation and guidance.
+
+-- ============================================================
+-- EXAMPLE — replace or extend with your actual schema
+-- ============================================================
+
 CREATE TABLE users (
-  id UUID PRIMARY KEY,
-  email VARCHAR(255) NOT NULL UNIQUE,
-  created_at TIMESTAMP DEFAULT NOW()
+  id          UUID         PRIMARY KEY,
+  email       VARCHAR(255) NOT NULL UNIQUE,
+  created_at  TIMESTAMP    NOT NULL DEFAULT NOW()
 );
-```
 
-Or document models in any format that matches the stack — TypeScript interfaces, Prisma schema, Pydantic models, GraphQL SDL. -->
-
-## API Contracts
-
-<!-- Document request/response shapes, route handlers, and external API integrations relevant to this product. -->
+-- Add additional tables, indexes, and JSON model shapes below.
+```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dreamcontext",
-  "version": "0.5.4",
+  "version": "0.7.0",
   "description": "dreamcontext — the persistent brain for your AI agents. Remembers what you built, knows how your project works.",
   "type": "module",
   "bin": {
@@ -13,6 +13,7 @@
     "agents",
     "README.md",
     "LICENSE",
+    "NOTICE",
     "install.sh"
   ],
   "scripts": {
@@ -23,6 +24,7 @@
     "dev:dashboard": "cd dashboard && npm run dev",
     "test": "vitest",
     "test:e2e": "playwright test",
+    "diagrams": "node scripts/diagrams/build-all.mjs",
     "prepublishOnly": "npm run build"
   },
   "keywords": [
@@ -36,7 +38,7 @@
     "gemini",
     "llm"
   ],
-  "license": "MIT",
+  "license": "Apache-2.0",
   "engines": {
     "node": ">=18"
   },