@lalalic/markcut 3.2.0 → 3.2.2

package/README.md

@@ -342,3 +342,27 @@ flowchart LR
 | `--show-prompts` | Print the prompts template file and exit |
 
 ## Architecture
+
+
+### Browser ChatGPT Vision backend
+
+Markcut can use the Neo `chatgpt-browser-worker` agent as the configurable ITT/VTT backend without changing the default local VLMs. Each invocation creates one unique durable file output plus unique Neo job/task correlation, then hands the complete one-shot request to the existing browser-worker agent runtime.
+
+For an installed package, select it with the existing CLI-template surface:
+
+```bash
+export MARKCUT_ITT_CLI='markcut vision-chatgpt --mode image --prompt "{prompt}" --input {input}'
+export MARKCUT_VTT_CLI='markcut vision-chatgpt --mode video --prompt "{prompt}" --input {input}'
+```
+
+For a source checkout, replace `markcut` above with `node src/render/cli.mjs`.
+
+The facade intentionally does not call `chatgpt-browser-worker/scripts/*` or automate Chrome itself. Set `MARKCUT_CHATGPT_BROWSER_WORKER_AGENT_CLI` to the local command that launches the Neo **browser-worker agent/runtime**. The older `MARKCUT_CHATGPT_BROWSER_WORKER_CLI` name remains as a compatibility fallback.
+
+- `MARKCUT_CHATGPT_PROMPT_FILE` — complete one-shot prompt, including the exact `Output: file` contract and worker-owned event IDs.
+- `MARKCUT_CHATGPT_MEDIA_FILES_JSON` — JSON array of attachments. Images are passed directly; video is deterministically reduced to a chronological contact sheet of representative frames plus timing context.
+- `MARKCUT_CHATGPT_OUTPUT_FILE` — authoritative result path that the delegated ChatGPT task must write.
+- `NEO_JOB_ID` / `NEO_TASK_ID` — unique correlation for the delegated worker's canonical `task.started` and terminal task event.
+- `MARKCUT_CHATGPT_TAB_CLOSE_POLICY=after-terminal` — requests that only the isolated worker-owned tab remain open until the exact task terminal event.
+
+The launcher may stay active through `after-terminal`; launcher/process exit is not treated as task completion. Markcut's only result is the declared durable output file, and it exits nonzero on launcher failure, timeout, preprocessing failure, or an empty/missing result. Carrier-only `task.process.launched`, `task.process.exited`, and `task.process.async_exited` events never substitute for worker-owned task lifecycle.

package/docs/research/2026-09-19-chatgpt-browser-vision.md

@@ -0,0 +1,250 @@
+# ChatGPT Browser Harness as a Markcut Vision Backend
+
+Date: 2026-09-19
+PR: #3
+Recommendation: **not currently viable**
+
+## Objective
+
+Evaluate whether Markcut's configurable Vision CLI can use an authenticated ChatGPT web session, operated through the `chatgpt-browser-worker` skill and `browser-harness`, as an image/video understanding backend without using the ChatGPT API or the legacy MacDeveloperBridge Chrome/token path.
+
+## Existing Markcut seam
+
+Markcut already has a narrow backend seam in `src/config.mjs`:
+
+- `DEFAULT_ITT_CLI` for image-to-text
+- `DEFAULT_VTT_CLI` for video-to-text
+- `DEFAULT_AGENT_CLI` for downstream agent work
+
+The Vision CLI substitutes `{prompt}` and `{input}` into these command templates, so a future browser-backed backend does not require rewriting the Vision pipeline. The correct integration shape would be a small executable command that accepts the same prompt/input contract and emits normalized text/JSON on stdout.
+
+```mermaid
+flowchart LR
+    M[markcut vision] --> C[configured ITT/VTT command]
+    C -->|current| L[mlx-vlm]
+    C -->|experimental| B[chatgpt-browser-worker adapter]
+    B --> H[browser-harness]
+    H --> W[authenticated ChatGPT web UI]
+    W --> R[verified assistant result]
+    R --> N[normalize JSON/text]
+    N --> M
+```
+
+## Browser-worker path tested
+
+The experiment used the `chatgpt-browser-worker` contract from the Agents Relay worktree. The worker uses `browser-harness` as the only browser interaction layer and preserves ChatGPT thread identity and Project identity.
+
+Observed worker thread:
+
+- Project: `agents-relay`
+- Thread: `6aaf32d4-4164-83ea-860a-c2453ce57e51`
+- Requested thinking: `high`
+- Effective thinking: `unknown` (UI could not verify it; no inference was made)
+
+A second experiment targeted the existing `markcut` ChatGPT Project. The helper selected the Project, but current ChatGPT UI drift prevented its create helper from reliably observing/sending the composer without manual self-healing.
+
+## Synthetic fixtures
+
+To avoid private media and make correctness objectively measurable, the test generated:
+
+1. `image-test-unique.png`
+   - red rectangle
+   - blue circle
+   - green triangle
+   - visible text identifying all three shapes and a unique fixture label
+2. `video-test.mp4`
+   - 6 seconds total
+   - 0-2s red phase
+   - 2-4s blue phase
+   - 4-6s green phase
+
+Fixtures were generated locally and kept out of Git.
+
+## Control: existing Markcut image backend
+
+The current configured image backend was run directly with the same image fixture using the existing `mlx-vlm`/MiniCPM path.
+
+Observed runtime: approximately 8 seconds.
+
+Observed result correctly identified:
+
+- red rectangle
+- blue circle
+- green triangle
+- `RED RECTANGLE | BLUE CIRCLE | GREEN TRIANGLE`
+- `MARKCUT VISION UNIQUE FIXTURE 2026-09-19`
+
+This establishes that the fixture and current Markcut backend are functioning.
+
+## ChatGPT image experiment
+
+The Browser Harness path successfully:
+
+- selected an authenticated ChatGPT Project;
+- created and preserved a real ChatGPT thread id;
+- uploaded an image into the composer;
+- submitted a media-analysis prompt through the browser UI;
+- observed assistant message ids from the actual ChatGPT DOM.
+
+However, the result path was not reliable enough for a backend:
+
+1. One submitted image turn produced a terminal-looking but truncated assistant message:
+
+   `{"description":"A`
+
+2. The existing `result` helper incorrectly considered that partial text stable/final because its generating-state detector did not match the current UI state.
+3. A later image retry entered a persistent `Stop answering` state while the attachment and prompt remained in the composer and no durable new user turn was observed.
+4. The current helper's composer logic also drifted from the current ChatGPT DOM. In the current UI, the visible composer is a `contenteditable` element while a separate textarea may coexist. The helper can fill the wrong semantic surface or fail to observe the send control.
+5. Direct self-healing through fresh DOM observation could send ordinary text turns, but media-turn behavior was still inconsistent.
+
+### Structured-output reliability
+
+Result: **fail** for backend-grade use.
+
+A Markcut backend needs deterministic success/failure signaling and complete output. A truncated JSON prefix that is misclassified as complete is unsafe because downstream parsing or cache persistence can accept corrupted evidence.
+
+## Video experiment
+
+Direct video upload was attempted as the preferred path. The experiment reached a second failure mode before a trustworthy model result could be obtained: after opening the `markcut` Project for a fresh direct-video test, the current helper/UI combination failed to observe the expected composer surface consistently.
+
+Because image media submission was already non-deterministic and the direct-video setup could not produce a verified, repeatable turn, the experiment did **not** treat a frame-sampling fallback as proof that direct video upload works.
+
+A frame-sampling fallback remains technically possible because Markcut already owns ffmpeg normalization/sampling and can present sampled images to a backend. But the browser-backed image transport itself must first become reliable.
+
+```mermaid
+flowchart TD
+    V[video input] --> D{direct web video upload reliable?}
+    D -->|current result: no| F[Markcut frame sampling]
+    F --> I[image batches]
+    I --> B[ChatGPT browser backend]
+    B --> Q{media turn + result verified?}
+    Q -->|current result: no| X[do not enable backend]
+    Q -->|future: yes| O[chronology / scene JSON]
+```
+
+## Failure modes observed
+
+| Area | Observation | Impact |
+| --- | --- | --- |
+| Composer detection | Current UI exposes both contenteditable and textarea-like surfaces | helper may target the wrong composer |
+| Send detection | helper could report no `Send prompt` even when manual re-observation later found a usable control | brittle create/continue lifecycle |
+| Media upload | attachment preview can exist without a reliable submitted turn | no durable proof input reached model |
+| Generating detection | partial result was considered final | corrupted JSON may be accepted |
+| Result verification | stable DOM text alone is insufficient when streaming state is misdetected | false completion |
+| Project page lifecycle | new Project chat can change composer state/URL after first send | helper assumptions stale |
+| Video | direct video path was not repeatably driven to a verified result | no production evidence |
+
+## Minimum changes required before re-benchmarking
+
+1. **Fix semantic composer selection**
+   - prefer the visible `contenteditable` composer by accessibility role/name;
+   - reject ambiguous multiple composer surfaces;
+   - verify typed text appears in the same visible surface before sending.
+
+2. **Fix send verification**
+   - re-observe the current send control after typing/upload;
+   - after click, require a new user-turn node or other durable conversation evidence;
+   - do not return `prompt_sent: true` from a click alone.
+
+3. **Fix streaming/final-result verification**
+   - do not infer completion only from absence of one historical stop selector;
+   - require the assistant message text to remain stable across a longer interval and require the conversation to return to an input-ready state;
+   - for structured output, parse the requested JSON before returning success.
+
+4. **Add media-specific lifecycle states**
+   - upload started
+   - attachment observed
+   - attachment ready
+   - turn submitted
+   - assistant started
+   - assistant complete
+   - output validated
+
+5. **Add regression fixtures**
+   - one small PNG
+   - one short MP4
+   - exact expected JSON fields
+   - timeout/stuck-generation case
+   - truncated-JSON case
+
+6. **Only then add a Markcut adapter command**
+   - keep it experimental and opt-in;
+   - preserve the existing ITT/VTT CLI template boundary;
+   - do not change the default backend.
+
+## Proposed future adapter shape
+
+If the browser worker is repaired, the narrowest Markcut integration remains a command adapter rather than changes throughout the Vision pipeline:
+
+```text
+markcut vision
+  -> MARKCUT_ITT_CLI / MARKCUT_VTT_CLI
+  -> chatgpt-browser-vision executable
+  -> chatgpt-browser-worker typed lifecycle
+  -> browser-harness
+  -> verified JSON stdout
+```
+
+The adapter should be responsible for media upload, schema validation, retry/resume of the same ChatGPT thread, and non-zero exit on unverified/truncated output.
+
+## Recommendation
+
+**Not currently viable.**
+
+The architectural seam is good and ChatGPT web Vision is reachable through the intended browser-harness stack, but the current `chatgpt-browser-worker` implementation does not yet provide backend-grade guarantees for media submission or final-result verification. The existing Markcut local Vision backend remains substantially more deterministic for this workflow.
+
+Do not switch Markcut's default Vision backend. Repair and regression-test the browser worker first, then rerun this PR's image and video fixtures. Once image media turns are deterministic, evaluate video in this order:
+
+1. direct MP4 upload;
+2. Markcut-owned frame sampling if direct upload is unsupported or unstable;
+3. compare output quality, JSON validity, p50/p95 latency, and retry rate against the existing backend.
+
+## Commands/evidence
+
+Control image command used:
+
+```sh
+uvx --from mlx-vlm mlx_vlm.generate \
+  --model mlx-community/MiniCPM-V-4.6-bf16 \
+  --max-tokens 512 \
+  --prompt 'Describe this image precisely, including colored shapes and visible text. Return concise JSON.' \
+  --image .tmp/pr3-vision/image-test-unique.png \
+  --temperature 0.0 \
+  --thinking-mode disabled
+```
+
+Browser path used the `chatgpt-browser-worker` skill plus `browser-harness`; no ChatGPT API call, copied cookie, or MacDeveloperBridge ChatGPT runtime was used.
+
+## Re-benchmark after browser-worker attachment and tab-lifecycle fixes
+
+A second pass was run after Neo main added verified attachment submission (`defe80d`) and operation-owned tab cleanup (`3b4331e`). This changed the result in two useful ways:
+
+- attachment submission itself is now materially safer: a prior smoke run observed the requested filename, prompt text, a new durable user turn, and a real user message ID before reporting success;
+- `result --expect-json` now rejects malformed/truncated assistant output instead of treating it as completed. A deliberately observed truncated response failed with `assistant result is not valid JSON`, which closes the false-completion path seen in the first pass;
+- operation-owned tab cleanup was verified by comparing page targets before and after an operation that opened a temporary thread tab; the target sets were identical after return.
+
+The re-benchmark also exposed a new blocking failure at the durable-thread boundary. Threads that are visible and usable when reached through the existing ChatGPT sidebar can load as a shell-only page when opened directly by their durable conversation URL in a fresh tab. In the observed failures, the URL and Project sidebar loaded, but the conversation had **0 visible user messages, 0 visible assistant messages, and 0 composer surfaces**. `operate_bh.py` therefore correctly failed with `ChatGPT composer was not observed` rather than sending into an ambiguous page.
+
+This matters because the current worker contract reopens a durable thread by URL for `send`, `status`, and `result`. A Vision backend cannot be considered reliable if durable thread recovery depends on whether the ChatGPT SPA was entered through sidebar navigation versus direct URL navigation.
+
+### Updated failure boundary
+
+```mermaid
+flowchart TD
+    A[Markcut media input] --> B[chatgpt-browser-worker send]
+    B --> C{attachment + durable user turn verified?}
+    C -->|yes| D[assistant generation]
+    C -->|no| X[fail safely]
+    D --> E{result JSON valid?}
+    E -->|no| X
+    E -->|yes| F[backend result]
+    R[resume/reopen thread by durable URL] --> G{conversation UI + composer restored?}
+    G -->|current rerun: not reliably| X
+    G -->|future| B
+```
+
+### Updated recommendation
+
+The browser-worker changes fix two important defects from the first pass—attachment-submit evidence and structured-result validation—but PR #3 should still remain research-only. The remaining blocker is now narrower and clearer: **reliable durable-thread reopening / SPA recovery**. Until the worker can reopen the exact `thread_id` and observe the same conversation state independent of navigation path, direct image/video benchmarking cannot be considered repeatable enough for a Markcut backend.
+
+The next browser-worker fix should therefore target thread reopening semantics: prefer semantic sidebar navigation to the exact observed thread when direct URL hydration yields an empty shell, verify the requested `thread_id` after navigation, and only then expose the composer or result operations. After that, rerun the same PNG and MP4 fixtures without changing Markcut itself.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lalalic/markcut",
-  "version": "3.2.0",
+  "version": "3.2.2",
   "description": "Markdown-to-video engine. Describe scenes in markdown, get a rendered video.",
   "bin": {
     "markcut": "bin/markcut"

package/src/render/cli.mjs

@@ -81,6 +81,11 @@ Commands:
     --label                            Add interactive labeling step before AI pipeline
     --instruct "text"                   Background context about people/places (injected into prompts)
 
+  vision-chatgpt                       Browser ChatGPT ITT/VTT facade
+    --mode image|video|auto            Media mode (default: auto)
+    --prompt "text"                    Vision instructions
+    --input <path...>                  Local media input(s)
+
   spots --waypoints "lat,lng;..."      Discover POIs along a route (Directions + Places API)
     --travelMode DRIVING               DRIVING | WALKING | BICYCLING (default DRIVING)
     --limit 8                          Max spots after ranking
@@ -223,6 +228,11 @@ edit=${DEFAULT_EDIT_CLI}`);
     process.exit(0);
   }
 
+  if (args.command === "vision-chatgpt") {
+    const { main: visionChatGptMain } = await import("../vision/chatgpt-browser-cli.mjs");
+    process.exit(visionChatGptMain(process.argv.slice(3)));
+  }
+
   if (args.command === "spots") {
     await import("../spots/cli.mjs");  // self-executing top-level script
     process.exit(0);

package/src/vision/candidate.mjs

@@ -0,0 +1,408 @@
+import { createHash } from "node:crypto";
+import {
+  existsSync, mkdirSync, readFileSync, statSync, writeFileSync,
+} from "node:fs";
+import { basename, dirname, extname, join, resolve } from "node:path";
+import { z } from "zod";
+import { execSync } from "node:child_process";
+import { VIDEO_EXTS, DEFAULT_VTT_SAMPLE_INTERVAL } from "../config.mjs";
+
+export const CANDIDATE_CONTRACT_VERSION = 1;
+
+const evidence = z.object({
+  start: z.number().finite().min(0),
+  end: z.number().finite().gt(0),
+  observation: z.string().min(1),
+  evidence: z.string().min(1),
+}).refine((item) => item.end > item.start, { message: "evidence end must exceed start" });
+
+const evidenceItem = z.object({
+  start: z.number().finite().min(0),
+  end: z.number().finite().gt(0),
+  label: z.string().min(1),
+  observation: z.string().min(1),
+  evidence: z.string().min(1),
+}).refine((item) => item.end > item.start, { message: "timestamped item end must exceed start" });
+
+const observationsSchema = z.object({
+  summary: z.string().min(1),
+  speech: z.object({
+    supplied: z.boolean(),
+    speakers: z.array(z.string()),
+    keyQuoteAlignment: z.array(z.object({
+      quote: z.string().min(1),
+      start: z.number().finite().min(0),
+      end: z.number().finite().gt(0),
+      alignment: z.string().min(1),
+      evidence: z.string().min(1),
+    }).refine((item) => item.end > item.start)),
+  }),
+  people: z.array(z.object({
+    label: z.string().min(1),
+    position: z.string().min(1),
+    faceVisible: z.enum(["yes", "partial", "no", "unclear"]),
+    clarity: z.enum(["high", "medium", "low", "unclear"]),
+    emotion: z.string(),
+    reaction: z.string(),
+    action: z.string(),
+    evidence: z.string().min(1),
+  })),
+  scenes: z.array(z.object({
+    start: z.number().finite().min(0),
+    end: z.number().finite().gt(0),
+    event: z.string().min(1),
+    onScreenText: z.array(z.string()),
+    shotChange: z.boolean(),
+    visualQuality: z.enum(["high", "acceptable", "low", "unclear"]),
+    audioQuality: z.enum(["high", "acceptable", "low", "unclear"]),
+    evidence: z.string().min(1),
+  }).refine((item) => item.end > item.start)),
+  narrative: z.object({
+    setup: evidence.nullable(),
+    tension: evidence.nullable(),
+    turn: evidence.nullable(),
+    payoff: evidence.nullable(),
+    selfContained: z.boolean(),
+    contextNeeded: z.string(),
+  }),
+  hooks: z.array(evidenceItem),
+  emotionalHighlights: z.array(evidenceItem),
+  weakRegions: z.array(evidenceItem),
+  suggestedCuts: z.array(z.object({
+    start: z.number().finite().min(0),
+    end: z.number().finite().gt(0),
+    rationale: z.string().min(1),
+    evidence: z.string().min(1),
+  }).refine((item) => item.end > item.start)),
+  verticalFit: z.object({
+    suitability: z.enum(["good", "acceptable", "poor", "unclear"]),
+    cropFeasibility: z.enum(["good", "acceptable", "poor", "unclear"]),
+    trackingFeasibility: z.enum(["good", "acceptable", "poor", "unclear"]),
+    subjectSafety: z.string().min(1),
+    evidence: z.string().min(1),
+  }),
+  editSuggestions: z.array(z.object({
+    type: z.enum(["punch-in", "b-roll", "caption-emphasis"]),
+    start: z.number().finite().min(0),
+    end: z.number().finite().gt(0),
+    suggestion: z.string().min(1),
+    evidence: z.string().min(1),
+  }).refine((item) => item.end > item.start)),
+  observableEvidence: z.array(z.object({
+    start: z.number().finite().min(0),
+    end: z.number().finite().gt(0),
+    observation: z.string().min(1),
+    evidence: z.string().min(1),
+  }).refine((item) => item.end > item.start)),
+  uncertainty: z.array(z.object({
+    claim: z.string().min(1),
+    reason: z.string().min(1),
+  })).min(1),
+}).strict();
+
+function emitInfo(message) { console.error(message); }
+
+function shQuote(value) {
+  return `'${String(value).replace(/'/g, "'\\''")}'`;
+}
+
+function run(command, options = {}) {
+  return execSync(command, {
+    encoding: "utf-8",
+    stdio: ["pipe", "pipe", "pipe"],
+    timeout: 300_000,
+    ...options,
+  });
+}
+
+function fingerprint(path) {
+  try {
+    const stats = statSync(path);
+    return `${stats.mtimeMs}:${stats.size}`;
+  } catch {
+    return "0:0";
+  }
+}
+
+function durationOf(path) {
+  const output = run(`ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 ${shQuote(path)}`, { timeout: 30_000 });
+  return Number.parseFloat(output.trim()) || 0;
+}
+
+function numericBound(value) {
+  const parsed = typeof value === "number"
+    ? value
+    : typeof value === "string" && value.trim() !== "" ? Number(value) : Number.NaN;
+  return Number.isFinite(parsed) ? parsed : Number.NaN;
+}
+
+function normalizeWhitespace(value) {
+  return String(value).replace(/\s+/g, " ").trim();
+}
+
+function strictJsonParse(raw) {
+  const text = String(raw ?? "").trim();
+  const fenceMatch = text.match(/^```(?:json)?\s*\n([\s\S]*?)\n?```\s*$/i);
+  const candidate = fenceMatch ? fenceMatch[1] : text;
+  if (!/^\{/.test(candidate) || !/\}$/.test(candidate)) {
+    throw new Error("model response is not a single JSON object");
+  }
+  return JSON.parse(candidate);
+}
+
+function normalizeTranscript(input, candidateDuration) {
+  if (!input) {
+    return { supplied: false, text: null };
+  }
+  const text = readFileSync(input, "utf-8").trim();
+  const cuePattern = /((?:\d{2}:)?\d{2}:\d{2}[.,]\d{3})\s*-->\s*((?:\d{2}:)?\d{2}:\d{2}[.,]\d{3})\s*\n([\s\S]*?)(?=\n\n|\n(?:\d{2}:)?\d{2}:\d{2}|$)/g;
+  const cues = [];
+  let match;
+  while ((match = cuePattern.exec(text)) !== null) {
+    const toSeconds = (timestamp) => {
+      const parts = timestamp.replace(",", ".").split(":").map(Number);
+      return parts.length === 3 ? parts[0] * 3600 + parts[1] * 60 + parts[2] : parts[0] * 60 + parts[1];
+    };
+    cues.push({ start: toSeconds(match[1]), end: toSeconds(match[2]), text: normalizeWhitespace(match[3]) });
+  }
+  if (cues.length > 0) {
+    return {
+      supplied: true,
+      text: cues.map((cue) => `[${cue.start.toFixed(3)}-${cue.end.toFixed(3)}] ${cue.text}`).join("\n"),
+      cues,
+    };
+  }
+  return { supplied: true, text: normalizeWhitespace(text).slice(0, 20_000), cues: [] };
+}
+
+function getPrompt(prompts, name, fallback) {
+  const custom = prompts?.get?.(name);
+  if (custom) return custom;
+  if (prompts?.size === 0) return fallback;
+  return fallback;
+}
+
+function buildPrompt(candidate, transcript, context, domainPrompt) {
+  return `${domainPrompt}
+
+TRANSCRIPT CONTEXT: ${transcript.supplied ? transcript.text : "(none supplied; do not invent speech or quote alignment)"}
+CONTEXT: ${context || "(none)"}
+SOURCE IDENTITY: ${candidate.source.id} (${candidate.source.start}s-${candidate.source.end}s)
+CANDIDATE IDENTITY: ${candidate.candidate.id}; candidate timeline 0-${candidate.candidate.duration}s
+
+STABLE JSON CONTRACT
+Return exactly one UTF-8 JSON object and no prose outside optional JSON code fences. Every timestamp is candidate-relative seconds. Every editorial observation must cite observable evidence. If perception cannot support a claim, put it in uncertainty. Do not create scores, rankings, viral ratings, or selection decisions. Do not invent speech; keyQuoteAlignment may be non-empty only when transcript text is supplied above and each quote must be an exact transcript substring.
+Required top-level keys and shapes:
+{"summary":string,"speech":{"supplied":boolean,"speakers":string[],"keyQuoteAlignment":[{"quote":string,"start":number,"end":number,"alignment":string,"evidence":string}]},"people":[{"label":string,"position":string,"faceVisible":"yes|partial|no|unclear","clarity":"high|medium|low|unclear","emotion":string,"reaction":string,"action":string,"evidence":string}],"scenes":[{"start":number,"end":number,"event":string,"onScreenText":string[],"shotChange":boolean,"visualQuality":"high|acceptable|low|unclear","audioQuality":"high|acceptable|low|unclear","evidence":string}],"narrative":{"setup":{"start":number,"end":number,"observation":string,"evidence":string}|null,"tension":same|null,"turn":same|null,"payoff":same|null,"selfContained":boolean,"contextNeeded":string},"hooks":[{"start":number,"end":number,"label":string,"observation":string,"evidence":string}],"emotionalHighlights":[hook],"weakRegions":[hook],"suggestedCuts":[{"start":number,"end":number,"rationale":string,"evidence":string}],"verticalFit":{"suitability":"good|acceptable|poor|unclear","cropFeasibility":"good|acceptable|poor|unclear","trackingFeasibility":"good|acceptable|poor|unclear","subjectSafety":string,"evidence":string},"editSuggestions":[{"type":"punch-in|b-roll|caption-emphasis","start":number,"end":number,"suggestion":string,"evidence":string}],"observableEvidence":[{"start":number,"end":number,"observation":string,"evidence":string}],"uncertainty":[{"claim":string,"reason":string}]}`;
+}
+
+function validateAgainstTranscript(parsed, transcript) {
+  if (!transcript.supplied && parsed.speech.keyQuoteAlignment.length > 0) {
+    throw new Error("structured evidence contains quote alignment without transcript");
+  }
+  const transcriptText = normalizeWhitespace(transcript.text || "").toLowerCase();
+  for (const alignment of parsed.speech.keyQuoteAlignment) {
+    const quote = normalizeWhitespace(alignment.quote).toLowerCase();
+    if (!transcriptText.includes(quote)) {
+      throw new Error("key quote alignment cites text absent from caller-supplied transcript");
+    }
+  }
+  return parsed;
+}
+
+function rejectSelectionSignals(value, path = "$") {
+  if (Array.isArray(value)) {
+    value.forEach((item, index) => rejectSelectionSignals(item, `${path}[${index}]`));
+    return;
+  }
+  if (!value || typeof value !== "object") return;
+  for (const [key, child] of Object.entries(value)) {
+    if (/^(viral[_-]?)?(score|rating|rank|ranking|selection)$/i.test(key) || /viral/i.test(key)) {
+      throw new Error(`structured evidence contains forbidden editorial decision field: ${path}.${key}`);
+    }
+    rejectSelectionSignals(child, `${path}.${key}`);
+  }
+}
+
+function cacheKey(candidate, transcript, context, prompt, modelCommand, contract) {
+  const parts = {
+    version: 2,
+    contract,
+    source: { id: candidate.source.id, fingerprint: fingerprint(candidate.source.path), bounds: [candidate.source.start, candidate.source.end] },
+    candidate: { id: candidate.candidate.id, fingerprint: fingerprint(candidate.candidate.path), bounds: [candidate.candidate.start, candidate.candidate.end] },
+    transcript: transcript.supplied ? {
+      supplied: true,
+      contentHash: createHash("sha256").update(transcript.text || "").digest("hex"),
+    } : { supplied: false },
+    context,
+    prompt,
+    modelCommand,
+  };
+  return createHash("sha256").update(JSON.stringify(parts)).digest("hex");
+}
+
+function prepareCandidate(candidate, workingDir) {
+  const normalizedDir = join(workingDir, ".markcut-candidate-vision");
+  mkdirSync(normalizedDir, { recursive: true });
+  let mediaPath = candidate.candidate.path;
+  if (candidate.candidate.needsSlice || !existsSync(mediaPath)) {
+    mediaPath = join(normalizedDir, `${candidate.candidate.id}.mp4`);
+    const duration = candidate.source.end - candidate.source.start;
+    run(`ffmpeg -y -ss ${candidate.source.start} -i ${shQuote(candidate.source.path)} -t ${duration} -c:v libx264 -preset fast -crf 26 -c:a aac -b:a 96k ${shQuote(mediaPath)}`, { timeout: 300_000 });
+  }
+  const actualDuration = durationOf(mediaPath);
+  if (actualDuration <= 0) throw new Error(`candidate media has no measurable duration: ${mediaPath}`);
+  const normalizedPath = join(normalizedDir, `${candidate.candidate.id}_sample.mp4`);
+  if (!existsSync(normalizedPath)) {
+    run(`ffmpeg -y -i ${shQuote(mediaPath)} -vf "fps=1,scale='min(360,iw)':'min(360,ih)':force_original_aspect_ratio=decrease,pad='ceil(iw/2)*2':'ceil(ih/2)*2':-1:-1" -an -c:v libx264 -preset fast -crf 28 ${shQuote(normalizedPath)}`, { timeout: 300_000 });
+  }
+  return {
+    ...candidate,
+    candidate: {
+      ...candidate.candidate,
+      path: mediaPath,
+      duration: actualDuration,
+      normalizedMediaPath: normalizedPath,
+      isTrimmedCandidate: true,
+    },
+  };
+}
+
+function artifact(candidate, parsed, transcript, modelCommandHash) {
+  return {
+    schemaVersion: 1,
+    type: "markcut.candidate-evidence",
+    contractVersion: CANDIDATE_CONTRACT_VERSION,
+    source: candidate.source,
+    candidate: candidate.candidate,
+    transcript: {
+      supplied: transcript.supplied,
+      text: transcript.text,
+      speakers: parsed.speech.speakers,
+      keyQuoteAlignment: parsed.speech.keyQuoteAlignment,
+    },
+    observations: parsed,
+    model: { commandHash: modelCommandHash },
+  };
+}
+
+export async function runCandidateVision(inputPath, options) {
+  const resolved = resolve(inputPath);
+  if (!existsSync(resolved)) throw new Error(`input not found: ${resolved}`);
+  const extension = extname(resolved).toLowerCase();
+  let manifest;
+  if (VIDEO_EXTS.has(extension)) {
+    const duration = durationOf(resolved);
+    const start = Number.isFinite(options.start) ? options.start : 0;
+    const end = Number.isFinite(options.end) ? options.end : duration;
+    if (end <= start) throw new Error("candidate end must exceed start");
+    manifest = {
+      source: { id: options.sourceId || basename(resolved, extension), path: resolved, start, end },
+      candidate: {
+        id: options.candidateId || `${basename(resolved, extension)}-${start}-${end}`,
+        path: resolved,
+        start,
+        end,
+        needsSlice: start > 0 || end < duration,
+      },
+    };
+  } else {
+    const input = JSON.parse(readFileSync(resolved, "utf-8"));
+    const sourceValue = input.source;
+    const candidateValue = input.candidate;
+    const sourcePath = resolve(dirname(resolved), sourceValue?.path || input.sourcePath || (typeof sourceValue === "string" ? sourceValue : ""));
+    const candidatePath = typeof candidateValue === "string"
+      ? ""
+      : candidateValue?.path || input.candidatePath || "";
+    const candidateId = candidateValue?.id || input.candidateId || (typeof candidateValue === "string" ? candidateValue : "");
+    const start = numericBound(sourceValue?.start ?? input.start ?? input.sourceStartSec);
+    const end = numericBound(sourceValue?.end ?? input.end ?? input.sourceEndSec);
+    const candidateStart = numericBound(candidateValue?.start ?? input.candidateStartSec);
+    const candidateEnd = numericBound(candidateValue?.end ?? input.candidateEndSec);
+    if (!sourcePath || !Number.isFinite(start) || !Number.isFinite(end) || end <= start) {
+      throw new Error("candidate manifest requires source path and numeric start/end bounds");
+    }
+    if (candidatePath && (!Number.isFinite(candidateStart) || !Number.isFinite(candidateEnd) || candidateEnd <= candidateStart)) {
+      throw new Error("candidate path manifests require complete numeric candidate bounds; zero is valid for candidateStartSec");
+    }
+    manifest = {
+      source: { id: input.source?.id || input.sourceId || basename(sourcePath), path: sourcePath, start, end },
+      candidate: {
+        id: candidateId || input.candidateId || `${input.sourceId || basename(sourcePath)}-${start}-${end}`,
+        path: candidatePath ? resolve(dirname(resolved), candidatePath) : sourcePath,
+        start: candidatePath ? candidateStart : start,
+        end: candidatePath ? candidateEnd : end,
+        needsSlice: !candidatePath,
+      },
+    };
+  }
+
+  if (!Number.isFinite(manifest.candidate.start)) manifest.candidate.start = manifest.source.start;
+  if (!Number.isFinite(manifest.candidate.end)) manifest.candidate.end = manifest.source.end;
+  manifest.candidate.isTrimmedCandidate = !manifest.candidate.needsSlice;
+
+  const prepared = prepareCandidate(
+    { ...manifest, candidate: { ...manifest.candidate, path: manifest.candidate.path || manifest.source.path } },
+    dirname(resolved),
+  );
+  const transcript = normalizeTranscript(options.transcriptFile ? resolve(options.transcriptFile) : null);
+  const domainPrompt = getPrompt(options.prompts, "candidate-evidence", `Produce reusable, timestamped visual/editorial evidence for an already-selected short-form candidate clip.`);
+  const prompt = buildPrompt(prepared, transcript, options.context || "", domainPrompt);
+  const contract = `candidate-evidence-v${CANDIDATE_CONTRACT_VERSION}`;
+  const modelCommand = options.modelCommand || process.env.MARKCUT_VISION_CANDIDATE_CLI || process.env.MARKCUT_VTT_CLI || "";
+  if (!modelCommand) throw new Error("candidate vision requires --model-command or MARKCUT_VISION_CANDIDATE_CLI");
+  const key = cacheKey(prepared, transcript, options.context || "", prompt, modelCommand, contract);
+  const commandHash = createHash("sha256").update(modelCommand).digest("hex").slice(0, 16);
+  const cachePath = join(dirname(resolved), ".markcut-candidate-vision", `${key}.json`);
+  const outputPath = resolve(options.output || join(dirname(resolved), `${prepared.candidate.id}.candidate-vision.json`));
+
+  let artifactValue;
+  if (existsSync(cachePath)) {
+    emitInfo(`Candidate evidence cached: ${key.slice(0, 12)}`);
+    artifactValue = JSON.parse(readFileSync(cachePath, "utf-8"));
+  } else {
+    const substituted = modelCommand
+      .replace(/\{input\}/g, shQuote(prepared.candidate.normalizedMediaPath))
+      .replace(/\{prompt\}/g, shQuote(prompt));
+    emitInfo(`Analyzing candidate ${prepared.candidate.id}...`);
+    let raw;
+    try {
+      raw = run(substituted, { timeout: 600_000 }).trim();
+    } catch (error) {
+      throw new Error(`candidate vision model failed: ${error.message}`);
+    }
+    let parsed;
+    try {
+      parsed = strictJsonParse(raw);
+    } catch (error) {
+      throw new Error(`candidate vision returned invalid JSON: ${error.message}`);
+    }
+    const shapeResult = observationsSchema.safeParse(parsed);
+    if (!shapeResult.success) {
+      throw new Error(`candidate vision returned invalid structured evidence: ${shapeResult.error.issues.map((issue) => `${issue.path.join(".")}: ${issue.message}`).join("; ")}`);
+    }
+    parsed = shapeResult.data;
+    if (parsed.speech.supplied !== transcript.supplied) {
+      throw new Error("structured evidence transcript-supplied flag contradicts caller input");
+    }
+    rejectSelectionSignals(parsed);
+    validateAgainstTranscript(parsed, transcript);
+    artifactValue = artifact(prepared, parsed, transcript, commandHash);
+    writeFileSync(cachePath, JSON.stringify(artifactValue, null, 2));
+  }
+
+  mkdirSync(dirname(outputPath), { recursive: true });
+  writeFileSync(outputPath, JSON.stringify(artifactValue, null, 2));
+  emitInfo(`Candidate evidence written: ${outputPath}`);
+  return outputPath;
+}
+
+export {
+  buildPrompt,
+  cacheKey,
+  normalizeTranscript,
+  observationsSchema,
+  rejectSelectionSignals,
+  strictJsonParse,
+  validateAgainstTranscript,
+};

package/src/vision/chatgpt-browser-cli.mjs

@@ -0,0 +1,122 @@
+#!/usr/bin/env node
+import { execFileSync, execSync } from "node:child_process";
+import { randomUUID } from "node:crypto";
+import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { basename, extname, join, resolve } from "node:path";
+
+const IMAGE_EXTS = new Set([".jpg", ".jpeg", ".png", ".webp", ".gif", ".bmp", ".tiff", ".heic", ".avif"]);
+const VIDEO_EXTS = new Set([".mp4", ".mov", ".avi", ".mkv", ".webm", ".m4v", ".wmv"]);
+
+function shellQuote(value) { return `'${String(value).replace(/'/g, `'\\''`)}'`; }
+
+export function parseArgs(argv) {
+  const out = { mode: "auto", inputs: [], prompt: "", timeoutMs: Number(process.env.MARKCUT_CHATGPT_VISION_TIMEOUT_MS) || 600_000, maxFrames: 8 };
+  for (let i = 0; i < argv.length; i++) {
+    const arg = argv[i];
+    if (arg === "--mode") out.mode = argv[++i] || "auto";
+    else if (arg === "--prompt") out.prompt = argv[++i] || "";
+    else if (arg === "--input") {
+      while (argv[i + 1] && !argv[i + 1].startsWith("--")) out.inputs.push(argv[++i]);
+    } else if (arg === "--timeout-ms") out.timeoutMs = Number(argv[++i]);
+    else if (arg === "--max-frames") out.maxFrames = Number(argv[++i]);
+    else if (!arg.startsWith("--")) out.inputs.push(arg);
+    else throw new Error(`Unknown argument: ${arg}`);
+  }
+  if (!out.prompt) throw new Error("--prompt is required");
+  if (out.inputs.length === 0) throw new Error("--input is required");
+  if (!Number.isFinite(out.timeoutMs) || out.timeoutMs <= 0) throw new Error("--timeout-ms must be > 0");
+  if (!Number.isFinite(out.maxFrames) || out.maxFrames < 1 || out.maxFrames > 16) throw new Error("--max-frames must be 1..16");
+  out.inputs = out.inputs.map((p) => resolve(p.replace(/^@/, "")));
+  for (const input of out.inputs) if (!existsSync(input)) throw new Error(`Input not found: ${input}`);
+  if (out.mode === "auto") out.mode = out.inputs.some((p) => VIDEO_EXTS.has(extname(p).toLowerCase())) ? "video" : "image";
+  if (!['image', 'video'].includes(out.mode)) throw new Error("--mode must be image, video, or auto");
+  return out;
+}
+
+function ffprobeDuration(videoPath) {
+  const raw = execFileSync("ffprobe", ["-v", "error", "-show_entries", "format=duration", "-of", "default=noprint_wrappers=1:nokey=1", videoPath], { encoding: "utf8" }).trim();
+  const value = Number(raw);
+  if (!Number.isFinite(value) || value <= 0) throw new Error(`Unable to read video duration: ${videoPath}`);
+  return value;
+}
+
+export function prepareMedia(mode, inputs, workDir, maxFrames = 8) {
+  if (mode === "image") return { files: inputs, context: inputs.map((p) => `Image: ${basename(p)}`).join("\n") };
+  if (inputs.length !== 1) throw new Error("video mode accepts exactly one input video");
+  const videoPath = inputs[0];
+  if (!VIDEO_EXTS.has(extname(videoPath).toLowerCase())) throw new Error(`Unsupported video input: ${videoPath}`);
+  const duration = ffprobeDuration(videoPath);
+  const count = Math.max(1, Math.min(maxFrames, Math.ceil(duration / 5)));
+  const framesDir = join(workDir, "frames");
+  mkdirSync(framesDir, { recursive: true });
+  const pattern = join(framesDir, "frame-%03d.jpg");
+  const fps = count / duration;
+  execFileSync("ffmpeg", ["-y", "-i", videoPath, "-vf", `fps=${fps},scale='min(640,iw)':-2`, "-frames:v", String(count), "-q:v", "3", pattern], { stdio: "ignore" });
+  const frames = readdirSync(framesDir).filter((f) => f.endsWith(".jpg")).sort().map((f) => join(framesDir, f));
+  if (frames.length === 0) throw new Error("No representative frames extracted from video");
+  const cols = Math.min(4, frames.length);
+  const contact = join(workDir, "contact-sheet.jpg");
+  if (frames.length === 1) {
+    execFileSync("ffmpeg", ["-y", "-i", frames[0], "-vf", "scale=320:-2", "-frames:v", "1", contact], { stdio: "ignore" });
+  } else {
+    execSync(`ffmpeg -y ${frames.map((p) => `-i ${shellQuote(p)}`).join(" ")} -filter_complex ${shellQuote(`xstack=inputs=${frames.length}:layout=${frames.map((_, i) => `${i % cols}*w0_${Math.floor(i / cols)}*h0`).join('|')},scale=${cols * 320}:-2`)} -frames:v 1 ${shellQuote(contact)}`, { stdio: "ignore" });
+  }
+  const timing = frames.map((p, i) => `${basename(p)} ≈ ${(i * duration / frames.length).toFixed(1)}s`).join(", ");
+  return { files: [contact], context: `Video: ${basename(videoPath)}\nDuration: ${duration.toFixed(2)}s\nRepresentative frames are chronological, left-to-right then top-to-bottom.\nTiming: ${timing}` };
+}
+
+function waitForResult(outputPath, timeoutMs) {
+  const deadline = Date.now() + timeoutMs;
+  while (Date.now() < deadline) {
+    if (existsSync(outputPath)) {
+      const text = readFileSync(outputPath, "utf8").trim();
+      if (text) return text;
+    }
+    Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 250);
+  }
+  throw new Error(`Timed out waiting for Browser ChatGPT result: ${outputPath}`);
+}
+
+export function runVision({ mode, inputs, prompt, timeoutMs, maxFrames }, env = process.env) {
+  const launcher = env.MARKCUT_CHATGPT_BROWSER_WORKER_AGENT_CLI || env.MARKCUT_CHATGPT_BROWSER_WORKER_CLI;
+  if (!launcher) {
+    throw new Error("MARKCUT_CHATGPT_BROWSER_WORKER_AGENT_CLI is required; it must launch the Neo browser-worker agent runtime");
+  }
+  const workDir = mkdtempSync(join(tmpdir(), "markcut-chatgpt-vision-"));
+  const outputPath = join(workDir, "result.txt");
+  try {
+    const media = prepareMedia(mode, inputs, workDir, maxFrames);
+    const jobId = `markcut-vision-${randomUUID()}`;
+    const taskId = `browser-vision-${randomUUID()}`;
+    const fullPrompt = `${prompt}\n\nMedia context:\n${media.context}\n\nAnalyze only the attached media. Preserve chronology for video. Write only the final answer to the declared file output.\n\nExecution event contract:\n- Job: ${jobId}\n- Task: ${taskId}\n- Publish exactly one worker-owned task.started before analysis.\n- Publish exactly one worker-owned task.completed only after the output file is durable; publish task.failed instead if execution cannot complete.\n- task.process.* events are carrier lifecycle only and never substitute for task lifecycle.\n\nOutput:\nfile\n${outputPath}`;
+    const promptPath = join(workDir, "prompt.txt");
+    writeFileSync(promptPath, fullPrompt, "utf8");
+    const childEnv = {
+      ...env,
+      MARKCUT_CHATGPT_PROMPT_FILE: promptPath,
+      MARKCUT_CHATGPT_MEDIA_FILES_JSON: JSON.stringify(media.files),
+      MARKCUT_CHATGPT_OUTPUT_FILE: outputPath,
+      MARKCUT_CHATGPT_TAB_CLOSE_POLICY: "after-terminal",
+      NEO_JOB_ID: jobId,
+      NEO_TASK_ID: taskId,
+    };
+    execSync(launcher, { env: childEnv, stdio: ["ignore", "pipe", "pipe"], timeout: timeoutMs });
+    return waitForResult(outputPath, timeoutMs);
+  } finally {
+    rmSync(workDir, { recursive: true, force: true });
+  }
+}
+
+export function main(argv = process.argv.slice(2), env = process.env) {
+  try {
+    const result = runVision(parseArgs(argv), env);
+    process.stdout.write(`${result}\n`);
+    return 0;
+  } catch (error) {
+    process.stderr.write(`markcut chatgpt vision: ${error instanceof Error ? error.message : String(error)}\n`);
+    return 1;
+  }
+}
+
+if (process.argv[1] && process.argv[1].endsWith("chatgpt-browser-cli.mjs")) process.exitCode = main();

package/src/vision/cli.mjs

@@ -32,6 +32,8 @@
  *
  * Prompt overrides:
  *   --<prompt-name> "text"  Override any prompt template from vision_prompts.md
+ *   Use `--candidate-evidence "..."` in candidate mode to replace only the domain
+ *   instructions; the machine-readable JSON envelope remains fixed.
  */
 
 import { execSync, spawn } from "node:child_process";
@@ -46,6 +48,7 @@ import {
   MAX_IMAGE_DIMENSION, MAX_VIDEO_DURATION, MAX_VIDEO_DIMENSION,
   DEFAULT_ITT_CLI, DEFAULT_VTT_SAMPLE_INTERVAL, DEFAULT_VTT_CLI, DEFAULT_STT_CLI, DEFAULT_AGENT_CLI,
 } from "../config.mjs";
+import { runCandidateVision } from "./candidate.mjs";
 
 // ── Paths ─────────────────────────────────────────────────────────────────
 
@@ -788,7 +791,7 @@ function buildMergedCues(userHint, cues, sceneChangesMs, totalDurationMs) {
   return merged;
 }
 
-function analyzeVideo(videoPath, normInfo, normDir, prompts, context = "", userHint = "", sampleInterval = DEFAULT_VTT_SAMPLE_INTERVAL, userHints = null) {
+function analyzeVideo(videoPath, normInfo, normDir, prompts, context = "", userHint = "", sampleInterval = DEFAULT_VTT_SAMPLE_INTERVAL, userHints = null, skipSTT = false) {
   let ctxParts = [];
   if (context) ctxParts.push(`Context: ${context}`);
   if (userHint && typeof userHint === "string") ctxParts.push(`User hint: ${userHint}`);
@@ -803,8 +806,13 @@ function analyzeVideo(videoPath, normInfo, normDir, prompts, context = "", userH
   perception.desc = descText.slice(0, 500) || looseJSONParse(descRaw)?.desc || descRaw.slice(0, 500);
 
   // 2. STT → VTT subtitle
-  emitInfo(`  Running speech-to-text...`);
-  perception.subtitle = runSTT(videoPath, normDir, DEFAULT_STT_CLI);
+  if (skipSTT) {
+    emitInfo(`  Skipping speech-to-text...`);
+    perception.subtitle = null;
+  } else {
+    emitInfo(`  Running speech-to-text...`);
+    perception.subtitle = runSTT(videoPath, normDir, DEFAULT_STT_CLI);
+  }
 
   // 3. Build merged cue timeline from VTT + user hints + ffprobe
   emitInfo(`  Building merged segment boundaries...`);
@@ -1073,7 +1081,7 @@ async function runNormalizeAndPercept(folder, metadataPath, prompts, context, pi
       perception = cache[cacheKey];
       emitInfo(`  (cached)`);
     } else {
-      perception = analyzeVideo(vidPath, normInfo, normDir, prompts, context, userHint, vttSampleInterval, userHints);
+      perception = analyzeVideo(vidPath, normInfo, normDir, prompts, context, userHint, vttSampleInterval, userHints, skipSTT);
       if (perception.desc) cache[cacheKey] = perception;
     }
 
@@ -1123,6 +1131,14 @@ export async function main(args) {
   let skipSTT = false;
   let dryRun = false;
   let doLabel = false;
+  let candidateMode = false;
+  let output = "";
+  let modelCommand = "";
+  let transcriptFile = "";
+  let candidateId = "";
+  let sourceId = "";
+  let start = Number.NaN;
+  let end = Number.NaN;
   const pickSet = new Set();
   const promptOverrides = new Map();
 
@@ -1134,10 +1150,18 @@ export async function main(args) {
     const flag = args[i++];
     if (flag === "--help") { printUsage(); return; }
     else if (flag === "--label") { doLabel = true; }
+    else if (flag === "--candidate") { candidateMode = true; }
+    else if (flag === "--output" && args[i]) { output = resolve(args[i++]); }
+    else if (flag === "--model-command" && args[i]) { modelCommand = args[i++]; }
+    else if (flag === "--transcript-file" && args[i]) { transcriptFile = resolve(args[i++]); }
+    else if (flag === "--candidate-id" && args[i]) { candidateId = args[i++]; }
+    else if (flag === "--source-id" && args[i]) { sourceId = args[i++]; }
+    else if (flag === "--start" && args[i]) { start = Number.parseFloat(args[i++]); }
+    else if (flag === "--end" && args[i]) { end = Number.parseFloat(args[i++]); }
 
     else if (flag === "--prompts-file" && args[i]) { promptsFile = resolve(args[i++]); }
     else if (flag === "--vtt-sample-interval" && args[i]) { vttSampleInterval = parseInt(args[i++], 10) || DEFAULT_VTT_SAMPLE_INTERVAL; }
-    else if (flag === "--instruct" && args[i]) { context = args[i++]; }
+    else if (flag === "--instruct" && args[i] || flag === "--context" && args[i]) { context = args[i++]; }
     else if (flag === "--show-prompts") { console.log(readFileSync(promptsFile, "utf-8")); return; }
     else if (flag === "--skip-stt") { skipSTT = true; }
     else if (flag === "--pick" && args[i]) { for (const f of args[i++].split(",")) pickSet.add(f.trim()); }
@@ -1152,6 +1176,25 @@ export async function main(args) {
   const prompts = loadPrompts(promptsFile);
   for (const [name, value] of promptOverrides) prompts.set(name, value);
 
+  if (candidateMode) {
+    if (statSync(folder).isFile() && !VIDEO_EXTS.has(extname(folder).toLowerCase()) && extname(folder).toLowerCase() !== ".json") {
+      emitError("Candidate mode accepts one video file or a JSON candidate manifest.");
+      process.exit(1);
+    }
+    await runCandidateVision(folder, {
+      context,
+      output,
+      modelCommand,
+      transcriptFile,
+      candidateId,
+      sourceId,
+      start,
+      end,
+      prompts,
+    });
+    return;
+  }
+
   if (doLabel) {
     // Full pipeline with interactive labeling
     await runFullPipeline(folder, prompts, context, pickSet, vttSampleInterval, skipSTT, dryRun);

package/src/vision/vision_prompts.md

@@ -64,4 +64,12 @@ Output format:
 {"0to5000": {"description": "Introduction and welcome"}, "5000to15250": {"description": "Starting to swim"}, "15250to30000": {"description": "Building sandcastle"}}
 
 If inputs are too sparse for segmentation, return an empty object {}.
-~~~
+~~~
+
+## candidate-evidence
+
+Observe one already-selected short-form candidate and return reusable editorial evidence.
+
+~~~md
+Observe this already-trimmed candidate clip for a downstream editorial judge. Describe only what is visually or aurally observable. Cover subjects, faces, emotion, actions, scenes, on-screen text, quality, narrative development, hooks, highlights, weak regions, suggested boundaries, vertical reframing, and edit opportunities. Distinguish observation from uncertainty. Do not score, rank, select, or reject the candidate.
+~~~

package/tests/vision-candidate.test.ts

@@ -0,0 +1,271 @@
+import { execSync } from "node:child_process";
+import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { basename, join, resolve } from "node:path";
+import { afterAll, beforeAll, describe, expect, it } from "vitest";
+
+process.env.MARKCUT_VTT_CLI = "printf 'whole video'";
+process.env.MARKCUT_AGENT_CLI = "printf '{}'";
+
+const { main } = await import("../src/vision/cli.mjs");
+const {
+  cacheKey,
+  normalizeTranscript,
+  validateAgainstTranscript,
+} = await import("../src/vision/candidate.mjs");
+
+const ROOT = resolve(__dirname, "tmp", `vision-candidate-${Date.now()}`);
+const SKIP_STT_ROOT = join(ROOT, "skip-stt");
+
+function quote(value: string): string {
+  return `'${value.replace(/'/g, "'\\''")}'`;
+}
+
+function createVideo(dir: string, name: string, duration = 3): string {
+  const path = join(dir, name);
+  execSync(
+    `ffmpeg -y -f lavfi -i testsrc=size=180x320:rate=10:duration=${duration} ` +
+    `-f lavfi -i sine=frequency=640:duration=${duration} -shortest ` +
+    `-c:v libx264 -preset ultrafast -crf 30 -pix_fmt yuv420p -c:a aac ${quote(path)}`,
+    { stdio: "pipe" },
+  );
+  return path;
+}
+
+function evidenceResponse(options: { uncertainty?: boolean; quote?: string } = {}): object {
+  return {
+    summary: "A short visible test moment",
+    speech: {
+      supplied: Boolean(options.quote),
+      speakers: options.quote ? ["Host"] : [],
+      keyQuoteAlignment: options.quote ? [{
+        quote: options.quote,
+        start: 0,
+        end: 1,
+        alignment: "The visible host is speaking during this cue.",
+        evidence: "0.0-1.0 transcript cue while the host faces camera",
+      }] : [],
+    },
+    people: [],
+    scenes: [{
+      start: 0,
+      end: 1,
+      event: "A stationary test pattern is visible",
+      onScreenText: [],
+      shotChange: false,
+      visualQuality: "acceptable",
+      audioQuality: "unclear",
+      evidence: "0.0-1.0 sampled frames show the pattern",
+    }],
+    narrative: {
+      setup: null,
+      tension: null,
+      turn: null,
+      payoff: null,
+      selfContained: true,
+      contextNeeded: "",
+    },
+    hooks: [],
+    emotionalHighlights: [],
+    weakRegions: [],
+    suggestedCuts: [],
+    verticalFit: {
+      suitability: "good",
+      cropFeasibility: "acceptable",
+      trackingFeasibility: "unclear",
+      subjectSafety: "The frame leaves room for a vertical crop",
+      evidence: "0.0-1.0 sampled frames",
+    },
+    editSuggestions: [],
+    observableEvidence: [{
+      start: 0,
+      end: 1,
+      observation: "The sample test pattern remains visible",
+      evidence: "sampled frames at 0.0 and 1.0 seconds",
+    }],
+    uncertainty: options.uncertainty === false ? [] : [{
+      claim: "Speech content cannot be verified from frames alone",
+      reason: "The candidate analysis path does not perform its own STT",
+    }],
+  };
+}
+
+function fixture(path: string, value: unknown): string {
+  writeFileSync(path, typeof value === "string" ? value : JSON.stringify(value));
+  return path;
+}
+
+describe("candidate-only vision", () => {
+  let candidate: string;
+  let validFixture: string;
+  let trimmedCandidate: string;
+
+  beforeAll(() => {
+    mkdirSync(ROOT, { recursive: true });
+    candidate = createVideo(ROOT, "candidate.mp4");
+    writeFileSync(
+      join(ROOT, "transcript.vtt"),
+      "WEBVTT\n\n00:00:00.000 --> 00:00:01.000\nThis is the best part\n",
+    );
+    validFixture = fixture(join(ROOT, "valid.json"), evidenceResponse({
+      quote: "This is the best part",
+    }));
+    trimmedCandidate = createVideo(ROOT, "trimmed.mp4", 2.5);
+  }, 30_000);
+
+  afterAll(() => {
+    rmSync(ROOT, { recursive: true, force: true });
+  });
+
+  it("analyzes only the candidate and emits bounded structured evidence", async () => {
+    const output = join(ROOT, "evidence.json");
+    const invocationCount = join(ROOT, "invocations.txt");
+    const countingModel = `printf 'invoked\\n' >> ${quote(invocationCount)}; cat ${quote(validFixture)}`;
+    await main([
+      "node", "cli.mjs", "vision", candidate, "--candidate",
+      "--source-id", "source-1", "--candidate-id", "candidate-9",
+      "--start", "0.5", "--end", "2.5",
+      "--transcript-file", join(ROOT, "transcript.vtt"),
+      "--model-command", countingModel,
+      "--output", output,
+    ]);
+
+    const artifact = JSON.parse(readFileSync(output, "utf-8"));
+    expect(artifact.type).toBe("markcut.candidate-evidence");
+    expect(artifact.source).toMatchObject({ id: "source-1", start: 0.5, end: 2.5 });
+    expect(artifact.candidate).toMatchObject({ id: "candidate-9", start: 0.5, end: 2.5 });
+    expect(artifact.observations).toHaveProperty("summary");
+    expect(artifact.observations.uncertainty.length).toBeGreaterThan(0);
+    expect(readFileSync(invocationCount, "utf-8").trim().split("\n")).toHaveLength(1);
+    expect(artifact.candidate.path).not.toBe(candidate);
+    const boundedDuration = Number(execSync(
+      `ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 ${quote(artifact.candidate.path)}`,
+      { encoding: "utf-8" },
+    ).trim());
+    expect(boundedDuration).toBeGreaterThanOrEqual(1.9);
+    expect(boundedDuration).toBeLessThanOrEqual(2.1);
+
+    const mediaDir = join(ROOT, ".markcut-candidate-vision");
+    const generated = execSync(`find ${quote(mediaDir)} -maxdepth 1 -type f -print`, { encoding: "utf-8" }).trim().split("\n");
+    expect(generated.some((file) => file.endsWith("_audio.mp3"))).toBe(false);
+    expect(generated.some((file) => file.endsWith(".vtt"))).toBe(false);
+
+    const sample = generated.find((file) => file.endsWith("_sample.mp4"));
+    expect(sample).toBeTruthy();
+    const streams = JSON.parse(execSync(
+      `ffprobe -v quiet -print_format json -show_streams ${quote(sample!)}`,
+      { encoding: "utf-8", stdio: ["pipe", "pipe", "pipe"] },
+    )).streams as Array<{ codec_type: string }>;
+    expect(streams.some((stream) => stream.codec_type === "audio")).toBe(false);
+  }, 60_000);
+
+  it("rejects malformed model output instead of writing valid evidence", async () => {
+    const badFixture = fixture(join(ROOT, "malformed.txt"), "This looks useful but is only prose.");
+    const output = join(ROOT, "invalid-evidence.json");
+    await expect(main([
+      "node", "cli.mjs", "vision", candidate, "--candidate",
+      "--model-command", `cat ${quote(badFixture)}`,
+      "--output", output,
+    ])).rejects.toThrow(/invalid JSON/);
+    expect(existsSync(output)).toBe(false);
+  });
+
+  it("accepts zero and preserves explicit pre-trimmed candidate intervals", async () => {
+    const manifestPath = join(ROOT, "zero-candidate-interval.json");
+    const output = join(ROOT, "zero-interval-evidence.json");
+    const noTranscriptFixture = fixture(join(ROOT, "valid-no-transcript.json"), evidenceResponse({ uncertainty: true }));
+    writeFileSync(manifestPath, JSON.stringify({
+      source: {
+        id: "source-22",
+        path: basename(trimmedCandidate),
+        start: 12,
+        end: 14.5,
+      },
+      candidatePath: basename(trimmedCandidate),
+      candidateId: "candidate-zero",
+      candidateStartSec: 0,
+      candidateEndSec: 2.5,
+    }));
+
+    await main([
+      "node", "cli.mjs", "vision", manifestPath, "--candidate",
+      "--model-command", `cat ${quote(noTranscriptFixture)}`,
+      "--output", output,
+    ]);
+
+    const artifact = JSON.parse(readFileSync(output, "utf-8"));
+    expect(artifact.source).toMatchObject({ id: "source-22", start: 12, end: 14.5 });
+    expect(artifact.candidate).toMatchObject({
+      id: "candidate-zero",
+      start: 0,
+      end: 2.5,
+      path: trimmedCandidate,
+    });
+  });
+
+  it("rejects fabricated transcript quotes", () => {
+    const transcript = normalizeTranscript(join(ROOT, "transcript.vtt"));
+    const parsed = evidenceResponse({ quote: "This sentence was never spoken" });
+    expect(() => validateAgainstTranscript(parsed as any, transcript)).toThrow(/absent from caller-supplied transcript/);
+  });
+
+  it("invalidates cache for prompt, contract, model, and candidate-bound changes", () => {
+    const identity = {
+      source: { id: "source", path: candidate, start: 0, end: 1 },
+      candidate: { id: "candidate", path: candidate, start: 0, end: 1 },
+    };
+    const transcript = { supplied: false, text: null };
+    const base = cacheKey(identity as any, transcript as any, "", "prompt", "model", "v1");
+
+    expect(cacheKey(identity as any, transcript as any, "changed", "prompt", "model", "v1")).not.toBe(base);
+    expect(cacheKey(identity as any, transcript as any, "", "changed prompt", "model", "v1")).not.toBe(base);
+    expect(cacheKey(identity as any, transcript as any, "", "prompt", "changed model", "v1")).not.toBe(base);
+    expect(cacheKey(identity as any, transcript as any, "", "prompt", "model", "v2")).not.toBe(base);
+
+    const changedBounds = {
+      source: { ...identity.source, start: 1, end: 2 },
+      candidate: { ...identity.candidate, start: 1, end: 2 },
+    };
+    expect(cacheKey(changedBounds as any, transcript as any, "", "prompt", "model", "v1")).not.toBe(base);
+  });
+
+  it("invalidates cache when normalized transcript content changes", () => {
+    const transcriptPath = join(ROOT, "cache-transcript.vtt");
+    const identity = {
+      source: { id: "source", path: candidate, start: 0, end: 1 },
+      candidate: { id: "candidate", path: candidate, start: 0, end: 1 },
+    };
+    writeFileSync(
+      transcriptPath,
+      "WEBVTT\n\n00:00:00.000 --> 00:00:01.000\nAlpha transcript line\n",
+    );
+    const firstTranscript = normalizeTranscript(transcriptPath);
+    const first = cacheKey(identity as any, firstTranscript as any, "", "prompt", "model", "v1");
+
+    writeFileSync(
+      transcriptPath,
+      "WEBVTT\n\n00:00:00.000 --> 00:00:01.000\nBravo transcript line\n",
+    );
+    const secondTranscript = normalizeTranscript(transcriptPath);
+    const second = cacheKey(identity as any, secondTranscript as any, "", "prompt", "model", "v1");
+
+    expect(secondTranscript.text).not.toBe(firstTranscript.text);
+    expect(second).not.toBe(first);
+  });
+
+  it("honors --skip-stt in normal vision mode", async () => {
+    mkdirSync(SKIP_STT_ROOT, { recursive: true });
+    const video = createVideo(SKIP_STT_ROOT, "long.mp4");
+    await main(["node", "cli.mjs", "vision", SKIP_STT_ROOT, "--skip-stt"]);
+
+    const metadataPath = join(SKIP_STT_ROOT, "metadata.json");
+    expect(existsSync(metadataPath)).toBe(true);
+    const metadata = JSON.parse(readFileSync(metadataPath, "utf-8"));
+    const entry = metadata.long;
+    expect(entry.perception.desc).toContain("whole video");
+    expect(entry.perception.subtitle).toBeNull();
+
+    const generated = execSync(`find ${quote(SKIP_STT_ROOT)} -type f \\( -name '*_audio.mp3' -o -name '*.vtt' \\) -print`, { encoding: "utf-8" }).trim();
+    expect(generated).toBe("");
+    expect(existsSync(video)).toBe(true);
+  }, 120_000);
+});

package/tests/vision-chatgpt-browser.test.ts

@@ -0,0 +1,50 @@
+import { afterEach, describe, expect, it } from "vitest";
+import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { execFileSync } from "node:child_process";
+import { parseArgs, prepareMedia, runVision } from "../src/vision/chatgpt-browser-cli.mjs";
+
+const roots: string[] = [];
+afterEach(() => { while (roots.length) rmSync(roots.pop()!, { recursive: true, force: true }); });
+function temp() { const p = mkdtempSync(join(tmpdir(), "markcut-browser-test-")); roots.push(p); return p; }
+function makeMock(root: string) {
+  const path = join(root, "mock-worker.mjs");
+  writeFileSync(path, `import {readFileSync,writeFileSync} from 'node:fs';\nconst p=readFileSync(process.env.MARKCUT_CHATGPT_PROMPT_FILE,'utf8');\nif(!p.includes('Output:\\nfile\\n'+process.env.MARKCUT_CHATGPT_OUTPUT_FILE)) process.exit(3);\nif(!process.env.NEO_JOB_ID?.startsWith('markcut-vision-')||!process.env.NEO_TASK_ID?.startsWith('browser-vision-')) process.exit(5);\nif(process.env.MARKCUT_CHATGPT_TAB_CLOSE_POLICY!=='after-terminal') process.exit(6);\nif(!p.includes('Publish exactly one worker-owned task.started')||!p.includes(process.env.NEO_JOB_ID)||!p.includes(process.env.NEO_TASK_ID)) process.exit(7);\nconst files=JSON.parse(process.env.MARKCUT_CHATGPT_MEDIA_FILES_JSON);\nif(!files.length||files.some(f=>!readFileSync(f))) process.exit(4);\nwriteFileSync(process.env.MARKCUT_CHATGPT_OUTPUT_FILE,'mock answer');\n`);
+  return `node ${JSON.stringify(path)}`;
+}
+
+it("parses Markcut template-style image arguments", () => {
+  const root = temp(); const a = join(root, "a.jpg"); const b = join(root, "b.jpg"); writeFileSync(a, "a"); writeFileSync(b, "b");
+  const args = parseArgs(["--mode", "image", "--prompt", "describe", "--input", `@${a}`, b]);
+  expect(args.mode).toBe("image"); expect(args.inputs).toEqual([a, b]);
+});
+
+it("uses a unique file output per invocation and propagates result", () => {
+  const root = temp(); const image = join(root, "a.jpg"); writeFileSync(image, "image"); const launcher = makeMock(root);
+  const env = { ...process.env, MARKCUT_CHATGPT_BROWSER_WORKER_AGENT_CLI: launcher } as NodeJS.ProcessEnv;
+  expect(runVision({ mode:"image", inputs:[image], prompt:"describe", timeoutMs:2000, maxFrames:8 }, env)).toBe("mock answer");
+  expect(runVision({ mode:"image", inputs:[image], prompt:"describe again", timeoutMs:2000, maxFrames:8 }, env)).toBe("mock answer");
+});
+
+it("extracts deterministic chronological video representation", () => {
+  const root = temp(); const video = join(root, "clip.mp4"); const work = join(root, "work");
+  execFileSync("ffmpeg", ["-y", "-f", "lavfi", "-i", "testsrc=s=64x64:d=2:r=4", "-c:v", "libx264", video], { stdio:"ignore" });
+  const prepared = prepareMedia("video", [video], work, 4);
+  expect(prepared.files).toHaveLength(1); expect(existsSync(prepared.files[0])).toBe(true);
+  expect(prepared.context).toContain("chronological"); expect(prepared.context).toContain("Duration:");
+});
+
+it("returns nonzero from CLI when launcher fails", () => {
+  const root = temp(); const image = join(root, "a.jpg"); writeFileSync(image, "image");
+  expect(() => execFileSync("node", ["src/vision/chatgpt-browser-cli.mjs", "--mode", "image", "--prompt", "x", "--input", image], { cwd: process.cwd(), env: { ...process.env, MARKCUT_CHATGPT_BROWSER_WORKER_CLI: "exit 7" }, stdio:"pipe" })).toThrow();
+});
+
+describe("concurrency", () => {
+  it("does not collide across simultaneous invocations", async () => {
+    const root = temp(); const image = join(root, "a.jpg"); writeFileSync(image, "image"); const launcher = makeMock(root);
+    const env = { ...process.env, MARKCUT_CHATGPT_BROWSER_WORKER_AGENT_CLI: launcher } as NodeJS.ProcessEnv;
+    const results = await Promise.all(Array.from({length:4}, (_, i) => Promise.resolve().then(() => runVision({ mode:"image", inputs:[image], prompt:`p${i}`, timeoutMs:2000, maxFrames:8 }, env))));
+    expect(results).toEqual(["mock answer", "mock answer", "mock answer", "mock answer"]);
+  });
+});