@juspay/neurolink 13.4.2 → 13.4.3

package/CHANGELOG.md

@@ -1,8 +1,8 @@
-## [13.4.2](https://github.com/juspay/neurolink/compare/v13.4.1...v13.4.2) (2026-10-18)
+## [13.4.3](https://github.com/juspay/neurolink/compare/v13.4.2...v13.4.3) (2026-10-18)
 
 ### Bug Fixes
 
-- **(office):** extract PPTX text without matching table tags, with entities and notes ([20b0368](https://github.com/juspay/neurolink/commit/20b03684f25ebcad337dfd110ff8e386631def99)), closes [#427](https://github.com/juspay/neurolink/issues/427)
+- **(auth):** annotate the auth provider emitter so declarations stay valid on newer @types/node ([fbd53a3](https://github.com/juspay/neurolink/commit/fbd53a37317e71bf79e1fb59063bf3521464510f)), closes [#1999](https://github.com/juspay/neurolink/issues/1999)
 
 ## [11.2.3](https://github.com/juspay/neurolink/compare/v11.2.2...v11.2.3) (2026-08-19)
 

package/README.md

@@ -928,7 +928,7 @@ const batcher = new RequestBatcher({ maxBatchSize: 10, maxWaitMs: 50 });
 | **Config**    | `.env`, `.ini`, `.toml`, `.cfg`                                                        | Secure parsing                                                      |
 | **Media**     | Images (PNG, JPEG, WebP, GIF), PDFs, CSV                                               | Provider-specific formatting                                        |
 | **Audio**     | `.mp3`, `.wav`, `.m4a`, `.ogg`, `.flac`, `.webm`                                       | Automatic transcription + duration metadata                         |
-| **Video**     | `.mp4`, `.webm`, `.mov`, `.mkv`, `.avi`                                                | Keyframe extraction, metadata, embedded subtitles                   |
+| **Video**     | `.mp4`, `.webm`, `.mov`, `.mkv`, `.avi`                                                | Native to Gemini (up to 14 MB); else keyframes, metadata, subtitles |
 | **Archive**   | `.zip`, `.tar`, `.gz`, `.tgz`, `.bz2`, `.tbz2`, `.jar`, `.xz`, `.txz`, `.zst`, `.tzst` | Entry listing, nested text extraction                               |
 
 ```typescript
@@ -950,8 +950,20 @@ const result = await neurolink.generate({
 ```
 
 Audio and video attach the same way. Audio is transcribed automatically before
-the model sees it; video is reduced to keyframes plus metadata and any embedded
-subtitle track:
+the model sees it. Video takes one of two routes, depending on the provider:
+
+- **Gemini (Google AI Studio and Vertex AI):** the clip itself goes into the
+  request, audio track included, so the model sees the motion and hears the
+  speech. The ceiling is 14 MB of source bytes for the whole request, shared
+  with every other inline image, PDF and audio part. A clip over that, in a
+  container Gemini does not read inline (MKV, for one), or longer than an hour
+  falls back to the keyframe route; `--debug` shows the reason. The clip has to
+  be a local path or a `Buffer`: a URL is fetched for keyframes only, without a
+  warning.
+- **Every other provider:** the video is reduced to timestamped keyframes plus
+  metadata and any embedded subtitle track. Add `--transcribe-audio`
+  (`videoOptions.transcribeAudio`) to include a transcript of the speech. Leave
+  it off for Gemini, which hears the clip itself.
 
 ```typescript
 const result = await neurolink.generate({
@@ -959,22 +971,35 @@ const result = await neurolink.generate({
     text: "What was decided in this meeting, and who owns each action item?",
     files: [
       "./standup.mp3", // transcribed, then folded into the prompt
-      "./demo.mp4", // keyframes + duration/codec metadata + subtitles
+      "./demo.mp4", // whole clip on Gemini; keyframes + metadata + subtitles elsewhere
       "./deck.pptx", // slides and speaker notes
     ],
   },
+  // Keyframe knobs; the clip itself is sent unchanged on Gemini.
+  videoOptions: {
+    frames: 16,
+    quality: 90,
+    format: "jpeg",
+  },
 });
 ```
 
 ```bash
 # Same thing from the CLI
 neurolink generate "Summarize this recording" --file ./standup.mp3
-neurolink generate "Describe what happens" --file ./demo.mp4
+neurolink generate "Describe what happens" --file ./demo.mp4 --provider google-ai
+
+# --video is the explicit flag; the others tune keyframes and the transcript
+neurolink generate "What is said, and what is shown?" --video ./demo.mp4 \
+  --provider openai --video-frames 16 --video-quality 90 --transcribe-audio
 ```
 
-> Audio transcription needs a provider with a speech model configured (OpenAI
-> Whisper by default). Video keyframe extraction requires `ffmpeg` — install it
-> separately or rely on the bundled `ffmpeg-static`.
+> Transcription runs on whichever speech-to-text engine is configured: OpenAI
+> Whisper when `OPENAI_API_KEY` is set, another hosted engine, or the built-in
+> local one (see the [transcribe guide](docs/features/transcribe.md)).
+> Keyframe extraction and `--transcribe-audio` need `ffmpeg`, from the optional
+> `ffmpeg-static` dependency or your `PATH`. Native Gemini delivery does not.
+> Full reference: **[Video Analysis guide](docs/features/video-analysis.md)**.
 
 **Key Features:**
 

package/dist/auth/providers/BaseAuthProvider.d.ts

@@ -87,7 +87,7 @@ export declare abstract class BaseAuthProvider implements AuthProvider {
     protected sessionStorage: SessionStorage;
     protected sessionConfig: SessionConfig;
     protected rbacConfig: RBACConfig;
-    protected emitter: EventEmitter<[never]>;
+    protected emitter: EventEmitter;
     constructor(config: AuthProviderConfig);
     /**
      * Validate and authenticate a token

package/docs-site/static/search-index.json

@@ -773,7 +773,7 @@
 {"objectID":"f007d6256ae9ce3cd888b757374cf819d6d20a2466de5280aa882385928dfa42","title":"Install globally","url":"/docs/cli/commands#install-globally","content":"npm install -g @juspay/neurolink","hierarchy":{"lvl0":"Cli","lvl1":"CLI Command Reference","lvl2":"Install globally","lvl3":""}},
 {"objectID":"fe8f683c688c22a4fb00ea52ec4f803cc988e40adedfb184ae651f06394d7d87","title":"Local project dependency","url":"/docs/cli/commands#local-project-dependency","content":"npm install @juspay/neurolink\n`","hierarchy":{"lvl0":"Cli","lvl1":"CLI Command Reference","lvl2":"Local project dependency","lvl3":""}},
 {"objectID":"6d0358f7c107fa0314373826a28ae5ced11a8739cb937a2e595a91d9b60e762c","title":"Command Map","url":"/docs/cli/commands#command-map","content":"| Command                | Description                                                        | Example                                                                                                                      |\n| ---------------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------- |\n| generate / gen     | One-shot content generation with optional multimodal input.        | npx @juspay/neurolink generate \"Draft release notes\" --image ./before.png                                                  |\n| stream               | Real-time streaming output with tool support.                      | npx @juspay/neurolink stream \"Narrate sprint demo\" --enableAnalytics                                                       |\n| decide [state]       | Typed, calibrated judgements from a decision model (no free text). | npx @juspay/neurolink decide \"Refund request\" --questions '{\"urgent\":{\"type\":\"boolean\",\"instructions\":\"Is this urgent?\"}}' |\n| transcribe <file\\|-> | Audio to text; local engine by default, dictionary and correction. | npx @juspay/neurolink transcribe meeting.wav --format srt                                                                  |\n| batch                | Process multiple prompts from a file.                              | npx @juspay/neurolink batch prompts.txt --format json                                                                      |\n| loop                 | Interactive session with persistent variables & memory.            | npx @juspay/neurolink loop --auto-redis                                                                                    |\n| auth <subcommand>    | Manage provider authentication (API key or OAuth).","hierarchy":{"lvl0":"Cli","lvl1":"CLI Command Reference","lvl2":"Command Map","lvl3":""}},
-{"objectID":"b149865b2facf38f974af3ecc7a0bfaee49be27a20c207ed01c32b26247fd099","title":"generate <input>","url":"/docs/cli/commands#generate","content":"Key flags:\n--provider, -p – provider slug (default auto).\n--model, -m – model name for the chosen provider.\n--image, -i – attach one or more image files/URLs for multimodal prompts.\n--pdf – attach one or more PDF files for document analysis.\n--csv, -c – attach one or more CSV files for data analysis.\n--file – attach any supported file type, auto-detected. Covers Office documents (Word .docx, Excel .xlsx/.xls, PowerPoint .pptx, RTF, OpenDocument), audio (.mp3, .wav, .m4a, … — transcribed automatically), video (.mp4, .webm, .mov, .mkv — keyframes plus metadata and any embedded subtitles), archives, JSON, YAML, XML, HTML, SVG, Markdown, and 50+ code languages. Repeatable. Not available on batch, where it would collide with the prompts-file positional — use generate or stream.\n--temperature, -t – creativity (default 0.7).\n--maxTokens, --max – response limit (default 1000).\n--system, -s – system prompt.\n--format, -f, --output-format – text (default), json, or table. With json, stdout carries only the JSON result; debug and info logs go to stderr, so the output stays parseable even with --debug.\n--output, -o – write response to file.\n--imageOutput, --image-output – custom path for generated image (default: generated-images/image-<timestamp>.png).\n--enableAnalytics / --enableEvaluation – capture metrics & quality scores.\n--evaluationDomain – domain hint for the judge model.\n--domainAware – use domain-aware evaluation (default false).\n--context – JSON string appended to analytics/evaluation context.\n--domain, -d – domain type for specialized processing: healthcare, finance, analytics, ecommerce, education, legal, technology, generic, auto.\n--disableTools – bypass MCP tools for this call.\n--timeout – seconds before aborting the request (default 120).\n--region, -r – Vertex AI region (e.g., us-central1, europe-west1, asia-northeast1).\n--debug, -v, --verbose – verbose logging and full JSON payloads.\n--quiet, -q – suppress non-essential output (default true).","hierarchy":{"lvl0":"Cli","lvl1":"CLI Command Reference","lvl2":"generate <input>","lvl3":""}},
+{"objectID":"b149865b2facf38f974af3ecc7a0bfaee49be27a20c207ed01c32b26247fd099","title":"generate <input>","url":"/docs/cli/commands#generate","content":"Key flags:\n--provider, -p – provider slug (default auto).\n--model, -m – model name for the chosen provider.\n--image, -i – attach one or more image files/URLs for multimodal prompts.\n--pdf – attach one or more PDF files for document analysis.\n--csv, -c – attach one or more CSV files for data analysis.\n--file – attach any supported file type, auto-detected. Covers Office documents (Word .docx, Excel .xlsx/.xls, PowerPoint .pptx, RTF, OpenDocument), audio (.mp3, .wav, .m4a, … — transcribed automatically), video (.mp4, .webm, .mov, .mkv — Gemini providers receive the clip itself, up to 14 MB; every other provider receives keyframes plus metadata and any embedded subtitles), archives, JSON, YAML, XML, HTML, SVG, Markdown, and 50+ code languages. Repeatable. Not available on batch, where it would collide with the prompts-file positional — use generate or stream.\n--temperature, -t – creativity (default 0.7).\n--maxTokens, --max – response limit (default 1000).\n--system, -s – system prompt.\n--format, -f, --output-format – text (default), json, or table. With json, stdout carries only the JSON result; debug and info logs go to stderr, so the output stays parseable even with --debug.\n--output, -o – write response to file.\n--imageOutput, --image-output – custom path for generated image (default: generated-images/image-<timestamp>.png).\n--enableAnalytics / --enableEvaluation – capture metrics & quality scores.\n--evaluationDomain – domain hint for the judge model.\n--domainAware – use domain-aware evaluation (default false).\n--context – JSON string appended to analytics/evaluation context.\n--domain, -d – domain type for specialized processing: healthcare, finance, analytics, ecommerce, education, legal, technology, generic, auto.\n--disableTools – bypass MCP tools for this call.\n--timeout – seconds before aborting the request (default 120).\n--region, -r – Vertex AI region (e.g., us-central1, europe-west1, asia-northeast1).","hierarchy":{"lvl0":"Cli","lvl1":"CLI Command Reference","lvl2":"generate <input>","lvl3":""}},
 {"objectID":"621800e7302c71eae24c4286bcb9e05a8e110cf656e2a2d339bd8ecbe998d32e","title":"Generate with explicit subscription tier","url":"/docs/cli/commands#generate-with-explicit-subscription-tier","content":"npx @juspay/neurolink generate \"Explain quantum computing\" \\\n  --provider anthropic --subscription-tier pro","hierarchy":{"lvl0":"Cli","lvl1":"CLI Command Reference","lvl2":"Generate with explicit subscription tier","lvl3":""}},
 {"objectID":"ad3c3b350fe961788a9aad769e12752ba4a17c25fbc14e3aeec6374cfb865d0c","title":"Generate with OAuth auth method","url":"/docs/cli/commands#generate-with-oauth-auth-method","content":"npx @juspay/neurolink generate \"Write a poem\" \\\n  --provider anthropic --authMethod oauth --enableBeta","hierarchy":{"lvl0":"Cli","lvl1":"CLI Command Reference","lvl2":"Generate with OAuth auth method","lvl3":""}},
 {"objectID":"e10205dca3731c8d19a1bad6da55856d73395d3512b695b335cffbb198d8498b","title":"Stream with max tier","url":"/docs/cli/commands#stream-with-max-tier","content":"npx @juspay/neurolink stream \"Tell me a story\" \\\n  --provider anthropic --subscriptionTier max\nbash","hierarchy":{"lvl0":"Cli","lvl1":"CLI Command Reference","lvl2":"Stream with max tier","lvl3":""}},
@@ -3459,7 +3459,7 @@
 {"objectID":"f955eabea1a54bcd03d671cb9ddcd4595938f94906230186f04b7f69468b3694","title":"Markup Files","url":"/docs/features/file-processors#markup-files","content":"| Type         | Extensions         | Processor           | Features                                      |\n| ------------ | ------------------ | ------------------- | --------------------------------------------- |\n| HTML     | .html, .htm    | HtmlProcessor     | OWASP-compliant sanitization, text extraction |\n| SVG      | .svg             | SvgProcessor      | XSS prevention, text injection (not binary)   |\n| Markdown | .md, .markdown | MarkdownProcessor | Formatting preservation, metadata extraction  |\n| Text     | .txt             | TextProcessor     | Plain text handling, encoding detection       |","hierarchy":{"lvl0":"Features","lvl1":"File Processors Guide","lvl2":"Markup Files","lvl3":""}},
 {"objectID":"4d831ad6d8c06bb6560d07251a8dd9e1da0c2b74cb85e7899d93bc2c5576a8b5","title":"Source Code","url":"/docs/features/file-processors#source-code","content":"| Type                | Extensions                 | Processor             | Features                            |\n| ------------------- | -------------------------- | --------------------- | ----------------------------------- |\n| TypeScript      | .ts, .tsx              | SourceCodeProcessor | Language detection, syntax metadata |\n| JavaScript      | .js, .jsx, .mjs      | SourceCodeProcessor | Module detection                    |\n| Python          | .py                      | SourceCodeProcessor | Docstring preservation              |\n| Java            | .java                    | SourceCodeProcessor | Package detection                   |\n| Go              | .go                      | SourceCodeProcessor | Module awareness                    |\n| Rust            | .rs                      | SourceCodeProcessor | Crate detection                     |\n| C/C++           | .c, .cpp, .h, .hpp | SourceCodeProcessor | Header handling                     |\n| C#              | .cs                      | SourceCodeProcessor | Namespace detection                 |\n| Ruby            | .rb                      | SourceCodeProcessor | Gem awareness                       |\n| PHP             | .php                     | SourceCodeProcessor | Tag handling                        |\n| Swift           | .swift                   | SourceCodeProcessor | Framework detection                 |\n| Kotlin          | .kt, .kts              | SourceCodeProcessor | Android/JVM awareness               |\n| Scala           | .scala                   | SourceCodeProcessor | SBT integration                     |\n| Shell           | .sh, .bash, .zsh     | SourceCodeProcessor | Shebang detection                   |\n| SQL             | .sql                     | SourceCodeProcessor | Dialect hints                       |\n| And 35+ more... | Various                    | SourceCodeProcessor | Automatic language detection        |","hierarchy":{"lvl0":"Features","lvl1":"File Processors Guide","lvl2":"Source Code","lvl3":""}},
 {"objectID":"c2f404f3c6f4e80c2a49514197d11bfd92868a6dab3916b92ae3803ba7cd8507","title":"Configuration Files","url":"/docs/features/file-processors#configuration-files","content":"| Type            | Extensions       | Processor         | Features                           |\n| --------------- | ---------------- | ----------------- | ---------------------------------- |\n| Environment | .env, .env.* | ConfigProcessor | Secret masking option              |\n| INI         | .ini, .cfg   | ConfigProcessor | Section parsing                    |\n| TOML        | .toml          | ConfigProcessor | Cargo.toml, pyproject.toml support |\n| Properties  | .properties    | ConfigProcessor | Java properties format             |","hierarchy":{"lvl0":"Features","lvl1":"File Processors Guide","lvl2":"Configuration Files","lvl3":""}},
-{"objectID":"3b187138dfa909c5c773a82bb2d1f86d489423608b02ec8bce492057f7a7fd86","title":"Media Files","url":"/docs/features/file-processors#media-files","content":"| Type      | Extensions                                              | Processor        | Features                                                                         |\n| --------- | ------------------------------------------------------- | ---------------- | -------------------------------------------------------------------------------- |\n| Video | .mp4, .mkv, .webm, .avi, .mov, .m4v         | VideoProcessor | Duration, resolution, codec, frame rate, bitrate extraction via music-metadata |\n| Audio | .mp3, .wav, .ogg, .flac, .aac, .m4a, .wma | AudioProcessor | Codec, bitrate, sample rate, channels, duration extraction via music-metadata  |\n\nVideo and audio files are not sent as binary to the AI provider. Instead, the processors extract structured metadata and return it as formatted text, keeping token usage minimal (~50-200 tokens per file).\n\nExample video output:\n\nExample audio output:","hierarchy":{"lvl0":"Features","lvl1":"File Processors Guide","lvl2":"Media Files","lvl3":""}},
+{"objectID":"3b187138dfa909c5c773a82bb2d1f86d489423608b02ec8bce492057f7a7fd86","title":"Media Files","url":"/docs/features/file-processors#media-files","content":"| Type      | Extensions                                              | Processor        | Features                                                                         |\n| --------- | ------------------------------------------------------- | ---------------- | -------------------------------------------------------------------------------- |\n| Video | .mp4, .mkv, .webm, .avi, .mov, .m4v         | VideoProcessor | Duration, resolution, codec, frame rate, bitrate extraction via music-metadata |\n| Audio | .mp3, .wav, .ogg, .flac, .aac, .m4a, .wma | AudioProcessor | Codec, bitrate, sample rate, channels, duration extraction via music-metadata  |\n\nFor most providers, audio and video files are not sent as binary. Instead, the processor extracts structured metadata and returns it as formatted text, keeping token usage minimal (~50-200 tokens per file). Video files produce that same metadata text and also bring timestamped keyframes and any embedded subtitles. The Gemini providers additionally receive the audio file itself and the video clip itself (a clip qualifies up to 14 MB of source bytes for the whole request); see the Video Analysis guide.\n\nExample video output:\n\nExample audio output:","hierarchy":{"lvl0":"Features","lvl1":"File Processors Guide","lvl2":"Media Files","lvl3":""}},
 {"objectID":"fb11a9eb041ca49290c486646aca74026a2bb5d538fb34eeff55de9a569bcc5f","title":"Archives","url":"/docs/features/file-processors#archives","content":"| Type    | Extensions               | Processor          | Features                                                               |\n| ------- | ------------------------ | ------------------ | ---------------------------------------------------------------------- |\n| ZIP | .zip                   | ArchiveProcessor | File listing with sizes, nested content extraction, ZIP bomb detection |\n| TAR | .tar                   | ArchiveProcessor | File listing with sizes                                                |\n| GZ  | .gz, .tar.gz, .tgz | ArchiveProcessor | Gzip decompression, tar content listing                                |\n\nArchive files return a structured listing of their contents with file sizes and optionally extract text from contained files (routing through existing processors).\n\nExample archive output:\n\nSecurity: Archive processing includes ZIP bomb detection (compression ratio limits), path traversal prevention, symlink blocking, entry count limits, and aggregate decompression size limits.","hierarchy":{"lvl0":"Features","lvl1":"File Processors Guide","lvl2":"Archives","lvl3":""}},
 {"objectID":"94d24822dfeda8ff141ce17dbc1c895f92f40bedda94050fae9e89b10d441eb9","title":"CLI Usage","url":"/docs/features/file-processors#cli-usage","content":"`bash","hierarchy":{"lvl0":"Features","lvl1":"File Processors Guide","lvl2":"CLI Usage","lvl3":""}},
 {"objectID":"236270eebf265535ca63a7b9c340fc62d5b82eb72e66811a946513db37985e61","title":"Single file","url":"/docs/features/file-processors#single-file","content":"neurolink generate \"Analyze this spreadsheet\" --file ./data.xlsx","hierarchy":{"lvl0":"Features","lvl1":"File Processors Guide","lvl2":"Single file","lvl3":""}},
@@ -3956,14 +3956,14 @@
 {"objectID":"0ffdeaf2a4d433739fb1f96c6487b2e35e254c4459ead6ecbfd3dcb3a98de8b3","title":"Explicit CSV with options","url":"/docs/features/multimodal#explicit-csv-with-options","content":"npx @juspay/neurolink generate \"Summarize data\" \\\n  --csv ./data.csv \\\n  --csv-max-rows 500 \\\n  --csv-format raw\ntypescript\n// Auto-detect (recommended)\nawait neurolink.generate({\n  input: {\n    text: \"Analyze this sales data\",\n    files: [\"./sales.csv\"], // Auto-detected as CSV\n  },\n});\n\n// Explicit CSV with options\nawait neurolink.generate({\n  input: {\n    text: \"Compare quarterly data\",\n    csvFiles: [\"./q1.csv\", \"./q2.csv\"],\n  },\n  csvOptions: {\n    maxRows: 1000,\n    formatStyle: \"json\", // or \"raw\", \"markdown\"\n  },\n});\n`","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Explicit CSV with options","lvl3":""}},
 {"objectID":"93535721732536c013e0204b266f2a131be12a7745f6c88a6e61cdeb76704728","title":"CSV Format Options","url":"/docs/features/multimodal#csv-format-options","content":"Three format styles:\nRaw format (default)\nBest for large files\nMinimal token usage\nPreserves original CSV structure\nJSON format\nStructured data processing\nEasier for AI to parse\nHigher token usage\nMarkdown format\nReadable tables\nGood for small datasets (\\<100 rows)\nModerate token usage","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"CSV Format Options","lvl3":""}},
 {"objectID":"427b303d134e7e002fe1fbb0ae86243f33506e795e3029cdfb7994c2014c0db4","title":"CSV Best Practices","url":"/docs/features/multimodal#csv-best-practices","content":"Use raw format for large files to minimize token usage\nUse JSON format for structured processing when AI needs to manipulate data\nLimit to 1000 rows by default (configurable up to 10,000)\nCombine CSV with visualization images for comprehensive analysis\nWorks across supported providers (not just vision-capable models)","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"CSV Best Practices","lvl3":""}},
-{"objectID":"5f2f649cdd7639b3fc3fb30871df07bf3331ad446ac4d23340aaab8976ccf1a7","title":"Frame-extraction options","url":"/docs/features/multimodal#frame-extraction-options","content":"These control the keyframes and are ignored by a provider on the native path.\n\nWithout an explicit frames, the interval is chosen from the clip's duration\n— roughly every second for clips under 10s, widening to every three minutes\nfor recordings over half an hour, always capped at 100 frames.","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Frame-extraction options","lvl3":""}},
+{"objectID":"5f2f649cdd7639b3fc3fb30871df07bf3331ad446ac4d23340aaab8976ccf1a7","title":"Frame-extraction options","url":"/docs/features/multimodal#frame-extraction-options","content":"frames, quality and format control the keyframes and do not change a clip sent natively. transcribeAudio does not look at the provider: on Gemini it still adds a transcript next to the audio the model hears, so set it for frame-path providers only.\n\nWithout an explicit frames, the interval is chosen from the clip's duration\n— roughly every second for clips under 10s, widening to one frame a minute\nfor recordings over half an hour, always capped at 100 frames.","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Frame-extraction options","lvl3":""}},
 {"objectID":"2c4c2bb10b6fea62d177994e6df1345d05606b65f8122b3c969cf214df3363f1","title":"CLI usage","url":"/docs/features/multimodal#cli-usage","content":"`bash","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"CLI usage","lvl3":""}},
 {"objectID":"915a9e9701a9e6b02890e744550ed66754549288fa044c90bd2dd873e6b48c2a","title":"Auto-detect","url":"/docs/features/multimodal#auto-detect","content":"neurolink generate \"Describe this video\" --file demo.mp4 --provider google-ai","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Auto-detect","lvl3":""}},
 {"objectID":"12ce623ba884b34886674b24daff6ce9e58465c68dc25f4f66c01229c6855513","title":"Explicit flag","url":"/docs/features/multimodal#explicit-flag","content":"neurolink generate \"Summarise this\" --video standup.mp4 --provider vertex","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Explicit flag","lvl3":""}},
-{"objectID":"08774b3c1d8fd456cb7d64ad98281449b9271854310a9a713048ae2112572d5c","title":"Frame-extraction knobs","url":"/docs/features/multimodal#frame-extraction-knobs","content":"neurolink generate \"Describe each scene\" --file demo.mp4 \\\n  --provider openai --video-frames 16 --video-quality 90 --video-format jpeg\n\n\n--transcribe-audio (or videoOptions.transcribeAudio) gives a frame-path\nprovider the clip's speech, which keyframes cannot carry. The audio track is\nextracted with ffmpeg, transcribed by OpenAI Whisper (whisper-1, using\nOPENAIAPIKEY, and OPENAIBASEURL when set), and sent alongside the\nkeyframes under a Spoken Audio (transcribed) heading. Transcription is\nadditive: when it cannot run — the clip has no audio track, OPENAIAPIKEY is\nunset, the extracted audio is over Whisper's 25 MB limit, or extraction or the\nupload fails — the request still succeeds with the keyframes and metadata, and a\nNo transcript for :  warning says which. A Gemini provider does\nnot need it while the clip is delivered natively, since that includes the audio\ntrack; a clip that falls back to keyframes (over the inline ceiling) loses its\nspeech unless --transcribe-audio is set.\n\nEmbedded **subtitle tracks** are extracted separately and included whenever\nthey exist, with or without transcribeAudio`.","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Frame-extraction knobs","lvl3":""}},
+{"objectID":"08774b3c1d8fd456cb7d64ad98281449b9271854310a9a713048ae2112572d5c","title":"Frame-extraction knobs","url":"/docs/features/multimodal#frame-extraction-knobs","content":"neurolink generate \"Describe each scene\" --file demo.mp4 \\\n  --provider openai --video-frames 16 --video-quality 90 --video-format jpeg\n\n\n--transcribe-audio (or videoOptions.transcribeAudio) gives a frame-path\nprovider the clip's speech, which keyframes cannot carry. The audio track is\nextracted with ffmpeg, transcribed by the configured speech-to-text engine\n(the same selection as neurolink.transcribe(); see the\n[transcribe guide](/docs/features/transcribe)), and sent alongside the keyframes under a\nSpoken Audio (transcribed) heading. Transcription is additive: when it cannot\nrun — the clip has no audio track, the speech-to-text backend is not\nconfigured, the extracted audio is over the 25 MB limit, or extraction or the\ntranscription request fails — the request still succeeds with the keyframes and\nmetadata, and a No transcript for :  warning says which. A\nGemini provider does not need it while the clip is delivered natively, since\nthat includes the audio track; a clip that falls back to keyframes (over the\ninline ceiling) loses its speech unless --transcribe-audio is set.\n\nEmbedded **subtitle tracks** are extracted separately and included whenever\nthey exist, with or without transcribeAudio`.","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Frame-extraction knobs","lvl3":""}},
 {"objectID":"d0faa964a225271bc41d6c279e3e925829ac8e8828518387c39ba60d97cd363f","title":"Asking which mechanism applies","url":"/docs/features/multimodal#asking-which-mechanism-applies","content":"The capability table is exported, so a caller can ask before sending — useful\nfor choosing a provider, or for budgeting.","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Asking which mechanism applies","lvl3":""}},
 {"objectID":"10a8f5bf0ea5129e242eb09d5db285a4d91aac091c171050471ebd7eb528cb72","title":"Best practices","url":"/docs/features/multimodal#best-practices","content":"Ask an audio question of a Gemini provider only. Nothing on the frame\n  path can hear the clip, so \"what was said\" has no answer there.\nKeep clips under 14 MB when you want native delivery. Re-encoding a\n  screen recording at a lower bitrate usually gets a long clip under the\n  ceiling without losing anything a model would read.\nRaise frames rather than quality for detail over time. Frames cost\n  roughly 250 tokens each regardless of quality; more of them is what buys\n  coverage of a clip where things change.\nLower frames for local runtimes. Ollama and llama.cpp hold every\n  frame in the same memory as the model; four is the practical default.","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Best practices","lvl3":""}},
-{"objectID":"7bd6ebb59bff22b19f26c8c358eb7012f8a73d4f90eed575ca136f632c6f9bc6","title":"Troubleshooting","url":"/docs/features/multimodal#troubleshooting","content":"| Symptom                                             | Cause                                                                                                                         | Fix                                                                    |\n| --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |\n| Model describes the file but not its content        | No keyframes extracted — ffmpeg missing — and the provider takes frames                                                       | Install ffmpeg, or use a Gemini provider, which needs none             |\n| Sending keyframes instead of the clip in the logs | The clip failed the inline gate; the reason is on the same line                                                               | Shorten or re-encode it, or accept the frames                          |\n| Model cannot hear speech on a Gemini provider       | The clip exceeded the inline ceiling and fell back to frames                                                                  | Get the source under 14 MB, or pass --transcribe-audio               |\n| --transcribe-audio produced no transcript         | The No transcript for warning names the reason: no audio track, OPENAI_API_KEY unset, or audio over Whisper's 25 MB limit | Set OPENAI_API_KEY, or split the recording                           |\n| Only the first seconds of a long clip are described | The frame budget was hit before the end                                                                                       | Raise frames; the interval then spreads evenly across the whole clip |","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Troubleshooting","lvl3":""}},
+{"objectID":"7bd6ebb59bff22b19f26c8c358eb7012f8a73d4f90eed575ca136f632c6f9bc6","title":"Troubleshooting","url":"/docs/features/multimodal#troubleshooting","content":"| Symptom                                             | Cause                                                                                                                       | Fix                                                                    |\n| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |\n| Model describes the file but not its content        | No keyframes extracted — ffmpeg missing — and the provider takes frames                                                     | Install ffmpeg, or use a Gemini provider, which needs none             |\n| Sending keyframes instead of the clip in the logs | The clip failed the inline gate; the reason is on the same line                                                             | Shorten or re-encode it, or accept the frames                          |\n| Model cannot hear speech on a Gemini provider       | The clip exceeded the inline ceiling and fell back to frames                                                                | Get the source under 14 MB, or pass --transcribe-audio               |\n| --transcribe-audio produced no transcript         | The No transcript for warning names the reason: no audio track, no speech-to-text backend configured, or audio over 25 MB | Configure an engine (see the transcribe guide), or split the recording |\n| Only the first seconds of a long clip are described | The frame budget was hit before the end                                                                                     | Raise frames; the interval then spreads evenly across the whole clip |","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Troubleshooting","lvl3":""}},
 {"objectID":"79318378a7df7cc816465eb447fa4bd2ba9e6344f17b0a9cf6661377bb03e7e2","title":"Combining Multiple Input Types","url":"/docs/features/multimodal#combining-multiple-input-types","content":"NeuroLink excels at combining different media types in a single request.","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Combining Multiple Input Types","lvl3":""}},
 {"objectID":"412f09080a5801a35250df393e2e0694602d5312e2b3d1b017fe469bf9dc0474","title":"Batch with Multimodal (CLI)","url":"/docs/features/multimodal#batch-with-multimodal-cli","content":"The batch command supports --image, --csv, --pdf, and --video. The file(s) are attached identically to every prompt in the batch (a one-line notice is printed to stderr, unconditionally — it is not suppressed by --quiet, so it never corrupts --format json output written to stdout):\n\n--file (auto-detect) is not available in batch, because it collides with the <file> positional (the prompts-list path). Use the explicit --image / --csv / --pdf / --video flags instead.","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"Batch with Multimodal (CLI)","lvl3":""}},
 {"objectID":"66d907236327d676dbe6e05193c2793f7c5990dce2ad113cf078eb1ff4c73216","title":"File validation & troubleshooting (CLI)","url":"/docs/features/multimodal#file-validation-troubleshooting-cli","content":"Before any provider call, the CLI validates local file inputs across generate, stream, and batch:\nA path that points at a directory, doesn't exist, or can't be read (e.g. a permissions error) is rejected up front with a clear error and troubleshooting hints — no cryptic EISDIR/EACCES deep in processing. This also applies to the batch <file> prompts-list positional itself.\nA large file (images > 10 MB, CSV/PDF > 50 MB) prints a non-blocking warning to stderr. Like the batch attachment notice, this is unconditional — visible without --debug and regardless of --quiet — and never mixes into stdout, so --format json output stays valid JSON even when large-file warnings fire.\n\nThis runs even under --dry-run and without API keys configured.","hierarchy":{"lvl0":"Features","lvl1":"Multimodal Capabilities Guide","lvl2":"File validation & troubleshooting (CLI)","lvl3":""}},
@@ -4599,24 +4599,29 @@
 {"objectID":"8a1f451537135b3f1ef892866610bb0dbabfe58ed1764bac2753046b8da98222","title":"MALFORMED_FUNCTION_CALL handling","url":"/docs/features/turn-time-budget#malformed_function_call-handling","content":"Gemini's MALFORMED_FUNCTION_CALL / UNEXPECTED_TOOL_CALL finish reasons\nmap to unified finishReason: \"error\" (never \"tool-calls\"). A\nMALFORMED_FUNCTION_CALL step is retried once with a corrective note;\nif it persists, the turn ends with stopReason: \"provider-error\" — usually\nworth a caller-side retry.","hierarchy":{"lvl0":"Features","lvl1":"Turn Time Budget","lvl2":"MALFORMED_FUNCTION_CALL handling","lvl3":""}},
 {"objectID":"b858770b3853e797dfaac8ecfbd0c1f86c320e6a94e46a2e60f8407f328f6090","title":"Other Gemini finish reasons","url":"/docs/features/turn-time-budget#other-gemini-finish-reasons","content":"The native Gemini loops (AI Studio and Vertex) map the SDK's finish reason onto\nthe unified values below and keep the verbatim value in rawFinishReason.\nVertex puts the unified value on finishReason; on AI Studio it is the\nunified field of the finish part that model middleware receives.\n\n| Gemini finish reason                                                                      | finishReason   |\n| ----------------------------------------------------------------------------------------- | ---------------- |\n| MAX_TOKENS, CONTINUATION (token limit reached before the model was done)              | length         |\n| TOO_MANY_TOOL_CALLS (the model called tools until the system cut the turn off)          | tool-calls     |\n| MALFORMED_FUNCTION_CALL, UNEXPECTED_TOOL_CALL                                         | error          |\n| SAFETY, RECITATION, BLOCKLIST, PROHIBITED_CONTENT, SPII, LANGUAGE, IMAGE_*¹ | content-filter |\n| STOP, OTHER, FINISH_REASON_UNSPECIFIED, IMAGE_OTHER, NO_IMAGE                   | stop           |\n\n¹ IMAGE_SAFETY, IMAGE_PROHIBITED_CONTENT and IMAGE_RECITATION.\n\ntool-calls also marks a turn that ran out of maxSteps, so branch on\nstopReason to tell the two apart: TOO_MANY_TOOL_CALLS leaves\nstopReason: \"completed\", a spent step budget reports step-cap. A value the\nSDK adds later reads stop until it is mapped.","hierarchy":{"lvl0":"Features","lvl1":"Turn Time Budget","lvl2":"Other Gemini finish reasons","lvl3":""}},
 {"objectID":"a975fa8c9a970f09409f0cd78f26180dbef30bebfc60396f0eebda7776d8a3c3","title":"Telemetry","url":"/docs/features/turn-time-budget#telemetry","content":"result.analytics (when enableAnalytics is on) carries stepsUsed,\n  toolCallCount, stopReason, elapsedMs, rawFinishReason.\nThe SDK emitter fires turn:lifecycle events (alongside\n  tool:start/tool:end) for non-completed exits, tool timeouts, and\n  malformed-call retries:","hierarchy":{"lvl0":"Features","lvl1":"Turn Time Budget","lvl2":"Telemetry","lvl3":""}},
-{"objectID":"4237dd71854673dfa0c8c155353abb6bf61149758ab7cd851436a49841b8dba3","title":"Video Analysis","url":"/docs/features/video-analysis","content":"Video Analysis\n\nComprehensive video analysis for NeuroLink, powered by Gemini 2.0 Flash. This feature goes beyond basic visual description—it provides a deep logical audit of video sequences to understand \"why\" and \"how\" events occur.\n\nKey Capabilities\nLogical Analysis: Dissect any video to extract the underlying intent, cause-and-effect, and logical progression.\nAction-Reaction Chain: A step-by-step audit of user or system actions and their immediate visual results.\nEvidence-Based Reporting: Detailed reasoning backed by structured visual indicators (colors, labels, text) in JSON format.\nStrategic Verdicts: High-level assessments of whether a workflow succeeded or failed logically.\n\nQuick Start\n\nHow It Works\nFrame Extraction: The system uses ffmpeg to extract high-quality keyframes from the video at calculated intervals.\nAnalysis Pipeline: These frames are sent to Gemini 2.0 Flash with a specialized system instruction focused on critical logic auditing.\nUnified Results: The resulting report is added directly to your standard generation output.\n\nUsage\n\nCLI Usage\n\nAnalyze any video file with a natural language prompt.\n\nSDK Usage\n\nIntegrate video analysis into your TypeScript/JavaScript projects.\n\nAdvanced SDK Examples\n\nCustom Model Configuration\nFine-tune the analysis by adjusting token limits and temperature.\n\nDisabling Tool Interference\nBy default, the model might try to use available tools. For pure video analysis, you can disable them.\n\nExamples\nUI/UX Bug Analysis\n\nIdentify why a user is unable to complete a form or where the interface is misleading.\n\nPrompt: \"Find why the user is getting stuck at the payment step. Look for validation errors or hidden UI elements.\"\nSilent Failure Detection\n\nDetect cases where an action is taken but the system provides no feedback (no loaders, no success messages).\n\nPrompt: \"Audit the 'Submit' button click. Is there a visual 'bond' between the click and the next state? Report any lag or missing loading indicators.\"\nWorkflow Validation\n\nVerify if a complex multi-step process follows the intended business logic.\n\nPrompt: \"Trace the logical progression from 'Item Selection' to 'Checkout'. Does every state change correspond to a user action?\"\nComparison Analysis\n\nCompare two recordings to find discrepancies in behavior.\n\nPrompt: \"Compare these two clips. The first one is the expected behavior and the second one has a bug. Identify the exact frame or timestamp where the logic deviates.\"\n\nCommand Gallery\n\nQuick CLI recipes for common tasks:\n\nThe Analysis Report\n\nThe output is structured into four major sections designed to give you a complete understanding of the video:\nStrategic Overview & Intent: Defines the core activity, expected logic, and provides a primary verdict.\nThe Action-Reaction Chain: A granular, step-by-step audit of attempts, results, and technical inferences.\nCritical Findings: Categorized milestones or anomalies with root cause analysis and visual evidence in JSON.\nFinal Assessment: A conclusive summary of the logical flow based on the observed evidence.\n\nBest Practices\nFrame Depth: Short videos (under 10s) get high-density frame coverage (1 per second), while long ones are intelligently sampled.\nPrompt Precision: While the model is a \"Critical Logic Auditor,\" you can guide it with specific questions about the activity.\nFormat: The analysis is returned as text in result.content, making it easy to store, display, or pipe to other tools.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"","lvl3":""}},
-{"objectID":"288c031674e78fd4fcd753b2d753bff8b085367a321fac3e9845113df6361a2d","title":"Video Analysis","url":"/docs/features/video-analysis#video-analysis","content":"Comprehensive video analysis for NeuroLink, powered by Gemini 2.0 Flash. This feature goes beyond basic visual description—it provides a deep logical audit of video sequences to understand \"why\" and \"how\" events occur.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Video Analysis","lvl3":""}},
-{"objectID":"44263c4ea6bfb64da5f25923894e2a071f7224ea60134b669551c7e1477b3404","title":"Key Capabilities","url":"/docs/features/video-analysis#key-capabilities","content":"Logical Analysis: Dissect any video to extract the underlying intent, cause-and-effect, and logical progression.\nAction-Reaction Chain: A step-by-step audit of user or system actions and their immediate visual results.\nEvidence-Based Reporting: Detailed reasoning backed by structured visual indicators (colors, labels, text) in JSON format.\nStrategic Verdicts: High-level assessments of whether a workflow succeeded or failed logically.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Key Capabilities","lvl3":""}},
-{"objectID":"f9cd53c8cff1bc1a463d950ae0d3d406a2d29f14e7070d2f8d77d7b041fb2571","title":"How It Works","url":"/docs/features/video-analysis#how-it-works","content":"Frame Extraction: The system uses ffmpeg to extract high-quality keyframes from the video at calculated intervals.\nAnalysis Pipeline: These frames are sent to Gemini 2.0 Flash with a specialized system instruction focused on critical logic auditing.\nUnified Results: The resulting report is added directly to your standard generation output.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"How It Works","lvl3":""}},
+{"objectID":"4237dd71854673dfa0c8c155353abb6bf61149758ab7cd851436a49841b8dba3","title":"Video Analysis","url":"/docs/features/video-analysis","content":"Video Analysis\n\nAttach a video file to generate() or stream() and ask about it. What the model receives depends on the provider:\nGemini (Google AI Studio and Vertex AI) receives the clip itself, inline, with its audio track. It sees continuous motion and hears the speech.\nEvery other provider receives a metadata summary, timestamped keyframes as ordinary images, and any embedded subtitle track. It sees a handful of stills and hears nothing, unless you ask for a transcript with --transcribe-audio.\n\nVideo analysis suits screen recordings in particular: checking whether a workflow worked, finding the moment a UI went wrong, or comparing two recordings. The prompts in this guide ask for a logical audit of that kind. The audit style comes from your prompt; the SDK imposes no report format.\n\nFor the cross-modality overview (images, PDFs, audio, video) see the Multimodal guide.\n\nQuick Start\n\ninput.videoFiles and input.files are equivalent for video: videoFiles is folded into files before file detection runs. Each entry is a local path, a Buffer or a URL. Only a local path or a Buffer can be sent to Gemini as the clip itself; a URL is fetched for keyframes and metadata alone (see What qualifies a clip).\n\nHow It Works\nDetection. The file type is recognized from the file's leading bytes as well as its extension (MP4 and MOV by their ftyp box, AVI by its RIFF header, and so on). Through generate(), stream() and the CLI a file can be at most 100 MB. On the Gemini providers a larger file is refused with an error-level log line (File too large: ... (max: 100.00 MB), printed even without --debug) and the request goes on without it, and without any file listed after it in the same call: the model receives no ## Video File section at all.\nProcessing. VideoProcessor reads the duration, resolution, codecs, frame rate and bitrate, extracts timestamped keyframes with ffmpeg, extracts the first embedded subtitle track if there is one, and, only if you ask, transcribes the speech. The summary becomes a ## Video File section in the prompt text and the keyframes are attached as images, each labelled with the moment it was sampled.\nDelivery. If the provider can take video, the clip itself is added to the request as inline data, in addition to the summary and the keyframes. If it cannot, or the clip does not qualify, the model works from the summary and the keyframes alone.\n\nA video that cannot be processed at all (a corrupt file, say) does not fail the request: the model receives a short placeholder describing the attachment.\n\nNative Delivery to Gemini\n\nGemini accepts video inline, and takes the audio track with it. Sending the file itself needs no ffmpeg, so on a machine without it Gemini still receives the whole clip. Keyframes and the metadata summary are still attached when they could be produced.\n\nWhich providers\n\nThe capability table is VIDEO_PROVIDER_CONFIGS, keyed by provider name or alias:\n\n| Provider names                                                                                                                                                                                                  | Delivery       | Inline ceiling | Longest clip | Hears audio |\n| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | -------------- | ------------ | ----------- |\n| vertex, google-vertex, googlevertex, google-ai-studio, googleaistudio, google-ai, googleai, gemini                                                                                              | Native, inline | 14 MB          | 3600 s       | yes         |\n| openai, azure, anthropic, bedrock, amazon-bedrock, mistral, litellm, openrouter, openai-compatible, huggingface, sagemaker, deepseek, nvidia-nim, lmstudio, cerebras, sambanova | Keyframes      | none           | none         | no          |\n| ollama, llamacpp, llama-cpp                                                                                                                                                                               | Keyframes      | none           | none         | no          |\n\nTwo details matter:\nA Vertex request for a model whose id contains claude is routed to the Anthropic SDK, which never sees a video part, so it gets the keyframe row, not the Gemini one.\nA provider that is not in the table keeps the keyframe behavior. The table says nothing about whether such a provider understands the frames; getVideoProviderConfig() returns null for it, and supportsNativeVideo() reads that as false.\n\nWhat qualifies a clip\n\nAll of these must hold, checked per clip:\nSource. The clip must be a local file path or a Buffer (or an object with a buffer). A URL is fetched for keyframes and the metadata summary only; its bytes are not kept for delivery, so it never goes to Gemini as a clip. This case is not reported as a fallback: n","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"","lvl3":""}},
+{"objectID":"288c031674e78fd4fcd753b2d753bff8b085367a321fac3e9845113df6361a2d","title":"Video Analysis","url":"/docs/features/video-analysis#video-analysis","content":"Attach a video file to generate() or stream() and ask about it. What the model receives depends on the provider:\nGemini (Google AI Studio and Vertex AI) receives the clip itself, inline, with its audio track. It sees continuous motion and hears the speech.\nEvery other provider receives a metadata summary, timestamped keyframes as ordinary images, and any embedded subtitle track. It sees a handful of stills and hears nothing, unless you ask for a transcript with --transcribe-audio.\n\nVideo analysis suits screen recordings in particular: checking whether a workflow worked, finding the moment a UI went wrong, or comparing two recordings. The prompts in this guide ask for a logical audit of that kind. The audit style comes from your prompt; the SDK imposes no report format.\n\nFor the cross-modality overview (images, PDFs, audio, video) see the Multimodal guide.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Video Analysis","lvl3":""}},
+{"objectID":"32013276172f573a97c140859e883ea31ef3885c90d864843b49a46cd5f6f820","title":"Quick Start","url":"/docs/features/video-analysis#quick-start","content":"input.videoFiles and input.files are equivalent for video: videoFiles is folded into files before file detection runs. Each entry is a local path, a Buffer or a URL. Only a local path or a Buffer can be sent to Gemini as the clip itself; a URL is fetched for keyframes and metadata alone (see What qualifies a clip).","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Quick Start","lvl3":""}},
+{"objectID":"f9cd53c8cff1bc1a463d950ae0d3d406a2d29f14e7070d2f8d77d7b041fb2571","title":"How It Works","url":"/docs/features/video-analysis#how-it-works","content":"Detection. The file type is recognized from the file's leading bytes as well as its extension (MP4 and MOV by their ftyp box, AVI by its RIFF header, and so on). Through generate(), stream() and the CLI a file can be at most 100 MB. On the Gemini providers a larger file is refused with an error-level log line (File too large: ... (max: 100.00 MB), printed even without --debug) and the request goes on without it, and without any file listed after it in the same call: the model receives no ## Video File section at all.\nProcessing. VideoProcessor reads the duration, resolution, codecs, frame rate and bitrate, extracts timestamped keyframes with ffmpeg, extracts the first embedded subtitle track if there is one, and, only if you ask, transcribes the speech. The summary becomes a ## Video File section in the prompt text and the keyframes are attached as images, each labelled with the moment it was sampled.\nDelivery. If the provider can take video, the clip itself is added to the request as inline data, in addition to the summary and the keyframes. If it cannot, or the clip does not qualify, the model works from the summary and the keyframes alone.\n\nA video that cannot be processed at all (a corrupt file, say) does not fail the request: the model receives a short placeholder describing the attachment.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"How It Works","lvl3":""}},
+{"objectID":"fe0b44ea5907afc3c6d8a349458f77c0d250aadc1129d1e9b242741e2b9ce0e4","title":"Native Delivery to Gemini","url":"/docs/features/video-analysis#native-delivery-to-gemini","content":"Gemini accepts video inline, and takes the audio track with it. Sending the file itself needs no ffmpeg, so on a machine without it Gemini still receives the whole clip. Keyframes and the metadata summary are still attached when they could be produced.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Native Delivery to Gemini","lvl3":""}},
+{"objectID":"79ca07ae077018944112070e21dc654f2dd5d62d80786f32637a03b6a2ef43ae","title":"Which providers","url":"/docs/features/video-analysis#which-providers","content":"The capability table is VIDEO_PROVIDER_CONFIGS, keyed by provider name or alias:\n\n| Provider names                                                                                                                                                                                                  | Delivery       | Inline ceiling | Longest clip | Hears audio |\n| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | -------------- | ------------ | ----------- |\n| vertex, google-vertex, googlevertex, google-ai-studio, googleaistudio, google-ai, googleai, gemini                                                                                              | Native, inline | 14 MB          | 3600 s       | yes         |\n| openai, azure, anthropic, bedrock, amazon-bedrock, mistral, litellm, openrouter, openai-compatible, huggingface, sagemaker, deepseek, nvidia-nim, lmstudio, cerebras, sambanova | Keyframes      | none           | none         | no          |\n| ollama, llamacpp, llama-cpp                                                                                                                                                                               | Keyframes      | none           | none         | no          |\n\nTwo details matter:\nA Vertex request for a model whose id contains claude is routed to the Anthropic SDK, which never sees a video part, so it gets the keyframe row, not the Gemini one.\nA provider that is not in the table keeps the keyframe behavior. The table says nothing about whether such a provider understands the frames; getVideoProviderConfig() returns null for it, and supportsNativeVideo() reads that as false.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Which providers","lvl3":""}},
+{"objectID":"25dce466026a630874944fc947a50a15ed5ea154014a5c239558963a35774f08","title":"What qualifies a clip","url":"/docs/features/video-analysis#what-qualifies-a-clip","content":"All of these must hold, checked per clip:\nSource. The clip must be a local file path or a Buffer (or an object with a buffer). A URL is fetched for keyframes and the metadata summary only; its bytes are not kept for delivery, so it never goes to Gemini as a clip. This case is not reported as a fallback: no warning is logged and no reason is given, so a URL entry that you expected to be heard simply arrives as keyframes. Download the file first to send it whole.\nContainer. Gemini reads these inline: video/mp4, video/mpeg, video/mpg, video/mov, video/quicktime, video/avi, video/x-msvideo, video/x-flv, video/webm, video/wmv, video/x-ms-wmv and video/3gpp. Anything else, MKV for example, falls back to keyframes. There is no transcode step: re-encoding a long recording inside a generation request costs minutes of CPU, for a payload that would usually breach the ceiling anyway.\nSize. The 14 MB ceiling is one budget for the whole request, counted in source bytes, not a per-clip limit. Before a clip ships, its bytes are added to the source bytes of every inline part already in the request, which means images (the keyframes among them), PDFs, audio and any clip accepted earlier. With nothing else attached that is the clip's own size.\nDuration. At most 3600 s. A clip whose duration could not be measured is not refused for it.\n\nWhy 14 MB when Gemini documents 20 MB: inline data travels as base64 in a JSON body, which costs 4 bytes per 3, and the 20 MB limit covers the whole request, prompt and system instructions included. 14 MB of source encodes to about 18.7 MB, leaving headroom. 15 MB would encode to exactly 20.0 MB and leave none.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"What qualifies a clip","lvl3":""}},
+{"objectID":"1a45a378b4628cb81c1a44566872ed9885ee0fd469a573fc6e54075d2801ddf1","title":"Falling back to keyframes","url":"/docs/features/video-analysis#falling-back-to-keyframes","content":"A clip that does not qualify is not an error. It is left out of the request, the model works from the summary and keyframes, and a warning names the reason. The one exception is a URL source (see above), which is silent. Warnings are only printed in debug mode (--debug, or NEUROLINK_DEBUG=true for the SDK), so run with one of those to see it:\n\nThe keyframes already in the request count against the budget, which is why that line speaks of the request's inline attachments. The reasons you can see are:\n<provider> does not accept video attachments\n<mime type> is not a container <provider> reads inline\nthe clip is X MB, over the 14MB inline ceiling for <provider>, or, when other inline parts already used part of the budget, the clip is X MB, which would bring this request's inline attachments to Y MB, over the 14MB inline ceiling shared by the whole request for <provider>\nthe clip runs N s, over the 3600s ceiling for <provider>\n\nA fallback loses the clip's audio, because keyframes carry none. If the speech matters, add --transcribe-audio (see Audio transcription) or shorten or compress the clip below the ceiling.\n\nClips above 14 MB are not uploaded through the Gemini Files API. That is a separate piece of work that has not been done, so the ceiling is a hard edge today.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Falling back to keyframes","lvl3":""}},
+{"objectID":"6df98e9bb24ba4251ea16a628607c8399b9117eca11629db7561d3f3e95d27c9","title":"Asking before you send","url":"/docs/features/video-analysis#asking-before-you-send","content":"The capability table is exported, so a caller can check a provider and a clip without making a request:\n\nestimateVideoTokens prices a native clip at 260 tokens per second, a keyframe at 250 tokens, and a minute of transcribed speech at 400. It is a budgeting estimate, not a quote.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Asking before you send","lvl3":""}},
+{"objectID":"f49e30185c2254b2d9f8d13f42cd434dea85d76d636375b71c6f228acef8d624","title":"Keyframes","url":"/docs/features/video-analysis#keyframes","content":"Every other provider, and Gemini when a clip falls back, works from keyframes.\nCadence. Without an explicit frames, the interval between frames follows the clip's duration:\n\n  | Clip length  | One frame every |\n  | ------------ | --------------- |\n  | up to 10 s   | 1 s             |\n  | up to 30 s   | 2 s             |\n  | up to 2 min  | 3 s             |\n  | up to 10 min | 6 s             |\n  | up to 30 min | 20 s            |\n  | longer       | 60 s            |\n\n  The total is capped at 100 frames; if a tier would produce more, the interval widens so the frames still cover the whole clip. The count does not depend on the provider. The recommendedFrameCount in the capability table (16, 8 or 4) is only what estimateVideoTokens assumes when you give it no frameCount; it does not set how many frames are extracted.\nSize. Frames are scaled to fit within 768 by 768 pixels, keeping the aspect ratio and never enlarging, then encoded as JPEG at quality 80 unless you set quality or format.\nTimestamps. Each frame carries the time ffmpeg actually sampled it at, in its image label and in a ## Keyframes list in the prompt, so \"what happens at 0:30?\" has an answer.\nSubtitles. The first embedded subtitle track is extracted as plain text and added under ## Subtitles / Captions, up to 10,000 characters. It is included whether or not you transcribe.\nNo ffmpeg, no frames. Frame extraction shells out to ffmpeg, which comes from the optional ffmpeg-static dependency or your PATH. Without it a keyframe provider receives the metadata summary and nothing visual, and no error is raised. Native Gemini delivery is unaffected.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Keyframes","lvl3":""}},
+{"objectID":"81795290c8f5734a415c1fe2aca479f392ed86267b697362eed7ac0380926363","title":"CLI Flags and videoOptions","url":"/docs/features/video-analysis#cli-flags-and-videooptions","content":"The flags are available on generate, stream and batch. --video attaches the clip (input.videoFiles). The other four flags each map to one field of the videoOptions object that generate() and stream() accept (type VideoProcessorOptions).\n\n| CLI flag                  | SDK field                      | Default                               | Meaning                                                                                                                                                                                                    |\n| ------------------------- | ------------------------------ | ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| --video <path>          | input.videoFiles             | none                                  | Attach a video; repeatable. --file <path> works too, but is not available on batch, where it would collide with the prompts-file positional.                                                           |\n| --video-frames <n>      | videoOptions.frames          | chosen from the duration, at most 100 | Keyframe budget. An explicit n is spread evenly across the whole clip and clamped to 100; zero, negative or non-numeric values are ignored. It is a ceiling: a clip with sparse frames can return fewer. |\n| --video-quality \\<1-100> | videoOptions.quality         | 80                                    | Encoder quality for keyframes. Rounded and clamped to 1 to 100.                                                                                                                                            |\n| --video-format <format> | videoOptions.format          | jpeg                                | Keyframe encoding: jpeg or png. On Google AI Studio (google-ai) use jpeg: its PNG keyframes are currently sent labelled image/jpeg.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"CLI Flags and videoOptions","lvl3":""}},
+{"objectID":"5c0b40920615303139197392d319ed23814495503c8c26a9ce62a5e57fa0fa83","title":"Audio Transcription","url":"/docs/features/video-analysis#audio-transcription","content":"Gemini hears the audio track of a clip it receives natively, so it does not need a transcript. Transcription is for the others: a recorded standup or a narrated demo reaches a keyframe provider with its entire spoken content missing, and \"summarize this meeting\" would be answered from a few screenshots. It also helps when a Gemini clip falls back to keyframes.\n\nSet --transcribe-audio, or videoOptions.transcribeAudio: true. It is off by default because it costs an ffmpeg pass plus a speech-to-text call, and most attached video is silent screen capture. The flag does not look at the provider: on Gemini, with the clip delivered natively, it still spends the ffmpeg pass and the speech-to-text call and adds the transcript to the prompt next to the audio Gemini already hears. Leave it off there.\n\nWhat happens. ffmpeg drops the video stream and writes the audio as mono 16 kHz MP3 (16-bit PCM WAV for an engine that does not read MP3). That goes to the speech-to-text engine, and the result is added to the prompt under ## Spoken Audio (transcribed), truncated at 20,000 characters. It is labelled as a machine transcription and kept apart from ## Subtitles / Captions, so the model can tell the two apart when both exist.\n\nWhich engine. The one named by NEUROLINK_STT_PROVIDER; otherwise the first configured hosted engine in this order: OpenAI-compatible Whisper (OPENAI_STT_API_KEY or OPENAI_API_KEY), Deepgram, ElevenLabs, Google Speech-to-Text, Azure Speech; otherwise the built-in local engine. Video uses the same engine selection as audio files and neurolink.transcribe(). See the transcribe guide for configuration.\n\nLimits. The extracted audio must be 25 MB or less. Audio extraction times out after 180 s and one transcription request after 120 s.\n\nFailure is not fatal. Transcription is best effort.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Audio Transcription","lvl3":""}},
 {"objectID":"5e168624a526526bde730f1fa41b194f50f69477e008053ecfeb9478b1bd8ee1","title":"CLI Usage","url":"/docs/features/video-analysis#cli-usage","content":"Analyze any video file with a natural language prompt.\n\n`bash","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"CLI Usage","lvl3":""}},
-{"objectID":"2beb7cdd65aad32b4627b566f1ad0158c97dc642e7c2b70a538b24bc9524ab65","title":"Basic video analysis","url":"/docs/features/video-analysis#basic-video-analysis","content":"neurolink generate \"Analyze the login workflow in this video\" \\\n  --file ./recordings/screen-capture.mp4 \\\n  --provider vertex \\\n  --model gemini-2.0-flash\n`","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Basic video analysis","lvl3":""}},
+{"objectID":"2beb7cdd65aad32b4627b566f1ad0158c97dc642e7c2b70a538b24bc9524ab65","title":"Basic video analysis","url":"/docs/features/video-analysis#basic-video-analysis","content":"neurolink generate \"Analyze the login workflow in this video\" \\\n  --video ./recordings/screen-capture.mp4 \\\n  --provider vertex \\\n  --model gemini-2.5-flash\n`","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Basic video analysis","lvl3":""}},
 {"objectID":"5bbcc85cacab5d8285b1d0dd3b7f9c8e8f87e2688789a71286b15b71f5dc0523","title":"SDK Usage","url":"/docs/features/video-analysis#sdk-usage","content":"Integrate video analysis into your TypeScript/JavaScript projects.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"SDK Usage","lvl3":""}},
-{"objectID":"477f1971b2ed446069626a085bb52d457acd8ce6de6d2d7eae3ded9f705f2396","title":"Advanced SDK Examples","url":"/docs/features/video-analysis#advanced-sdk-examples","content":"Custom Model Configuration\nFine-tune the analysis by adjusting token limits and temperature.\n\nDisabling Tool Interference\nBy default, the model might try to use available tools. For pure video analysis, you can disable them.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Advanced SDK Examples","lvl3":""}},
+{"objectID":"477f1971b2ed446069626a085bb52d457acd8ce6de6d2d7eae3ded9f705f2396","title":"Advanced SDK Examples","url":"/docs/features/video-analysis#advanced-sdk-examples","content":"Custom Model Configuration\nFine-tune the analysis by adjusting token limits and temperature.\n\nDisabling Tool Interference\nBy default, the model might try to use available tools. For pure video analysis, you can disable them.\n\nA provider without video support, with speech\nSend keyframes and a transcript to a provider that cannot take a clip.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Advanced SDK Examples","lvl3":""}},
 {"objectID":"e8da7529d5ab85823deea2db05394f31e5ecf862d0a6323227b0ee768b1cff21","title":"1. UI/UX Bug Analysis","url":"/docs/features/video-analysis#1-uiux-bug-analysis","content":"Identify why a user is unable to complete a form or where the interface is misleading.\n\nPrompt: \"Find why the user is getting stuck at the payment step. Look for validation errors or hidden UI elements.\"","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"1. UI/UX Bug Analysis","lvl3":""}},
 {"objectID":"0c9601bbe95e54b4a3aad6f81fe3f51b79c276a3a5538f60a95f6185163665ca","title":"2. Silent Failure Detection","url":"/docs/features/video-analysis#2-silent-failure-detection","content":"Detect cases where an action is taken but the system provides no feedback (no loaders, no success messages).\n\nPrompt: \"Audit the 'Submit' button click. Is there a visual 'bond' between the click and the next state? Report any lag or missing loading indicators.\"","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"2. Silent Failure Detection","lvl3":""}},
 {"objectID":"b1cd293e3e616fd973c2b9a32d377f5e52c83efe91780479a771578461500d52","title":"3. Workflow Validation","url":"/docs/features/video-analysis#3-workflow-validation","content":"Verify if a complex multi-step process follows the intended business logic.\n\nPrompt: \"Trace the logical progression from 'Item Selection' to 'Checkout'. Does every state change correspond to a user action?\"","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"3. Workflow Validation","lvl3":""}},
 {"objectID":"4d6474550ede4e74baaa46e74cf53f70f1f7f8835faadd163bb74bc0133371e5","title":"4. Comparison Analysis","url":"/docs/features/video-analysis#4-comparison-analysis","content":"Compare two recordings to find discrepancies in behavior.\n\nPrompt: \"Compare these two clips. The first one is the expected behavior and the second one has a bug. Identify the exact frame or timestamp where the logic deviates.\"","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"4. Comparison Analysis","lvl3":""}},
-{"objectID":"7db25ca977eed8a31bb24ee361f45e0a2f0f4b9644d011da50b6e3d5d31174e7","title":"Command Gallery","url":"/docs/features/video-analysis#command-gallery","content":"Quick CLI recipes for common tasks:\n\n`bash","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Command Gallery","lvl3":""}},
-{"objectID":"109bff6e17ad4d0552e775c0bdeccfa59307355392abbd7e9c0b8aa4805cae06","title":"Debugging with full technical detail","url":"/docs/features/video-analysis#debugging-with-full-technical-detail","content":"neurolink generate \"Audit this video\" --file bug.mp4 --debug","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Debugging with full technical detail","lvl3":""}},
-{"objectID":"b294dca05cfd4f9265805dc2ba9bc1651631379fe6f851cf847df918e3f446f9","title":"Using a specifically tuned model","url":"/docs/features/video-analysis#using-a-specifically-tuned-model","content":"neurolink generate \"Analyze logic\" --file demo.mov --model gemini-2.0-flash","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Using a specifically tuned model","lvl3":""}},
-{"objectID":"e8f28fab18b5c5d56913e4eb89da371dc583211559e0683819e32f3ce3c5f110","title":"Forcing a specific provider","url":"/docs/features/video-analysis#forcing-a-specific-provider","content":"neurolink generate \"Extract patterns\" --file test.mp4 --provider vertex\n`","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Forcing a specific provider","lvl3":""}},
-{"objectID":"b5c761aaea72621d94a6f8ab469ae99b8612b622b83277f0a725b3bb82bbf1d1","title":"The Analysis Report","url":"/docs/features/video-analysis#the-analysis-report","content":"The output is structured into four major sections designed to give you a complete understanding of the video:\nStrategic Overview & Intent: Defines the core activity, expected logic, and provides a primary verdict.\nThe Action-Reaction Chain: A granular, step-by-step audit of attempts, results, and technical inferences.\nCritical Findings: Categorized milestones or anomalies with root cause analysis and visual evidence in JSON.\nFinal Assessment: A conclusive summary of the logical flow based on the observed evidence.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"The Analysis Report","lvl3":""}},
-{"objectID":"1f2bd84873465739c002846fe135f629903002f80a472a77931ccf676831a4dd","title":"Best Practices","url":"/docs/features/video-analysis#best-practices","content":"Frame Depth: Short videos (under 10s) get high-density frame coverage (1 per second), while long ones are intelligently sampled.\nPrompt Precision: While the model is a \"Critical Logic Auditor,\" you can guide it with specific questions about the activity.\nFormat: The analysis is returned as text in result.content, making it easy to store, display, or pipe to other tools.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Best Practices","lvl3":""}},
+{"objectID":"7db25ca977eed8a31bb24ee361f45e0a2f0f4b9644d011da50b6e3d5d31174e7","title":"Command Gallery","url":"/docs/features/video-analysis#command-gallery","content":"Quick CLI recipes for common tasks.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Command Gallery","lvl3":""}},
+{"objectID":"7a9101f88fa3f594b059acbcc51b6bb30449a055ff5bc9a387ad514f4e5cf85e","title":"Debug delivery and fallback decisions","url":"/docs/features/video-analysis#debug-delivery-and-fallback-decisions","content":"Full technical detail, including whether the clip or keyframes were sent and why:","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Debug delivery and fallback decisions","lvl3":""}},
+{"objectID":"1f2bd84873465739c002846fe135f629903002f80a472a77931ccf676831a4dd","title":"Best Practices","url":"/docs/features/video-analysis#best-practices","content":"Prefer Gemini for anything with sound or fast motion. Only the native path sees what happens between frames and hears the audio.\nKeep clips under the ceiling. A recording that is a few megabytes at most travels whole; a longer one falls back to keyframes. Trimming to the relevant stretch, or re-encoding at a lower bitrate, keeps it on the native path. Remember the budget is shared with every other inline attachment in the same request.\nFrame depth. On the keyframe path, short videos (under 10 s) get one frame per second and long ones are sampled more sparsely. Set --video-frames to spend your frame budget evenly across a clip you care about.\nAdd a transcript only where it helps. --transcribe-audio is for providers that cannot hear the clip. It is not provider-aware, so on Gemini it only adds a redundant transcript and a speech-to-text call.\nPrompt precision. Guide the model with specific questions about the activity, and ask for the structure you want (a timeline, an action-reaction table, a verdict) in the prompt.\nFormat. The analysis is returned as text in result.content, making it easy to store, display, or pipe to other tools.","hierarchy":{"lvl0":"Features","lvl1":"Video Analysis","lvl2":"Best Practices","lvl3":""}},
 {"objectID":"0af2cd2e57086d14cba79ae945f4f934c12cace5d06368f2d6846db441d467a8","title":"Video Director Mode – Multi-Clip Generation & Merging","url":"/docs/features/video-director-mode","content":"Video Director Mode\n\nDirector Mode extends NeuroLink's video generation capability to produce multi-segment videos with seamless AI-generated transitions. Instead of a single clip, you define an array of segments — each with its own prompt and image — and NeuroLink orchestrates the full pipeline: generating each clip, extracting boundary frames, producing transition videos (with individually configurable durations) using Veo 3.1's first-and-last-frame interpolation, and merging everything into one continuous video.\n\nOverview\n\nDirector Mode is triggered automatically when you supply an input.segments array to the video generation API. Each segment is a self-documenting { prompt, image } object, mapping cleanly to the pipeline concept of ordered video segments.\n\nHow It Works\nParallel clip generation – All main clips are generated concurrently (fixed concurrency of 2) via Veo 3.1's image-to-video endpoint, with a circuit breaker that trips after 2 consecutive failures to avoid wasted API calls\nFrame extraction – The last frame of clip N and first frame of clip N+1 are extracted from generated video buffers (with MP4 ftyp header validation)\nParallel transition generation – Veo 3.1 Fast's first-and-last-frame interpolation API generates transitions between each pair of adjacent clips in parallel (same concurrency limit), with individually configurable duration (4, 6, or 8 seconds each)\nSequential merge – Clips and transitions are concatenated: Clip₁ → Trans₁₋₂ → Clip₂ → Trans₂₋₃ → Clip₃ → …\nSingle output – The merged result is returned as one VideoGenerationResult buffer\n\nKey Technology: Veo First-and-Last-Frame Interpolation\n\nThe transition clips use Veo 3.1's native lastFrame parameter in the predictLongRunning API. Instead of generating from a single image, you provide two images — the first frame and the last frame — and Veo generates a video that smoothly interpolates between them:\n\nThis produces a physically coherent, AI-generated morph — far superior to simple crossfade or dissolve effects. The durationSeconds value is set independently for each transition (from the transitionDurations array), allowing shorter or longer interpolations per segment boundary.\n\nWhat You Get\nMulti-segment video – Chain any number of video segments into a single continuous output\nAI transitions – Per-transition configurable duration (4, 6, or 8 seconds each) generated by Veo 3.1 frame interpolation (not simple crossfades)\nParallel generation – Both main clips and transitions are generated concurrently (fixed concurrency of 2) with a circuit breaker for clip failures\nMixed image inputs – Each segment's image field accepts a Buffer, file path, URL, or ImageWithAltText\nConsistent settings – Resolution, aspect ratio, and audio settings apply uniformly across all segments and transitions\nPer-segment customization – Each segment is a self-contained { prompt, image } object\nBuffer validation – All video buffers are validated for MP4 ftyp headers before frame extraction and merging\nSDK only – Use programmatically via generate() (CLI not supported for Director Mode)\n\nSupported Provider & Model\n\n| Provider | Model                                            | Interpolation Support | Transition Duration | Max Segments | Concurrency |\n| -------- | ------------------------------------------------ | --------------------- | ------------------- | ------------ | ----------- |\n| vertex | veo-3.1 (clips) / veo-3.1-fast (transitions) | First + Last Frame    | 4-8s per transition | 10           | 2 (fixed)   |\n\nNote: The lastFrame parameter is supported by veo-2.0-generate-001, veo-3.1-generate-001, and veo-3.1-fast-generate-001. NeuroLink uses veo-3.1-generate-001 for main clips and veo-3.1-fast-generate-001 for transition clips (faster generation with minimal quality difference for short interpolations).\n\nPrerequisites\n\nSame as Video Generation prerequisites, plus:\nSufficient quota – Director Mode generates N + (N-1) video operations (N clips + N-1 transitions). Ensure your Vertex AI project has adequate quota.\nAdequate timeout – Multi-segment generation takes proportionally longer. Set timeout accordingly (recommended: 5-10 minutes for 3+ segments).\n\nQuick Start\n\nSDK Usage\n\nUsing Image URLs\n\nMixed Input Types\n\nEach segment's image field accepts a Buffer, file path, URL, or ImageWithAltText:\n\nNote: Director Mode is SDK-only. CLI support is not available for this generation type. Use the standard --outputMode video CLI flags for single-clip video generation.\n\nComprehensive Examples\n\nExample 1: Product Commercial (3 Segments)\n\nExample 2: Social Media Story (Portrait, 4 Segments)\n\nExample 3: AI-Driven Storyboard\n\nExample 4: Batch Director Mode\n\nExample 5: Error Handling in Director Mode\n\n⚠️ Full-job retry warning: The generateWithRetry function below retries the entire neurolink.generate() call on any retriable VideoError. This means all segments and transitions are re-generated from scratch, incurring full cost each attempt ($10-60+ depending on set","hierarchy":{"lvl0":"Features","lvl1":"Video Director Mode – Multi-Clip Generation & Merging","lvl2":"","lvl3":""}},
 {"objectID":"9dd83315c72382cd60863330c64a615aec84cadf693d7751bf8676806e359c94","title":"Video Director Mode","url":"/docs/features/video-director-mode#video-director-mode","content":"Director Mode extends NeuroLink's video generation capability to produce multi-segment videos with seamless AI-generated transitions. Instead of a single clip, you define an array of segments — each with its own prompt and image — and NeuroLink orchestrates the full pipeline: generating each clip, extracting boundary frames, producing transition videos (with individually configurable durations) using Veo 3.1's first-and-last-frame interpolation, and merging everything into one continuous video.","hierarchy":{"lvl0":"Features","lvl1":"Video Director Mode – Multi-Clip Generation & Merging","lvl2":"Video Director Mode","lvl3":""}},
 {"objectID":"df7c4473d5ef1434c82a40e880da4d6ab470fcd00a0383199ed8c08fdca6e739","title":"Overview","url":"/docs/features/video-director-mode#overview","content":"Director Mode is triggered automatically when you supply an input.segments array to the video generation API. Each segment is a self-documenting { prompt, image } object, mapping cleanly to the pipeline concept of ordered video segments.","hierarchy":{"lvl0":"Features","lvl1":"Video Director Mode – Multi-Clip Generation & Merging","lvl2":"Overview","lvl3":""}},

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@juspay/neurolink",
-  "version": "13.4.2",
+  "version": "13.4.3",
   "packageManager": "pnpm@10.15.1",
   "description": "The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002/call) for routing, tool selection and context budgeting. MCP-native (4 transports), voice TTS/STT/realtime, RAG, agents, memory, compaction, 9 observability exporters. OpenAI · Anthropic · Gemini · Bedrock · Azure · Ollama · TypeSafe Jev and more.",
   "author": {