@ash_j1997/ffmpeg-mcp-server 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ko.md +129 -0
- package/README.md +18 -7
- package/dist/index.js +53 -1
- package/dist/tools/burn_subtitles.js +5 -3
- package/dist/tools/convert.js +1 -0
- package/dist/tools/detect_scenes.js +132 -0
- package/dist/tools/mux.js +102 -0
- package/dist/tools/reformat.js +1 -0
- package/dist/tools/subtitles.js +12 -2
- package/dist/tools/thumbnail.js +6 -1
- package/dist/tools/trim.js +14 -3
- package/dist/tools/waveform.js +6 -1
- package/dist/utils/hwaccel.js +54 -23
- package/package.json +1 -1
package/README.ko.md
ADDED
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
# ffmpeg-mcp-server
|
|
2
|
+
|
|
3
|
+
[English](README.md) | [한국어](README.ko.md)
|
|
4
|
+
|
|
5
|
+
[](https://github.com/Ashutos1997/ffmpeg-mcp-server/actions) [](https://github.com/Ashutos1997/ffmpeg-mcp-server/releases) [](LICENSE) [](https://www.npmjs.com/package/@ash_j1997/ffmpeg-mcp-server) [](https://www.npmjs.com/package/@ash_j1997/ffmpeg-mcp-server) [](https://www.npmjs.com/package/@ash_j1997/ffmpeg-mcp-server) [](https://glama.ai/mcp/servers/@Ashutos1997/ffmpeg-mcp-server) [](https://mcprush.com/to/ashutos1997-ffmpeg-mcp?ref=badge)
|
|
6
|
+
|
|
7
|
+
AI 에이전트와 도구가 로컬에서 비디오 및 오디오를 처리할 수 있도록 FFmpeg 및 FFprobe 기능을 제공하는 Model Context Protocol (MCP) 서버입니다.
|
|
8
|
+
|
|
9
|
+
크로스 플랫폼 GPU 하드웨어 가속(Apple Silicon VideoToolbox, NVIDIA NVENC, Intel QuickSync, AMD AMF), MCP 진행률 토큰을 통한 실시간 진행 상황 보고, `AbortSignal` 기반의 즉각적인 작업 취소, 멀티모달 비전 모델을 위한 이미지 프리뷰 인라인 반환을 완벽하게 지원합니다.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## 개발 배경 (Why This Exists)
|
|
14
|
+
|
|
15
|
+
AI 에이전트에게 영상 처리를 직접 명령하면 잘못된 플래그, 문법 오류, 지원되지 않는 픽셀 포맷 등을 만들어내어 작업이 실패하기 쉽습니다.
|
|
16
|
+
|
|
17
|
+
본 서버는 Zod 스키마로 정의된 **19가지 타입 안전한 도구(Typed Tools)**를 제공합니다. AI가 `{ inputPath: "video.mov", preset: "web_mp4" }`와 같은 구조화된 매개변수를 전달하면, 서버가 유효한 FFmpeg 명령어를 로컬에서 직접 실행하며 안정적인 에러 처리, 하드웨어 가속 및 진행률 알림을 수행합니다.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## 주요 기능 (Key Features)
|
|
22
|
+
|
|
23
|
+
* **범용 하드웨어 가속:** macOS(VideoToolbox), Linux/Windows GPU(NVIDIA NVENC, Intel QuickSync, AMD AMF, VA-API)를 자동 감지하여 낮은 CPU 점유율로 5배~20배 빠른 인코딩을 지원합니다.
|
|
24
|
+
* **실시간 진행률 보고:** 표준 MCP 진행률 토큰(`notifications/progress`)을 통해 실시간 백분율(%)을 전송하여 클라이언트 UI에 진행 바를 표시합니다.
|
|
25
|
+
* **즉각적인 작업 취소:** 클라이언트의 `AbortSignal` 이벤트를 감지하여 사용자가 생성을 중단하면 실행 중인 하위 FFmpeg 프로세스를 즉시 종료합니다.
|
|
26
|
+
* **멀티모달 비전 모델용 프리뷰:** 썸네일, 오디오 파형, 멀티 프레임 콘택트 시트를 base64 이미지 데이터로 인라인 반환하여 Claude 3.7, GPT-4o, Gemini 등의 비전 모델이 미디어를 직접 "볼" 수 있습니다.
|
|
27
|
+
* **숏폼/소셜 포맷 자동화:** 가로 영상을 배경 블러 처리된 9:16 세로 숏폼 영상으로 변환하고, 스타일이 적용된 자막을 하드코딩(버닝)하며, 틱톡/릴스용 상하 분할 스플릿 스크린 영상을 제작합니다.
|
|
28
|
+
* **오디오 엔지니어링:** 스트리밍/방송 표준(EBU R128 / -16 LUFS)에 맞춰 음량을 정규화하고, 80Hz 럼블 제거 필터로 음성을 선명하게 보정하며, 립싱크를 유지하면서 무음 구간을 자동으로 잘라냅니다.
|
|
29
|
+
* **장면 감지 및 스트림 결합:** 시각적 장면 전환(컷 편집 지점)을 감지하여 타임스탬프와 콘택트 시트를 반환하고, 무손실 스트림 복사로 비디오와 오디오를 즉시 결합/교체합니다.
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## 제공 도구 목록 (총 19개)
|
|
34
|
+
|
|
35
|
+
| 도구명 | 설명 | 주요 매개변수 |
|
|
36
|
+
| :--- | :--- | :--- |
|
|
37
|
+
| **`media_probe`** | ffprobe로 미디어 파일을 검사하여 메타데이터, 스트림, 코덱, 해상도, 재생 시간을 JSON으로 반환합니다. | `inputPath` |
|
|
38
|
+
| **`media_convert`** | 비디오 및 오디오 포맷을 변환합니다. 웹용 MP4, WebM, 애니메이션 GIF 프리셋 및 GPU 가속을 지원합니다. | `inputPath`, `outputPath`, `preset`, `crf`, `resolution`, `fps`, `hardwareAcceleration` |
|
|
39
|
+
| **`trim_media`** | 스트림 복사(무손실/초고속) 또는 재인코딩 방식으로 비디오/오디오의 특정 구간을 자릅니다. | `inputPath`, `outputPath`, `startTime`, `endTime`, `duration`, `accurateReencode` |
|
|
40
|
+
| **`extract_audio`** | 비디오에서 오디오 트랙을 추출합니다. Whisper 음성 인식에 최적화된 16kHz 모노 WAV 프리셋을 포함합니다. | `inputPath`, `outputPath`, `format`, `bitrate` |
|
|
41
|
+
| **`extract_thumbnail`** | 특정 타임스탬프 또는 중간 지점의 스틸 컷을 캡처합니다. 비전 모델을 위해 base64 이미지 데이터를 함께 반환할 수 있습니다. | `inputPath`, `outputPath`, `timestamp`, `resolution`, `includeBase64` |
|
|
42
|
+
| **`compress_video`** | 대상 크기(예: GitHub PR용 10MB 미만, Discord용 25MB 미만)에 맞게 비디오를 압축합니다. | `inputPath`, `outputPath`, `targetSizeMB`, `preset`, `crf`, `maxWidth`, `hardwareAcceleration` |
|
|
43
|
+
| **`reformat_aspect_ratio`** | 배경 블러 패딩 또는 중앙 크롭을 적용해 영상 비율을 9:16(세로), 1:1(정사각형), 4:5(포트레이트)로 변환합니다. | `inputPath`, `outputPath`, `targetRatio`, `mode`, `blurAmount`, `padColor`, `hardwareAcceleration` |
|
|
44
|
+
| **`create_split_screen`** | 두 개의 영상을 상하(9:16 숏폼/릴스) 또는 좌우로 배치하고 오디오 믹싱 및 자막을 합성합니다. | `firstVideoPath`, `secondVideoPath`, `outputPath`, `layout`, `audioSource`, `subtitlesPath` |
|
|
45
|
+
| **`trim_silence`** | 영상과 음성의 립싱크를 완벽히 유지하면서 녹음/녹화 영상의 무음 구간(침묵)을 감지해 자동으로 잘라냅니다. | `inputPath`, `outputPath`, `mode`, `thresholdDb`, `minDurationSeconds` |
|
|
46
|
+
| **`concat_media`** | 여러 비디오 또는 오디오 파일을 순서대로 병합합니다. 스트림 복사 및 재인코딩 방식을 지원합니다. | `inputPaths`, `outputPath`, `mode` |
|
|
47
|
+
| **`generate_waveform`** | 오디오 파형(Waveform) PNG 이미지를 생성합니다. 비전 모델을 위한 base64 데이터를 반환할 수 있습니다. | `inputPath`, `outputPath`, `width`, `height`, `color`, `scale`, `includeBase64` |
|
|
48
|
+
| **`add_subtitles`** | 비디오 재인코딩 없이 SRT 또는 VTT 자막 트랙을 MP4/MKV 파일에 소프트웨어 자막으로 삽입합니다. | `videoPath`, `subtitlesPath`, `outputPath`, `language`, `title` |
|
|
49
|
+
| **`burn_subtitles`** | 폰트 크기, 글자 색상, 외곽선, 여백을 커스텀하여 비디오 프레임에 자막을 직접 하드코딩(버닝)합니다. | `videoPath`, `subtitlesPath`, `outputPath`, `fontSize`, `fontColor`, `outlineColor`, `marginV` |
|
|
50
|
+
| **`extract_keyframes`** | 영상 전반에 걸쳐 대표 키프레임을 추출하고 비전 AI가 한눈에 볼 수 있는 그리드형 콘택트 시트를 생성합니다. | `inputPath`, `outputPath`, `count`, `layout`, `includeBase64` |
|
|
51
|
+
| **`normalize_audio`** | EBU R128 표준(기본 -16 LUFS)을 적용해 오디오 음량을 균일하게 정규화하고 80Hz 고역 통과 필터로 음성을 선명하게 다듬습니다. | `inputPath`, `outputPath`, `targetLufs`, `targetPeakDb`, `voiceClarity` |
|
|
52
|
+
| **`add_overlay`** | 워터마크 이미지나 로고 PNG를 영상 모서리에 오버레이하며 위치, 여백, 크기 비율, 투명도를 조절할 수 있습니다. | `videoPath`, `overlayPath`, `outputPath`, `position`, `padding`, `scaleWidth`, `opacity` |
|
|
53
|
+
| **`change_speed`** | 피치(음높이) 왜곡 없이 재생 속도를 배속(2배속 고속, 0.5배속 슬로우 모션 등)으로 조절합니다. | `inputPath`, `outputPath`, `speed`, `muteAudio` |
|
|
54
|
+
| **`mux_media`** *(신규)* | 비디오와 오디오 스트림을 결합하거나 교체(더빙/음악 삽입)합니다. 오디오 지연 오프셋 및 무손실 스트림 복사를 지원합니다. | `videoPath`, `audioPath`, `outputPath`, `mode`, `audioOffsetSeconds`, `streamCopyVideo`, `shortest` |
|
|
55
|
+
| **`detect_scenes`** *(신규)* | 비디오의 장면 전환 및 컷 편집 지점을 감지합니다. 타임스탬프, 구간별 재생 시간, 콘택트 시트 프리뷰를 JSON 및 이미지로 반환합니다. | `inputPath`, `threshold`, `minDurationSeconds`, `outputPath`, `includeBase64` |
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## MCP 리소스 및 프롬프트 (Resources & Prompts)
|
|
60
|
+
|
|
61
|
+
### 리소스 (Resources)
|
|
62
|
+
* **`media://capabilities`**: 로컬 FFmpeg 설치 상태, 버전, 감지된 하드웨어 가속기(VideoToolbox, NVENC, QSV, AMF, VAAPI) 및 지원 비디오 인코더를 조회합니다.
|
|
63
|
+
* **`media://presets`**: 소셜 미디어 종횡비, 플랫폼별 업로드 용량 제한, 방송 음량 타깃 기준표를 제공합니다.
|
|
64
|
+
|
|
65
|
+
### 프롬프트 (Prompts)
|
|
66
|
+
* **`social_clip_generator`**: 원본 가로 영상을 블러 패딩, 무음 구간 편집, 자막 버닝이 적용된 세로형 숏폼 클립으로 변환하는 인터랙티브 워크플로우입니다.
|
|
67
|
+
* **`whisper_audio_prep`**: Whisper 음성 인식 모델에 최적화된 16kHz 모노 WAV 오디오 추출 및 음량 정규화 파이프라인입니다.
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## 요구 사항 (Requirements)
|
|
72
|
+
|
|
73
|
+
* **Node.js** >= 18.0.0
|
|
74
|
+
* 로컬에 **FFmpeg & FFprobe** 설치 필요:
|
|
75
|
+
* **macOS:** `brew install ffmpeg`
|
|
76
|
+
* **Linux (Ubuntu/Debian):** `sudo apt update && sudo apt install ffmpeg libass-dev`
|
|
77
|
+
* **Windows:** `winget install Gyan.FFmpeg` 또는 `choco install ffmpeg`
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## 설치 및 설정 (Installation)
|
|
82
|
+
|
|
83
|
+
### 빠른 시작 (Claude Desktop, Cursor, Antigravity)
|
|
84
|
+
|
|
85
|
+
별도의 저장소 클론이나 빌드 없이 `npx`로 바로 실행할 수 있습니다:
|
|
86
|
+
|
|
87
|
+
```json
|
|
88
|
+
{
|
|
89
|
+
"mcpServers": {
|
|
90
|
+
"ffmpeg": {
|
|
91
|
+
"command": "npx",
|
|
92
|
+
"args": ["-y", "@ash_j1997/ffmpeg-mcp-server"]
|
|
93
|
+
}
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
#### 설정 파일 위치 안내:
|
|
99
|
+
* **Claude Desktop (macOS):** `~/Library/Application Support/Claude/claude_desktop_config.json`
|
|
100
|
+
* **Claude Desktop (Windows):** `%APPDATA%\Claude\claude_desktop_config.json`
|
|
101
|
+
* **Antigravity IDE:** `~/.gemini/config/mcp_config.json`
|
|
102
|
+
* **Cursor / Windsurf:** stdio 서버 유형 선택 후 Command: `npx`, Args: `["-y", "@ash_j1997/ffmpeg-mcp-server"]` 추가
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 로컬 개발 및 테스트 (Local Development)
|
|
107
|
+
|
|
108
|
+
1. **의존성 설치:**
|
|
109
|
+
```bash
|
|
110
|
+
npm install
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
2. **빌드 및 자동화 테스트 실행:**
|
|
114
|
+
```bash
|
|
115
|
+
npm test
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
---
|
|
119
|
+
|
|
120
|
+
## 감사의 글 (Acknowledgements & Community)
|
|
121
|
+
|
|
122
|
+
단순 포맷 변환을 넘어 스트림 멀티플렉싱(`mux_media`) 및 장면 전환 감지 기능(`detect_scenes`)을 제안해 주신 [@rajarshidattapy](https://github.com/rajarshidattapy) 님께 감사드립니다.
|
|
123
|
+
|
|
124
|
+
---
|
|
125
|
+
|
|
126
|
+
## 라이선스 (License)
|
|
127
|
+
|
|
128
|
+
MIT © [Ashutos1997](https://github.com/Ashutos1997)
|
|
129
|
+
|
package/README.md
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
# ffmpeg-mcp-server
|
|
2
2
|
|
|
3
|
-
[
|
|
3
|
+
[English](README.md) | [한국어](README.ko.md)
|
|
4
|
+
|
|
5
|
+
[](https://github.com/Ashutos1997/ffmpeg-mcp-server/actions) [](https://github.com/Ashutos1997/ffmpeg-mcp-server/releases) [](LICENSE) [](https://www.npmjs.com/package/@ash_j1997/ffmpeg-mcp-server) [](https://www.npmjs.com/package/@ash_j1997/ffmpeg-mcp-server) [](https://www.npmjs.com/package/@ash_j1997/ffmpeg-mcp-server) [](https://glama.ai/mcp/servers/@Ashutos1997/ffmpeg-mcp-server) [](https://mcprush.com/to/ashutos1997-ffmpeg-mcp?ref=badge)
|
|
4
6
|
|
|
5
7
|
A Model Context Protocol (MCP) server that gives AI tools access to FFmpeg and FFprobe to process video and audio locally.
|
|
6
8
|
|
|
@@ -12,7 +14,7 @@ Supports cross-platform GPU hardware encoding (Apple Silicon, NVIDIA NVENC, Inte
|
|
|
12
14
|
|
|
13
15
|
When AI agents process media, they often hallucinate invalid flags, syntax errors, or incompatible pixel formats.
|
|
14
16
|
|
|
15
|
-
This server provides **
|
|
17
|
+
This server provides **19 typed tools** using Zod schemas. The AI passes parameters (such as `{ inputPath: "video.mov", preset: "web_mp4" }`), and the server runs the valid FFmpeg command locally with proper error handling, hardware acceleration, and progress notifications.
|
|
16
18
|
|
|
17
19
|
---
|
|
18
20
|
|
|
@@ -24,10 +26,11 @@ This server provides **17 typed tools** using Zod schemas. The AI passes paramet
|
|
|
24
26
|
* **Multimodal Vision Previews:** Inlines base64 image data for thumbnails, audio waveforms, and multi-frame contact sheets so vision models (Claude 3.7, GPT-4o, Gemini) can "see" media content.
|
|
25
27
|
* **Social Formats & Automation:** Reformats landscape video to 9:16 vertical video with blurred backgrounds, burns customizable hardcoded subtitles, and stacks top/bottom split-screen reels for TikTok and Instagram Reels.
|
|
26
28
|
* **Audio Engineering:** Normalizes loudness to broadcast/streaming standards (EBU R128 / -16 LUFS), cleans vocal audio with 80Hz rumble removal, and trims dead air while preserving lip-sync.
|
|
29
|
+
* **Scene Intelligence & Multiplexing:** Detects visual scene changes and cut points for smart editing workflows, and combines or replaces video/audio tracks with zero-loss stream copy.
|
|
27
30
|
|
|
28
31
|
---
|
|
29
32
|
|
|
30
|
-
## Included Tools (
|
|
33
|
+
## Included Tools (19 Total)
|
|
31
34
|
|
|
32
35
|
| Tool | Description | Key Parameters |
|
|
33
36
|
| :--- | :--- | :--- |
|
|
@@ -43,11 +46,13 @@ This server provides **17 typed tools** using Zod schemas. The AI passes paramet
|
|
|
43
46
|
| **`concat_media`** | Joins multiple video or audio files in order using stream copy or re-encoding. | `inputPaths`, `outputPath`, `mode` |
|
|
44
47
|
| **`generate_waveform`** | Generates a PNG waveform image. Returns base64 image data for vision models. | `inputPath`, `outputPath`, `width`, `height`, `color`, `scale`, `includeBase64` |
|
|
45
48
|
| **`add_subtitles`** | Embeds SRT or VTT subtitle tracks into MP4/MKV video files without re-encoding video. | `videoPath`, `subtitlesPath`, `outputPath`, `language`, `title` |
|
|
46
|
-
| **`burn_subtitles`**
|
|
47
|
-
| **`extract_keyframes`**
|
|
48
|
-
| **`normalize_audio`**
|
|
49
|
-
| **`add_overlay`**
|
|
49
|
+
| **`burn_subtitles`** | Hardcodes subtitles directly onto video frames with customizable font size, colors, and margins. | `videoPath`, `subtitlesPath`, `outputPath`, `fontSize`, `fontColor`, `outlineColor`, `marginV` |
|
|
50
|
+
| **`extract_keyframes`** | Extracts multiple representative frames across the video and generates a contact sheet grid for vision AI. | `inputPath`, `outputPath`, `count`, `layout`, `includeBase64` |
|
|
51
|
+
| **`normalize_audio`** | Normalizes audio loudness using EBU R128 (default -16 LUFS) with optional speech clarity high-pass filter. | `inputPath`, `outputPath`, `targetLufs`, `targetPeakDb`, `voiceClarity` |
|
|
52
|
+
| **`add_overlay`** | Places a watermark image or logo PNG over video with customizable corner position, scale, and opacity. | `videoPath`, `overlayPath`, `outputPath`, `position`, `padding`, `scaleWidth`, `opacity` |
|
|
50
53
|
| **`change_speed`** | Adjusts playback speed (such as 2x fast forward or 0.5x slow motion) with pitch correction. | `inputPath`, `outputPath`, `speed`, `muteAudio` |
|
|
54
|
+
| **`mux_media`** *(New)* | Combines or replaces video and audio streams with optional audio offset delay and zero-loss stream copy. | `videoPath`, `audioPath`, `outputPath`, `mode`, `audioOffsetSeconds`, `streamCopyVideo`, `shortest` |
|
|
55
|
+
| **`detect_scenes`** *(New)* | Detects video cut points and scene transitions with adjustable sensitivity. Returns timestamps, scene durations, and an optional contact sheet. | `inputPath`, `threshold`, `minDurationSeconds`, `outputPath`, `includeBase64` |
|
|
51
56
|
|
|
52
57
|
---
|
|
53
58
|
|
|
@@ -112,6 +117,12 @@ Run directly with `npx` (no manual cloning or building required):
|
|
|
112
117
|
|
|
113
118
|
---
|
|
114
119
|
|
|
120
|
+
## Acknowledgements & Community
|
|
121
|
+
|
|
122
|
+
Special thanks to [@rajarshidattapy](https://github.com/rajarshidattapy) for suggesting the stream multiplexing (`mux_media`) and scene transition detection (`detect_scenes`) capabilities to expand the server beyond format conversion.
|
|
123
|
+
|
|
124
|
+
---
|
|
125
|
+
|
|
115
126
|
## License
|
|
116
127
|
|
|
117
128
|
MIT © [Ashutos1997](https://github.com/Ashutos1997)
|
package/dist/index.js
CHANGED
|
@@ -19,11 +19,13 @@ import { burnSubtitles, burnSubtitlesSchema } from "./tools/burn_subtitles.js";
|
|
|
19
19
|
import { extractKeyframes, extractKeyframesSchema } from "./tools/keyframes.js";
|
|
20
20
|
import { normalizeAudio, normalizeAudioSchema } from "./tools/normalize.js";
|
|
21
21
|
import { addOverlay, addOverlaySchema } from "./tools/overlay.js";
|
|
22
|
+
import { muxMedia, muxMediaSchema } from "./tools/mux.js";
|
|
23
|
+
import { detectScenes, detectScenesSchema } from "./tools/detect_scenes.js";
|
|
22
24
|
import { detectHwAccel } from "./utils/hwaccel.js";
|
|
23
25
|
import { runCommand } from "./utils/exec.js";
|
|
24
26
|
const server = new McpServer({
|
|
25
27
|
name: "ffmpeg-mcp-server",
|
|
26
|
-
version: "0.
|
|
28
|
+
version: "0.6.0",
|
|
27
29
|
});
|
|
28
30
|
function createProgressCallback(extra) {
|
|
29
31
|
return (p) => {
|
|
@@ -407,6 +409,56 @@ server.tool("add_overlay", "Place a watermark image, logo PNG, or branding graph
|
|
|
407
409
|
};
|
|
408
410
|
}
|
|
409
411
|
});
|
|
412
|
+
// Tool 18: Mux Media
|
|
413
|
+
server.tool("mux_media", "Combine or replace video and audio streams. Takes a video file and an audio track, with optional audio delay offset, stream-copy mode for zero quality loss, and 'replace' or 'add' stream mapping.", muxMediaSchema.shape, async (args, extra) => {
|
|
414
|
+
try {
|
|
415
|
+
const onProgress = createProgressCallback(extra);
|
|
416
|
+
const result = await muxMedia(args, { signal: extra.signal, onProgress });
|
|
417
|
+
return {
|
|
418
|
+
content: [
|
|
419
|
+
{
|
|
420
|
+
type: "text",
|
|
421
|
+
text: `Successfully muxed media:\nOutput: ${result.outputPath}\nMode: ${result.mode}\nEncoder: ${result.encoderUsed} (Hardware Accelerated: ${result.hardwareAccelerated})\nAudio Offset: ${result.audioOffsetSeconds}s\nCommand: ${result.commandExecuted}`,
|
|
422
|
+
},
|
|
423
|
+
],
|
|
424
|
+
};
|
|
425
|
+
}
|
|
426
|
+
catch (err) {
|
|
427
|
+
return {
|
|
428
|
+
isError: true,
|
|
429
|
+
content: [{ type: "text", text: `Error muxing media: ${err.message}` }],
|
|
430
|
+
};
|
|
431
|
+
}
|
|
432
|
+
});
|
|
433
|
+
// Tool 19: Detect Scenes
|
|
434
|
+
server.tool("detect_scenes", "Detect video cuts and scene transitions. Returns structured timestamps, scene durations, and cut boundaries as JSON, with optional contact-sheet image preview for vision models.", detectScenesSchema.shape, async (args, extra) => {
|
|
435
|
+
try {
|
|
436
|
+
const onProgress = createProgressCallback(extra);
|
|
437
|
+
const result = await detectScenes(args, { signal: extra.signal, onProgress });
|
|
438
|
+
const content = [];
|
|
439
|
+
if (result.base64 && result.mimeType) {
|
|
440
|
+
content.push({
|
|
441
|
+
type: "image",
|
|
442
|
+
data: result.base64,
|
|
443
|
+
mimeType: result.mimeType,
|
|
444
|
+
});
|
|
445
|
+
}
|
|
446
|
+
const scenesList = result.scenes
|
|
447
|
+
.map((s) => ` • Scene ${s.sceneIndex}: ${s.startTimeSeconds}s - ${s.endTimeSeconds}s (${s.durationSeconds}s)`)
|
|
448
|
+
.join("\n");
|
|
449
|
+
content.push({
|
|
450
|
+
type: "text",
|
|
451
|
+
text: `Successfully detected ${result.totalScenes} scenes (${result.totalCuts} cuts):\nCut Timestamps: [${result.cutTimestamps.join(", ")}]\n\nScenes:\n${scenesList}${result.contactSheetPath ? `\n\nContact Sheet: ${result.contactSheetPath}` : ""}\nCommand: ${result.commandExecuted}`,
|
|
452
|
+
});
|
|
453
|
+
return { content };
|
|
454
|
+
}
|
|
455
|
+
catch (err) {
|
|
456
|
+
return {
|
|
457
|
+
isError: true,
|
|
458
|
+
content: [{ type: "text", text: `Error detecting scenes: ${err.message}` }],
|
|
459
|
+
};
|
|
460
|
+
}
|
|
461
|
+
});
|
|
410
462
|
// Resource 1: Capabilities & Hardware Info
|
|
411
463
|
server.registerResource("system_capabilities", "media://capabilities", {
|
|
412
464
|
title: "FFmpeg System Capabilities",
|
|
@@ -9,11 +9,13 @@ export const burnSubtitlesSchema = z.object({
|
|
|
9
9
|
fontSize: z.number().int().min(8).max(120).optional().default(24).describe("Subtitle font size in points. Default: 24."),
|
|
10
10
|
fontColor: z
|
|
11
11
|
.string()
|
|
12
|
+
.regex(/^(white|yellow|cyan|red|green|blue|black|&[hH][0-9a-fA-F]{6,8})$/i, "Color must be a recognized color name or ASS hex code (e.g. '&H00FFFFFF')")
|
|
12
13
|
.optional()
|
|
13
14
|
.default("white")
|
|
14
15
|
.describe("Primary subtitle text color: 'white', 'yellow', 'cyan', or ASS hex e.g. '&H00FFFFFF'. Default: 'white'."),
|
|
15
16
|
outlineColor: z
|
|
16
17
|
.string()
|
|
18
|
+
.regex(/^(white|yellow|cyan|red|green|blue|black|&[hH][0-9a-fA-F]{6,8})$/i, "Outline color must be a recognized color name or ASS hex code (e.g. '&H00000000')")
|
|
17
19
|
.optional()
|
|
18
20
|
.default("black")
|
|
19
21
|
.describe("Subtitle border/outline color: 'black', 'white', or ASS hex. Default: 'black'."),
|
|
@@ -26,8 +28,8 @@ export const burnSubtitlesSchema = z.object({
|
|
|
26
28
|
.describe("Use hardware-accelerated video encoding when available. Default: true."),
|
|
27
29
|
});
|
|
28
30
|
function toAssColor(color) {
|
|
29
|
-
const c = color.
|
|
30
|
-
if (c
|
|
31
|
+
const c = color.trim();
|
|
32
|
+
if (/^&[hH][0-9a-fA-F]{6,8}$/.test(c))
|
|
31
33
|
return color;
|
|
32
34
|
const map = {
|
|
33
35
|
white: "&H00FFFFFF",
|
|
@@ -38,7 +40,7 @@ function toAssColor(color) {
|
|
|
38
40
|
green: "&H0000FF00",
|
|
39
41
|
blue: "&H00FF0000",
|
|
40
42
|
};
|
|
41
|
-
return map[c] || "&H00FFFFFF";
|
|
43
|
+
return map[c.toLowerCase()] || "&H00FFFFFF";
|
|
42
44
|
}
|
|
43
45
|
export async function burnSubtitles(params, options) {
|
|
44
46
|
const { videoPath, subtitlesPath, outputPath, fontSize = 24, fontColor = "white", outlineColor = "black", outlineWidth = 2, marginV = 35, hardwareAcceleration = true, } = params;
|
package/dist/tools/convert.js
CHANGED
|
@@ -17,6 +17,7 @@ export const convertMediaSchema = z.object({
|
|
|
17
17
|
.describe("Quality factor from 0 to 51. Lower numbers mean higher quality. Default: 23 for MP4, 30 for WebM."),
|
|
18
18
|
resolution: z
|
|
19
19
|
.string()
|
|
20
|
+
.regex(/^\d{2,5}x\d{2,5}$/, "Resolution must be formatted as 'WIDTHxHEIGHT' (e.g. '1920x1080')")
|
|
20
21
|
.optional()
|
|
21
22
|
.describe("Target size formatted as 'WIDTHxHEIGHT', such as '1920x1080' or '1280x720'"),
|
|
22
23
|
fps: z
|
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
import fs from "node:fs/promises";
|
|
2
|
+
import { z } from "zod";
|
|
3
|
+
import { ensureFileExists, ensureOutputDir, runFfmpeg } from "../utils/exec.js";
|
|
4
|
+
import { probeMedia } from "./probe.js";
|
|
5
|
+
export const detectScenesSchema = z.object({
|
|
6
|
+
inputPath: z.string().describe("Path to the source video file to analyze for scene transitions."),
|
|
7
|
+
threshold: z
|
|
8
|
+
.number()
|
|
9
|
+
.min(0.05)
|
|
10
|
+
.max(1.0)
|
|
11
|
+
.optional()
|
|
12
|
+
.default(0.3)
|
|
13
|
+
.describe("Scene cut sensitivity threshold between 0.05 and 1.0. Lower values detect subtle transitions; higher values detect only dramatic scene cuts. Default: 0.3."),
|
|
14
|
+
minDurationSeconds: z
|
|
15
|
+
.number()
|
|
16
|
+
.min(0.1)
|
|
17
|
+
.max(60)
|
|
18
|
+
.optional()
|
|
19
|
+
.default(0.5)
|
|
20
|
+
.describe("Minimum scene duration in seconds to filter out rapid flashes or micro-cuts. Default: 0.5."),
|
|
21
|
+
outputPath: z
|
|
22
|
+
.string()
|
|
23
|
+
.optional()
|
|
24
|
+
.describe("Optional destination path for a contact-sheet montage image (.png or .jpg) visualizing detected scenes."),
|
|
25
|
+
includeBase64: z
|
|
26
|
+
.boolean()
|
|
27
|
+
.optional()
|
|
28
|
+
.default(true)
|
|
29
|
+
.describe("Return base64 contact sheet image data in response for vision models when outputPath is specified. Default: true."),
|
|
30
|
+
});
|
|
31
|
+
export async function detectScenes(params, options) {
|
|
32
|
+
const { inputPath, threshold = 0.3, minDurationSeconds = 0.5, outputPath, includeBase64 = true, } = params;
|
|
33
|
+
const input = await ensureFileExists(inputPath);
|
|
34
|
+
const probe = await probeMedia(input, { signal: options?.signal });
|
|
35
|
+
const totalDuration = Math.max(0.1, probe.format.durationSeconds);
|
|
36
|
+
// Run scene change detection using select filter with showinfo output
|
|
37
|
+
const detectArgs = [
|
|
38
|
+
"-i", input,
|
|
39
|
+
"-vf", `select='gt(scene\\,${threshold})',showinfo`,
|
|
40
|
+
"-f", "null",
|
|
41
|
+
"-",
|
|
42
|
+
];
|
|
43
|
+
const res = await runFfmpeg(detectArgs, {
|
|
44
|
+
signal: options?.signal,
|
|
45
|
+
onProgress: options?.onProgress,
|
|
46
|
+
expectedDurationSeconds: totalDuration,
|
|
47
|
+
});
|
|
48
|
+
// Extract cut timestamps from showinfo log lines
|
|
49
|
+
const rawCuts = [];
|
|
50
|
+
const lines = res.stderr.split(/\r?\n/);
|
|
51
|
+
for (const line of lines) {
|
|
52
|
+
if (line.includes("pts_time:")) {
|
|
53
|
+
const match = line.match(/pts_time:([0-9.]+)/);
|
|
54
|
+
if (match && match[1]) {
|
|
55
|
+
const time = parseFloat(match[1]);
|
|
56
|
+
if (!isNaN(time) && time > 0 && time < totalDuration) {
|
|
57
|
+
rawCuts.push(Number(time.toFixed(3)));
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
// Deduplicate and filter out cuts too close to each other
|
|
63
|
+
const validCuts = [];
|
|
64
|
+
let lastCut = 0;
|
|
65
|
+
for (const cut of rawCuts) {
|
|
66
|
+
if (cut - lastCut >= minDurationSeconds) {
|
|
67
|
+
validCuts.push(cut);
|
|
68
|
+
lastCut = cut;
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
// Build structured scene segments
|
|
72
|
+
const scenes = [];
|
|
73
|
+
let currentStart = 0;
|
|
74
|
+
for (let i = 0; i < validCuts.length; i++) {
|
|
75
|
+
const cutTime = validCuts[i];
|
|
76
|
+
const duration = Number((cutTime - currentStart).toFixed(3));
|
|
77
|
+
scenes.push({
|
|
78
|
+
sceneIndex: i + 1,
|
|
79
|
+
startTimeSeconds: currentStart,
|
|
80
|
+
endTimeSeconds: cutTime,
|
|
81
|
+
durationSeconds: duration,
|
|
82
|
+
});
|
|
83
|
+
currentStart = cutTime;
|
|
84
|
+
}
|
|
85
|
+
// Final scene segment to end of video
|
|
86
|
+
const lastDuration = Number((totalDuration - currentStart).toFixed(3));
|
|
87
|
+
scenes.push({
|
|
88
|
+
sceneIndex: scenes.length + 1,
|
|
89
|
+
startTimeSeconds: currentStart,
|
|
90
|
+
endTimeSeconds: Number(totalDuration.toFixed(3)),
|
|
91
|
+
durationSeconds: lastDuration,
|
|
92
|
+
});
|
|
93
|
+
// Optional Contact Sheet generation
|
|
94
|
+
let contactSheetPath;
|
|
95
|
+
let base64;
|
|
96
|
+
let mimeType;
|
|
97
|
+
if (outputPath) {
|
|
98
|
+
contactSheetPath = await ensureOutputDir(outputPath);
|
|
99
|
+
const count = scenes.length;
|
|
100
|
+
const cols = count <= 3 ? count : Math.ceil(Math.sqrt(count));
|
|
101
|
+
const rows = Math.ceil(count / cols);
|
|
102
|
+
const sheetArgs = [
|
|
103
|
+
"-i", input,
|
|
104
|
+
"-vf", `select='eq(n\\,0)+gt(scene\\,${threshold})',scale=320:180:force_original_aspect_ratio=decrease,pad=320:180:(ow-iw)/2:(oh-ih)/2,tile=${cols}x${rows}`,
|
|
105
|
+
"-frames:v", "1",
|
|
106
|
+
"-y", contactSheetPath,
|
|
107
|
+
];
|
|
108
|
+
await runFfmpeg(sheetArgs, { signal: options?.signal });
|
|
109
|
+
if (includeBase64) {
|
|
110
|
+
try {
|
|
111
|
+
const buffer = await fs.readFile(contactSheetPath);
|
|
112
|
+
if (buffer.length < 8 * 1024 * 1024) {
|
|
113
|
+
base64 = buffer.toString("base64");
|
|
114
|
+
mimeType = contactSheetPath.toLowerCase().endsWith(".png") ? "image/png" : "image/jpeg";
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
catch { }
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
return {
|
|
121
|
+
success: true,
|
|
122
|
+
totalScenes: scenes.length,
|
|
123
|
+
totalCuts: validCuts.length,
|
|
124
|
+
cutTimestamps: validCuts,
|
|
125
|
+
scenes,
|
|
126
|
+
contactSheetPath,
|
|
127
|
+
base64,
|
|
128
|
+
mimeType,
|
|
129
|
+
commandExecuted: res.command,
|
|
130
|
+
durationSeconds: res.durationSeconds,
|
|
131
|
+
};
|
|
132
|
+
}
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
import { z } from "zod";
|
|
2
|
+
import { ensureFileExists, ensureOutputDir, runFfmpeg } from "../utils/exec.js";
|
|
3
|
+
import { detectHwAccel } from "../utils/hwaccel.js";
|
|
4
|
+
import { probeMedia } from "./probe.js";
|
|
5
|
+
export const muxMediaSchema = z.object({
|
|
6
|
+
videoPath: z.string().describe("Path to the source video file."),
|
|
7
|
+
audioPath: z.string().describe("Path to the source audio file (.mp3, .wav, .aac, .m4a, etc.) to mux."),
|
|
8
|
+
outputPath: z.string().describe("Target output video path (.mp4, .mov, or .mkv)."),
|
|
9
|
+
mode: z
|
|
10
|
+
.enum(["replace", "add"])
|
|
11
|
+
.optional()
|
|
12
|
+
.default("replace")
|
|
13
|
+
.describe("Muxing mode: 'replace' drops existing video audio and uses new audio; 'add' keeps original audio and adds new audio as secondary track. Default: 'replace'."),
|
|
14
|
+
audioOffsetSeconds: z
|
|
15
|
+
.number()
|
|
16
|
+
.optional()
|
|
17
|
+
.default(0)
|
|
18
|
+
.describe("Audio delay or advance in seconds. Positive numbers delay audio; negative numbers advance it. Default: 0."),
|
|
19
|
+
shortest: z
|
|
20
|
+
.boolean()
|
|
21
|
+
.optional()
|
|
22
|
+
.default(true)
|
|
23
|
+
.describe("End output when the shorter input stream finishes. Default: true."),
|
|
24
|
+
streamCopyVideo: z
|
|
25
|
+
.boolean()
|
|
26
|
+
.optional()
|
|
27
|
+
.default(true)
|
|
28
|
+
.describe("Use stream copy (-c:v copy) for instantaneous video multiplexing without quality loss. Default: true."),
|
|
29
|
+
hardwareAcceleration: z
|
|
30
|
+
.boolean()
|
|
31
|
+
.optional()
|
|
32
|
+
.default(true)
|
|
33
|
+
.describe("Use hardware-accelerated video encoding when streamCopyVideo is false. Default: true."),
|
|
34
|
+
});
|
|
35
|
+
export async function muxMedia(params, options) {
|
|
36
|
+
const { videoPath, audioPath, outputPath, mode = "replace", audioOffsetSeconds = 0, shortest = true, streamCopyVideo = true, hardwareAcceleration = true, } = params;
|
|
37
|
+
const video = await ensureFileExists(videoPath);
|
|
38
|
+
const audio = await ensureFileExists(audioPath);
|
|
39
|
+
const output = await ensureOutputDir(outputPath);
|
|
40
|
+
let expectedDuration;
|
|
41
|
+
try {
|
|
42
|
+
const probe = await probeMedia(video, { signal: options?.signal });
|
|
43
|
+
expectedDuration = probe.format.durationSeconds;
|
|
44
|
+
}
|
|
45
|
+
catch { }
|
|
46
|
+
const hwInfo = await detectHwAccel();
|
|
47
|
+
const useHw = hardwareAcceleration && hwInfo.isAvailable;
|
|
48
|
+
const isHwUsed = !streamCopyVideo && useHw;
|
|
49
|
+
const videoEncoder = streamCopyVideo ? "copy" : (isHwUsed ? hwInfo.h264Encoder : "libx264");
|
|
50
|
+
const args = [];
|
|
51
|
+
// Input 0: Video
|
|
52
|
+
args.push("-i", video);
|
|
53
|
+
// Input 1: Audio (with optional offset)
|
|
54
|
+
if (audioOffsetSeconds !== 0) {
|
|
55
|
+
args.push("-itsoffset", String(audioOffsetSeconds));
|
|
56
|
+
}
|
|
57
|
+
args.push("-i", audio);
|
|
58
|
+
// Explicit Stream Mapping
|
|
59
|
+
if (mode === "replace") {
|
|
60
|
+
// Map video stream from input 0 and audio stream from input 1
|
|
61
|
+
args.push("-map", "0:v:0", "-map", "1:a:0");
|
|
62
|
+
}
|
|
63
|
+
else {
|
|
64
|
+
// Keep original audio from input 0 if present, and append audio from input 1
|
|
65
|
+
args.push("-map", "0:v:0", "-map", "0:a?", "-map", "1:a:0");
|
|
66
|
+
}
|
|
67
|
+
// Video codec
|
|
68
|
+
if (streamCopyVideo) {
|
|
69
|
+
args.push("-c:v", "copy");
|
|
70
|
+
}
|
|
71
|
+
else {
|
|
72
|
+
args.push("-c:v", videoEncoder);
|
|
73
|
+
if (isHwUsed) {
|
|
74
|
+
args.push("-b:v", "4M", "-pix_fmt", "yuv420p");
|
|
75
|
+
}
|
|
76
|
+
else {
|
|
77
|
+
args.push("-crf", "22", "-preset", "fast", "-pix_fmt", "yuv420p");
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
// Audio codec
|
|
81
|
+
args.push("-c:a", "aac", "-b:a", "192k");
|
|
82
|
+
if (shortest) {
|
|
83
|
+
args.push("-shortest");
|
|
84
|
+
}
|
|
85
|
+
args.push("-movflags", "+faststart", "-y", output);
|
|
86
|
+
const res = await runFfmpeg(args, {
|
|
87
|
+
signal: options?.signal,
|
|
88
|
+
onProgress: options?.onProgress,
|
|
89
|
+
expectedDurationSeconds: expectedDuration,
|
|
90
|
+
});
|
|
91
|
+
return {
|
|
92
|
+
success: true,
|
|
93
|
+
outputPath: output,
|
|
94
|
+
mode,
|
|
95
|
+
encoderUsed: videoEncoder,
|
|
96
|
+
hardwareAccelerated: isHwUsed,
|
|
97
|
+
audioOffsetSeconds,
|
|
98
|
+
shortest,
|
|
99
|
+
commandExecuted: res.command,
|
|
100
|
+
durationSeconds: res.durationSeconds,
|
|
101
|
+
};
|
|
102
|
+
}
|
package/dist/tools/reformat.js
CHANGED
|
@@ -15,6 +15,7 @@ export const reformatAspectSchema = z.object({
|
|
|
15
15
|
.describe("Format mode: 'blur_background' adds blurred video borders, 'fit_pad' adds solid color padding, and 'crop_center' zooms and crops the center. Default: 'blur_background'."),
|
|
16
16
|
padColor: z
|
|
17
17
|
.string()
|
|
18
|
+
.regex(/^(0x[0-9a-fA-F]{6}|#[0-9a-fA-F]{6}|[a-zA-Z]{3,20})$/, "padColor must be hex (e.g. '0x1e1e2e') or color name (e.g. 'black', 'white')")
|
|
18
19
|
.default("black")
|
|
19
20
|
.describe("Background color for fit_pad mode, such as 'black', 'white', or hex '0x1e1e2e'. Default: 'black'."),
|
|
20
21
|
blurAmount: z
|
package/dist/tools/subtitles.js
CHANGED
|
@@ -4,8 +4,18 @@ export const addSubtitlesSchema = z.object({
|
|
|
4
4
|
videoPath: z.string().describe("Path to the source video file"),
|
|
5
5
|
subtitlesPath: z.string().describe("Path to subtitle file (.srt or .vtt)"),
|
|
6
6
|
outputPath: z.string().describe("Target output video path (.mp4, .mkv, or .mov)"),
|
|
7
|
-
language: z
|
|
8
|
-
|
|
7
|
+
language: z
|
|
8
|
+
.string()
|
|
9
|
+
.regex(/^[a-zA-Z]{2,4}$/, "Must be an ISO 2-4 letter language code (e.g. 'eng')")
|
|
10
|
+
.optional()
|
|
11
|
+
.default("eng")
|
|
12
|
+
.describe("ISO 639-2 three-letter language code (e.g. 'eng', 'spa', 'fra', 'deu')"),
|
|
13
|
+
title: z
|
|
14
|
+
.string()
|
|
15
|
+
.max(100)
|
|
16
|
+
.optional()
|
|
17
|
+
.default("English")
|
|
18
|
+
.describe("Subtitle track label displayed in video players"),
|
|
9
19
|
});
|
|
10
20
|
export async function addSubtitles(params, options) {
|
|
11
21
|
const { videoPath, subtitlesPath, outputPath, language = "eng", title = "English" } = params;
|
package/dist/tools/thumbnail.js
CHANGED
|
@@ -7,9 +7,14 @@ export const extractThumbnailSchema = z.object({
|
|
|
7
7
|
outputPath: z.string().describe("Destination image path ending in .png or .jpg (e.g. './thumb.png')."),
|
|
8
8
|
timestamp: z
|
|
9
9
|
.string()
|
|
10
|
+
.regex(/^((\d{1,2}:)?\d{1,2}:\d{1,2}(\.\d+)?|\d+(\.\d+)?|midpoint)$/i, "Timestamp must be 'HH:MM:SS', seconds (e.g. '10'), or 'midpoint'")
|
|
10
11
|
.optional()
|
|
11
12
|
.describe("Timestamp to capture, such as '00:00:05', '10', or 'midpoint' for the middle frame. Default: '00:00:01'."),
|
|
12
|
-
resolution: z
|
|
13
|
+
resolution: z
|
|
14
|
+
.string()
|
|
15
|
+
.regex(/^\d{2,5}x\d{2,5}$/, "Resolution must be formatted as 'WIDTHxHEIGHT' (e.g. '1280x720')")
|
|
16
|
+
.optional()
|
|
17
|
+
.describe("Optional scaled size formatted as 'WIDTHxHEIGHT', such as '1280x720'. Defaults to source resolution."),
|
|
13
18
|
includeBase64: z.boolean().optional().describe("Return base64 image data in response for vision models. Default: true."),
|
|
14
19
|
});
|
|
15
20
|
export async function extractThumbnail(params, options) {
|
package/dist/tools/trim.js
CHANGED
|
@@ -4,9 +4,20 @@ import { detectHwAccel } from "../utils/hwaccel.js";
|
|
|
4
4
|
export const trimMediaSchema = z.object({
|
|
5
5
|
inputPath: z.string().describe("Path to the source media file to cut."),
|
|
6
6
|
outputPath: z.string().describe("Destination path for the trimmed output file (e.g. './clip.mp4')."),
|
|
7
|
-
startTime: z
|
|
8
|
-
|
|
9
|
-
|
|
7
|
+
startTime: z
|
|
8
|
+
.string()
|
|
9
|
+
.regex(/^((\d{1,2}:)?\d{1,2}:\d{1,2}(\.\d+)?|\d+(\.\d+)?)$/, "startTime must be 'HH:MM:SS' (e.g. '00:01:30') or seconds (e.g. '90')")
|
|
10
|
+
.describe("Start timestamp formatted as 'HH:MM:SS' (e.g. '00:01:30') or seconds (e.g. '90')."),
|
|
11
|
+
endTime: z
|
|
12
|
+
.string()
|
|
13
|
+
.regex(/^((\d{1,2}:)?\d{1,2}:\d{1,2}(\.\d+)?|\d+(\.\d+)?)$/, "endTime must be 'HH:MM:SS' or seconds")
|
|
14
|
+
.optional()
|
|
15
|
+
.describe("Optional end timestamp formatted as 'HH:MM:SS' or seconds. Do not combine with duration."),
|
|
16
|
+
duration: z
|
|
17
|
+
.string()
|
|
18
|
+
.regex(/^(\d+(\.\d+)?)$/, "duration must be a positive number of seconds (e.g. '30')")
|
|
19
|
+
.optional()
|
|
20
|
+
.describe("Optional duration in seconds from startTime (e.g. '30'). If neither endTime nor duration is set, trims to end of file."),
|
|
10
21
|
accurateReencode: z
|
|
11
22
|
.boolean()
|
|
12
23
|
.optional()
|
package/dist/tools/waveform.js
CHANGED
|
@@ -6,7 +6,12 @@ export const generateWaveformSchema = z.object({
|
|
|
6
6
|
outputPath: z.string().describe("Destination path for the PNG image."),
|
|
7
7
|
width: z.number().int().positive().optional().default(1200).describe("Image width in pixels. Default: 1200."),
|
|
8
8
|
height: z.number().int().positive().optional().default(280).describe("Image height in pixels. Default: 280."),
|
|
9
|
-
color: z
|
|
9
|
+
color: z
|
|
10
|
+
.string()
|
|
11
|
+
.regex(/^(0x[0-9a-fA-F]{6}|#[0-9a-fA-F]{6}|[a-zA-Z]{3,20})$/, "Color must be hex (e.g. '0x6366f1', '#6366f1') or color name (e.g. 'white', 'black')")
|
|
12
|
+
.optional()
|
|
13
|
+
.default("0x6366f1")
|
|
14
|
+
.describe("Waveform color in hex or name (such as '0x6366f1', 'white', or 'black'). Default: '0x6366f1'."),
|
|
10
15
|
scale: z.enum(["lin", "log", "sqrt"]).optional().default("lin").describe("Amplitude scale: 'lin', 'log', or 'sqrt'. Default: 'lin'."),
|
|
11
16
|
includeBase64: z.boolean().optional().default(true).describe("Return base64 image data in the response for vision models. Default: true."),
|
|
12
17
|
});
|
package/dist/utils/hwaccel.js
CHANGED
|
@@ -1,6 +1,22 @@
|
|
|
1
1
|
import os from "node:os";
|
|
2
2
|
import { runCommand } from "./exec.js";
|
|
3
3
|
let cachedInfo = null;
|
|
4
|
+
async function canEncodeWith(encoder) {
|
|
5
|
+
try {
|
|
6
|
+
await runCommand("ffmpeg", [
|
|
7
|
+
"-v", "quiet",
|
|
8
|
+
"-f", "lavfi",
|
|
9
|
+
"-i", "color=c=black:s=16x16:d=0.04",
|
|
10
|
+
"-c:v", encoder,
|
|
11
|
+
"-f", "null",
|
|
12
|
+
"-"
|
|
13
|
+
]);
|
|
14
|
+
return true;
|
|
15
|
+
}
|
|
16
|
+
catch {
|
|
17
|
+
return false;
|
|
18
|
+
}
|
|
19
|
+
}
|
|
4
20
|
/**
|
|
5
21
|
* Detects available hardware-accelerated video encoders across macOS, Linux, and Windows.
|
|
6
22
|
* Supports Apple Silicon (VideoToolbox), NVIDIA (NVENC), Intel (QuickSync), AMD (AMF), and Linux (VAAPI).
|
|
@@ -74,35 +90,50 @@ export async function detectHwAccel() {
|
|
|
74
90
|
let chosenType = "none";
|
|
75
91
|
let h264Encoder = "libx264";
|
|
76
92
|
let hevcEncoder = "libx265";
|
|
93
|
+
const candidates = [];
|
|
77
94
|
if (isDarwin && detectedTypes.includes("videotoolbox")) {
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
95
|
+
candidates.push({
|
|
96
|
+
type: "videotoolbox",
|
|
97
|
+
h264: "h264_videotoolbox",
|
|
98
|
+
hevc: detectedHevc.includes("hevc_videotoolbox") ? "hevc_videotoolbox" : "libx265",
|
|
99
|
+
});
|
|
81
100
|
}
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
101
|
+
if (detectedTypes.includes("nvenc")) {
|
|
102
|
+
candidates.push({
|
|
103
|
+
type: "nvenc",
|
|
104
|
+
h264: "h264_nvenc",
|
|
105
|
+
hevc: detectedHevc.includes("hevc_nvenc") ? "hevc_nvenc" : "libx265",
|
|
106
|
+
});
|
|
86
107
|
}
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
108
|
+
if (detectedTypes.includes("qsv")) {
|
|
109
|
+
candidates.push({
|
|
110
|
+
type: "qsv",
|
|
111
|
+
h264: "h264_qsv",
|
|
112
|
+
hevc: detectedHevc.includes("hevc_qsv") ? "hevc_qsv" : "libx265",
|
|
113
|
+
});
|
|
91
114
|
}
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
115
|
+
if (detectedTypes.includes("amf")) {
|
|
116
|
+
candidates.push({
|
|
117
|
+
type: "amf",
|
|
118
|
+
h264: "h264_amf",
|
|
119
|
+
hevc: detectedHevc.includes("hevc_amf") ? "hevc_amf" : "libx265",
|
|
120
|
+
});
|
|
96
121
|
}
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
122
|
+
if (detectedTypes.includes("vaapi")) {
|
|
123
|
+
candidates.push({
|
|
124
|
+
type: "vaapi",
|
|
125
|
+
h264: "h264_vaapi",
|
|
126
|
+
hevc: detectedHevc.includes("hevc_vaapi") ? "hevc_vaapi" : "libx265",
|
|
127
|
+
});
|
|
101
128
|
}
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
129
|
+
// Verify that the driver/device actually initializes (e.g. NVENC compiled in ffmpeg on runner without GPU)
|
|
130
|
+
for (const candidate of candidates) {
|
|
131
|
+
if (await canEncodeWith(candidate.h264)) {
|
|
132
|
+
chosenType = candidate.type;
|
|
133
|
+
h264Encoder = candidate.h264;
|
|
134
|
+
hevcEncoder = candidate.hevc;
|
|
135
|
+
break;
|
|
136
|
+
}
|
|
106
137
|
}
|
|
107
138
|
cachedInfo = {
|
|
108
139
|
isAvailable: chosenType !== "none",
|