@avclabs.ai/media-mcp 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +398 -184
- package/README_CN.md +633 -0
- package/config.json +1 -0
- package/dist/image-enhancement.d.ts +3 -0
- package/dist/image-enhancement.d.ts.map +1 -0
- package/dist/image-enhancement.js +185 -0
- package/dist/image-enhancement.js.map +1 -0
- package/dist/sam3.d.ts +7 -0
- package/dist/sam3.d.ts.map +1 -1
- package/dist/sam3.js +148 -104
- package/dist/sam3.js.map +1 -1
- package/dist/server.d.ts +1 -1
- package/dist/server.js +80 -56
- package/dist/server.js.map +1 -1
- package/dist/service-config.d.ts +62 -0
- package/dist/service-config.d.ts.map +1 -0
- package/dist/service-config.js +58 -0
- package/dist/service-config.js.map +1 -0
- package/dist/tooling.d.ts +42 -0
- package/dist/tooling.d.ts.map +1 -0
- package/dist/tooling.js +153 -0
- package/dist/tooling.js.map +1 -0
- package/dist/tos.d.ts +35 -0
- package/dist/tos.d.ts.map +1 -0
- package/dist/tos.js +281 -0
- package/dist/tos.js.map +1 -0
- package/dist/upload.d.ts +79 -0
- package/dist/upload.d.ts.map +1 -0
- package/dist/upload.js +190 -0
- package/dist/upload.js.map +1 -0
- package/dist/video-enhancement.d.ts.map +1 -1
- package/dist/video-enhancement.js +95 -175
- package/dist/video-enhancement.js.map +1 -1
- package/package.json +11 -6
- package/README_EN.md +0 -419
package/README.md
CHANGED
|
@@ -1,65 +1,74 @@
|
|
|
1
1
|
# media-mcp (Node.js)
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
English | [中文](https://github.com/avclabs/media-mcp/blob/main/README_CN.md)
|
|
4
4
|
|
|
5
5
|
[](https://www.npmjs.com/package/@avclabs.ai/media-mcp)
|
|
6
6
|
[](https://nodejs.org/)
|
|
7
7
|
[](https://opensource.org/licenses/MIT)
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
A video enhancement, image enhancement/colorization/denoising, and image segmentation service based on the MCP protocol, acting as an MCP Client-Server to interact with backend HTTP Servers.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
> **Release status:** the published npm package (`latest` = `0.2.1`) includes 3 video tools and 2 SAM3 tools. Image enhancement tools are in active development and will ship in an upcoming release — they are documented below for early adopters, but are not part of the published package yet.
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
## Features
|
|
14
14
|
|
|
15
|
-
|
|
16
|
-
- `create_task` - 创建视频增强任务(支持 URL 或本地文件上传)
|
|
17
|
-
- `get_task_status` - 查询任务状态
|
|
18
|
-
- `enhance_video_sync` - 同步增强视频(阻塞等待,默认50秒截断)
|
|
15
|
+
Provides the following MCP Tools:
|
|
19
16
|
|
|
20
|
-
|
|
21
|
-
- `
|
|
17
|
+
**Video Enhancement**
|
|
18
|
+
- `create_task` - Create a video enhancement task (supports URL or local file upload)
|
|
19
|
+
- `get_task_status` - Query task status
|
|
20
|
+
- `enhance_video_sync` - Synchronously enhance video (blocking wait, truncated at ~45s by default)
|
|
22
21
|
|
|
23
|
-
|
|
22
|
+
**Image Enhancement (coming soon — not in the published package yet)**
|
|
23
|
+
- `enhance_image_sync` - Enhance image quality and optimize faces (supports URL or local file upload)
|
|
24
|
+
- `colorize_image_sync` - Colorize black-and-white photos (supports URL or local file upload)
|
|
25
|
+
- `denoise_image_sync` - Remove noise from images (supports URL or local file upload)
|
|
26
|
+
- `get_image_task_status` - Query image task status (for polling after sync timeout)
|
|
24
27
|
|
|
25
|
-
|
|
26
|
-
-
|
|
28
|
+
**Image Segmentation (SAM3)**
|
|
29
|
+
- `sam3_predict` - SAM3 image segmentation (supports local path, URL, or Base64 image)
|
|
30
|
+
- `get_sam3_task_status` - Query SAM3 task status (for polling after sync timeout)
|
|
27
31
|
|
|
28
|
-
##
|
|
32
|
+
## Prerequisites
|
|
29
33
|
|
|
30
|
-
|
|
34
|
+
- **Node.js >= 18** (check: `node --version`)
|
|
35
|
+
- **API Key** (required for authentication)
|
|
36
|
+
|
|
37
|
+
## Lazy Install (Recommended)
|
|
38
|
+
|
|
39
|
+
If your AI Agent has a known MCP config path, just copy the line below and send it to your AI:
|
|
31
40
|
|
|
32
41
|
```
|
|
33
|
-
|
|
42
|
+
Install the npm package @avclabs.ai/media-mcp as an MCP server. My API Key is: sk-xxxxxxxx.
|
|
34
43
|
```
|
|
35
44
|
|
|
36
|
-
AI
|
|
37
|
-
1.
|
|
38
|
-
2.
|
|
39
|
-
3.
|
|
40
|
-
4.
|
|
45
|
+
The AI will automatically:
|
|
46
|
+
1. Detect your MCP client
|
|
47
|
+
2. Find the config file path
|
|
48
|
+
3. Write the correct configuration
|
|
49
|
+
4. Prompt you to restart the client
|
|
41
50
|
|
|
42
|
-
##
|
|
51
|
+
## Manual Install
|
|
43
52
|
|
|
44
|
-
|
|
53
|
+
No installation needed. Use `npx` directly in your MCP client config.
|
|
45
54
|
|
|
46
|
-
### 1. Claude Code
|
|
55
|
+
### 1. Claude Code (CLI)
|
|
47
56
|
|
|
48
|
-
|
|
57
|
+
Run in Claude Code:
|
|
49
58
|
|
|
50
59
|
```
|
|
51
60
|
/mcp
|
|
52
61
|
```
|
|
53
62
|
|
|
54
|
-
|
|
63
|
+
Check the output for the **"User MCPs"** section to find the config file path, then edit that file.
|
|
55
64
|
|
|
56
|
-
|
|
65
|
+
Common paths (if `/mcp` is unavailable):
|
|
57
66
|
- **Windows**: `%USERPROFILE%\.claude.json`
|
|
58
67
|
- **macOS**: `~/.claude.json`
|
|
59
68
|
- **Linux**: `~/.claude.json`
|
|
60
|
-
-
|
|
69
|
+
- **Legacy/Alternative**: `~/.claude/mcp.json`
|
|
61
70
|
|
|
62
|
-
|
|
71
|
+
Paste this (replace `your-api-key`):
|
|
63
72
|
|
|
64
73
|
```json
|
|
65
74
|
{
|
|
@@ -75,20 +84,20 @@ AI 会自动完成:
|
|
|
75
84
|
}
|
|
76
85
|
```
|
|
77
86
|
|
|
78
|
-
|
|
87
|
+
Save and run `/mcp` to verify it's loaded.
|
|
79
88
|
|
|
80
89
|
### 2. Cursor
|
|
81
90
|
|
|
82
|
-
|
|
91
|
+
Go to **Settings > Tools & MCPs > Add New MCP Server**:
|
|
83
92
|
|
|
84
|
-
- **Name
|
|
85
|
-
- **Type
|
|
86
|
-
- **Command
|
|
93
|
+
- **Name**: `video-enhancement`
|
|
94
|
+
- **Type**: `command`
|
|
95
|
+
- **Command**:
|
|
87
96
|
```bash
|
|
88
97
|
env API_KEY=your-api-key npx -y @avclabs.ai/media-mcp@latest
|
|
89
98
|
```
|
|
90
99
|
|
|
91
|
-
|
|
100
|
+
Or edit `~/.cursor/mcp.json`:
|
|
92
101
|
|
|
93
102
|
```json
|
|
94
103
|
{
|
|
@@ -104,100 +113,112 @@ AI 会自动完成:
|
|
|
104
113
|
}
|
|
105
114
|
```
|
|
106
115
|
|
|
107
|
-
##
|
|
116
|
+
## Verify Installation
|
|
108
117
|
|
|
109
|
-
|
|
118
|
+
After restarting your client, check if the tools are available:
|
|
110
119
|
|
|
111
|
-
1.
|
|
112
|
-
2.
|
|
120
|
+
1. Or ask: "What tools do you have available?"
|
|
121
|
+
2. With the current npm `latest=0.2.1`, you should see `create_task`, `get_task_status`, `enhance_video_sync`, `sam3_predict`, and `get_sam3_task_status`.
|
|
122
|
+
3. Once the image tools ship in an upcoming release, you will additionally see `enhance_image_sync`, `colorize_image_sync`, `denoise_image_sync`, and `get_image_task_status`.
|
|
113
123
|
|
|
114
|
-
##
|
|
124
|
+
## Configuration Options
|
|
115
125
|
|
|
116
|
-
|
|
|
126
|
+
| Variable | Required | Default | Description |
|
|
117
127
|
|---|---|---|---|
|
|
118
|
-
| `API_KEY` |
|
|
119
|
-
| `HTTP_API_BASE_URL` |
|
|
120
|
-
| `
|
|
121
|
-
| `
|
|
122
|
-
| `
|
|
128
|
+
| `API_KEY` | **Yes** | - | API authentication key (shared by video, image, and SAM3 services) |
|
|
129
|
+
| `HTTP_API_BASE_URL` | No | `https://mcp.avc.ai/enhance` | Shared video and image HTTP service endpoint |
|
|
130
|
+
| `IMAGE_API_BASE_URL` | No | Same as `HTTP_API_BASE_URL` | Optional image endpoint override; omit for the shared production server |
|
|
131
|
+
| `SAM3_API_BASE_URL` | No | `https://mcp.avc.ai/sam` | SAM3 service endpoint |
|
|
132
|
+
| `SAM3_POLL_INTERVAL_MS` | No | `2000` | SAM3 polling interval in milliseconds (`SAM3_POLL_INTERVAL` is a deprecated alias) |
|
|
133
|
+
| `SAM3_POLL_MAX_ATTEMPTS` | No | `25` | SAM3 maximum status queries; the time budget is interval × attempts, and whichever limit is hit first truncates the wait |
|
|
123
134
|
|
|
124
|
-
|
|
135
|
+
`IMAGE_API_BASE_URL` is an optional override for pointing image calls at a custom image endpoint. When omitted, image and video calls share the `HTTP_API_BASE_URL` service — the recommended setup for the production server.
|
|
136
|
+
|
|
137
|
+
### Custom Endpoint
|
|
125
138
|
|
|
126
139
|
```json
|
|
127
140
|
{
|
|
128
141
|
"env": {
|
|
129
|
-
"HTTP_API_BASE_URL": "https://your-endpoint.com",
|
|
142
|
+
"HTTP_API_BASE_URL": "https://your-media-endpoint.com",
|
|
130
143
|
"API_KEY": "your-api-key",
|
|
131
144
|
"SAM3_API_BASE_URL": "https://your-sam3-endpoint.com"
|
|
132
145
|
}
|
|
133
146
|
}
|
|
134
147
|
```
|
|
135
148
|
|
|
136
|
-
|
|
149
|
+
Or via CLI args:
|
|
137
150
|
```bash
|
|
138
|
-
npx -y @avclabs.ai/media-mcp@latest --base-url https://your-endpoint.com --api-key your-api-key --sam3-base-url https://your-sam3-endpoint.com
|
|
151
|
+
npx -y @avclabs.ai/media-mcp@latest --base-url https://your-media-endpoint.com --api-key your-api-key --sam3-base-url https://your-sam3-endpoint.com
|
|
139
152
|
```
|
|
140
153
|
|
|
141
|
-
|
|
154
|
+
The optional `--image-base-url` flag is available for deployments that intentionally split the public image endpoint; the current npm `latest` release does not use it (image calls share the main endpoint).
|
|
155
|
+
|
|
156
|
+
## Recommended Workflow
|
|
142
157
|
|
|
143
|
-
|
|
158
|
+
This project provides both **synchronous** and **asynchronous** modes.
|
|
144
159
|
|
|
145
|
-
|
|
160
|
+
**Because MCP Agents typically enforce a ~60-second timeout per tool call**, tasks with longer processing times (video enhancement) are strongly recommended to use **asynchronous mode**:
|
|
146
161
|
|
|
147
|
-
###
|
|
162
|
+
### Asynchronous Mode (Recommended)
|
|
148
163
|
|
|
149
|
-
|
|
150
|
-
1.
|
|
151
|
-
2.
|
|
152
|
-
3.
|
|
153
|
-
4.
|
|
154
|
-
5.
|
|
164
|
+
**Video Enhancement:**
|
|
165
|
+
1. Call `create_task` to create a task → immediately get `task_id`
|
|
166
|
+
2. Wait a few seconds, then call `get_task_status` to query the status
|
|
167
|
+
3. If `status` is `processing`, continue waiting and repeat step 2
|
|
168
|
+
4. If `status` is `completed`, the task is done and the result contains `video_url`
|
|
169
|
+
5. If `status` is `failed`, the task failed and the result contains `error_message`
|
|
155
170
|
|
|
156
|
-
###
|
|
171
|
+
### Synchronous Mode (Simple Scenarios)
|
|
157
172
|
|
|
158
|
-
|
|
159
|
-
-
|
|
160
|
-
-
|
|
161
|
-
-
|
|
162
|
-
-
|
|
173
|
+
**Video Enhancement:**
|
|
174
|
+
- Call `enhance_video_sync` → the server polls internally
|
|
175
|
+
- Defaults to a maximum wait of 45 seconds
|
|
176
|
+
- If completed within 45 seconds, returns the result directly
|
|
177
|
+
- If not completed within 45 seconds, returns `task_id` and instructions for the Agent to switch to `get_task_status`
|
|
163
178
|
|
|
164
|
-
|
|
165
|
-
-
|
|
166
|
-
-
|
|
167
|
-
-
|
|
168
|
-
-
|
|
179
|
+
**Image Segmentation (SAM3):**
|
|
180
|
+
- Call `sam3_predict` → the server polls internally
|
|
181
|
+
- Defaults to at most 25 status queries within a ~50-second budget (25 × 2-second interval; whichever limit is reached first ends the wait)
|
|
182
|
+
- If the task completes within the budget, returns the segmentation result directly
|
|
183
|
+
- If not completed within the budget, returns a truncation notice (with `task_id`) indicating the task is still processing
|
|
169
184
|
|
|
170
|
-
##
|
|
185
|
+
## Usage Examples
|
|
171
186
|
|
|
172
|
-
|
|
187
|
+
Once configured, ask your AI agent naturally:
|
|
173
188
|
|
|
174
|
-
> "
|
|
189
|
+
> "Enhance this video to 1080p: https://example.com/video.mp4"
|
|
175
190
|
|
|
176
|
-
> "
|
|
191
|
+
> "Improve the quality of /Users/me/Desktop/video.mp4 to 2k"
|
|
177
192
|
|
|
178
|
-
> "
|
|
193
|
+
> "Enhance this image: https://example.com/photo.jpg"
|
|
179
194
|
|
|
180
|
-
> "
|
|
195
|
+
> "Colorize this black-and-white photo: /Users/me/Desktop/old_photo.png"
|
|
181
196
|
|
|
182
|
-
|
|
197
|
+
> "Remove noise from this image: C:\\Users\\xxx\\noisy.jpg"
|
|
183
198
|
|
|
184
|
-
|
|
199
|
+
> "Analyze this image and find all objects: C:\\Users\\xxx\\photo.png"
|
|
185
200
|
|
|
186
|
-
|
|
201
|
+
> "Use SAM3 to segment this image, prompt: 'find all cars'"
|
|
202
|
+
|
|
203
|
+
The agent will automatically choose sync or async tools based on task complexity.
|
|
204
|
+
|
|
205
|
+
## Provided Tools
|
|
206
|
+
|
|
207
|
+
### Video Enhancement
|
|
187
208
|
|
|
188
209
|
#### create_task
|
|
189
210
|
|
|
190
|
-
|
|
211
|
+
Create an asynchronous video enhancement task.
|
|
191
212
|
|
|
192
|
-
>
|
|
213
|
+
> **Recommended for most use cases.** Ideal for longer videos (over 10 seconds) to avoid timeouts and blocking the connection.
|
|
193
214
|
|
|
194
|
-
|
|
|
215
|
+
| Parameter | Type | Required | Default | Description |
|
|
195
216
|
|---|---|---|---|---|
|
|
196
|
-
| `video_source` | string |
|
|
197
|
-
| `type` | string |
|
|
198
|
-
| `resolution` | string |
|
|
217
|
+
| `video_source` | string | Yes | - | Video URL or local file path (URL must be publicly accessible, links requiring login or signatures are not supported) |
|
|
218
|
+
| `type` | string | No | `url` | `url` or `local` |
|
|
219
|
+
| `resolution` | string | No | `720p` | `480p`, `540p`, `720p`, `1080p`, `2k` |
|
|
199
220
|
|
|
200
|
-
|
|
221
|
+
**Returns:**
|
|
201
222
|
```json
|
|
202
223
|
{
|
|
203
224
|
"success": true,
|
|
@@ -208,15 +229,15 @@ AI 会根据任务复杂度自动选择同步或异步工具完成任务。
|
|
|
208
229
|
|
|
209
230
|
#### get_task_status
|
|
210
231
|
|
|
211
|
-
|
|
232
|
+
Query video enhancement task status.
|
|
212
233
|
|
|
213
|
-
>
|
|
234
|
+
> The returned `status` field can be: `processing`, `completed`, or `failed`. If `status` is `processing`, you need to wait a few seconds and call this tool again.
|
|
214
235
|
|
|
215
|
-
|
|
|
236
|
+
| Parameter | Type | Required |
|
|
216
237
|
|---|---|---|
|
|
217
|
-
| `task_id` | string |
|
|
238
|
+
| `task_id` | string | Yes |
|
|
218
239
|
|
|
219
|
-
|
|
240
|
+
**Returns:**
|
|
220
241
|
```json
|
|
221
242
|
{
|
|
222
243
|
"success": true,
|
|
@@ -224,76 +245,190 @@ AI 会根据任务复杂度自动选择同步或异步工具完成任务。
|
|
|
224
245
|
"status": "completed",
|
|
225
246
|
"progress": 100,
|
|
226
247
|
"video_url": "https://...",
|
|
227
|
-
"message": "
|
|
248
|
+
"message": "Task is still processing, please check again later"
|
|
228
249
|
}
|
|
229
250
|
```
|
|
230
251
|
|
|
231
|
-
|
|
252
|
+
The `message` field only appears when `status` is `processing`, prompting the Agent to continue waiting.
|
|
232
253
|
|
|
233
254
|
#### enhance_video_sync
|
|
234
255
|
|
|
235
|
-
|
|
256
|
+
Synchronously enhance video (blocks until completion).
|
|
236
257
|
|
|
237
|
-
>
|
|
258
|
+
> **Best for short videos (estimated processing time < 1 minute).** If the task is not completed within the default 45-second wait budget, the tool returns early with a `task_id`, and you need to use `get_task_status` to continue querying.
|
|
238
259
|
|
|
239
|
-
|
|
|
260
|
+
| Parameter | Type | Required | Default | Description |
|
|
240
261
|
|---|---|---|---|---|
|
|
241
|
-
| `video_source` | string |
|
|
242
|
-
| `type` | string |
|
|
243
|
-
| `resolution` | string |
|
|
244
|
-
| `poll_interval` | number |
|
|
245
|
-
| `timeout` | number |
|
|
262
|
+
| `video_source` | string | Yes | - | Video URL or local file path |
|
|
263
|
+
| `type` | string | No | `url` | `url` or `local` |
|
|
264
|
+
| `resolution` | string | No | `720p` | Target resolution |
|
|
265
|
+
| `poll_interval` | number | No | `5` | Poll interval (seconds, 0.5-30) |
|
|
266
|
+
| `timeout` | number | No | `45` | Sync wait timeout in seconds (1-45), returns early when exceeded |
|
|
267
|
+
|
|
268
|
+
**Truncated return example (not completed within the default 45-second budget):**
|
|
269
|
+
```json
|
|
270
|
+
{
|
|
271
|
+
"success": true,
|
|
272
|
+
"status": "processing",
|
|
273
|
+
"task_id": "xxx",
|
|
274
|
+
"message": "Task is still processing (waited 45 seconds, 9 status queries). Please use get_task_status to continue polling.",
|
|
275
|
+
"note": "The synchronous wait for this long-running task has been truncated. Switch to get_task_status polling."
|
|
276
|
+
}
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
### Image Enhancement
|
|
280
|
+
|
|
281
|
+
Three image processing tools are provided, each targeting a specific use case:
|
|
282
|
+
|
|
283
|
+
| Tool | Function | Use Case |
|
|
284
|
+
|---|---|---|
|
|
285
|
+
| `enhance_image_sync` | Image quality enhancement & face optimization | Blurry, low-resolution, or degraded photos |
|
|
286
|
+
| `colorize_image_sync` | Black-and-white photo colorization | Restoring old B&W photos with realistic colors |
|
|
287
|
+
| `denoise_image_sync` | Image noise removal | Noisy/grainy photos taken in low light |
|
|
288
|
+
|
|
289
|
+
All three tools share the same parameters and behavior pattern. They are **synchronous** — the tool blocks until the image is processed or the timeout is reached.
|
|
290
|
+
|
|
291
|
+
**Supported image formats**: PNG, JPG, JPEG, BMP, WebP, etc.
|
|
292
|
+
|
|
293
|
+
**Two upload methods**:
|
|
294
|
+
1. **URL upload**: provide a publicly accessible image URL (`type: "url"`)
|
|
295
|
+
2. **Local upload**: provide a local file path, the MCP Server auto-uploads to TOS object storage (`type: "local"`, max file size: 100MB)
|
|
296
|
+
|
|
297
|
+
#### enhance_image_sync
|
|
298
|
+
|
|
299
|
+
Synchronously enhance an image to improve quality and optimize faces.
|
|
300
|
+
|
|
301
|
+
> The tool internally creates a task and polls for the result. If processing completes within the timeout (default 45s), the result is returned directly. If not, the tool returns early with a `task_id` — use `get_image_task_status` to continue polling.
|
|
302
|
+
|
|
303
|
+
| Parameter | Type | Required | Default | Description |
|
|
304
|
+
|---|---|---|---|---|
|
|
305
|
+
| `image_source` | string | Yes | - | Image URL or local file path (URL must be publicly accessible, links requiring login or signatures are not supported) |
|
|
306
|
+
| `type` | string | No | `url` | `url` or `local` |
|
|
307
|
+
| `scale` | number | No | `2` | Enhancement scale multiplier (e.g. `2` for 2x, `4` for 4x upscaling) |
|
|
308
|
+
| `poll_interval` | number | No | `5` | Poll interval in seconds (0.5-30) |
|
|
309
|
+
| `timeout` | number | No | `45` | Sync wait timeout in seconds (1-45), returns early when exceeded |
|
|
310
|
+
|
|
311
|
+
**Normal completion return:**
|
|
312
|
+
```json
|
|
313
|
+
{
|
|
314
|
+
"success": true,
|
|
315
|
+
"task_id": "xxx",
|
|
316
|
+
"status": "completed",
|
|
317
|
+
"progress": 100,
|
|
318
|
+
"image_url": "https://..."
|
|
319
|
+
}
|
|
320
|
+
```
|
|
246
321
|
|
|
247
|
-
|
|
322
|
+
**Truncated return (not completed within the default 45-second budget):**
|
|
248
323
|
```json
|
|
249
324
|
{
|
|
250
325
|
"success": true,
|
|
251
326
|
"status": "processing",
|
|
252
327
|
"task_id": "xxx",
|
|
253
|
-
"message": "
|
|
254
|
-
"note": "
|
|
328
|
+
"message": "Task is still processing (waited 45 seconds, 9 status queries). Please use get_image_task_status to continue polling.",
|
|
329
|
+
"note": "The synchronous wait for this long-running task has been truncated. Switch to get_image_task_status polling."
|
|
255
330
|
}
|
|
256
331
|
```
|
|
257
332
|
|
|
258
|
-
|
|
333
|
+
#### colorize_image_sync
|
|
334
|
+
|
|
335
|
+
Synchronously colorize a black-and-white photo with AI.
|
|
336
|
+
|
|
337
|
+
> Best for old black-and-white photos. The AI will add realistic colors to the image. Supports the same parameters and return format as `enhance_image_sync`.
|
|
338
|
+
|
|
339
|
+
| Parameter | Type | Required | Default | Description |
|
|
340
|
+
|---|---|---|---|---|
|
|
341
|
+
| `image_source` | string | Yes | - | Image URL or local file path (URL must be publicly accessible, links requiring login or signatures are not supported) |
|
|
342
|
+
| `type` | string | No | `url` | `url` or `local` |
|
|
343
|
+
| `poll_interval` | number | No | `5` | Poll interval in seconds (0.5-30) |
|
|
344
|
+
| `timeout` | number | No | `45` | Sync wait timeout in seconds (1-45), returns early when exceeded |
|
|
345
|
+
|
|
346
|
+
**Returns:** Same format as `enhance_image_sync`.
|
|
347
|
+
|
|
348
|
+
#### denoise_image_sync
|
|
349
|
+
|
|
350
|
+
Synchronously remove noise from an image.
|
|
351
|
+
|
|
352
|
+
> Best for grainy/noisy photos taken in low-light conditions or with high ISO settings. Supports the same parameters and return format as `enhance_image_sync`.
|
|
353
|
+
|
|
354
|
+
| Parameter | Type | Required | Default | Description |
|
|
355
|
+
|---|---|---|---|---|
|
|
356
|
+
| `image_source` | string | Yes | - | Image URL or local file path (URL must be publicly accessible, links requiring login or signatures are not supported) |
|
|
357
|
+
| `type` | string | No | `url` | `url` or `local` |
|
|
358
|
+
| `poll_interval` | number | No | `5` | Poll interval in seconds (0.5-30) |
|
|
359
|
+
| `timeout` | number | No | `45` | Sync wait timeout in seconds (1-45), returns early when exceeded |
|
|
360
|
+
|
|
361
|
+
**Returns:** Same format as `enhance_image_sync`.
|
|
362
|
+
|
|
363
|
+
#### get_image_task_status
|
|
364
|
+
|
|
365
|
+
Query image processing task status. Used to poll for results when a sync tool times out.
|
|
366
|
+
|
|
367
|
+
> The returned `status` field can be: `processing`, `completed`, or `failed`. If `status` is `processing`, wait a few seconds and call this tool again.
|
|
368
|
+
|
|
369
|
+
| Parameter | Type | Required |
|
|
370
|
+
|---|---|---|
|
|
371
|
+
| `task_id` | string | Yes |
|
|
372
|
+
|
|
373
|
+
**Returns:**
|
|
374
|
+
```json
|
|
375
|
+
{
|
|
376
|
+
"success": true,
|
|
377
|
+
"task_id": "xxx",
|
|
378
|
+
"status": "completed",
|
|
379
|
+
"progress": 100,
|
|
380
|
+
"image_url": "https://...",
|
|
381
|
+
"message": "Task is still processing, please check again later"
|
|
382
|
+
}
|
|
383
|
+
```
|
|
384
|
+
|
|
385
|
+
The `message` field only appears when `status` is `processing`, prompting the Agent to continue waiting.
|
|
386
|
+
|
|
387
|
+
#### Recommended Workflow for Image Tools
|
|
388
|
+
|
|
389
|
+
1. **For most images**: Call `enhance_image_sync` / `colorize_image_sync` / `denoise_image_sync` directly — the tool handles everything and returns the result
|
|
390
|
+
2. **If truncated**: The tool returns a `task_id`, then use `get_image_task_status` to poll until `status` becomes `completed` or `failed`
|
|
391
|
+
3. **If failed**: Check the `error_message` field for details
|
|
392
|
+
|
|
393
|
+
### Image Segmentation (SAM3)
|
|
259
394
|
|
|
260
395
|
#### sam3_predict
|
|
261
396
|
|
|
262
|
-
|
|
397
|
+
Analyze an image using the SAM3 segmentation API to generate inference results (masks, boxes, scores).
|
|
263
398
|
|
|
264
|
-
|
|
399
|
+
**Parameters:**
|
|
265
400
|
|
|
266
|
-
|
|
401
|
+
Image input (choose one, must provide exactly one):
|
|
267
402
|
|
|
268
|
-
- `imagePath` (string):
|
|
269
|
-
-
|
|
270
|
-
-
|
|
403
|
+
- `imagePath` (string): Absolute path of a local image file. Supports common formats (PNG, JPG, JPEG).
|
|
404
|
+
- Example: `"C:\\Users\\xxx\\photo.png"`, `"/home/user/images/cat.jpg"`
|
|
405
|
+
- Use when: The user explicitly provides a local file path
|
|
271
406
|
|
|
272
|
-
- `imageUrl` (string):
|
|
273
|
-
-
|
|
274
|
-
-
|
|
275
|
-
-
|
|
407
|
+
- `imageUrl` (string): Publicly accessible URL of the image.
|
|
408
|
+
- Example: `"https://example.com/photo.jpg"`
|
|
409
|
+
- Use when: The image is already online and the user provides a link
|
|
410
|
+
- Note: The URL must be publicly accessible. Links requiring login or signatures are not supported
|
|
276
411
|
|
|
277
|
-
- `imageBase64` (string): Base64
|
|
278
|
-
-
|
|
279
|
-
-
|
|
280
|
-
-
|
|
412
|
+
- `imageBase64` (string): Base64-encoded image data.
|
|
413
|
+
- Example: `"iVBORw0KGgoAAAANSUhEUgAA..."`
|
|
414
|
+
- Use when: The user drags or uploads an image attachment, and the Agent encodes it as base64
|
|
415
|
+
- Note: Large images will produce very large base64 strings, which may slow transmission
|
|
281
416
|
|
|
282
|
-
|
|
417
|
+
Other parameters:
|
|
283
418
|
|
|
284
|
-
- `prompt` (string, required):
|
|
419
|
+
- `prompt` (string, required): English text prompt specifying the target object to segment. Since the SAM3 model only accepts English prompts, provide an English description. If the user provides Chinese or other non-English text, the Agent will automatically translate it before calling the tool.
|
|
285
420
|
|
|
286
|
-
|
|
421
|
+
**Normal completion return:**
|
|
287
422
|
|
|
288
|
-
|
|
423
|
+
After inference completes, returns a JSON string containing three fields:
|
|
289
424
|
|
|
290
|
-
- **`masks
|
|
291
|
-
- **`boxes
|
|
425
|
+
- **`masks`**: 2D array. Each element is a binary mask (values 0 or 1) with the same dimensions as the input image, marking the pixel-level location of detected objects. The i-th mask corresponds to the i-th detected object instance.
|
|
426
|
+
- **`boxes`**: 2D array. Each element is a bounding box in `[x1, y1, x2, y2]` format, representing the rectangular region of the detected object. `x1`, `y1` are the top-left coordinates; `x2`, `y2` are the bottom-right coordinates.
|
|
292
427
|
|
|
293
|
-
|
|
294
|
-
- **`scores
|
|
428
|
+
Coordinate system: The top-left corner of the image is the origin `(0, 0)`. The x-axis increases to the right, and the y-axis increases downward, in pixels. For example, `[120, 80, 300, 450]` means the region starts 120px from the left edge and 80px from the top edge, extending to 300px from the left and 450px from the top. Width = `x2 - x1 = 180px`, Height = `y2 - y1 = 370px`.
|
|
429
|
+
- **`scores`**: 1D array. Each element is a confidence score for the corresponding detection result, ranging from 0 to 1. Higher scores indicate greater model confidence.
|
|
295
430
|
|
|
296
|
-
|
|
431
|
+
Example result JSON:
|
|
297
432
|
|
|
298
433
|
```json
|
|
299
434
|
{
|
|
@@ -309,111 +444,190 @@ AI 会根据任务复杂度自动选择同步或异步工具完成任务。
|
|
|
309
444
|
}
|
|
310
445
|
```
|
|
311
446
|
|
|
312
|
-
|
|
447
|
+
**Truncated return example (not completed within 50s):**
|
|
313
448
|
```json
|
|
314
449
|
{
|
|
315
450
|
"success": true,
|
|
316
451
|
"status": "processing",
|
|
317
452
|
"task_id": "xxx",
|
|
318
|
-
"message": "Task is still processing (waited
|
|
319
|
-
"note": "The synchronous wait for this long-running task has been truncated."
|
|
453
|
+
"message": "Task is still processing (waited 50 seconds, 25 status queries). Please use get_sam3_task_status to continue polling.",
|
|
454
|
+
"note": "The synchronous wait for this long-running task has been truncated. Switch to task status polling."
|
|
320
455
|
}
|
|
321
456
|
```
|
|
322
457
|
|
|
323
|
-
|
|
458
|
+
**Result download failure (completed task, missing result):**
|
|
459
|
+
If the task completed but the result file could not be downloaded, the tool returns `success: false` with the `task_id` and `status: "completed"`. Use `get_sam3_task_status` with that `task_id` to recover the result. Do not resubmit the task.
|
|
460
|
+
```json
|
|
461
|
+
{
|
|
462
|
+
"success": false,
|
|
463
|
+
"task_id": "xxx",
|
|
464
|
+
"status": "completed",
|
|
465
|
+
"error": "Failed to download the completed result for task xxx: ...",
|
|
466
|
+
"note": "The task has completed; only the result download failed. Do not resubmit. Use get_sam3_task_status with this task_id to recover the result."
|
|
467
|
+
}
|
|
468
|
+
```
|
|
324
469
|
|
|
325
|
-
|
|
470
|
+
#### get_sam3_task_status
|
|
326
471
|
|
|
327
|
-
|
|
472
|
+
Query SAM3 segmentation task status. Used to poll for results when `sam3_predict` times out.
|
|
328
473
|
|
|
329
|
-
|
|
474
|
+
> The returned `status` field can be: `processing`, `completed`, or `failed`. If `status` is `processing`, wait a few seconds and call this tool again.
|
|
330
475
|
|
|
331
|
-
|
|
476
|
+
| Parameter | Type | Required |
|
|
477
|
+
|---|---|---|
|
|
478
|
+
| `task_id` | string | Yes |
|
|
332
479
|
|
|
333
|
-
|
|
480
|
+
**Completed return:**
|
|
481
|
+
```json
|
|
482
|
+
{
|
|
483
|
+
"success": true,
|
|
484
|
+
"task_id": "xxx",
|
|
485
|
+
"status": "completed",
|
|
486
|
+
"result_url": "https://..."
|
|
487
|
+
}
|
|
488
|
+
```
|
|
334
489
|
|
|
335
|
-
|
|
490
|
+
**Processing return:**
|
|
491
|
+
```json
|
|
492
|
+
{
|
|
493
|
+
"success": true,
|
|
494
|
+
"task_id": "xxx",
|
|
495
|
+
"status": "processing",
|
|
496
|
+
"message": "Task is still processing, please check again later."
|
|
497
|
+
}
|
|
498
|
+
```
|
|
336
499
|
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
500
|
+
**Failed return:**
|
|
501
|
+
```json
|
|
502
|
+
{
|
|
503
|
+
"success": false,
|
|
504
|
+
"task_id": "xxx",
|
|
505
|
+
"status": "failed",
|
|
506
|
+
"error": "Task failed"
|
|
507
|
+
}
|
|
508
|
+
```
|
|
509
|
+
|
|
510
|
+
## FAQ
|
|
511
|
+
|
|
512
|
+
### Is AVCLabs Media MCP free to use?
|
|
513
|
+
|
|
514
|
+
You get a free quota when you register — no credit card required. Beyond the free quota, pricing is pay-as-you-go credit packs (from $19.95) and monthly subscriptions (from $9.95/month). See https://mcp.avc.ai/pricing/ for the current plans.
|
|
515
|
+
|
|
516
|
+
### Which AI clients work with it?
|
|
517
|
+
|
|
518
|
+
Any MCP-compatible client. One-command setup guides are available for Claude Code, Cursor, and Windsurf at https://mcp.avc.ai/docs/ (the package speaks standard MCP stdio, so other agents work by adding the same `npx` config).
|
|
519
|
+
|
|
520
|
+
### Does it need a GPU or powerful hardware?
|
|
521
|
+
|
|
522
|
+
No. Video enhancement and SAM3 inference run on AVCLabs' cloud infrastructure. Your machine only runs the lightweight MCP stdio server process (Node.js 18+), so a plain laptop is enough.
|
|
523
|
+
|
|
524
|
+
### Where do I get an API key?
|
|
525
|
+
|
|
526
|
+
Register at https://mcp.avc.ai/ and create a key in the dashboard. The key authenticates video enhancement, image tools, and SAM3 segmentation.
|
|
527
|
+
|
|
528
|
+
### What does `sam3_predict` return?
|
|
342
529
|
|
|
343
|
-
|
|
530
|
+
Pixel-level segmentation masks, bounding boxes, and per-instance confidence scores for the objects described by your text prompt — ready to plug into compositing, annotation, or further analysis.
|
|
344
531
|
|
|
345
|
-
|
|
532
|
+
### Agent reports timeout when calling tools?
|
|
346
533
|
|
|
347
|
-
|
|
534
|
+
This is the primary issue this project addresses. MCP Agents (such as Claude, Cursor) typically enforce a ~60-second timeout per tool call. If task processing exceeds this limit, the Agent will error and disconnect.
|
|
348
535
|
|
|
349
|
-
|
|
350
|
-
> "请处理这张图片 `D:\\photos\\cat.jpg`,找出里面的猫"
|
|
536
|
+
**Solutions:**
|
|
351
537
|
|
|
352
|
-
|
|
538
|
+
1. **Prefer asynchronous tools**: For video enhancement and other time-consuming tasks, always use `create_task` + `get_task_status`. These tools return instantly on each call and will not trigger timeouts.
|
|
353
539
|
|
|
354
|
-
|
|
540
|
+
2. **Sync tool truncation mechanism**: `enhance_video_sync` has an internal 45-second truncation limit (default `timeout`). If the task is not completed within the budget, the tool proactively returns a `task_id` and instructs the Agent to use `get_task_status` to follow up.
|
|
355
541
|
|
|
356
|
-
|
|
542
|
+
3. **SAM3 truncation mechanism**: `sam3_predict` is bounded by two independent limits derived from its config: at most `SAM3_POLL_MAX_ATTEMPTS` status queries (default 25) and a time budget of `SAM3_POLL_INTERVAL_MS × attempts` (default 25 × 2 s ≈ 50 s). Whichever runs out first truncates the wait; the returned notice carries the `task_id` and the actual elapsed time and query count, and the task keeps running remotely.
|
|
357
543
|
|
|
358
|
-
|
|
544
|
+
4. **Adjust SAM3 polling parameters** (advanced): The two SAM3 limits are independent, so you can trade interval for attempt count freely (e.g. `SAM3_POLL_INTERVAL_MS=1000` with `SAM3_POLL_MAX_ATTEMPTS=50` still gives a ~50-second budget with twice as many queries). Lower the interval if the task usually finishes in a few seconds and you want quicker completion detection. Invalid explicit values (non-integer or outside the allowed range) abort startup before any task can be submitted. The wait budget covers only status polling after the task is created — source preparation, upload, task submission, and the final result download are not part of it, so the total tool call can take longer than the budget.
|
|
359
545
|
|
|
360
|
-
-
|
|
361
|
-
- 你提供了网络链接 → 使用 `imageUrl`
|
|
362
|
-
- 你拖拽了附件且没有路径 → 尝试 `imageBase64`
|
|
546
|
+
### Drag-and-drop attachment says file not found?
|
|
363
547
|
|
|
364
|
-
|
|
548
|
+
This is a known limitation of stdio MCP. When dragging or uploading an attachment through the Agent interface, the file path is usually not automatically passed to the MCP Server.
|
|
365
549
|
|
|
366
|
-
|
|
550
|
+
**Solutions:**
|
|
367
551
|
|
|
368
|
-
|
|
552
|
+
1. **Provide the path simultaneously** (recommended): After dragging the image, add the local absolute path in your message:
|
|
553
|
+
> "Please analyze this image `D:\\photos\\cat.jpg` and find the cat"
|
|
369
554
|
|
|
370
|
-
|
|
555
|
+
2. **Wait for auto-encoding**: Claude may automatically encode the image as base64. If successful, no extra action is needed.
|
|
371
556
|
|
|
372
|
-
|
|
557
|
+
3. **Reply to path inquiry**: If Claude asks for the image path, simply reply with the local absolute path.
|
|
373
558
|
|
|
374
|
-
|
|
559
|
+
### Is there a priority among the three input methods?
|
|
375
560
|
|
|
376
|
-
|
|
377
|
-
2. 或先将图片压缩后再编码
|
|
561
|
+
There is no strict priority. Claude will automatically choose the most appropriate method based on conversation context:
|
|
378
562
|
|
|
379
|
-
|
|
563
|
+
- You provided a local path → uses `imagePath`
|
|
564
|
+
- You provided a web link → uses `imageUrl`
|
|
565
|
+
- You dragged an attachment without a path → tries `imageBase64`
|
|
380
566
|
|
|
381
|
-
|
|
382
|
-
1. 读取本地文件
|
|
383
|
-
2. 通过预签名 URL 直传到 TOS 对象存储
|
|
384
|
-
3. **最大文件大小:100MB**
|
|
567
|
+
### What image formats are supported?
|
|
385
568
|
|
|
386
|
-
|
|
569
|
+
Common formats: PNG, JPG, JPEG, BMP, WebP, etc. PNG or JPG is recommended.
|
|
570
|
+
|
|
571
|
+
### What if URL image download fails?
|
|
572
|
+
|
|
573
|
+
Ensure the URL is **publicly accessible**, requiring no login, cookies, or signatures. If the image is on a service requiring authentication (e.g., private S3 Bucket, login-required image host), download it locally first and use `imagePath`.
|
|
574
|
+
|
|
575
|
+
### What if the base64 image is too large?
|
|
576
|
+
|
|
577
|
+
If the image is very large (e.g., 4K resolution), the base64-encoded data will be very large and may slow transmission. Suggestions:
|
|
578
|
+
|
|
579
|
+
1. Use `imagePath` instead
|
|
580
|
+
2. Or compress the image before encoding
|
|
581
|
+
|
|
582
|
+
## File Upload Notes
|
|
583
|
+
|
|
584
|
+
When `type` is `"local"`:
|
|
585
|
+
1. File is read locally by the MCP Server
|
|
586
|
+
2. Uploaded directly to TOS object storage via pre-signed URL
|
|
587
|
+
3. **Max file size: 100MB**
|
|
588
|
+
|
|
589
|
+
## Troubleshooting
|
|
387
590
|
|
|
388
591
|
### "command not found: npx"
|
|
389
592
|
|
|
390
|
-
|
|
593
|
+
Install Node.js >= 18: https://nodejs.org/
|
|
391
594
|
|
|
392
|
-
### "
|
|
595
|
+
### "API key is required: pass --api-key or set the API_KEY environment variable"
|
|
393
596
|
|
|
394
|
-
API Key
|
|
597
|
+
Your API Key is missing. Double-check the `env.API_KEY` in your config.
|
|
395
598
|
|
|
396
|
-
### MCP Server
|
|
599
|
+
### MCP Server shows red/error in client
|
|
397
600
|
|
|
398
|
-
|
|
399
|
-
- **Claude Desktop macOS
|
|
400
|
-
- **Claude Desktop Windows
|
|
401
|
-
- **Cursor
|
|
601
|
+
Check logs:
|
|
602
|
+
- **Claude Desktop macOS**: `~/Library/Logs/Claude/mcp*.log`
|
|
603
|
+
- **Claude Desktop Windows**: `%APPDATA%\Claude\logs\mcp*.log`
|
|
604
|
+
- **Cursor**: Output panel > MCP
|
|
402
605
|
|
|
403
|
-
### "TOS
|
|
606
|
+
### "TOS upload failed"
|
|
404
607
|
|
|
405
|
-
|
|
608
|
+
Usually a signature mismatch. Ensure your `IMAGE_API_BASE_URL` (or its `HTTP_API_BASE_URL` fallback) and `API_KEY` are correct and active.
|
|
406
609
|
|
|
407
|
-
##
|
|
610
|
+
## Global Install (Alternative)
|
|
408
611
|
|
|
409
|
-
|
|
612
|
+
If you prefer not using `npx` every time:
|
|
410
613
|
|
|
411
614
|
```bash
|
|
412
615
|
npm install -g @avclabs.ai/media-mcp
|
|
413
616
|
```
|
|
414
617
|
|
|
415
|
-
|
|
618
|
+
Then use `"command": "media-mcp"` with `"args": ["--api-key", "your-api-key"]` in your config.
|
|
619
|
+
|
|
620
|
+
## Development and Release
|
|
621
|
+
|
|
622
|
+
This package runs locally in the MCP client and is released through npm; it is not a remote Node daemon. To verify a working tree before publishing:
|
|
623
|
+
|
|
624
|
+
```bash
|
|
625
|
+
npm ci
|
|
626
|
+
npm run release:verify
|
|
627
|
+
```
|
|
628
|
+
|
|
629
|
+
Maintainers: see the [release guide](https://github.com/avclabs/media-mcp/blob/main/docs/RELEASE.md) for the current release status, version synchronization, publish order, smoke tests, and the relationship with the portal/backend deployment.
|
|
416
630
|
|
|
417
631
|
## License
|
|
418
632
|
|
|
419
|
-
MIT License -
|
|
633
|
+
MIT License - See [LICENSE](LICENSE) file for details
|