@mobilenext/mobile-mcp 1.0.3 → 1.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ja.md +12 -7
- package/README.md +11 -7
- package/README.zh-CN.md +13 -8
- package/lib/coordinate-mapping.js +27 -0
- package/lib/index.js +65 -39
- package/lib/jpeg.js +41 -0
- package/lib/server.js +33 -10
- package/package.json +3 -2
package/README.ja.md
CHANGED
|
@@ -112,6 +112,7 @@ https://github.com/user-attachments/assets/bb084777-beb3-4930-ae6f-8d3fe694ddde
|
|
|
112
112
|
- **`mobile_get_device_logs`** - デバイスのライブログを収集します(Android は logcat、iOS は unified log)。ファイルへの保存も可能です
|
|
113
113
|
- **`mobile_list_crashes`** - デバイス上で利用可能なクラッシュレポートを一覧表示します
|
|
114
114
|
- **`mobile_get_crash`** - ID を指定してクラッシュレポートの全文を取得します
|
|
115
|
+
- **`mobile_batch_commands`** - 複数のツールを 1 回の呼び出しで順番に実行します(例: クリック、入力、クリック)。最後に画面上の要素を一覧表示することもできます
|
|
115
116
|
|
|
116
117
|
## 🏗️ Mobile MCP のアーキテクチャ
|
|
117
118
|
|
|
@@ -387,9 +388,9 @@ npx @mobilenext/mobile-mcp@latest
|
|
|
387
388
|
log in to mobile next cloud and then show me which remote devices are available to me
|
|
388
389
|
```
|
|
389
390
|
|
|
390
|
-
###
|
|
391
|
+
### Streamable HTTP サーバーモード
|
|
391
392
|
|
|
392
|
-
Mobile MCP はデフォルトで stdio 上で動作します。代わりに
|
|
393
|
+
Mobile MCP はデフォルトで stdio 上で動作します。代わりに [Streamable HTTP](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#streamable-http) サーバーを起動するには、`--listen` フラグを使用します:
|
|
393
394
|
|
|
394
395
|
```bash
|
|
395
396
|
npx @mobilenext/mobile-mcp@latest --listen 3000
|
|
@@ -401,17 +402,21 @@ npx @mobilenext/mobile-mcp@latest --listen 3000
|
|
|
401
402
|
npx @mobilenext/mobile-mcp@latest --listen 0.0.0.0:3000
|
|
402
403
|
```
|
|
403
404
|
|
|
404
|
-
その後、MCP クライアントが `http://<host>:3000/mcp`
|
|
405
|
+
その後、MCP クライアントが `http://<host>:3000/mcp`(TLS 配下なら `https://…/mcp`)に接続するよう設定します。エンドポイントは Streamable HTTP(`/mcp` への `POST`)を受け付けます。リモートモードは **ステートレス** で(セッション親和性不要)、Smithery などの水平スケールするホストとも相性が良いです。
|
|
406
|
+
|
|
407
|
+
> **移行メモ:** 以前の `--listen` は非推奨の HTTP+SSE を `/mcp` で提供していました。クライアントは `http(s)://host:port/mcp` に対して Streamable HTTP を使う必要があります。以前の純粋な SSE フローは `/mcp` では利用できなくなりました。
|
|
408
|
+
|
|
409
|
+
localhost にバインドする場合、Host ヘッダーによる DNS リバインディング保護が自動的に有効になります。
|
|
405
410
|
|
|
406
411
|
#### 認可
|
|
407
412
|
|
|
408
|
-
|
|
413
|
+
HTTP サーバーで Bearer トークンによる認可を要求するには、環境変数 `MOBILEMCP_AUTH` を設定します:
|
|
409
414
|
|
|
410
415
|
```bash
|
|
411
416
|
MOBILEMCP_AUTH=my-secret-token npx @mobilenext/mobile-mcp@latest --listen 3000
|
|
412
417
|
```
|
|
413
418
|
|
|
414
|
-
設定すると、すべてのリクエストにヘッダー `Authorization: Bearer my-secret-token`
|
|
419
|
+
設定すると、すべてのリクエストにヘッダー `Authorization: Bearer my-secret-token` を含める必要があります。未設定の場合は認証なしで受け付け、警告を出力します。
|
|
415
420
|
|
|
416
421
|
### 🛠️ 使い方
|
|
417
422
|
|
|
@@ -484,7 +489,7 @@ Gmail to contacts "team@example.com".
|
|
|
484
489
|
|
|
485
490
|
| 変数 | 説明 | 例 |
|
|
486
491
|
|---|---|---|
|
|
487
|
-
| `MOBILEMCP_AUTH` |
|
|
492
|
+
| `MOBILEMCP_AUTH` | Streamable HTTP サーバー(`--listen`)で Bearer トークンを必須にします。設定すると、すべてのリクエストが `Authorization: Bearer <token>` を送信する必要があります。 | `MOBILEMCP_AUTH=my-secret-token` |
|
|
488
493
|
| `MOBILEMCP_DISABLE_TELEMETRY` | 匿名の利用テレメトリを無効にします。 | `MOBILEMCP_DISABLE_TELEMETRY=1` |
|
|
489
494
|
| `MOBILEMCP_ALLOW_UNSAFE_URLS` | `mobile_open_url` が標準外の URL スキームを開くことを許可します(デフォルトではブロックされます)。 | `MOBILEMCP_ALLOW_UNSAFE_URLS=1` |
|
|
490
495
|
| `MOBILEMCP_LEGACY_ROBOT` | Android デバイスおよび iOS 実機に対して、従来のプラットフォーム固有のロボットを使用します。iOS シミュレーターは引き続き `mobilecli` を使用します。 | `MOBILEMCP_LEGACY_ROBOT=1` |
|
|
@@ -500,7 +505,7 @@ Mobile Next Mobile MCP を実行する前に、モバイルプラットフォー
|
|
|
500
505
|
|
|
501
506
|
### テレメトリ
|
|
502
507
|
|
|
503
|
-
Mobile MCP は PostHog を通じて匿名の利用テレメトリを収集します。無効にするには、環境変数 `MOBILEMCP_DISABLE_TELEMETRY` を設定します:
|
|
508
|
+
Mobile MCP は PostHog と Scarf を通じて匿名の利用テレメトリを収集します。無効にするには、環境変数 `MOBILEMCP_DISABLE_TELEMETRY` を設定します:
|
|
504
509
|
|
|
505
510
|
```bash
|
|
506
511
|
MOBILEMCP_DISABLE_TELEMETRY=1 npx @mobilenext/mobile-mcp@latest
|
package/README.md
CHANGED
|
@@ -388,9 +388,9 @@ In your Agent, prompt:
|
|
|
388
388
|
log in to mobile next cloud and then show me which remote devices are available to me
|
|
389
389
|
```
|
|
390
390
|
|
|
391
|
-
###
|
|
391
|
+
### Streamable HTTP Server Mode
|
|
392
392
|
|
|
393
|
-
By default, Mobile MCP runs over stdio. To start
|
|
393
|
+
By default, Mobile MCP runs over stdio. To start a [Streamable HTTP](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#streamable-http) server instead, use the `--listen` flag:
|
|
394
394
|
|
|
395
395
|
```bash
|
|
396
396
|
npx @mobilenext/mobile-mcp@latest --listen 3000
|
|
@@ -402,17 +402,21 @@ This binds to `localhost:3000`. To bind to a specific interface:
|
|
|
402
402
|
npx @mobilenext/mobile-mcp@latest --listen 0.0.0.0:3000
|
|
403
403
|
```
|
|
404
404
|
|
|
405
|
-
Then configure your MCP client to connect to `http://<host>:3000/mcp
|
|
405
|
+
Then configure your MCP client to connect to `http://<host>:3000/mcp` (or `https://…/mcp` behind TLS). The endpoint accepts Streamable HTTP (`POST` on `/mcp`); remote mode is **stateless** (no session affinity required), which works well with Smithery and other horizontal hosts.
|
|
406
|
+
|
|
407
|
+
> **Migration note:** `--listen` previously served the deprecated HTTP+SSE transport on `/mcp`. Clients must use Streamable HTTP against `http(s)://host:port/mcp`. The old pure-SSE flow on `/mcp` is no longer available.
|
|
408
|
+
|
|
409
|
+
When binding to localhost, Host-header DNS rebinding protection is enabled automatically.
|
|
406
410
|
|
|
407
411
|
#### Authorization
|
|
408
412
|
|
|
409
|
-
To require Bearer token authorization on the
|
|
413
|
+
To require Bearer token authorization on the HTTP server, set the `MOBILEMCP_AUTH` environment variable:
|
|
410
414
|
|
|
411
415
|
```bash
|
|
412
416
|
MOBILEMCP_AUTH=my-secret-token npx @mobilenext/mobile-mcp@latest --listen 3000
|
|
413
417
|
```
|
|
414
418
|
|
|
415
|
-
When set, all requests must include the header `Authorization: Bearer my-secret-token`.
|
|
419
|
+
When set, all requests must include the header `Authorization: Bearer my-secret-token`. When unset, the server accepts unauthenticated connections and logs a warning.
|
|
416
420
|
|
|
417
421
|
### 🛠️ How to Use
|
|
418
422
|
|
|
@@ -485,7 +489,7 @@ Gmail to contacts "team@example.com".
|
|
|
485
489
|
|
|
486
490
|
| Variable | Description | Example |
|
|
487
491
|
|---|---|---|
|
|
488
|
-
| `MOBILEMCP_AUTH` | Require a Bearer token on the
|
|
492
|
+
| `MOBILEMCP_AUTH` | Require a Bearer token on the Streamable HTTP server (`--listen`) — every request must then send `Authorization: Bearer <token>`. | `MOBILEMCP_AUTH=my-secret-token` |
|
|
489
493
|
| `MOBILEMCP_DISABLE_TELEMETRY` | Disable anonymous usage telemetry. | `MOBILEMCP_DISABLE_TELEMETRY=1` |
|
|
490
494
|
| `MOBILEMCP_ALLOW_UNSAFE_URLS` | Allow `mobile_open_url` to open non-standard URL schemes (blocked by default). | `MOBILEMCP_ALLOW_UNSAFE_URLS=1` |
|
|
491
495
|
| `MOBILEMCP_LEGACY_ROBOT` | Use the legacy platform-specific robots for Android devices and physical iOS devices. iOS simulators continue to use `mobilecli`. | `MOBILEMCP_LEGACY_ROBOT=1` |
|
|
@@ -501,7 +505,7 @@ Make sure you have your mobile platform SDKs (Xcode, Android SDK) installed and
|
|
|
501
505
|
|
|
502
506
|
### Telemetry
|
|
503
507
|
|
|
504
|
-
Mobile MCP collects anonymous usage telemetry via PostHog. To disable it, set the `MOBILEMCP_DISABLE_TELEMETRY` environment variable:
|
|
508
|
+
Mobile MCP collects anonymous usage telemetry via PostHog and Scarf. To disable it, set the `MOBILEMCP_DISABLE_TELEMETRY` environment variable:
|
|
505
509
|
|
|
506
510
|
```bash
|
|
507
511
|
MOBILEMCP_DISABLE_TELEMETRY=1 npx @mobilenext/mobile-mcp@latest
|
package/README.zh-CN.md
CHANGED
|
@@ -112,6 +112,7 @@ https://github.com/user-attachments/assets/bb084777-beb3-4930-ae6f-8d3fe694ddde
|
|
|
112
112
|
- **`mobile_get_device_logs`** - 采集设备实时日志(Android 上为 logcat,iOS 上为 unified log),可选择保存到文件
|
|
113
113
|
- **`mobile_list_crashes`** - 列出设备上可用的崩溃报告
|
|
114
114
|
- **`mobile_get_crash`** - 按 ID 获取崩溃报告的完整内容
|
|
115
|
+
- **`mobile_batch_commands`** - 在一次调用中按顺序运行多个工具(例如点击、输入、点击),可选择在最后列出屏幕元素
|
|
115
116
|
|
|
116
117
|
## 🏗️ Mobile MCP 架构
|
|
117
118
|
|
|
@@ -387,9 +388,9 @@ npx @mobilenext/mobile-mcp@latest
|
|
|
387
388
|
log in to mobile next cloud and then show me which remote devices are available to me
|
|
388
389
|
```
|
|
389
390
|
|
|
390
|
-
###
|
|
391
|
+
### Streamable HTTP 服务器模式
|
|
391
392
|
|
|
392
|
-
Mobile MCP 默认通过 stdio 运行。若要改为启动
|
|
393
|
+
Mobile MCP 默认通过 stdio 运行。若要改为启动 [Streamable HTTP](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#streamable-http) 服务器,请使用 `--listen` 参数:
|
|
393
394
|
|
|
394
395
|
```bash
|
|
395
396
|
npx @mobilenext/mobile-mcp@latest --listen 3000
|
|
@@ -401,17 +402,21 @@ npx @mobilenext/mobile-mcp@latest --listen 3000
|
|
|
401
402
|
npx @mobilenext/mobile-mcp@latest --listen 0.0.0.0:3000
|
|
402
403
|
```
|
|
403
404
|
|
|
404
|
-
然后将你的 MCP 客户端配置为连接 `http://<host>:3000/mcp
|
|
405
|
+
然后将你的 MCP 客户端配置为连接 `http://<host>:3000/mcp`(TLS 后可用 `https://…/mcp`)。该端点接受 Streamable HTTP(对 `/mcp` 的 `POST`);远程模式为 **无状态**(无需会话亲和性),非常适合 Smithery 等水平扩展的托管平台。
|
|
406
|
+
|
|
407
|
+
> **迁移说明:** 此前 `--listen` 在 `/mcp` 上提供已弃用的 HTTP+SSE。客户端需对 `http(s)://host:port/mcp` 使用 Streamable HTTP。`/mcp` 上原有的纯 SSE 流程已不再可用。
|
|
408
|
+
|
|
409
|
+
绑定到 localhost 时,会自动启用基于 Host 请求头的 DNS 重绑定防护。
|
|
405
410
|
|
|
406
411
|
#### 认证授权
|
|
407
412
|
|
|
408
|
-
若要在
|
|
413
|
+
若要在 HTTP 服务器上强制使用 Bearer token 授权,请设置环境变量 `MOBILEMCP_AUTH`:
|
|
409
414
|
|
|
410
415
|
```bash
|
|
411
416
|
MOBILEMCP_AUTH=my-secret-token npx @mobilenext/mobile-mcp@latest --listen 3000
|
|
412
417
|
```
|
|
413
418
|
|
|
414
|
-
设置之后,所有请求都必须包含请求头 `Authorization: Bearer my-secret-token
|
|
419
|
+
设置之后,所有请求都必须包含请求头 `Authorization: Bearer my-secret-token`。未设置时接受未认证连接并输出警告。
|
|
415
420
|
|
|
416
421
|
### 🛠️ 如何使用
|
|
417
422
|
|
|
@@ -484,7 +489,7 @@ Gmail to contacts "team@example.com".
|
|
|
484
489
|
|
|
485
490
|
| 变量 | 说明 | 示例 |
|
|
486
491
|
|---|---|---|
|
|
487
|
-
| `MOBILEMCP_AUTH` | 要求
|
|
492
|
+
| `MOBILEMCP_AUTH` | 要求 Streamable HTTP 服务器(`--listen`)使用 Bearer token —— 设置后每个请求都必须发送 `Authorization: Bearer <token>`。 | `MOBILEMCP_AUTH=my-secret-token` |
|
|
488
493
|
| `MOBILEMCP_DISABLE_TELEMETRY` | 关闭匿名使用情况遥测。 | `MOBILEMCP_DISABLE_TELEMETRY=1` |
|
|
489
494
|
| `MOBILEMCP_ALLOW_UNSAFE_URLS` | 允许 `mobile_open_url` 打开非标准的 URL scheme(默认被阻止)。 | `MOBILEMCP_ALLOW_UNSAFE_URLS=1` |
|
|
490
495
|
| `MOBILEMCP_LEGACY_ROBOT` | 对 Android 设备和 iOS 真机使用旧版的平台专用 robot。iOS 模拟器仍然使用 `mobilecli`。 | `MOBILEMCP_LEGACY_ROBOT=1` |
|
|
@@ -500,7 +505,7 @@ Gmail to contacts "team@example.com".
|
|
|
500
505
|
|
|
501
506
|
### 遥测
|
|
502
507
|
|
|
503
|
-
Mobile MCP 通过 PostHog 收集匿名的使用情况遥测数据。若要关闭,请设置环境变量 `MOBILEMCP_DISABLE_TELEMETRY`:
|
|
508
|
+
Mobile MCP 通过 PostHog 和 Scarf 收集匿名的使用情况遥测数据。若要关闭,请设置环境变量 `MOBILEMCP_DISABLE_TELEMETRY`:
|
|
504
509
|
|
|
505
510
|
```bash
|
|
506
511
|
MOBILEMCP_DISABLE_TELEMETRY=1 npx @mobilenext/mobile-mcp@latest
|
|
@@ -569,4 +574,4 @@ Mobile MCP 是驱动真实移动设备这套工具集中的一环:
|
|
|
569
574
|
|
|
570
575
|
Mobile MCP 在本地运行,仅与你所连接的设备通信。
|
|
571
576
|
有关数据收集、使用、保留以及联系方式,请查看 Mobile Next 隐私政策:
|
|
572
|
-
https://mobilenext.ai/privacy。
|
|
577
|
+
[https://mobilenext.ai/privacy](https://mobilenext.ai/privacy)。
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.describeCoordinateMapping = void 0;
|
|
4
|
+
const RATIO_DECIMALS = 3;
|
|
5
|
+
const formatRatio = (ratio) => Number(ratio.toFixed(RATIO_DECIMALS)).toString();
|
|
6
|
+
const isPositive = (size) => size.width > 0 && size.height > 0;
|
|
7
|
+
/**
|
|
8
|
+
* Plain-words instruction for a model that reads pixel positions off a
|
|
9
|
+
* screenshot and taps in screen coordinates. The screenshot is usually
|
|
10
|
+
* downscaled, and on iOS the screen is measured in points while a full-size
|
|
11
|
+
* screenshot is in pixels, so the two spaces rarely agree.
|
|
12
|
+
*
|
|
13
|
+
* Returns null when either size is unknown, in which case no instruction is
|
|
14
|
+
* better than a wrong one.
|
|
15
|
+
*/
|
|
16
|
+
const describeCoordinateMapping = (screenshot, screen) => {
|
|
17
|
+
if (!isPositive(screenshot) || !isPositive(screen)) {
|
|
18
|
+
return null;
|
|
19
|
+
}
|
|
20
|
+
if (screenshot.width === screen.width && screenshot.height === screen.height) {
|
|
21
|
+
return `Screenshot is ${screenshot.width}x${screenshot.height} and its coordinates match the screen.`;
|
|
22
|
+
}
|
|
23
|
+
const x = formatRatio(screen.width / screenshot.width);
|
|
24
|
+
const y = formatRatio(screen.height / screenshot.height);
|
|
25
|
+
return `Screenshot is ${screenshot.width}x${screenshot.height}. Screen coordinates are ${screen.width}x${screen.height}. To tap something you see in this screenshot, multiply its x by ${x} and y by ${y}.`;
|
|
26
|
+
};
|
|
27
|
+
exports.describeCoordinateMapping = describeCoordinateMapping;
|
package/lib/index.js
CHANGED
|
@@ -1,21 +1,31 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
"use strict";
|
|
3
|
-
var __importDefault = (this && this.__importDefault) || function (mod) {
|
|
4
|
-
return (mod && mod.__esModule) ? mod : { "default": mod };
|
|
5
|
-
};
|
|
6
3
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
7
|
-
const
|
|
4
|
+
const streamableHttp_js_1 = require("@modelcontextprotocol/sdk/server/streamableHttp.js");
|
|
8
5
|
const stdio_js_1 = require("@modelcontextprotocol/sdk/server/stdio.js");
|
|
6
|
+
const express_js_1 = require("@modelcontextprotocol/sdk/server/express.js");
|
|
9
7
|
const server_1 = require("./server");
|
|
10
8
|
const logger_1 = require("./logger");
|
|
11
|
-
const express_1 = __importDefault(require("express"));
|
|
12
9
|
const commander_1 = require("commander");
|
|
13
|
-
const
|
|
14
|
-
|
|
15
|
-
|
|
10
|
+
const startHttpServer = async (host, port) => {
|
|
11
|
+
// createMcpExpressApp applies express.json() and Host-header DNS rebinding
|
|
12
|
+
// protection automatically when binding to localhost / 127.0.0.1 / ::1.
|
|
13
|
+
const app = (0, express_js_1.createMcpExpressApp)({ host });
|
|
14
|
+
// Migration hint for clients still pointing at the removed SSE transport.
|
|
15
|
+
// Registered before auth so unauthenticated clients see the reason, not a 401.
|
|
16
|
+
app.all("/sse", (_req, res) => {
|
|
17
|
+
res.status(410).json({
|
|
18
|
+
jsonrpc: "2.0",
|
|
19
|
+
error: {
|
|
20
|
+
code: -32000,
|
|
21
|
+
message: "SSE transport removed. mobile-mcp now serves MCP Streamable HTTP at /mcp. See https://github.com/mobile-next/mobile-mcp#streamable-http-server-mode",
|
|
22
|
+
},
|
|
23
|
+
id: null,
|
|
24
|
+
});
|
|
25
|
+
});
|
|
16
26
|
const authToken = process.env.MOBILEMCP_AUTH;
|
|
17
27
|
if (!authToken) {
|
|
18
|
-
(0, logger_1.error)("WARNING: MOBILEMCP_AUTH is not set. The
|
|
28
|
+
(0, logger_1.error)("WARNING: MOBILEMCP_AUTH is not set. The HTTP server will accept unauthenticated connections. Set MOBILEMCP_AUTH to require Bearer token authentication.");
|
|
19
29
|
}
|
|
20
30
|
if (authToken) {
|
|
21
31
|
app.use((req, res, next) => {
|
|
@@ -26,37 +36,53 @@ const startSseServer = async (host, port) => {
|
|
|
26
36
|
next();
|
|
27
37
|
});
|
|
28
38
|
}
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
transport.handlePostMessage(req, res);
|
|
39
|
+
const handleMcpRequest = async (req, res) => {
|
|
40
|
+
// Stateless Streamable HTTP: fresh server + transport per request
|
|
41
|
+
// (Smithery / horizontal-host friendly; no session affinity).
|
|
42
|
+
const server = (0, server_1.createMcpServer)();
|
|
43
|
+
try {
|
|
44
|
+
const transport = new streamableHttp_js_1.StreamableHTTPServerTransport({
|
|
45
|
+
sessionIdGenerator: undefined,
|
|
46
|
+
});
|
|
47
|
+
// Register cleanup first: the response can close before handleRequest resolves.
|
|
48
|
+
res.on("close", () => {
|
|
49
|
+
transport.close();
|
|
50
|
+
server.close();
|
|
51
|
+
});
|
|
52
|
+
await server.connect(transport);
|
|
53
|
+
await transport.handleRequest(req, res, req.body);
|
|
45
54
|
}
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
55
|
+
catch (err) {
|
|
56
|
+
(0, logger_1.error)("Error handling MCP request: " + (err instanceof Error ? err.stack : String(err)));
|
|
57
|
+
if (!res.headersSent) {
|
|
58
|
+
res.status(500).json({
|
|
59
|
+
jsonrpc: "2.0",
|
|
60
|
+
error: {
|
|
61
|
+
code: -32603,
|
|
62
|
+
message: "Internal server error",
|
|
63
|
+
},
|
|
64
|
+
id: null,
|
|
65
|
+
});
|
|
66
|
+
}
|
|
51
67
|
}
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
68
|
+
};
|
|
69
|
+
app.post("/mcp", handleMcpRequest);
|
|
70
|
+
// Stateless mode has no long-lived sessions; GET (SSE stream) and DELETE
|
|
71
|
+
// (session teardown) are not applicable. Return 405 per SDK guidance.
|
|
72
|
+
const methodNotAllowed = (_req, res) => {
|
|
73
|
+
res.status(405).json({
|
|
74
|
+
jsonrpc: "2.0",
|
|
75
|
+
error: {
|
|
76
|
+
code: -32000,
|
|
77
|
+
message: "Method not allowed.",
|
|
78
|
+
},
|
|
79
|
+
id: null,
|
|
80
|
+
});
|
|
81
|
+
};
|
|
82
|
+
app.get("/mcp", methodNotAllowed);
|
|
83
|
+
app.delete("/mcp", methodNotAllowed);
|
|
58
84
|
app.listen(port, host, () => {
|
|
59
|
-
(0, logger_1.error)(`mobile-mcp ${(0, server_1.getAgentVersion)()}
|
|
85
|
+
(0, logger_1.error)(`mobile-mcp ${(0, server_1.getAgentVersion)()} streamable http server listening on http://${host}:${port}/mcp`);
|
|
60
86
|
});
|
|
61
87
|
};
|
|
62
88
|
const startStdioServer = async () => {
|
|
@@ -84,7 +110,7 @@ const startStdioServer = async () => {
|
|
|
84
110
|
const main = async () => {
|
|
85
111
|
commander_1.program
|
|
86
112
|
.version((0, server_1.getAgentVersion)())
|
|
87
|
-
.option("--listen <listen>", "Start
|
|
113
|
+
.option("--listen <listen>", "Start Streamable HTTP server on [host:]port")
|
|
88
114
|
.option("--stdio", "Start stdio server (default)")
|
|
89
115
|
.parse(process.argv);
|
|
90
116
|
const options = commander_1.program.opts();
|
|
@@ -105,7 +131,7 @@ const main = async () => {
|
|
|
105
131
|
(0, logger_1.error)(`Invalid --listen value "${listen}". Expected [host:]port with port 1-65535.`);
|
|
106
132
|
process.exit(1);
|
|
107
133
|
}
|
|
108
|
-
await
|
|
134
|
+
await startHttpServer(host, port);
|
|
109
135
|
}
|
|
110
136
|
else {
|
|
111
137
|
await startStdioServer();
|
package/lib/jpeg.js
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.getJpegDimensions = void 0;
|
|
4
|
+
const robot_1 = require("./robot");
|
|
5
|
+
const JPEG_SOI = 0xffd8;
|
|
6
|
+
const MARKER_PREFIX = 0xff;
|
|
7
|
+
// length(2) precision(1) height(2) width(2) components(1)
|
|
8
|
+
const SOF_MIN_SEGMENT_LENGTH = 8;
|
|
9
|
+
// SOFn markers carry the frame header with the image dimensions. 0xc4 (DHT),
|
|
10
|
+
// 0xc8 (JPG) and 0xcc (DAC) sit in the same range but are not frame headers.
|
|
11
|
+
const isStartOfFrameMarker = (marker) => marker >= 0xc0 && marker <= 0xcf && marker !== 0xc4 && marker !== 0xc8 && marker !== 0xcc;
|
|
12
|
+
const getJpegDimensions = (buffer) => {
|
|
13
|
+
if (buffer.length < 4 || buffer.readUInt16BE(0) !== JPEG_SOI) {
|
|
14
|
+
throw new robot_1.ActionableError("Invalid JPEG");
|
|
15
|
+
}
|
|
16
|
+
let offset = 2;
|
|
17
|
+
while (offset + 4 <= buffer.length) {
|
|
18
|
+
if (buffer[offset] !== MARKER_PREFIX) {
|
|
19
|
+
throw new robot_1.ActionableError("Invalid JPEG");
|
|
20
|
+
}
|
|
21
|
+
// a marker may be preceded by any number of 0xff fill bytes
|
|
22
|
+
while (buffer[offset + 1] === MARKER_PREFIX && offset + 4 < buffer.length) {
|
|
23
|
+
offset++;
|
|
24
|
+
}
|
|
25
|
+
const marker = buffer[offset + 1];
|
|
26
|
+
const segmentLength = buffer.readUInt16BE(offset + 2);
|
|
27
|
+
if (isStartOfFrameMarker(marker)) {
|
|
28
|
+
// segment: length(2) precision(1) height(2) width(2) ...
|
|
29
|
+
if (segmentLength < SOF_MIN_SEGMENT_LENGTH || offset + 9 > buffer.length) {
|
|
30
|
+
throw new robot_1.ActionableError("Invalid JPEG");
|
|
31
|
+
}
|
|
32
|
+
return {
|
|
33
|
+
height: buffer.readUInt16BE(offset + 5),
|
|
34
|
+
width: buffer.readUInt16BE(offset + 7),
|
|
35
|
+
};
|
|
36
|
+
}
|
|
37
|
+
offset += 2 + segmentLength;
|
|
38
|
+
}
|
|
39
|
+
throw new robot_1.ActionableError("Invalid JPEG");
|
|
40
|
+
};
|
|
41
|
+
exports.getJpegDimensions = getJpegDimensions;
|
package/lib/server.js
CHANGED
|
@@ -15,6 +15,8 @@ const android_1 = require("./android");
|
|
|
15
15
|
const robot_1 = require("./robot");
|
|
16
16
|
const ios_1 = require("./ios");
|
|
17
17
|
const png_1 = require("./png");
|
|
18
|
+
const jpeg_1 = require("./jpeg");
|
|
19
|
+
const coordinate_mapping_1 = require("./coordinate-mapping");
|
|
18
20
|
const mobilecli_1 = require("./mobilecli");
|
|
19
21
|
const mobile_device_1 = require("./mobile-device");
|
|
20
22
|
const utils_1 = require("./utils");
|
|
@@ -59,6 +61,7 @@ const createMcpServer = () => {
|
|
|
59
61
|
inputSchema: paramsSchema,
|
|
60
62
|
annotations,
|
|
61
63
|
}, (async (args, _extra) => {
|
|
64
|
+
scarf();
|
|
62
65
|
try {
|
|
63
66
|
(0, logger_1.trace)(`Invoking ${name} with args: ${JSON.stringify(args)}`);
|
|
64
67
|
const start = +new Date();
|
|
@@ -130,6 +133,15 @@ const createMcpServer = () => {
|
|
|
130
133
|
// ignore
|
|
131
134
|
}
|
|
132
135
|
};
|
|
136
|
+
// ponytail: scarf.sh pixel, fired once per process on first tool invocation
|
|
137
|
+
let scarfSent = false;
|
|
138
|
+
const scarf = () => {
|
|
139
|
+
if (scarfSent || process.env.MOBILEMCP_DISABLE_TELEMETRY) {
|
|
140
|
+
return;
|
|
141
|
+
}
|
|
142
|
+
scarfSent = true;
|
|
143
|
+
fetch("https://static.scarf.sh/a.png?x-pxid=f238110d-fcbb-479a-bdda-d47716ec4f07").catch(() => { });
|
|
144
|
+
};
|
|
133
145
|
const mobilecli = new mobilecli_1.Mobilecli();
|
|
134
146
|
const activeRecordings = new Map();
|
|
135
147
|
const agentVerifiedSimulators = new Set();
|
|
@@ -551,7 +563,7 @@ const createMcpServer = () => {
|
|
|
551
563
|
});
|
|
552
564
|
server.registerTool("mobile_take_screenshot", {
|
|
553
565
|
title: "Take Screenshot",
|
|
554
|
-
description: "Take a screenshot of the mobile device. Use this to understand what's on screen, if you need to press an element that is available through view hierarchy then you must list elements on screen instead. Do not cache this result.",
|
|
566
|
+
description: "Take a screenshot of the mobile device. Use this to understand what's on screen, if you need to press an element that is available through view hierarchy then you must list elements on screen instead. The screenshot is usually smaller than the screen, so when the device reports its screen size the result also states how to convert positions in the screenshot into screen coordinates before tapping. Do not cache this result.",
|
|
555
567
|
inputSchema: {
|
|
556
568
|
device: zod_1.z.string().describe("The device identifier to use. Use mobile_list_available_devices to find which devices are available to you."),
|
|
557
569
|
maxSize: zod_1.z.number().int().positive().optional().describe(`Maximum width/height in pixels, keeping aspect ratio. Defaults to ${DEFAULT_SCREENSHOT_MAX_SIZE}.`),
|
|
@@ -579,25 +591,36 @@ const createMcpServer = () => {
|
|
|
579
591
|
const screenshot = await robot.getScreenshot(options);
|
|
580
592
|
const isJpeg = screenshot.length > 2 && screenshot[0] === 0xff && screenshot[1] === 0xd8;
|
|
581
593
|
let mimeType = "image/jpeg";
|
|
582
|
-
|
|
594
|
+
let screenshotSize;
|
|
595
|
+
if (isJpeg) {
|
|
596
|
+
screenshotSize = (0, jpeg_1.getJpegDimensions)(screenshot);
|
|
597
|
+
}
|
|
598
|
+
else {
|
|
583
599
|
mimeType = "image/png";
|
|
584
600
|
// validate we received a png, will throw exception otherwise
|
|
585
601
|
const image = new png_1.PNG(screenshot);
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
602
|
+
screenshotSize = image.getDimensions();
|
|
603
|
+
}
|
|
604
|
+
if (screenshotSize.width <= 0 || screenshotSize.height <= 0) {
|
|
605
|
+
throw new robot_1.ActionableError("Screenshot is invalid. Please try again.");
|
|
590
606
|
}
|
|
607
|
+
// the screenshot is downscaled (and on ios the screen is in points while pixels are not),
|
|
608
|
+
// so tell the model how to map what it sees onto tap coordinates. see #29.
|
|
609
|
+
const screenSize = await robot.getScreenSize();
|
|
610
|
+
const mapping = (0, coordinate_mapping_1.describeCoordinateMapping)(screenshotSize, screenSize);
|
|
591
611
|
const screenshot64 = screenshot.toString("base64");
|
|
592
|
-
(0, logger_1.trace)(`Screenshot taken: ${screenshot.length} bytes`);
|
|
612
|
+
(0, logger_1.trace)(`Screenshot taken: ${screenshot.length} bytes, ${screenshotSize.width}x${screenshotSize.height}`);
|
|
593
613
|
posthog("tool_invoked", {
|
|
594
614
|
"ToolName": "mobile_take_screenshot",
|
|
595
615
|
"ScreenshotFilesize": screenshot64.length,
|
|
596
616
|
"ScreenshotMimeType": mimeType,
|
|
597
617
|
}).then();
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
618
|
+
const content = [];
|
|
619
|
+
if (mapping !== null) {
|
|
620
|
+
content.push({ type: "text", text: mapping });
|
|
621
|
+
}
|
|
622
|
+
content.push({ type: "image", data: screenshot64, mimeType });
|
|
623
|
+
return { content };
|
|
601
624
|
}
|
|
602
625
|
catch (err) {
|
|
603
626
|
(0, logger_1.error)(`Error taking screenshot: ${err.message} ${err.stack}`);
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mobilenext/mobile-mcp",
|
|
3
3
|
"mcpName": "io.github.mobile-next/mobile-mcp",
|
|
4
|
-
"version": "1.0.
|
|
4
|
+
"version": "1.0.4",
|
|
5
5
|
"description": "Mobile MCP",
|
|
6
6
|
"repository": {
|
|
7
7
|
"type": "git",
|
|
@@ -13,6 +13,7 @@
|
|
|
13
13
|
"license": "Apache-2.0",
|
|
14
14
|
"scripts": {
|
|
15
15
|
"build": "tsc && chmod +x lib/index.js",
|
|
16
|
+
"start": "node lib/index.js",
|
|
16
17
|
"lint": "eslint .",
|
|
17
18
|
"fixlint": "eslint . --fix",
|
|
18
19
|
"test": "c8 playwright test",
|
|
@@ -24,7 +25,7 @@
|
|
|
24
25
|
"lib"
|
|
25
26
|
],
|
|
26
27
|
"dependencies": {
|
|
27
|
-
"@modelcontextprotocol/sdk": "1.
|
|
28
|
+
"@modelcontextprotocol/sdk": "1.30.0",
|
|
28
29
|
"ajv": "^8.18.0",
|
|
29
30
|
"commander": "14.0.0",
|
|
30
31
|
"express": "5.1.0",
|