ta-studio-mcp 1.2.3 → 1.2.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +83 -23
- package/dist/index.js +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,38 +1,98 @@
|
|
|
1
|
-
# ta-studio-mcp
|
|
1
|
+
# ta-studio-mcp 🚀
|
|
2
2
|
|
|
3
|
-
The definitive domain knowledge layer for AI agents building mobile test automation at TA Studio
|
|
3
|
+
**The definitive domain knowledge layer for AI agents building mobile test automation at TA Studio.**
|
|
4
4
|
|
|
5
|
-
ta-studio-mcp
|
|
5
|
+
AI agents often struggle with project-specific context, unique navigation patterns, and "tribal knowledge" about past bugs. `ta-studio-mcp` solves this by giving your agent (Claude, Cursor, Windsurf) structured, programmatic access to the team's methodologies, codebase maps, and verified bug fixes.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
---
|
|
8
8
|
|
|
9
|
+
## 📋 Prerequisites
|
|
10
|
+
|
|
11
|
+
- **Node.js**: `v18.0.0` or higher.
|
|
12
|
+
- **MCP Client**: An IDE or tool that supports the [Model Context Protocol](https://modelcontextprotocol.io) (e.g., Claude Desktop, Cursor, Windsurf, VS Code).
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## ⚡ Quick Start
|
|
17
|
+
|
|
18
|
+
```bash
|
|
9
19
|
npx ta-studio-mcp
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 🧠 Expert Knowledge & Deep Technical Lore
|
|
25
|
+
|
|
26
|
+
This section documents the state-of-the-art implementations used by the TA Studio team.
|
|
27
|
+
|
|
28
|
+
### 1. Set-of-Mark (SoM) Screenshot Annotation
|
|
29
|
+
Based on OmniParser's SoM approach, we use color-coded, type-aware bounding boxes to provide visual anchors.
|
|
30
|
+
- **Type-Aware Palette**: 9 distinct colors (e.g., **Dodger Blue** for buttons, **Orange** for inputs, **Purple** for toggles).
|
|
31
|
+
- **PIL Threading**: `asyncio.to_thread(_draw_bounding_boxes_threaded)` for non-blocking UI drawing.
|
|
32
|
+
- **TOON Optimization**: **Token Optimized Object Notation** reduces prompt tokens by 40% by stripping redundant XML.
|
|
33
|
+
- **Scaling Correction**: Screenshots are compressed to ~45% resolution. We apply `native_coord * (img_width / native_width)` for pixel-perfect alignment.
|
|
10
34
|
|
|
11
|
-
|
|
35
|
+
### 2. Deep Subagent Handoff Protocol
|
|
36
|
+
Our "Deep Agent Pattern" orchestrates specialized specialists via a strict chain of custody:
|
|
37
|
+
1. **Perceptor** (`Screen Classifier`): Returns structured state and **TOON** elements.
|
|
38
|
+
2. **Planner** (`Device Agent`): Proposes action based on the identified UI state.
|
|
39
|
+
3. **Guardrail** (`Action Verifier`): Applies **Boolean Verification** (Safe/Relevant/Executable).
|
|
40
|
+
4. **Actor** (`Mobile MCP`): Executes the approved action on the target device.
|
|
41
|
+
5. **Doctor** (`Failure Diagnosis`): Categorizes failures and suggests recovery (Jitter/Wait/Backtrack).
|
|
12
42
|
|
|
13
|
-
###
|
|
14
|
-
|
|
15
|
-
-
|
|
16
|
-
-
|
|
43
|
+
### 3. Boolean Verification vs. Numerical Scoring
|
|
44
|
+
We reject "0.85 confidence" scores. Every action must pass three binary checks:
|
|
45
|
+
- **is_safe**: Does this action cause data loss or unauthorized access? (YES/NO)
|
|
46
|
+
- **is_relevant**: Does this move the needle on the task goal? (YES/NO)
|
|
47
|
+
- **is_executable**: Is the target realistically reachable on the current screen? (YES/NO)
|
|
48
|
+
- **Logic**: Action executes ONLY if ALL checks are YES. Reject and propose an `alternative_action` otherwise.
|
|
17
49
|
|
|
18
|
-
###
|
|
19
|
-
|
|
50
|
+
### 4. Real-Time HUD & Parallel Execution
|
|
51
|
+
- **Observation Pipeline**: Achieve `<200ms` lag via `on_step` async callbacks that emit SSE events to the frontend.
|
|
52
|
+
- **Concurrency Control**: `asyncio.Semaphore` and per-simulation `asyncio.Lock` manage multiple parallel device streams without resource collision.
|
|
53
|
+
- **Retention**: Automated 24h age or 100 total simulations cleanup before auto-purge.
|
|
20
54
|
|
|
21
|
-
###
|
|
22
|
-
|
|
55
|
+
### 5. Model Tiering (Jan 2026 Standard)
|
|
56
|
+
- **Thinking Tier (GPT-5.2)**: High-level orchestration (Coordinator) and complex visual reasoning. reasoning effort: `high`.
|
|
57
|
+
- **Core Tier (GPT-5-mini)**: Specialized agents (Classifier, Verifier, Diagnosis). *Never use nano for classification.*
|
|
58
|
+
- **Utility Tier (GPT-5-nano)**: MCP tool call formatting and data distillation.
|
|
23
59
|
|
|
24
|
-
|
|
25
|
-
- <200ms lag via async callbacks.
|
|
26
|
-
- Parallel execution using asyncio.Semaphore.
|
|
60
|
+
---
|
|
27
61
|
|
|
28
|
-
|
|
29
|
-
- GPT-5.2: Orchestration.
|
|
30
|
-
- GPT-5-mini: Specialists.
|
|
31
|
-
- GPT-5-nano: Formatting.
|
|
62
|
+
## 🐞 Critical Bug Fixes (Implementation Level)
|
|
32
63
|
|
|
33
|
-
|
|
64
|
+
| Severity | Issue | Root Cause & Expert Fix |
|
|
65
|
+
|----------|-------|-------------------------|
|
|
66
|
+
| **CRITICAL** | Bbox Misalignment | **RC**: 45% Scaling Delta. **Fix**: Apply `img_width / native_width` factor. |
|
|
67
|
+
| **CRITICAL** | Async to_thread | **RC**: CORO vs CALL mismatch. **Fix**: Remove `async` from functions passed to `asyncio.to_thread`. |
|
|
68
|
+
| **CRITICAL** | Race Condition | **RC**: Parallel session state collision. **Fix**: Set `parallel_tool_calls=False`. |
|
|
69
|
+
| **HIGH** | Simulation Leak | **RC**: Memory persistence. **Fix**: 24h/100-run auto-purge with `asyncio.Lock`. |
|
|
70
|
+
| **HIGH** | Mobile MCP Bug | **RC**: Offline device fail (v0.0.36). **Fix**: Full ADB bridge fallback (screencap/uiautomator). |
|
|
34
71
|
|
|
72
|
+
---
|
|
73
|
+
|
|
74
|
+
## 🔄 Core Workflows
|
|
75
|
+
|
|
76
|
+
### The Ralph Loop (Closed-Loop Verification)
|
|
77
|
+
1. **CODE** → Implement feature or fix.
|
|
78
|
+
2. **LINT** → `mypy` / `eslint` verification.
|
|
79
|
+
3. **UNIT TEST** → Specific module verification.
|
|
80
|
+
4. **CHECK ASYNC** → Confirm `to_thread` safety.
|
|
81
|
+
5. **VERIFY HUD** → Watch the emulator stream while agent runs autonomously.
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## 📦 Installation & Setup
|
|
86
|
+
|
|
87
|
+
### Claude Desktop
|
|
88
|
+
```bash
|
|
35
89
|
claude mcp add ta-studio -- npx -y ta-studio-mcp@latest
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### Cursor / Windsurf
|
|
93
|
+
Add `npx -y ta-studio-mcp@latest` as a command-type MCP server.
|
|
94
|
+
|
|
95
|
+
---
|
|
36
96
|
|
|
37
|
-
## License
|
|
38
|
-
MIT
|
|
97
|
+
## 📜 License
|
|
98
|
+
MIT © 2026 TA Studios.
|
package/dist/index.js
CHANGED
|
@@ -14,7 +14,7 @@ import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
|
|
|
14
14
|
import { registerAllTools } from './tools/register-all.js';
|
|
15
15
|
const server = new McpServer({
|
|
16
16
|
name: 'ta-studio-mcp',
|
|
17
|
-
version: '1.2.
|
|
17
|
+
version: '1.2.4',
|
|
18
18
|
}, {
|
|
19
19
|
capabilities: {
|
|
20
20
|
logging: {},
|
package/package.json
CHANGED