mobile-debug-mcp 0.21.4 → 0.22.0

package/AGENTS.md

@@ -0,0 +1,74 @@
+# AGENTS.md
+
+This file is the **cold-start entrypoint for autonomous agents** that arrive in this public repository without prior context.
+
+It is intentionally vendor-neutral and should be useful to Copilot, Codex, Claude, or custom agent systems.
+
+## What this repository is
+
+`mobile-debug-mcp` is an MCP server for mobile app debugging on Android and iOS. The codebase is TypeScript and the built server entrypoint is `dist/server.js`.
+
+Primary areas:
+
+- `src/server.ts` — runtime entrypoint
+- `src/server-core.ts` — tool registration and dispatch logic
+- `src/interact/` — tap, swipe, typing, waits, and interaction helpers
+- `src/observe/` — logs, screenshots, UI tree, fingerprints, snapshots
+- `src/manage/` — build, install, start, terminate, restart, reset-app-data
+- `src/system/` — environment and toolchain health checks
+- `src/utils/` — shared helpers and platform-specific utility code
+
+## First things an agent should know
+
+1. Prefer existing helpers and conventions over introducing new frameworks or abstractions.
+2. The repository already has a test structure; place tests in the correct tree instead of inventing new ones.
+3. Use existing validation commands:
+   - `npm run build`
+   - `npm run lint`
+   - `npm run test:unit`
+   - `npm run test:device`
+4. Device tests are split intentionally:
+   - `test/device/automated/...` = automated smoke/integration coverage
+   - `test/device/manual/...` = helper/debug/manual scripts
+5. Unit tests use the current lightweight self-running `tsx` pattern rather than a dedicated framework.
+
+## Where to look for deeper guidance
+
+### Skills
+
+Portable agent skills live under `skills/`.
+
+- `skills/README.md` — repo-wide skill convention
+- `skills/mcp-builder/` — build/install/toolchain guidance
+- `skills/test-authoring/` — test creation and placement guidance
+
+If the task is about **creating or updating tests**, load `skills/test-authoring/SKILL.md` first.
+
+If the task is about **building, installing, or diagnosing native tooling**, load `skills/mcp-builder/SKILL.md` first.
+
+### Repository docs
+
+- `README.md` — high-level repo overview and commands
+- `docs/tools/TOOLS.md` — tool documentation and contracts
+- `docs/CHANGELOG.md` — recent behavioral changes
+
+## Testing guidance
+
+Use these defaults unless the task clearly requires something else:
+
+- Add automated unit tests under `test/unit/...`
+- Add automated device smoke tests under `test/device/automated/...`
+- Add ad hoc or environment-specific helper scripts under `test/device/manual/...`
+
+For test authoring details, rely on the `test-authoring` skill package rather than duplicating its rules here.
+
+## Output expectations for agents
+
+- Make focused changes
+- Preserve existing repo behavior unless the task requires a change
+- Keep docs aligned when structure or usage changes
+- Prefer deterministic validation over ad hoc verification
+
+## Notes for maintainers
+
+This file is intentionally short. Keep task-specific guidance in `skills/...` so multiple agent systems can reuse the same instructions.

package/README.md

@@ -2,8 +2,10 @@
 
 A minimal, secure MCP server for AI-assisted mobile development. Build, install, and inspect Android/iOS apps from an MCP-compatible client.
 
-> **Note:**
-> * iOS only tested on simulator.
+> **Support:**
+> * Android support
+> * iOS only tested on simulator
+> * KMP support
 > * Flutter iOS projects not fetching logs
 > * React native not tested
 
@@ -32,15 +34,32 @@ You will need to add ADB_PATH for Android and XCRUN_PATH and IDB_PATH for iOS.
 ## Usage
 
 Examples: 
-* I have a crash on the app, can you diagnose it, fix and validate using the mcp tools available
-* Add a button, hook into the repository and confirm API request successful
+
+Crash fixing:
+> I have a crash on the app, can you diagnose it, fix and validate using the mcp tools available
+
+Feature building:
+> Add a button, hook into the repository and confirm API request successful
 
 ## Docs
 
 - Tools: [Tools](docs/tools/TOOLS.md) — full input/response examples
 - Changelog: [Changelog](docs/CHANGELOG.md)
+- Agents: [AGENTS.md](AGENTS.md) — cold-start guidance for autonomous agents entering the public repo
+- Skills: [skills/README.md](skills/README.md) — portable Markdown skill packages for agents such as Copilot, Codex, Claude, or custom systems
+
+## Testing
+
+- `npm run test:unit` runs every automated unit test under `test/unit/...`
+- `npm run test:device` runs the automated device smoke checks under `test/device/automated/...`
+- Manual and debug-oriented device scripts live under `test/device/manual/...` and are not part of the default test commands
+
+## Agent skills
+
+- `skills/mcp-builder/` contains reusable build/install guidance for agents
+- `skills/test-authoring/` contains reusable test-creation guidance aligned to this repo's current test structure
+- Skills are written as plain Markdown packages so they can be consumed by different agent systems rather than one vendor-specific runtime
 
 ## License
 
 MIT
-

package/dist/interact/index.js

@@ -1,9 +1,145 @@
+import { createHash } from 'crypto';
 import { AndroidInteract } from './android.js';
 import { iOSInteract } from './ios.js';
 export { AndroidInteract, iOSInteract };
 import { resolveTargetDevice } from '../utils/resolve-device.js';
 import { ToolsObserve } from '../observe/index.js';
 export class ToolsInteract {
+    static _maxResolvedUiElements = 256;
+    static _resolvedUiElements = new Map();
+    static _normalize(s) {
+        if (s === null || s === undefined)
+            return '';
+        try {
+            return String(s).toLowerCase().trim();
+        }
+        catch {
+            return '';
+        }
+    }
+    static _normalizeBounds(bounds) {
+        if (!Array.isArray(bounds) || bounds.length < 4)
+            return null;
+        const normalized = bounds.slice(0, 4).map((value) => Number(value));
+        if (normalized.some((value) => Number.isNaN(value)))
+            return null;
+        return normalized;
+    }
+    static _isVisibleElement(el) {
+        const bounds = ToolsInteract._normalizeBounds(el.bounds);
+        return !!el.visible && !!bounds && bounds[2] > bounds[0] && bounds[3] > bounds[1];
+    }
+    static _computeElementId(platform, deviceId, el, index) {
+        const identity = {
+            platform,
+            deviceId: deviceId || '',
+            text: ToolsInteract._normalize(el.text ?? el.label ?? el.value ?? ''),
+            resourceId: ToolsInteract._normalize(el.resourceId ?? el.resourceID ?? el.id ?? ''),
+            accessibilityId: ToolsInteract._normalize(el.contentDescription ?? el.contentDesc ?? el.accessibilityLabel ?? el.label ?? ''),
+            class: ToolsInteract._normalize(el.type ?? el.class ?? ''),
+            bounds: ToolsInteract._normalizeBounds(el.bounds) ?? [0, 0, 0, 0],
+            index
+        };
+        return `el_${createHash('sha1').update(JSON.stringify(identity)).digest('hex').slice(0, 24)}`;
+    }
+    static _buildResolvedElement(platform, deviceId, el, index) {
+        const bounds = ToolsInteract._normalizeBounds(el.bounds);
+        const elementId = ToolsInteract._computeElementId(platform, deviceId, el, index);
+        ToolsInteract._rememberResolvedElement(elementId, {
+            elementId,
+            platform,
+            deviceId,
+            bounds,
+            index
+        });
+        return {
+            text: el.text ?? null,
+            resource_id: el.resourceId ?? el.resourceID ?? el.id ?? null,
+            accessibility_id: el.contentDescription ?? el.contentDesc ?? el.accessibilityLabel ?? el.label ?? null,
+            class: el.type ?? el.class ?? null,
+            bounds,
+            index,
+            elementId
+        };
+    }
+    static _rememberResolvedElement(elementId, context) {
+        if (ToolsInteract._resolvedUiElements.has(elementId)) {
+            ToolsInteract._resolvedUiElements.delete(elementId);
+        }
+        ToolsInteract._resolvedUiElements.set(elementId, context);
+        while (ToolsInteract._resolvedUiElements.size > ToolsInteract._maxResolvedUiElements) {
+            const oldestElementId = ToolsInteract._resolvedUiElements.keys().next().value;
+            if (!oldestElementId)
+                break;
+            ToolsInteract._resolvedUiElements.delete(oldestElementId);
+        }
+    }
+    static _resetResolvedUiElementsForTests() {
+        ToolsInteract._resolvedUiElements.clear();
+    }
+    static _findCurrentResolvedElement(elements, platform, deviceId, resolved) {
+        const indexedCandidate = elements[resolved.index];
+        if (indexedCandidate && ToolsInteract._computeElementId(platform, deviceId, indexedCandidate, resolved.index) === resolved.elementId) {
+            return { el: indexedCandidate, index: resolved.index };
+        }
+        return null;
+    }
+    static _resolveActionableAncestor(elements, chosen) {
+        if (!chosen)
+            return null;
+        if (chosen.el.clickable || chosen.el.focusable)
+            return chosen;
+        let current = chosen;
+        let safety = 0;
+        while (safety < 20 && current.el && !(current.el.clickable || current.el.focusable) && current.el.parentId !== undefined && current.el.parentId !== null) {
+            const parentId = current.el.parentId;
+            let parentIndex = null;
+            if (typeof parentId === 'number')
+                parentIndex = parentId;
+            else if (typeof parentId === 'string' && /^\d+$/.test(parentId))
+                parentIndex = Number(parentId);
+            if (parentIndex !== null && elements[parentIndex]) {
+                current = { el: elements[parentIndex], idx: parentIndex };
+                if (current.el.clickable || current.el.focusable)
+                    return current;
+            }
+            else if (typeof parentId === 'string') {
+                const foundIndex = elements.findIndex((el) => el.resourceId === parentId || el.id === parentId);
+                if (foundIndex === -1)
+                    break;
+                current = { el: elements[foundIndex], idx: foundIndex };
+                if (current.el.clickable || current.el.focusable)
+                    return current;
+            }
+            else {
+                break;
+            }
+            safety++;
+        }
+        const childBounds = ToolsInteract._normalizeBounds(chosen.el.bounds);
+        if (!childBounds)
+            return null;
+        const [cl, ct, cr, cb] = childBounds;
+        let best = null;
+        let bestArea = Infinity;
+        for (let i = 0; i < elements.length; i++) {
+            const el = elements[i];
+            if (!el || !(el.clickable || el.focusable))
+                continue;
+            const bounds = ToolsInteract._normalizeBounds(el.bounds);
+            if (!bounds)
+                continue;
+            const [pl, pt, pr, pb] = bounds;
+            if (pl <= cl && pt <= ct && pr >= cr && pb >= cb) {
+                const area = (pr - pl) * (pb - pt);
+                if (area < bestArea) {
+                    bestArea = area;
+                    best = { el, idx: i };
+                }
+            }
+        }
+        return best;
+    }
     static async getInteractionService(platform, deviceId) {
         const effectivePlatform = platform || 'android';
         const resolved = await resolveTargetDevice({ platform: effectivePlatform, deviceId });
@@ -14,6 +150,90 @@ export class ToolsInteract {
         const { interact, resolved } = await ToolsInteract.getInteractionService(platform, deviceId);
         return await interact.tap(x, y, resolved.id);
     }
+    static async tapElementHandler({ elementId }) {
+        const action = 'tap';
+        const resolved = ToolsInteract._resolvedUiElements.get(elementId);
+        if (!resolved) {
+            return {
+                success: false,
+                elementId,
+                action,
+                error: {
+                    code: 'element_not_found',
+                    message: 'Element ID was not found in the current UI context'
+                }
+            };
+        }
+        const tree = await ToolsObserve.getUITreeHandler({ platform: resolved.platform, deviceId: resolved.deviceId });
+        const treePlatform = tree?.device?.platform === 'ios' ? 'ios' : resolved.platform;
+        const treeDeviceId = tree?.device?.id || resolved.deviceId;
+        const elements = Array.isArray(tree?.elements) ? tree.elements : [];
+        const currentMatch = ToolsInteract._findCurrentResolvedElement(elements, treePlatform, treeDeviceId, resolved);
+        if (!currentMatch) {
+            return {
+                success: false,
+                elementId,
+                action,
+                error: {
+                    code: 'element_not_found',
+                    message: 'Element ID is not present in the current UI context'
+                }
+            };
+        }
+        if (!ToolsInteract._isVisibleElement(currentMatch.el)) {
+            return {
+                success: false,
+                elementId,
+                action,
+                error: {
+                    code: 'element_not_visible',
+                    message: 'Element is not visible'
+                }
+            };
+        }
+        if (currentMatch.el.enabled === false) {
+            return {
+                success: false,
+                elementId,
+                action,
+                error: {
+                    code: 'element_not_enabled',
+                    message: 'Element is not enabled'
+                }
+            };
+        }
+        const bounds = ToolsInteract._normalizeBounds(currentMatch.el.bounds) ?? resolved.bounds;
+        if (!bounds || bounds[2] <= bounds[0] || bounds[3] <= bounds[1]) {
+            return {
+                success: false,
+                elementId,
+                action,
+                error: {
+                    code: 'element_not_visible',
+                    message: 'Element does not have valid visible bounds'
+                }
+            };
+        }
+        const x = Math.floor((bounds[0] + bounds[2]) / 2);
+        const y = Math.floor((bounds[1] + bounds[3]) / 2);
+        const tapResult = await ToolsInteract.tapHandler({ platform: resolved.platform, x, y, deviceId: resolved.deviceId });
+        if (!tapResult.success) {
+            return {
+                success: false,
+                elementId,
+                action,
+                error: {
+                    code: 'tap_failed',
+                    message: tapResult.error || 'Tap failed'
+                }
+            };
+        }
+        return {
+            success: true,
+            elementId,
+            action
+        };
+    }
     static async swipeHandler({ platform = 'android', x1, y1, x2, y2, duration, deviceId }) {
         const { interact, resolved } = await ToolsInteract.getInteractionService(platform, deviceId);
         return await interact.swipe(x1, y1, x2, y2, duration, resolved.id);
@@ -34,7 +254,7 @@ export class ToolsInteract {
         // Try to use observe layer to fetch the current UI tree and perform a fast semantic search
         const start = Date.now();
         const deadline = start + timeoutMs;
-        const normalize = (s) => (s === null || s === undefined) ? '' : String(s).toLowerCase().trim();
+        const normalize = ToolsInteract._normalize;
         const q = normalize(query);
         if (!q)
             return { found: false, error: 'Empty query' };
@@ -217,15 +437,19 @@ export class ToolsInteract {
     }
     static async waitForUIHandler({ selector, condition = 'exists', timeout_ms = 60000, poll_interval_ms = 300, match, retry = { max_attempts: 1, backoff_ms: 0 }, platform, deviceId }) {
         const overallStart = Date.now();
-        // Validate selector: require at least one of text, resource_id, or accessibility_id
-        if (!selector || (typeof selector === 'object' && Object.keys(selector).length === 0)) {
-            return { status: 'timeout', error: { code: 'INVALID_SELECTOR', message: 'At least one selector field must be provided (text, resource_id, or accessibility_id)' }, metrics: { latency_ms: Date.now() - overallStart, poll_count: 0, attempts: 0 } };
-        }
-        const hasText = selector && typeof selector.text === 'string' && selector.text.trim().length > 0;
-        const hasResId = selector && typeof selector.resource_id === 'string' && selector.resource_id.trim().length > 0;
-        const hasAccId = selector && typeof selector.accessibility_id === 'string' && selector.accessibility_id.trim().length > 0;
+        // Validate selector: require at least one non-empty field (text, resource_id, or accessibility_id)
+        const hasText = typeof selector?.text === 'string' && selector.text.trim().length > 0;
+        const hasResId = typeof selector?.resource_id === 'string' && selector.resource_id.trim().length > 0;
+        const hasAccId = typeof selector?.accessibility_id === 'string' && selector.accessibility_id.trim().length > 0;
         if (!hasText && !hasResId && !hasAccId) {
-            return { status: 'timeout', error: { code: 'INVALID_SELECTOR', message: 'Selector must include at least one of: text, resource_id, accessibility_id' }, metrics: { latency_ms: Date.now() - overallStart, poll_count: 0, attempts: 0 } };
+            return {
+                status: 'timeout',
+                error: {
+                    code: 'INVALID_SELECTOR',
+                    message: 'Selector must include at least one non-empty field: text, resource_id, or accessibility_id'
+                },
+                metrics: { latency_ms: Date.now() - overallStart, poll_count: 0, attempts: 0 }
+            };
         }
         // Validate condition
         if (!['exists', 'not_exists', 'visible', 'clickable'].includes(condition)) {
@@ -241,11 +465,11 @@ export class ToolsInteract {
         let attempts = 0;
         let totalPollCount = 0;
         // Precompute normalized selector values and helpers (constant across polls)
-        const normalize = (s) => (s === null || s === undefined) ? '' : String(s).toLowerCase().trim();
-        const containsFlag = !!selector.contains;
-        const selText = normalize(selector.text);
-        const selRid = normalize(selector.resource_id);
-        const selAid = normalize(selector.accessibility_id);
+        const normalize = ToolsInteract._normalize;
+        const containsFlag = !!selector?.contains;
+        const selText = normalize(selector?.text);
+        const selRid = normalize(selector?.resource_id);
+        const selAid = normalize(selector?.accessibility_id);
         try {
             while (attempts < maxAttempts) {
                 attempts++;
@@ -301,30 +525,32 @@ export class ToolsInteract {
                         }
                         // Evaluate condition
                         const matchedCount = matches.length;
-                        const pickIndexProvided = (match && typeof match.index === 'number');
-                        const pickIndex = pickIndexProvided ? Number(match.index) : 0;
+                        const pickIndex = (typeof match?.index === 'number') ? match.index : undefined;
                         let chosen = null;
-                        if (matches.length === 0) {
-                            chosen = null;
-                        }
-                        else if (pickIndexProvided) {
-                            // If a specific index is requested but out of bounds, treat as not matched for this poll (deterministic)
-                            if (pickIndex >= 0 && pickIndex < matches.length)
-                                chosen = matches[pickIndex];
-                            else
-                                chosen = null;
+                        if (matches.length > 0) {
+                            if (pickIndex !== undefined) {
+                                // If a specific index is requested but out of bounds, treat as not matched for this poll (deterministic)
+                                if (pickIndex >= 0 && pickIndex < matches.length)
+                                    chosen = matches[pickIndex];
+                                else
+                                    chosen = null;
+                            }
+                            else {
+                                chosen = matches[0];
+                            }
                         }
                         else {
-                            chosen = matches[0];
+                            chosen = null;
                         }
                         let conditionMet = false;
+                        let matchedElement = chosen;
                         if (condition === 'exists') {
                             // when an index is specified, existence requires that specific index be present
-                            conditionMet = pickIndexProvided ? (chosen !== null) : (matchedCount >= 1);
+                            conditionMet = (pickIndex !== undefined) ? (chosen !== null) : (matchedCount >= 1);
                         }
                         else if (condition === 'not_exists') {
                             // when an index is specified, not_exists is true if that index is absent
-                            conditionMet = pickIndexProvided ? (chosen === null) : (matchedCount === 0);
+                            conditionMet = (pickIndex !== undefined) ? (chosen === null) : (matchedCount === 0);
                         }
                         else if (condition === 'visible') {
                             if (chosen) {
@@ -336,11 +562,12 @@ export class ToolsInteract {
                                 conditionMet = false;
                         }
                         else if (condition === 'clickable') {
-                            if (chosen) {
-                                const b = chosen.el.bounds;
-                                const visibleFlag = !!chosen.el.visible && Array.isArray(b) && b.length >= 4 && (b[2] > b[0] && b[3] > b[1]);
-                                const enabled = !!chosen.el.enabled;
-                                const clickable = !!chosen.el.clickable || !!chosen.el._interactable;
+                            matchedElement = chosen ? (ToolsInteract._resolveActionableAncestor(elements, chosen) || chosen) : null;
+                            if (matchedElement) {
+                                const b = matchedElement.el.bounds;
+                                const visibleFlag = !!matchedElement.el.visible && Array.isArray(b) && b.length >= 4 && (b[2] > b[0] && b[3] > b[1]);
+                                const enabled = !!matchedElement.el.enabled;
+                                const clickable = !!matchedElement.el.clickable || !!matchedElement.el._interactable || !!matchedElement.el.focusable;
                                 conditionMet = visibleFlag && enabled && clickable;
                             }
                             else
@@ -350,14 +577,9 @@ export class ToolsInteract {
                             const now = Date.now();
                             const latency_ms = now - overallStart;
                             // Build element output per spec
-                            const outEl = chosen ? {
-                                text: chosen.el.text ?? null,
-                                resource_id: chosen.el.resourceId ?? chosen.el.resourceID ?? chosen.el.id ?? null,
-                                accessibility_id: chosen.el.contentDescription ?? chosen.el.contentDesc ?? chosen.el.accessibilityLabel ?? chosen.el.label ?? null,
-                                class: chosen.el.type ?? chosen.el.class ?? null,
-                                bounds: Array.isArray(chosen.el.bounds) && chosen.el.bounds.length >= 4 ? chosen.el.bounds : null,
-                                index: chosen.idx
-                            } : null;
+                            const resolvedPlatform = tree?.device?.platform === 'ios' ? 'ios' : (platform || 'android');
+                            const resolvedDeviceId = tree?.device?.id || deviceId;
+                            const outEl = matchedElement ? ToolsInteract._buildResolvedElement(resolvedPlatform, resolvedDeviceId, matchedElement.el, matchedElement.idx) : null;
                             return {
                                 status: 'success',
                                 matched: matchedCount,

package/dist/observe/ios.js

@@ -7,6 +7,13 @@ import { parseLogLine } from '../utils/android/utils.js';
 import { computeScreenFingerprint } from '../utils/ui/index.js';
 import { parsePngSize } from '../utils/image.js';
 const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms));
+let iosExecCommand = execCommand;
+export function _setIOSExecCommandForTests(fn) {
+    iosExecCommand = fn;
+}
+export function _resetIOSExecCommandForTests() {
+    iosExecCommand = execCommand;
+}
 function parseIDBFrame(frame) {
     if (!frame)
         return [0, 0, 0, 0];
@@ -122,7 +129,7 @@ export class iOSObserve {
                 const parts = appId.split('.');
                 const simpleName = parts[parts.length - 1];
                 try {
-                    const pgrepRes = await execCommand(['simctl', 'spawn', deviceId, 'pgrep', '-f', simpleName], deviceId);
+                    const pgrepRes = await iosExecCommand(['simctl', 'spawn', deviceId, 'pgrep', '-f', simpleName], deviceId);
                     const out = pgrepRes && pgrepRes.output ? pgrepRes.output.trim() : '';
                     const firstLine = out.split(/\r?\n/).find(Boolean);
                     if (firstLine) {
@@ -136,7 +143,7 @@ export class iOSObserve {
                 }
             }
             const effectivePid = pid || detectedPid || null;
-            const result = await execCommand(args, deviceId);
+            const result = await iosExecCommand(args, deviceId);
             const device = await getIOSDeviceMetadata(deviceId);
             const rawLines = result.output ? result.output.split(/\r?\n/).filter(Boolean) : [];
             // Parse lines into structured entries. iOS log format: timestamp [PID:tid] <level> subsystem:category: message
@@ -243,7 +250,7 @@ export class iOSObserve {
         const device = await getIOSDeviceMetadata(deviceId);
         const tmpFile = `/tmp/mcp-ios-screenshot-${Date.now()}.png`;
         try {
-            await execCommand(['simctl', 'io', deviceId, 'screenshot', tmpFile], deviceId);
+            await iosExecCommand(['simctl', 'io', deviceId, 'screenshot', tmpFile], deviceId);
             const buffer = await fs.readFile(tmpFile);
             const base64 = buffer.toString('base64');
             const dims = parsePngSize(buffer);