nerve-mcp 0.1.1 → 0.2.1

package/README.md

@@ -0,0 +1,208 @@
+# Nerve
+
+Nerve gives AI agents eyes and hands inside iOS apps.
+
+Add the MCP server to your AI agent, and it can see every element on screen, tap buttons, fill forms, scroll, inspect state, intercept network calls, and debug your iOS app — all through natural language. No code changes needed.
+
+## Setup
+
+### 1. Install the MCP Server
+
+```bash
+npx nerve-mcp@latest
+```
+
+Or install globally:
+
+```bash
+npm install -g nerve-mcp
+```
+
+Or clone and build from source:
+
+```bash
+git clone https://github.com/luchi0208/nerve-ios.git
+cd nerve/mcp-server && npm install && npm run build
+```
+
+### 2. Configure Your AI Agent
+
+**Claude Code** — add to your project's `.mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "nerve": {
+      "command": "npx",
+      "args": ["nerve-mcp@latest"]
+    }
+  }
+}
+```
+
+**Claude Desktop / Cursor / Other MCP clients:**
+
+```json
+{
+  "mcpServers": {
+    "nerve": {
+      "command": "npx",
+      "args": ["nerve-mcp@latest"]
+    }
+  }
+}
+```
+
+If installed from source:
+
+```json
+{
+  "mcpServers": {
+    "nerve": {
+      "command": "node",
+      "args": ["/path/to/nerve/mcp-server/dist/index.js"]
+    }
+  }
+}
+```
+
+That's it. Tell your AI agent to build and run your app — Nerve auto-injects on the Simulator with no code changes needed.
+
+## Example
+
+Things you can ask your AI agent with Nerve:
+
+> "Run my app on the simulator. Go to the checkout screen and try submitting an empty form — what validation errors show up?"
+
+> "There's a bug where the cart badge doesn't update after removing an item. Can you reproduce it and check the console logs?"
+
+> "Navigate through every screen in the app and find any buttons that don't respond to taps."
+
+> "The login screen looks broken on iPhone SE. Run it on that simulator and screenshot just the login form so I can see what's wrong."
+
+> "Trace all calls to `CartManager.addItem` and then add three items to the cart. Show me what arguments are being passed."
+
+> "Check what's stored in UserDefaults after onboarding completes. I think we're saving the auth token in the wrong key."
+
+> "Intercept the network requests when I pull to refresh on the orders screen. Show me the response bodies — I think the API is returning stale data."
+
+These are just starting points. The agent combines Nerve's tools on its own — you describe what you want in plain English, and it figures out the sequence of taps, inspections, and checks to get there.
+
+## How It Works
+
+Nerve auto-injects into the app at launch on the Simulator — no code changes needed. It runs inside the app process, starts a WebSocket server, and the MCP server on the Mac connects to it. AI agent tool calls are translated into commands executed inside the app.
+
+Because it runs in-process, Nerve has access to the full view hierarchy, the Objective-C runtime, live objects, network delegates, and the HID event system.
+
+```
+AI Agent  →  MCP Server (Mac)  →  WebSocket  →  Nerve (in-app)  →  UIKit/SwiftUI
+```
+
+## Tools
+
+### See the Screen
+
+| Tool | Description |
+|------|-------------|
+| `nerve_view` | See all visible elements with type, label, ID, tap point, and position |
+| `nerve_tree` | Full view hierarchy (UIKit + SwiftUI) |
+| `nerve_inspect` | Detailed properties of a specific element |
+| `nerve_screenshot` | Capture the screen as an image |
+
+### Interact
+
+| Tool | Description |
+|------|-------------|
+| `nerve_tap` | Tap an element by `@eN` ref, `#identifier`, `@label`, or coordinates |
+| `nerve_type` | Type text into the focused field |
+| `nerve_scroll` | Scroll in any direction |
+| `nerve_swipe` | Swipe gesture |
+| `nerve_long_press` | Long press |
+| `nerve_double_tap` | Double tap |
+| `nerve_drag_drop` | Drag from one element to another |
+| `nerve_pull_to_refresh` | Pull to refresh |
+| `nerve_pinch` | Pinch/zoom |
+| `nerve_context_menu` | Open context menu |
+| `nerve_back` | Navigate back |
+| `nerve_dismiss` | Dismiss keyboard or modal |
+
+### Navigate
+
+| Tool | Description |
+|------|-------------|
+| `nerve_map` | See all discovered screens and transitions |
+| `nerve_navigate` | Auto-navigate to a known screen |
+| `nerve_scroll_to_find` | Scroll until an element appears |
+| `nerve_deeplink` | Open a URL scheme |
+
+### Inspect & Debug
+
+| Tool | Description |
+|------|-------------|
+| `nerve_console` | App logs (stdout/stderr) |
+| `nerve_network` | Intercepted HTTP traffic with response bodies |
+| `nerve_heap` | Find live object instances by class name |
+| `nerve_storage` | Read UserDefaults, Keychain, cookies, files |
+| `nerve_trace` | Swizzle any method to log calls |
+| `nerve_highlight` | Draw colored borders on elements for visual debugging |
+| `nerve_modify` | Change view properties at runtime |
+| `nerve_lldb` | Full LLDB debugger access |
+
+### Build & Launch
+
+| Tool | Description |
+|------|-------------|
+| `nerve_run` | Build, install, and launch on the simulator (auto-injects Nerve) |
+| `nerve_build` | Build only |
+| `nerve_status` | Show connected targets |
+| `nerve_list_simulators` | List available simulators |
+| `nerve_boot_simulator` | Boot a simulator by name or UDID |
+| `nerve_appearance` | Switch between light and dark mode |
+| `nerve_grant_permissions` | Pre-grant iOS permissions |
+
+## Element Queries
+
+Nerve supports several query formats for targeting elements:
+
+| Format | Example | Description |
+|--------|---------|-------------|
+| `@eN` | `@e2` | Element ref from `nerve_view` output |
+| `#id` | `#login-btn` | Accessibility identifier |
+| `@label` | `@Settings` | Accessibility label |
+| `.type:index` | `.field:0` | Element type with index |
+| `x,y` | `195,160` | Screen coordinates |
+
+The `nerve_view` output shows each element with its ref and tap point:
+
+```
+@e1 btn "Product A" #product-a tap=195,222 x=16 y=195 w=358 h=54
+@e2 field val=Email tap=195,160 x=32 y=149 w=326 h=22
+```
+
+Use `@e2` to tap that field — Nerve uses the element's activation point (center), which is always the correct hittable position.
+
+## Architecture
+
+```
+Nerve/
+  Sources/
+    Nerve/           Swift framework — commands, element resolution, inspection
+    NerveObjC/       ObjC/C bridge — touch synthesis, heap walking, swizzling
+  Example/           Example app with test views
+  Tests/
+    E2E/             End-to-end tests (83 tests)
+    NerveTests/      Unit tests
+  mcp-server/        MCP server (TypeScript)
+  cli/               CLI tool
+```
+
+## Requirements
+
+- macOS 14+
+- Xcode 16+
+- iOS Simulator (iOS 16+)
+- Node.js 18+
+
+## License
+
+MIT

package/dist/index.js

@@ -43,6 +43,7 @@ const types_js_1 = require("@modelcontextprotocol/sdk/types.js");
 const ws_1 = __importDefault(require("ws"));
 const child_process_1 = require("child_process");
 const path = __importStar(require("path"));
+const fs = __importStar(require("fs"));
 const net = __importStar(require("net"));
 // --- Port Resolution ---
 // djb2 hash — must match the Swift side exactly
@@ -177,13 +178,13 @@ The tap= coordinate is the center point where the element is reliably hittable.
 ### Verify
 - nerve_view to see updated screen state — this is your PRIMARY inspection tool. It returns structured element data with refs, identifiers, and tap coordinates you can act on directly.
 - nerve_console with filter="[nerve]" and since="last_action" for your trace logs.
-- nerve_screenshot ONLY when you need to verify visual layout, colors, or spatial relationships that text can't convey. Do NOT use screenshot as a substitute for nerve_view.
+- Do NOT use nerve_screenshot unless nerve_view is insufficient (e.g., verifying colors, gradients, or visual layout). When you must screenshot, ALWAYS crop to the relevant element: nerve_screenshot with element="#my-element" instead of capturing the full screen.
 - nerve_heap to inspect live objects (e.g., check ViewModel state).
 
 ### Tips
 - Always call nerve_view before interacting — don't guess element identifiers.
 - Use @eN refs from nerve_view output to tap elements without identifiers.
-- nerve_view is lightweight (~1 line per element) and gives you everything needed to interact. Prefer it over nerve_screenshot for all inspection tasks.
+- nerve_view is lightweight (~1 line per element) and gives you everything needed to interact. NEVER use nerve_screenshot when nerve_view can answer the question.
 - If an element isn't visible, try nerve_scroll_to_find before giving up.
 - The navigation map builds automatically and persists across sessions.
 - Do NOT add sleep/delay between commands — Nerve handles waiting automatically.
@@ -371,13 +372,16 @@ const TOOLS = [
     },
     {
         name: "nerve_screenshot",
-        description: "Capture a screenshot of the current screen. Returns base64-encoded PNG. Prefer nerve_view for understanding screen state and finding elements — it returns structured data with element refs and tap coordinates. Only use screenshot for visual layout verification when text output isn't enough.",
+        description: "Capture a screenshot. WARNING: Screenshots consume significant tokens — use nerve_view for all normal inspection. Only use for visual checks (colors, gradients, images, layout). Use 'element' to crop to a specific element (cheapest), or 'region' to crop to a normalized area. Avoid full-screen screenshots when possible.",
         inputSchema: {
             type: "object",
             properties: {
                 target: { type: "string" },
+                element: { type: "string", description: "Crop to a specific element. Use @eN ref, #identifier, or @label from nerve_view. This is the most token-efficient way to verify visual appearance." },
+                region: { type: "string", description: "Crop to a normalized region: \"x1,y1,x2,y2\" where values are 0-1. Example: \"0,0,0.5,0.5\" for top-left quarter." },
+                padding: { type: "number", description: "Padding in points around the element crop. Default: 20." },
                 scale: { type: "number", description: "Image scale. Default: 1.0." },
-                maxDimension: { type: "number", description: "Resize so longest side fits within this value (in points). Normalizes across device sizes. Example: 800. Overrides scale when set." },
+                maxDimension: { type: "number", description: "Resize so longest side fits within this value (in points). Example: 800. Overrides scale when set." },
             },
         },
     },
@@ -530,7 +534,7 @@ const TOOLS = [
     },
     {
         name: "nerve_run",
-        description: "Build, install, and launch an iOS app on the simulator. The app must include the Nerve SPM package. After launching, call nerve_view to see the initial screen, then navigate and interact as needed.",
+        description: "Build, install, and launch an iOS app on the simulator. Nerve is auto-injected if the app doesn't include it via SPM — no code changes needed. After launching, call nerve_view to see the initial screen.",
         inputSchema: {
             type: "object",
             properties: {
@@ -1087,6 +1091,60 @@ function runShell(cmd, timeoutMs = 120000) {
         proc.on("error", reject);
     });
 }
+// --- Nerve Framework Injection ---
+// Resolve paths for finding the Nerve framework
+// mcp-server/dist/index.js -> mcp-server/ (npm package root)
+// mcp-server/src/index.ts  -> mcp-server/ (development)
+const mpcPackageRoot = path.resolve(__dirname, "..");
+const nerveRepoRoot = path.resolve(mpcPackageRoot, "..");
+function findNerveFramework() {
+    const candidates = [
+        // 1. Bundled with npm package (npm install nerve-mcp)
+        path.join(mpcPackageRoot, "framework", "Nerve.framework", "Nerve"),
+        // 2. Homebrew installation
+        "/opt/homebrew/lib/nerve/Nerve.framework/Nerve",
+        "/usr/local/lib/nerve/Nerve.framework/Nerve",
+        // 3. Repo .build/inject/ (development from source)
+        path.join(nerveRepoRoot, ".build", "inject", "Nerve.framework", "Nerve"),
+    ];
+    return candidates.find(p => fs.existsSync(p)) ?? null;
+}
+async function ensureNerveFramework() {
+    const existing = findNerveFramework();
+    if (existing)
+        return existing;
+    // Auto-build from source (development mode)
+    const buildScript = path.join(nerveRepoRoot, "scripts", "build-framework.sh");
+    if (fs.existsSync(buildScript)) {
+        await runShell(`bash "${buildScript}"`, 180000);
+        const built = findNerveFramework();
+        if (built)
+            return built;
+    }
+    throw new Error("Nerve.framework not found. If installed via npm, reinstall nerve-mcp. " +
+        "If developing from source, run: scripts/build-framework.sh");
+}
+async function appContainsNerve(appPath) {
+    // Check if the app binary contains Nerve symbols (works for both static and dynamic SPM linking)
+    try {
+        const appName = path.basename(appPath, ".app");
+        const binary = path.join(appPath, appName);
+        const symbols = await runShell(`nm -gU "${binary}" 2>/dev/null | grep nerve_auto_start || true`);
+        if (symbols.trim())
+            return true;
+        // Also check Frameworks/ for dynamic linking case
+        const frameworks = path.join(appPath, "Frameworks");
+        if (fs.existsSync(frameworks)) {
+            const entries = fs.readdirSync(frameworks);
+            if (entries.some(e => e.startsWith("Nerve")))
+                return true;
+        }
+        return false;
+    }
+    catch {
+        return false;
+    }
+}
 async function findSimulatorUDID(name) {
     const json = await runShell("xcrun simctl list devices available -j");
     const data = JSON.parse(json);
@@ -1177,9 +1235,19 @@ async function handleBuildRun(command, params) {
         catch {
             // Not running
         }
-        // Launch
-        await runShell(`xcrun simctl launch "${udid}" "${bundleId}"`);
-        log.push("Launched.");
+        // Launch — detect SPM vs inject mode
+        const hasNerve = await appContainsNerve(appPath);
+        let injected = false;
+        if (hasNerve) {
+            await runShell(`xcrun simctl launch "${udid}" "${bundleId}"`);
+            log.push("Launched (SPM mode).");
+        }
+        else {
+            const frameworkBinary = await ensureNerveFramework();
+            await runShell(`SIMCTL_CHILD_DYLD_INSERT_LIBRARIES="${frameworkBinary}" xcrun simctl launch "${udid}" "${bundleId}"`);
+            log.push("Launched (inject mode).");
+            injected = true;
+        }
         // Set active target
         activeTarget = { udid, bundleId };
         const port = nervePort(udid, bundleId);
@@ -1194,7 +1262,12 @@ async function handleBuildRun(command, params) {
             await new Promise(r => setTimeout(r, 250));
         }
         if (!nerveReady) {
-            log.push("Nerve did not start. Ensure your app includes the Nerve SPM package with Nerve.start() in #if DEBUG.");
+            if (injected) {
+                log.push("Nerve did not start after injection. The framework may be incompatible — try rebuilding: delete .build/inject/ and re-run.");
+            }
+            else {
+                log.push("Nerve did not start. Ensure your app calls Nerve.start() in #if DEBUG.");
+            }
         }
         return { content: [{ type: "text", text: log.join("\n") }] };
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "nerve-mcp",
-  "version": "0.1.1",
+  "version": "0.2.1",
   "description": "MCP server for Nerve — gives AI agents runtime access to iOS apps",
   "main": "dist/index.js",
   "bin": {
@@ -10,14 +10,15 @@
     "build": "tsc",
     "start": "node dist/index.js",
     "dev": "tsx src/index.ts",
-    "prepublishOnly": "npm run build"
+    "prepublishOnly": "npm run build && npm run bundle-framework && cp ../README.md README.md",
+    "bundle-framework": "bash ../scripts/build-framework.sh && rm -rf framework && mkdir -p framework/Nerve.framework && cp ../.build/inject/Nerve.framework/Nerve framework/Nerve.framework/ && cp ../.build/inject/Nerve.framework/Info.plist framework/Nerve.framework/"
   },
   "repository": {
     "type": "git",
     "url": "git+https://github.com/luchi0208/nerve-ios.git",
     "directory": "mcp-server"
   },
-  "files": ["dist", "README.md"],
+  "files": ["dist", "framework", "README.md"],
   "keywords": ["mcp", "ios", "automation", "simulator", "nerve"],
   "dependencies": {
     "@modelcontextprotocol/sdk": "^1.0.0",