@mmerterden/multi-agent-toolkit-mcp 3.5.0 → 3.6.0

package/CHANGELOG.md

@@ -15,6 +15,40 @@ Releases before this file exists are recorded in the git tags and commit history
 
 ---
 
+## 3.6.0
+
+### Added
+
+- **`ios_accessibility_audit_deep`: Apple's own accessibility audit.** Contrast,
+  Dynamic Type and clipped text cannot be reached from an accessibility tree
+  dump, whatever you do to the dumper. `XCUIAccessibilityAudit` can see them,
+  and it runs inside an XCUITest, so this is a separate tool rather than a mode
+  on the existing one: an `xcodebuild test` run, minutes rather than seconds,
+  needing a test target the tree-dump audit does not.
+
+  Verified end to end rather than written from the header. A throwaway SwiftUI
+  app with three planted defects was generated, built and driven on a live
+  simulator, and Apple's audit returned "Contrast failed", "Contrast nearly
+  passed" and "Element has no description". The parser was written against that
+  output, and the test fixture is those three messages in the exact result-bundle
+  shape `xcresulttool` emits.
+
+  The findings are classified by audit type. Anything the keyword table does not
+  recognise keeps its raw text under `unclassified` rather than being forced into
+  a bucket it may not belong to.
+
+  **The boundary, stated because it cannot be closed from here:** a test that
+  never calls `performAccessibilityAudit()` passes, and a passing test looks
+  exactly like an audit that found nothing. So the result reports whether the
+  named test RAN, the tool description says the test must perform the audit
+  itself, and `measurable` is false when the test did not run. What the tool
+  cannot do is prove the test called the API.
+
+### Note
+
+Stage 3 of 4. The Android side is last and stays unverifiable here: this machine
+has no adb and no emulator.
+
 ## 3.5.0
 
 ### Added

package/README.md

@@ -7,7 +7,7 @@
 
 🇹🇷 Türkçe: [README.tr.md](./README.tr.md)
 
-**86 tools** for iOS Simulator, Android Emulator, and headless web control. MCP server that lets your AI coding assistant see, interact with, and audit your mobile apps - plus drive browsers, run an 18-rule App Store compliance audit, and orchestrate multi-step batch flows.
+**87 tools** for iOS Simulator, Android Emulator, and headless web control. MCP server that lets your AI coding assistant see, interact with, and audit your mobile apps - plus drive browsers, run an 18-rule App Store compliance audit, and orchestrate multi-step batch flows.
 
 Distributed on the **public npm registry** - `npx @mmerterden/multi-agent-toolkit-mcp` resolves with no auth, no token, no `~/.npmrc` setup.
 
@@ -19,7 +19,7 @@ That distinction is worth keeping straight. This line once called five hosts "th
 - **Device Control** (59 tools) - screenshot, tap, swipe, type, navigate, dark mode, locale, orientation, location, permissions, push notifications, real start/stop screen recording
 - **Memory** (2 tools) - `ios_leaks` runs /usr/bin/leaks against a simulator or host process, snapshot or diff against a saved memory graph; `android_meminfo` reads dumpsys meminfo, snapshot or diff. Both report `measurable:false` rather than a clean result when they could not look
 - **Crash Diagnostics** (2 tools) - `ios_list_crashes` reads the host's DiagnosticReports, `android_list_crashes` dumps the adb crash buffer, both tail-bounded
-- **Accessibility Audit** (2 tools) - auto-detect missing labels, small tap targets, missing identifiers (scope filtering supported)
+- **Accessibility Audit** (3 tools) - `ios_accessibility_audit` / `android_accessibility_audit` read the live tree for missing labels, controls a screen reader cannot name, small tap targets, missing identifiers and a reading order that does not follow the visual layout. `ios_accessibility_audit_deep` runs Apple's own XCUIAccessibilityAudit through your XCUITest for the things a tree cannot show: contrast, Dynamic Type, clipped text. All three report `measurable:false` with a reason rather than a clean result when they could not look
 - **Store Compliance** (5 tools) - App Store / Play Store readiness; **18-rule deep `ios_app_store_audit`** cross-references Apple ITMS error codes + App Store Review Guidelines (privacy manifest, required-reason API, Info.plist, code signing, entitlements, embedded SDK, IPv6, debug-tool leak, ...)
 - **Web Automation** (8 tools) - Playwright-powered: goto, click (CSS selectors), type, eval JS, wait for selector, extract text, screenshot. Chromium / WebKit / Firefox engines. **Requires `playwright` peer dependency.**
 - **Design Audit** (6 tools) - mock-mode vs Figma conformance: scenario inventory, mock detection, mock launch, live UI geometry, pixel/geometry/typography compare, and the HTML/PDF report with its coverage gate
@@ -190,7 +190,7 @@ Run a scripted login flow in one MCP call:
 
 Returns per-step status so the agent can self-correct on failure.
 
-### iOS Tools (39)
+### iOS Tools (40)
 
 | Tool | Description |
 |------|-------------|

package/README.tr.md

@@ -7,7 +7,7 @@
 
 🇬🇧 English: [README.md](./README.md)
 
-iOS Simulator, Android Emulator ve headless web kontrolü için **86 araç**. AI kodlama asistanının mobil uygulamalarını görmesini, onlarla etkileşime girmesini ve denetlemesini sağlayan bir MCP sunucusu - ayrıca tarayıcıları sürer, 18-kurallık bir App Store uyumluluk denetimi çalıştırır ve çok-adımlı batch akışlarını orkestre eder.
+iOS Simulator, Android Emulator ve headless web kontrolü için **87 araç**. AI kodlama asistanının mobil uygulamalarını görmesini, onlarla etkileşime girmesini ve denetlemesini sağlayan bir MCP sunucusu - ayrıca tarayıcıları sürer, 18-kurallık bir App Store uyumluluk denetimi çalıştırır ve çok-adımlı batch akışlarını orkestre eder.
 
 **Public npm registry** üzerinden dağıtılır - `npx @mmerterden/multi-agent-toolkit-mcp`, auth'suz, token'sız, `~/.npmrc` ayarı gerekmeden çözülür.
 
@@ -189,7 +189,7 @@ Tek bir MCP çağrısında scriptlenmiş bir login akışı çalıştır:
 
 Agent'ın hata durumunda kendini düzeltebilmesi için adım-başına durum döner.
 
-### iOS Araçları (39)
+### iOS Araçları (40)
 
 | Araç | Açıklama |
 |------|-------------|

package/index.js

@@ -30,7 +30,7 @@ import {
 import { DESIGN_TOOLS, handleDesign } from "./tools/design-check/index.js";
 import { parseLaunchOutput } from "./tools/launch-time/index.js";
 import { parseLeaksOutput, parseMeminfoOutput, diffMeminfo } from "./tools/memory/index.js";
-import { auditIosTree, auditAndroidDump } from "./tools/a11y/index.js";
+import { auditIosTree, auditAndroidDump, parseAuditResults } from "./tools/a11y/index.js";
 import { interactiveElements } from "./tools/ui-inspect/index.js";
 import { selectCrashReports } from "./tools/crash-logs/index.js";
 import {
@@ -361,6 +361,7 @@ const IOS_TOOLS = [
   { name: "ios_xcresult", description: "Drill into a previous ios_xcodebuild result by xcresult ID. Modes: summary (counts), errors (file:line + message), warnings, log (last N lines), tests (failed), metrics (XCTMetric performance results as JSON). Use this instead of dumping the whole build log into context.", inputSchema: { type: "object", properties: { id: { type: "string", description: "xcresult ID returned by ios_xcodebuild" }, mode: { type: "string", enum: ["summary", "errors", "warnings", "log", "tests", "metrics"], description: "Default: summary" }, log_lines: { type: "number", description: "Lines of raw log to return when mode=log (default 200)" }, test_id: { type: "string", description: "mode=metrics only: scope to one test case or suite instead of every measured test" } }, required: ["id"] } },
   { name: "ios_visual_diff", description: "Compare two PNG screenshots. Returns JSON with diff_pct, pass/fail vs threshold, and an optional diff image. Use for snapshot regression checks across light/dark, locale, dynamic type variants.", inputSchema: { type: "object", properties: { baseline: { type: "string", description: "Path to baseline PNG" }, current: { type: "string", description: "Path to current PNG" }, threshold: { type: "number", description: "Per-pixel color threshold 0..1 (default 0.1, lower = stricter)" }, max_diff_pct: { type: "number", description: "Fail if diff exceeds this percent (default 1.0)" }, output: { type: "string", description: "Path to write diff PNG (optional)" } }, required: ["baseline", "current"] } },
   { name: "ios_leaks", description: "Look for leaked memory in a running simulator (or host) process with /usr/bin/leaks. mode=snapshot reports the current leak count and bytes; mode=diff reports only leaks new since a saved memory graph, which is the shape a regression gate wants. Reports measurable:false when the target lacks get-task-allow rather than reporting it as clean - leaks exits 0 in that case, so an unmeasurable target and a clean one are indistinguishable by exit status. Debug builds are debuggable; Apple-signed apps are not.", inputSchema: { type: "object", properties: { pid: { type: "number", description: "Process id. Either this or bundle_id." }, bundle_id: { type: "string", description: "Bundle id of an app running on the booted simulator; its pid is resolved for you." }, device_id: { type: "string" }, mode: { type: "string", enum: ["snapshot", "diff"], description: "Default: snapshot" }, baseline_graph: { type: "string", description: "mode=diff: path to the memory graph saved by an earlier call" }, output_graph: { type: "string", description: "Save a memory graph here to use as a later baseline" } }, required: [] } },
+  { name: "ios_accessibility_audit_deep", description: "Run Apple's own accessibility audit (XCUIAccessibilityAudit) through an existing XCUITest and report its findings: contrast, Dynamic Type, clipped text, traits, hit regions, element descriptions. These cannot be reached from an accessibility tree dump, which is why this is a separate tool from ios_accessibility_audit and does not replace it. It is an xcodebuild test run, minutes rather than seconds, and the named test MUST itself call performAccessibilityAudit() - a test that does not is indistinguishable from an audit that found nothing, so the result reports whether the test ran and never invents a clean verdict.", inputSchema: { type: "object", properties: { project: { type: "string", description: "Path to the .xcodeproj. Give this or workspace." }, workspace: { type: "string", description: "Path to the .xcworkspace." }, scheme: { type: "string" }, test_identifier: { type: "string", description: "e.g. AuditTest/testAccessibilityAudit(). Omit to read every test that ran." }, device_id: { type: "string", description: "Simulator udid. Defaults to the booted one." } }, required: ["scheme"] } },
   { name: "ios_list_crashes", description: "List recent crash reports from the host's ~/Library/Logs/DiagnosticReports - where simulator app crashes land. Filter by process name, bound by age and count.", inputSchema: { type: "object", properties: { app: { type: "string", description: "Only reports whose file name (the crashed process) contains this substring" }, since_min: { type: "number", description: "Only reports newer than this many minutes" }, limit: { type: "number", description: "Max reports returned, newest first (default 20)" } } } },
 ];
 
@@ -838,6 +839,34 @@ async function handleIOS(name, args, ctx = {}) {
         graph_saved_to: args.output_graph || null,
       }, null, 2);
     }
+    case "ios_accessibility_audit_deep": {
+      if (!args.project && !args.workspace) return "ERROR: pass project or workspace";
+      const container = args.workspace ? `-workspace ${shq(args.workspace)}` : `-project ${shq(args.project)}`;
+      const d = iosDevice(args.device_id);
+      const bundle = join(SCREENSHOT_DIR, `a11ydeep_${Date.now()}.xcresult`);
+      const only = args.test_identifier ? ` -only-testing:${shq(args.test_identifier)}` : "";
+      // xcodebuild exits non-zero when the audit finds anything, because each
+      // finding is an XCTest failure. That is a result, not a failure to run.
+      runCapture(
+        `xcodebuild test ${container} -scheme ${shq(args.scheme)} -destination ${shq(`platform=iOS Simulator,id=${d}`)} -resultBundlePath ${shq(bundle)}${only} 2>&1`,
+        { timeout: 900000 },
+      );
+      if (!existsSync(bundle)) {
+        return JSON.stringify({ measurable: false, reason: "xcodebuild produced no result bundle; the build failed before any test ran", test_ran: false, findings: [] }, null, 2);
+      }
+      const json = runCapture(`xcrun xcresulttool get test-results tests --path ${shq(bundle)} --format json 2>&1`, { timeout: 60000 });
+      const r = parseAuditResults(json, args.test_identifier || null);
+      return JSON.stringify({
+        measurable: r.measurable,
+        reason: r.reason,
+        test_ran: r.testRan,
+        test_result: r.testResult,
+        by_type: r.byType,
+        total_findings: r.measurable ? r.findings.length : null,
+        findings: r.findings,
+        result_bundle: bundle,
+      }, null, 2);
+    }
     case "ios_list_crashes": {
       const dir = join(homedir(), "Library", "Logs", "DiagnosticReports");
       if (!existsSync(dir)) return JSON.stringify({ dir, count: 0, reports: [] }, null, 2);

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@mmerterden/multi-agent-toolkit-mcp",
-  "version": "3.5.0",
-  "description": "MCP server for iOS Simulator, Android Emulator and headless web control. 86 tools: device automation (tap/swipe/type), accessibility audits, visual diff, crash logs, App Store / Play Store pre-submission compliance. Runs standalone over stdio with any MCP client.",
+  "version": "3.6.0",
+  "description": "MCP server for iOS Simulator, Android Emulator and headless web control. 87 tools: device automation (tap/swipe/type), accessibility audits, visual diff, crash logs, App Store / Play Store pre-submission compliance. Runs standalone over stdio with any MCP client.",
   "type": "module",
   "main": "index.js",
   "bin": {

package/tools/a11y/index.js

@@ -333,3 +333,109 @@ function unmeasurable(scope, reason, elementsSkipped = 0) {
     issues: [],
   };
 }
+
+// Apple's audit reports its findings as XCTest failure messages. These three
+// were captured from a real run against a deliberately inaccessible app; the
+// rest are classified by keyword and anything unrecognised keeps its raw text
+// rather than being forced into a bucket it may not belong to.
+const AUDIT_TYPE_PATTERNS = [
+  [/contrast/i, "contrast"],
+  [/has no description|description is|element description/i, "sufficientElementDescription"],
+  [/clipped|truncat/i, "textClipped"],
+  [/dynamic type|text size/i, "dynamicType"],
+  [/hit region|touch target|tap target/i, "hitRegion"],
+  [/trait/i, "trait"],
+  [/element detection|not detected/i, "elementDetection"],
+  [/parent|child/i, "parentChild"],
+  [/action/i, "action"],
+];
+
+/**
+ * Classify one audit failure message.
+ *
+ * @param {string} message
+ * @returns {string} audit type, or "unclassified"
+ */
+export function classifyAuditMessage(message) {
+  for (const [re, type] of AUDIT_TYPE_PATTERNS) if (re.test(message)) return type;
+  return "unclassified";
+}
+
+/**
+ * Pull accessibility audit findings out of `xcresulttool get test-results tests`.
+ *
+ * Shape captured from a real run: testNodes nest plan > bundle > suite > case,
+ * and a failing case carries children of nodeType "Failure Message" whose name
+ * is "<file>:<line>: <message>".
+ *
+ * The honest boundary, stated because it cannot be closed from here: a test that
+ * never calls performAccessibilityAudit passes, and a passing test is
+ * indistinguishable from an audit that found nothing. So the result reports
+ * whether the named test RAN, and the caller is told that the test must perform
+ * the audit itself. `measurable` is false when the test did not run at all.
+ *
+ * @param {object|string} results - parsed or raw JSON from xcresulttool
+ * @param {string|null} [testIdentifier] - e.g. "AuditTest/testAccessibilityAudit()"
+ * @returns {{measurable: boolean, reason: string|null, testRan: boolean,
+ *            testResult: string|null, findings: object[], byType: object}}
+ */
+export function parseAuditResults(results, testIdentifier = null) {
+  let doc = results;
+  if (typeof doc === "string") {
+    try {
+      doc = JSON.parse(doc);
+    } catch {
+      return { measurable: false, reason: "xcresulttool output was not JSON", testRan: false, testResult: null, findings: [], byType: {} };
+    }
+  }
+  if (!doc || !Array.isArray(doc.testNodes)) {
+    return { measurable: false, reason: "no testNodes in the result bundle; the build or the test run failed", testRan: false, testResult: null, findings: [], byType: {} };
+  }
+
+  const cases = [];
+  const walk = (n) => {
+    if (!n || typeof n !== "object") return;
+    if (n.nodeType === "Test Case") cases.push(n);
+    (n.children || []).forEach(walk);
+  };
+  doc.testNodes.forEach(walk);
+
+  const matched = testIdentifier
+    ? cases.filter((c) => c.nodeIdentifier === testIdentifier || c.name === testIdentifier)
+    : cases;
+
+  if (matched.length === 0) {
+    return {
+      measurable: false,
+      reason: testIdentifier
+        ? `no test case named "${testIdentifier}" ran; check the identifier and that the target built`
+        : "no test case ran",
+      testRan: false,
+      testResult: null,
+      findings: [],
+      byType: {},
+    };
+  }
+
+  const findings = [];
+  for (const c of matched) {
+    for (const child of c.children || []) {
+      if (child.nodeType !== "Failure Message") continue;
+      const raw = String(child.name || "");
+      const message = raw.replace(/^.*?:\d+:\s*/, "").trim();
+      findings.push({ test: c.nodeIdentifier || c.name || null, type: classifyAuditMessage(message), message, raw });
+    }
+  }
+
+  const byType = {};
+  for (const f of findings) byType[f.type] = (byType[f.type] || 0) + 1;
+
+  return {
+    measurable: true,
+    reason: null,
+    testRan: true,
+    testResult: matched[0].result || null,
+    findings,
+    byType,
+  };
+}