@mmerterden/multi-agent-toolkit-mcp 3.5.0 → 3.7.0

package/CHANGELOG.md

@@ -15,6 +15,70 @@ Releases before this file exists are recorded in the git tags and commit history
 
 ---
 
+## 3.7.0
+
+### Added
+
+- **`android_accessibility_audit` gains screen-reader reachability and reading
+  order.** A clickable node with `focusable="false"` can be tapped by a sighted
+  user and never reached by TalkBack at all: the control is invisible to the
+  screen reader rather than merely unnamed, so it is a `critical` finding, above
+  a missing contentDescription. The reading-order check is the same
+  rows-then-columns comparison the iOS side uses, sharing one implementation,
+  with `rtl` for right-to-left layouts.
+
+### Fixed
+
+- **A claim of mine in three places was wrong.** The notes on `android_meminfo`
+  and the 3.3.0 entry said this machine has no adb. It has adb; what it has no
+  Android device or emulator to run against. The distinction matters to anyone
+  reading those notes to decide what still needs verifying, so all three now say
+  what is actually missing.
+
+### Note
+
+Stage 4 of 4, and the last one that could not be measured here. The iOS work was
+verified against a live simulator at every step, including a generated app with
+planted defects for Apple's own audit. The Android checks are written against
+the documented `uiautomator dump` shape and pinned by fixtures; nothing on this
+machine could run them against a device. The first real run is their measurement,
+and the hierarchy attribute they depend on, `focusable`, is one uiautomator has
+always emitted.
+
+## 3.6.0
+
+### Added
+
+- **`ios_accessibility_audit_deep`: Apple's own accessibility audit.** Contrast,
+  Dynamic Type and clipped text cannot be reached from an accessibility tree
+  dump, whatever you do to the dumper. `XCUIAccessibilityAudit` can see them,
+  and it runs inside an XCUITest, so this is a separate tool rather than a mode
+  on the existing one: an `xcodebuild test` run, minutes rather than seconds,
+  needing a test target the tree-dump audit does not.
+
+  Verified end to end rather than written from the header. A throwaway SwiftUI
+  app with three planted defects was generated, built and driven on a live
+  simulator, and Apple's audit returned "Contrast failed", "Contrast nearly
+  passed" and "Element has no description". The parser was written against that
+  output, and the test fixture is those three messages in the exact result-bundle
+  shape `xcresulttool` emits.
+
+  The findings are classified by audit type. Anything the keyword table does not
+  recognise keeps its raw text under `unclassified` rather than being forced into
+  a bucket it may not belong to.
+
+  **The boundary, stated because it cannot be closed from here:** a test that
+  never calls `performAccessibilityAudit()` passes, and a passing test looks
+  exactly like an audit that found nothing. So the result reports whether the
+  named test RAN, the tool description says the test must perform the audit
+  itself, and `measurable` is false when the test did not run. What the tool
+  cannot do is prove the test called the API.
+
+### Note
+
+Stage 3 of 4. The Android side is last and stays unverifiable here: this machine
+has adb but no device or emulator to run it against.
+
 ## 3.5.0
 
 ### Added
@@ -51,7 +115,7 @@ Releases before this file exists are recorded in the git tags and commit history
 Stage 2 of 4. Contrast, Dynamic Type and clipped text still cannot be reached
 from a tree dump; they need Apple's own `XCUIAccessibilityAudit`, which is an
 XCUITest run rather than a simctl call. The Android side is stage 4 and is
-unverifiable on a machine with no adb.
+unverifiable without a device or emulator to run adb against.
 
 ## 3.4.0
 
@@ -127,7 +191,8 @@ order, traits and hints come next, then Apple's own `XCUIAccessibilityAudit`
 ### Note
 
 `android_meminfo`'s parser was written against the documented output shape on a
-machine with no adb and no emulator. The `leaks` parser was written against live
+machine with adb installed but no device or emulator to run it against. The
+`leaks` parser was written against live
 output captured from real runs, including the leaking and the unmeasurable case.
 The tests pin both shapes; the first real Android run is that half's measurement.
 

package/README.md

@@ -7,7 +7,7 @@
 
 🇹🇷 Türkçe: [README.tr.md](./README.tr.md)
 
-**86 tools** for iOS Simulator, Android Emulator, and headless web control. MCP server that lets your AI coding assistant see, interact with, and audit your mobile apps - plus drive browsers, run an 18-rule App Store compliance audit, and orchestrate multi-step batch flows.
+**87 tools** for iOS Simulator, Android Emulator, and headless web control. MCP server that lets your AI coding assistant see, interact with, and audit your mobile apps - plus drive browsers, run an 18-rule App Store compliance audit, and orchestrate multi-step batch flows.
 
 Distributed on the **public npm registry** - `npx @mmerterden/multi-agent-toolkit-mcp` resolves with no auth, no token, no `~/.npmrc` setup.
 
@@ -19,7 +19,7 @@ That distinction is worth keeping straight. This line once called five hosts "th
 - **Device Control** (59 tools) - screenshot, tap, swipe, type, navigate, dark mode, locale, orientation, location, permissions, push notifications, real start/stop screen recording
 - **Memory** (2 tools) - `ios_leaks` runs /usr/bin/leaks against a simulator or host process, snapshot or diff against a saved memory graph; `android_meminfo` reads dumpsys meminfo, snapshot or diff. Both report `measurable:false` rather than a clean result when they could not look
 - **Crash Diagnostics** (2 tools) - `ios_list_crashes` reads the host's DiagnosticReports, `android_list_crashes` dumps the adb crash buffer, both tail-bounded
-- **Accessibility Audit** (2 tools) - auto-detect missing labels, small tap targets, missing identifiers (scope filtering supported)
+- **Accessibility Audit** (3 tools) - `ios_accessibility_audit` / `android_accessibility_audit` read the live tree for missing labels, controls a screen reader cannot name, small tap targets, missing identifiers and a reading order that does not follow the visual layout. `ios_accessibility_audit_deep` runs Apple's own XCUIAccessibilityAudit through your XCUITest for the things a tree cannot show: contrast, Dynamic Type, clipped text. All three report `measurable:false` with a reason rather than a clean result when they could not look
 - **Store Compliance** (5 tools) - App Store / Play Store readiness; **18-rule deep `ios_app_store_audit`** cross-references Apple ITMS error codes + App Store Review Guidelines (privacy manifest, required-reason API, Info.plist, code signing, entitlements, embedded SDK, IPv6, debug-tool leak, ...)
 - **Web Automation** (8 tools) - Playwright-powered: goto, click (CSS selectors), type, eval JS, wait for selector, extract text, screenshot. Chromium / WebKit / Firefox engines. **Requires `playwright` peer dependency.**
 - **Design Audit** (6 tools) - mock-mode vs Figma conformance: scenario inventory, mock detection, mock launch, live UI geometry, pixel/geometry/typography compare, and the HTML/PDF report with its coverage gate
@@ -190,7 +190,7 @@ Run a scripted login flow in one MCP call:
 
 Returns per-step status so the agent can self-correct on failure.
 
-### iOS Tools (39)
+### iOS Tools (40)
 
 | Tool | Description |
 |------|-------------|

package/README.tr.md

@@ -7,7 +7,7 @@
 
 🇬🇧 English: [README.md](./README.md)
 
-iOS Simulator, Android Emulator ve headless web kontrolü için **86 araç**. AI kodlama asistanının mobil uygulamalarını görmesini, onlarla etkileşime girmesini ve denetlemesini sağlayan bir MCP sunucusu - ayrıca tarayıcıları sürer, 18-kurallık bir App Store uyumluluk denetimi çalıştırır ve çok-adımlı batch akışlarını orkestre eder.
+iOS Simulator, Android Emulator ve headless web kontrolü için **87 araç**. AI kodlama asistanının mobil uygulamalarını görmesini, onlarla etkileşime girmesini ve denetlemesini sağlayan bir MCP sunucusu - ayrıca tarayıcıları sürer, 18-kurallık bir App Store uyumluluk denetimi çalıştırır ve çok-adımlı batch akışlarını orkestre eder.
 
 **Public npm registry** üzerinden dağıtılır - `npx @mmerterden/multi-agent-toolkit-mcp`, auth'suz, token'sız, `~/.npmrc` ayarı gerekmeden çözülür.
 
@@ -189,7 +189,7 @@ Tek bir MCP çağrısında scriptlenmiş bir login akışı çalıştır:
 
 Agent'ın hata durumunda kendini düzeltebilmesi için adım-başına durum döner.
 
-### iOS Araçları (39)
+### iOS Araçları (40)
 
 | Araç | Açıklama |
 |------|-------------|

package/index.js

@@ -30,7 +30,7 @@ import {
 import { DESIGN_TOOLS, handleDesign } from "./tools/design-check/index.js";
 import { parseLaunchOutput } from "./tools/launch-time/index.js";
 import { parseLeaksOutput, parseMeminfoOutput, diffMeminfo } from "./tools/memory/index.js";
-import { auditIosTree, auditAndroidDump } from "./tools/a11y/index.js";
+import { auditIosTree, auditAndroidDump, parseAuditResults } from "./tools/a11y/index.js";
 import { interactiveElements } from "./tools/ui-inspect/index.js";
 import { selectCrashReports } from "./tools/crash-logs/index.js";
 import {
@@ -361,6 +361,7 @@ const IOS_TOOLS = [
   { name: "ios_xcresult", description: "Drill into a previous ios_xcodebuild result by xcresult ID. Modes: summary (counts), errors (file:line + message), warnings, log (last N lines), tests (failed), metrics (XCTMetric performance results as JSON). Use this instead of dumping the whole build log into context.", inputSchema: { type: "object", properties: { id: { type: "string", description: "xcresult ID returned by ios_xcodebuild" }, mode: { type: "string", enum: ["summary", "errors", "warnings", "log", "tests", "metrics"], description: "Default: summary" }, log_lines: { type: "number", description: "Lines of raw log to return when mode=log (default 200)" }, test_id: { type: "string", description: "mode=metrics only: scope to one test case or suite instead of every measured test" } }, required: ["id"] } },
   { name: "ios_visual_diff", description: "Compare two PNG screenshots. Returns JSON with diff_pct, pass/fail vs threshold, and an optional diff image. Use for snapshot regression checks across light/dark, locale, dynamic type variants.", inputSchema: { type: "object", properties: { baseline: { type: "string", description: "Path to baseline PNG" }, current: { type: "string", description: "Path to current PNG" }, threshold: { type: "number", description: "Per-pixel color threshold 0..1 (default 0.1, lower = stricter)" }, max_diff_pct: { type: "number", description: "Fail if diff exceeds this percent (default 1.0)" }, output: { type: "string", description: "Path to write diff PNG (optional)" } }, required: ["baseline", "current"] } },
   { name: "ios_leaks", description: "Look for leaked memory in a running simulator (or host) process with /usr/bin/leaks. mode=snapshot reports the current leak count and bytes; mode=diff reports only leaks new since a saved memory graph, which is the shape a regression gate wants. Reports measurable:false when the target lacks get-task-allow rather than reporting it as clean - leaks exits 0 in that case, so an unmeasurable target and a clean one are indistinguishable by exit status. Debug builds are debuggable; Apple-signed apps are not.", inputSchema: { type: "object", properties: { pid: { type: "number", description: "Process id. Either this or bundle_id." }, bundle_id: { type: "string", description: "Bundle id of an app running on the booted simulator; its pid is resolved for you." }, device_id: { type: "string" }, mode: { type: "string", enum: ["snapshot", "diff"], description: "Default: snapshot" }, baseline_graph: { type: "string", description: "mode=diff: path to the memory graph saved by an earlier call" }, output_graph: { type: "string", description: "Save a memory graph here to use as a later baseline" } }, required: [] } },
+  { name: "ios_accessibility_audit_deep", description: "Run Apple's own accessibility audit (XCUIAccessibilityAudit) through an existing XCUITest and report its findings: contrast, Dynamic Type, clipped text, traits, hit regions, element descriptions. These cannot be reached from an accessibility tree dump, which is why this is a separate tool from ios_accessibility_audit and does not replace it. It is an xcodebuild test run, minutes rather than seconds, and the named test MUST itself call performAccessibilityAudit() - a test that does not is indistinguishable from an audit that found nothing, so the result reports whether the test ran and never invents a clean verdict.", inputSchema: { type: "object", properties: { project: { type: "string", description: "Path to the .xcodeproj. Give this or workspace." }, workspace: { type: "string", description: "Path to the .xcworkspace." }, scheme: { type: "string" }, test_identifier: { type: "string", description: "e.g. AuditTest/testAccessibilityAudit(). Omit to read every test that ran." }, device_id: { type: "string", description: "Simulator udid. Defaults to the booted one." } }, required: ["scheme"] } },
   { name: "ios_list_crashes", description: "List recent crash reports from the host's ~/Library/Logs/DiagnosticReports - where simulator app crashes land. Filter by process name, bound by age and count.", inputSchema: { type: "object", properties: { app: { type: "string", description: "Only reports whose file name (the crashed process) contains this substring" }, since_min: { type: "number", description: "Only reports newer than this many minutes" }, limit: { type: "number", description: "Max reports returned, newest first (default 20)" } } } },
 ];
 
@@ -838,6 +839,34 @@ async function handleIOS(name, args, ctx = {}) {
         graph_saved_to: args.output_graph || null,
       }, null, 2);
     }
+    case "ios_accessibility_audit_deep": {
+      if (!args.project && !args.workspace) return "ERROR: pass project or workspace";
+      const container = args.workspace ? `-workspace ${shq(args.workspace)}` : `-project ${shq(args.project)}`;
+      const d = iosDevice(args.device_id);
+      const bundle = join(SCREENSHOT_DIR, `a11ydeep_${Date.now()}.xcresult`);
+      const only = args.test_identifier ? ` -only-testing:${shq(args.test_identifier)}` : "";
+      // xcodebuild exits non-zero when the audit finds anything, because each
+      // finding is an XCTest failure. That is a result, not a failure to run.
+      runCapture(
+        `xcodebuild test ${container} -scheme ${shq(args.scheme)} -destination ${shq(`platform=iOS Simulator,id=${d}`)} -resultBundlePath ${shq(bundle)}${only} 2>&1`,
+        { timeout: 900000 },
+      );
+      if (!existsSync(bundle)) {
+        return JSON.stringify({ measurable: false, reason: "xcodebuild produced no result bundle; the build failed before any test ran", test_ran: false, findings: [] }, null, 2);
+      }
+      const json = runCapture(`xcrun xcresulttool get test-results tests --path ${shq(bundle)} --format json 2>&1`, { timeout: 60000 });
+      const r = parseAuditResults(json, args.test_identifier || null);
+      return JSON.stringify({
+        measurable: r.measurable,
+        reason: r.reason,
+        test_ran: r.testRan,
+        test_result: r.testResult,
+        by_type: r.byType,
+        total_findings: r.measurable ? r.findings.length : null,
+        findings: r.findings,
+        result_bundle: bundle,
+      }, null, 2);
+    }
     case "ios_list_crashes": {
       const dir = join(homedir(), "Library", "Logs", "DiagnosticReports");
       if (!existsSync(dir)) return JSON.stringify({ dir, count: 0, reports: [] }, null, 2);
@@ -894,7 +923,7 @@ const ANDROID_TOOLS = [
   { name: "android_get_screen_size", description: "Get Android screen resolution", inputSchema: { type: "object", properties: { device_id: { type: "string" } } } },
   { name: "android_open_url", description: "Open URL or deep link on Android", inputSchema: { type: "object", properties: { url: { type: "string" }, device_id: { type: "string" } }, required: ["url"] } },
   { name: "android_clear_app_data", description: "Clear all data for Android app", inputSchema: { type: "object", properties: { package_name: { type: "string" }, device_id: { type: "string" } }, required: ["package_name"] } },
-  { name: "android_accessibility_audit", description: "Audit Android app accessibility: missing contentDescription, small touch targets (<48dp), missing resource-id. Use scope to filter by resource-id prefix.", inputSchema: { type: "object", properties: { device_id: { type: "string" }, scope: { type: "string", description: "Filter: only audit elements whose resource-id contains this prefix (e.g. 'login_', 'com.example:id/login_'). Omit to audit all." } } } },
+  { name: "android_accessibility_audit", description: "Audit Android app accessibility on the connected device: missing contentDescription, clickable nodes TalkBack cannot focus, touch targets under 48dp, missing resource-ids, and whether the reading order follows the visual layout. Reports measurable:false with a reason rather than a clean result when the dump could not be read.", inputSchema: { type: "object", properties: { device_id: { type: "string" }, scope: { type: "string", description: "Filter: only audit elements whose resource-id contains this prefix (e.g. 'login_', 'com.example:id/login_'). Omit to audit all." }, rtl: { type: "boolean", description: "Expect right-to-left reading within a row. Default false." } } } },
   { name: "android_launch_time", description: "Measure Android app launch time: force-stops the package, starts it with am start -W, and reports TotalTime/WaitTime in ms plus the platform's own LaunchState (COLD/WARM/HOT). Below Android 10 there is no LaunchState and cold_start is null rather than assumed.", inputSchema: { type: "object", properties: { package_name: { type: "string" }, activity: { type: "string" }, device_id: { type: "string" } }, required: ["package_name"] } },
   { name: "android_apk_audit", description: "Audit APK/AAB for Play Store compliance: debug flag, target SDK, permissions, signing, ProGuard", inputSchema: { type: "object", properties: { apk_path: { type: "string", description: "Path to .apk file" } }, required: ["apk_path"] } },
   { name: "android_meminfo", description: "Read an Android app's memory via `adb shell dumpsys meminfo` (KB, no root). mode=snapshot returns the App Summary rows and totals; mode=diff compares two snapshots so growth across the same flow is visible, which is the signal a leak actually produces - a single absolute number says almost nothing. Reports measurable:false when the package has no running process rather than returning zeros.", inputSchema: { type: "object", properties: { package_name: { type: "string" }, device_id: { type: "string" }, mode: { type: "string", enum: ["snapshot", "diff"], description: "Default: snapshot" }, baseline_json: { type: "string", description: "mode=diff: the JSON returned by an earlier snapshot call" } }, required: ["package_name"] } },
@@ -1055,13 +1084,14 @@ async function handleAndroid(name, args, ctx = {}) {
       run(`adb ${df} pull /sdcard/_mcp_a11y.xml ${shq(f)}`);
       run(`adb ${df} shell rm /sdcard/_mcp_a11y.xml`);
       const xml = existsSync(f) ? readFileSync(f, "utf-8") : "";
-      const r = auditAndroidDump({ xml, scope: args.scope || null });
+      const r = auditAndroidDump({ xml, scope: args.scope || null, rtl: args.rtl === true });
       return JSON.stringify({
         scope: r.scope,
         measurable: r.measurable,
         reason: r.reason,
         elements_scanned: r.elementsScanned,
         elements_skipped: r.elementsSkipped,
+        reading_order_ok: r.readingOrderOk ?? null,
         total_issues: r.totalIssues,
         critical: r.critical,
         important: r.important,

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@mmerterden/multi-agent-toolkit-mcp",
-  "version": "3.5.0",
-  "description": "MCP server for iOS Simulator, Android Emulator and headless web control. 86 tools: device automation (tap/swipe/type), accessibility audits, visual diff, crash logs, App Store / Play Store pre-submission compliance. Runs standalone over stdio with any MCP client.",
+  "version": "3.7.0",
+  "description": "MCP server for iOS Simulator, Android Emulator and headless web control. 87 tools: device automation (tap/swipe/type), accessibility audits, visual diff, crash logs, App Store / Play Store pre-submission compliance. Runs standalone over stdio with any MCP client.",
   "type": "module",
   "main": "index.js",
   "bin": {

package/tools/a11y/index.js

@@ -250,7 +250,7 @@ export function auditIosTree({ tree, scope = null, rtl = false }) {
  * @param {string|null} [params.scope] - only audit resource-ids containing this
  * @returns {object} same shape as auditIosTree
  */
-export function auditAndroidDump({ xml, scope = null }) {
+export function auditAndroidDump({ xml, scope = null, rtl = false }) {
   const text = typeof xml === "string" ? xml : "";
   if (!/<node\b/.test(text)) {
     return unmeasurable(scope, "the uiautomator dump held no nodes; the dump failed or the screen was not ready");
@@ -259,6 +259,7 @@ export function auditAndroidDump({ xml, scope = null }) {
   const issues = [];
   let elementsScanned = 0;
   let elementsSkipped = 0;
+  const sequence = [];
 
   for (const match of text.matchAll(/<node[^>]*>/g)) {
     const node = match[0];
@@ -276,13 +277,25 @@ export function auditAndroidDump({ xml, scope = null }) {
     }
     elementsScanned++;
 
+    // TalkBack moves between FOCUSABLE nodes. A clickable node that is not
+    // focusable can be tapped by a sighted user and never reached by a screen
+    // reader at all, which is a harder failure than a missing label: the
+    // control is invisible to the user rather than merely unnamed.
+    if (/focusable="false"/.test(node)) {
+      issues.push({ severity: "critical", issue: "Clickable but not focusable: TalkBack cannot reach this control", element: cls, resourceId: rid || null });
+    }
+
     if (!desc && !label) issues.push({ severity: "critical", issue: "Missing contentDescription", element: cls, resourceId: rid || null });
     if (!rid) issues.push({ severity: "warning", issue: "Missing resource-id (UI testing)", element: cls });
 
     const bounds = node.match(/bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"/);
     if (bounds) {
-      const w = parseInt(bounds[3], 10) - parseInt(bounds[1], 10);
-      const h = parseInt(bounds[4], 10) - parseInt(bounds[2], 10);
+      const x = parseInt(bounds[1], 10);
+      const y = parseInt(bounds[2], 10);
+      const w = parseInt(bounds[3], 10) - x;
+      const h = parseInt(bounds[4], 10) - y;
+      // Same shape the iOS side uses, so one reading-order check serves both.
+      sequence.push({ role: cls, identifier: rid || null, title: desc || label || null, frame: { x, y, w, h } });
       if (w < ANDROID_MIN_TAP_DP || h < ANDROID_MIN_TAP_DP) {
         issues.push({ severity: "important", issue: `Touch target too small: ${w}x${h}dp (min ${ANDROID_MIN_TAP_DP}x${ANDROID_MIN_TAP_DP})`, element: cls, resourceId: rid || null });
       }
@@ -297,12 +310,25 @@ export function auditAndroidDump({ xml, scope = null }) {
     );
   }
 
+  const order = checkReadingOrder(sequence, rtl);
+  if (!order.ok) {
+    const el = order.firstOutOfPlace;
+    issues.push({
+      severity: "important",
+      issue: `Reading order does not follow the visual layout: this element is read at position ${sequence.indexOf(el) + 1} but sits at position ${order.expectedIndex + 1} on screen`,
+      element: el.role,
+      resourceId: el.identifier || null,
+      label: el.title || null,
+    });
+  }
+
   return {
     measurable: true,
     reason: null,
     scope: scope || "all",
     elementsScanned,
     elementsSkipped,
+    readingOrderOk: order.ok,
     ...tally(issues),
   };
 }
@@ -333,3 +359,109 @@ function unmeasurable(scope, reason, elementsSkipped = 0) {
     issues: [],
   };
 }
+
+// Apple's audit reports its findings as XCTest failure messages. These three
+// were captured from a real run against a deliberately inaccessible app; the
+// rest are classified by keyword and anything unrecognised keeps its raw text
+// rather than being forced into a bucket it may not belong to.
+const AUDIT_TYPE_PATTERNS = [
+  [/contrast/i, "contrast"],
+  [/has no description|description is|element description/i, "sufficientElementDescription"],
+  [/clipped|truncat/i, "textClipped"],
+  [/dynamic type|text size/i, "dynamicType"],
+  [/hit region|touch target|tap target/i, "hitRegion"],
+  [/trait/i, "trait"],
+  [/element detection|not detected/i, "elementDetection"],
+  [/parent|child/i, "parentChild"],
+  [/action/i, "action"],
+];
+
+/**
+ * Classify one audit failure message.
+ *
+ * @param {string} message
+ * @returns {string} audit type, or "unclassified"
+ */
+export function classifyAuditMessage(message) {
+  for (const [re, type] of AUDIT_TYPE_PATTERNS) if (re.test(message)) return type;
+  return "unclassified";
+}
+
+/**
+ * Pull accessibility audit findings out of `xcresulttool get test-results tests`.
+ *
+ * Shape captured from a real run: testNodes nest plan > bundle > suite > case,
+ * and a failing case carries children of nodeType "Failure Message" whose name
+ * is "<file>:<line>: <message>".
+ *
+ * The honest boundary, stated because it cannot be closed from here: a test that
+ * never calls performAccessibilityAudit passes, and a passing test is
+ * indistinguishable from an audit that found nothing. So the result reports
+ * whether the named test RAN, and the caller is told that the test must perform
+ * the audit itself. `measurable` is false when the test did not run at all.
+ *
+ * @param {object|string} results - parsed or raw JSON from xcresulttool
+ * @param {string|null} [testIdentifier] - e.g. "AuditTest/testAccessibilityAudit()"
+ * @returns {{measurable: boolean, reason: string|null, testRan: boolean,
+ *            testResult: string|null, findings: object[], byType: object}}
+ */
+export function parseAuditResults(results, testIdentifier = null) {
+  let doc = results;
+  if (typeof doc === "string") {
+    try {
+      doc = JSON.parse(doc);
+    } catch {
+      return { measurable: false, reason: "xcresulttool output was not JSON", testRan: false, testResult: null, findings: [], byType: {} };
+    }
+  }
+  if (!doc || !Array.isArray(doc.testNodes)) {
+    return { measurable: false, reason: "no testNodes in the result bundle; the build or the test run failed", testRan: false, testResult: null, findings: [], byType: {} };
+  }
+
+  const cases = [];
+  const walk = (n) => {
+    if (!n || typeof n !== "object") return;
+    if (n.nodeType === "Test Case") cases.push(n);
+    (n.children || []).forEach(walk);
+  };
+  doc.testNodes.forEach(walk);
+
+  const matched = testIdentifier
+    ? cases.filter((c) => c.nodeIdentifier === testIdentifier || c.name === testIdentifier)
+    : cases;
+
+  if (matched.length === 0) {
+    return {
+      measurable: false,
+      reason: testIdentifier
+        ? `no test case named "${testIdentifier}" ran; check the identifier and that the target built`
+        : "no test case ran",
+      testRan: false,
+      testResult: null,
+      findings: [],
+      byType: {},
+    };
+  }
+
+  const findings = [];
+  for (const c of matched) {
+    for (const child of c.children || []) {
+      if (child.nodeType !== "Failure Message") continue;
+      const raw = String(child.name || "");
+      const message = raw.replace(/^.*?:\d+:\s*/, "").trim();
+      findings.push({ test: c.nodeIdentifier || c.name || null, type: classifyAuditMessage(message), message, raw });
+    }
+  }
+
+  const byType = {};
+  for (const f of findings) byType[f.type] = (byType[f.type] || 0) + 1;
+
+  return {
+    measurable: true,
+    reason: null,
+    testRan: true,
+    testResult: matched[0].result || null,
+    findings,
+    byType,
+  };
+}

package/tools/memory/index.js

@@ -86,7 +86,8 @@ const MEMINFO_ROWS = [
  * absolute number says almost nothing.
  *
  * NOT VERIFIED ON A DEVICE. Written against the documented output shape on a
- * machine with no adb and no emulator, unlike the leaks parser above, which was
+ * machine with adb installed but no device or emulator to run it against,
+ * unlike the leaks parser above, which was
  * checked against live output. The tests below pin the shape this expects; the
  * first real run is the measurement.
  *