@mmerterden/multi-agent-toolkit-mcp 3.3.0 → 3.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -15,6 +15,83 @@ Releases before this file exists are recorded in the git tags and commit history
15
15
 
16
16
  ---
17
17
 
18
+ ## 3.5.0
19
+
20
+ ### Added
21
+
22
+ - **`ios_accessibility_audit` checks the reading order.** A screen with perfect
23
+ labels is still unusable if VoiceOver walks it in the wrong order, and nothing
24
+ was looking. The audit compares the reading sequence against the visual layout
25
+ and reports the first element that arrives out of place.
26
+
27
+ The comparison is rows-then-columns, not a naive top-to-bottom sort: a home
28
+ screen is a grid, a toolbar is a row, and sorting by `y` alone calls every
29
+ correct grid broken. Elements share a row when their vertical extents overlap
30
+ by more than half the shorter one, which absorbs the few points of jitter real
31
+ layouts carry. `rtl: true` expects right-to-left within a row.
32
+
33
+ It is an `important` finding and never a `critical` one: a deliberate reading
34
+ order that differs from the visual one is legitimate, so this is a question
35
+ worth asking rather than a proven defect. Verified on a live home screen grid:
36
+ 13 elements, reading order correct, no false positive.
37
+
38
+ - **Traits and hints.** The dumper now reads `AXRoleDescription` (what a screen
39
+ reader announces the element AS) and `AXHelp` (where the iOS
40
+ `accessibilityHint` bridges to). A control with no role announcement is read
41
+ out as bare text and the user cannot tell it is actionable, so that is an
42
+ `important` finding. A missing hint is not: hints are not required on every
43
+ element, so the result reports `hints_present` as a count and leaves the
44
+ judgement to the reader.
45
+
46
+ `AXSubrole` was probed first and carries nothing at all through this bridge,
47
+ so it is not read. Guessing at it would have produced a check that never fires.
48
+
49
+ ### Note
50
+
51
+ Stage 2 of 4. Contrast, Dynamic Type and clipped text still cannot be reached
52
+ from a tree dump; they need Apple's own `XCUIAccessibilityAudit`, which is an
53
+ XCUITest run rather than a simctl call. The Android side is stage 4 and is
54
+ unverifiable on a machine with no adb.
55
+
56
+ ## 3.4.0
57
+
58
+ ### Fixed
59
+
60
+ - **The accessibility audits reported a clean screen they had never read.** On a
61
+ tree that came back empty they returned `elements_scanned: 0, total_issues: 0,
62
+ critical: 0, important: 0, warning: 0` - a full clean bill of health,
63
+ indistinguishable from an accessible screen, and the first reading is the one
64
+ anyone believes. Both audits now carry `measurable` and a `reason`, and an
65
+ unmeasurable run reports `total_issues: null`, never 0. A count of zero is a
66
+ measurement; none was taken.
67
+
68
+ - **The iOS tree was empty until the Simulator window had been activated.**
69
+ Measured on a live simulator: 1 node before activation, 28 after, and it stays
70
+ populated once the window is backgrounded again. The dumper now activates the
71
+ window itself. It also says what to do when Simulator.app is not running at
72
+ all, because booting a device with `simctl` is not enough - the accessibility
73
+ bridge lives in the UI app.
74
+
75
+ - **The iOS audit scored Simulator.app's own chrome as app findings.** The
76
+ dumped window carries the hardware buttons, the toolbar and the title text
77
+ beside the device screen, so Apple's 17x65pt Volume Up button was reported as
78
+ an app tap-target violation. The audit now scopes to the device screen, the
79
+ largest AXGroup child of the window. On a live Home screen this took the
80
+ result from 23 findings over 23 elements to 1 finding over 13: 22 of the 23
81
+ were the simulator's own interface.
82
+
83
+ - The scoring moved to `tools/a11y/` with a suite covering the degenerate tree,
84
+ the chrome exclusion, a scope that matches nothing, and the rule that an
85
+ element with no identifier is audited whatever the scope - you cannot scope
86
+ what you cannot identify, and skipping it would hide the finding that says so.
87
+
88
+ ### Note
89
+
90
+ This is the first of four stages. It buys nothing new for a blind user yet; it
91
+ makes the existing checks honest so the next stages can be trusted. Reading
92
+ order, traits and hints come next, then Apple's own `XCUIAccessibilityAudit`
93
+ (contrast, Dynamic Type, clipped text), then the Android side.
94
+
18
95
  ## 3.3.0
19
96
 
20
97
  ### Added
package/index.js CHANGED
@@ -30,6 +30,7 @@ import {
30
30
  import { DESIGN_TOOLS, handleDesign } from "./tools/design-check/index.js";
31
31
  import { parseLaunchOutput } from "./tools/launch-time/index.js";
32
32
  import { parseLeaksOutput, parseMeminfoOutput, diffMeminfo } from "./tools/memory/index.js";
33
+ import { auditIosTree, auditAndroidDump } from "./tools/a11y/index.js";
33
34
  import { interactiveElements } from "./tools/ui-inspect/index.js";
34
35
  import { selectCrashReports } from "./tools/crash-logs/index.js";
35
36
  import {
@@ -350,7 +351,7 @@ const IOS_TOOLS = [
350
351
  { name: "ios_get_app_container", description: "Get iOS app container path", inputSchema: { type: "object", properties: { bundle_id: { type: "string" }, container: { type: "string", enum: ["app", "data", "groups"] }, device_id: { type: "string" } }, required: ["bundle_id"] } },
351
352
  { name: "ios_erase_device", description: "Factory reset iOS simulator", inputSchema: { type: "object", properties: { device_id: { type: "string" } } } },
352
353
  { name: "ios_get_ui_tree", description: "Get iOS accessibility UI tree via macOS AX APIs. Pass `path` to write the raw dump to a file and return only its location - the full tree can be tens of KB.", inputSchema: { type: "object", properties: { max_depth: { type: "number" }, path: { type: "string", description: "Absolute file path to write the raw tree JSON to. The parent directory must already exist." } } } },
353
- { name: "ios_accessibility_audit", description: "Audit iOS app accessibility: missing labels, small tap targets (<44pt), missing identifiers. Use scope to filter by identifier prefix (e.g. 'login_' only checks login screen elements).", inputSchema: { type: "object", properties: { max_depth: { type: "number" }, scope: { type: "string", description: "Filter: only audit elements whose identifier starts with this prefix (e.g. 'login_', 'settings_'). Omit to audit all." } } } },
354
+ { name: "ios_accessibility_audit", description: "Audit iOS app accessibility on the booted simulator: missing labels, controls a screen reader cannot name, tap targets under 44pt, missing identifiers, and whether the reading order follows the visual layout. Reports measurable:false with a reason rather than a clean result when the tree could not be read - booting a device is not enough, Simulator.app must be running. Scopes to the device screen, never the simulator own chrome.", inputSchema: { type: "object", properties: { max_depth: { type: "number" }, scope: { type: "string", description: "Filter: only audit elements whose identifier starts with this prefix (e.g. 'login_', 'settings_'). Omit to audit all." }, rtl: { type: "boolean", description: "Expect right-to-left reading within a row. Default false." } } } },
354
355
  { name: "ios_biometric", description: "Simulate Face ID / Touch ID on iOS simulator (match or nomatch)", inputSchema: { type: "object", properties: { match: { type: "boolean", description: "true=success, false=failure" }, device_id: { type: "string" } }, required: ["match"] } },
355
356
  { name: "ios_archive_audit", description: "DEPRECATED - use ios_app_store_audit (18-rule deep scan). Lighter 6-check audit kept for backward compatibility; will be removed in the next major.", inputSchema: { type: "object", properties: { archive_path: { type: "string", description: "Path to .xcarchive" } }, required: ["archive_path"] } },
356
357
  { name: "ios_export_ipa", description: "Export a .xcarchive to a signed .ipa via xcodebuild -exportArchive. Generates the exportOptions.plist from the arguments (method defaults to app-store-connect), so callers do not have to hand-maintain one. Returns the .ipa path plus parsed errors; a run that exits 0 without producing an .ipa is reported as a failure. Pair with ios_testflight_validate for the pre-submission gate.", inputSchema: { type: "object", properties: { archive_path: { type: "string", description: "Absolute path to the .xcarchive" }, output_dir: { type: "string", description: "Directory to write the .ipa into" }, method: { type: "string", description: "Export method: app-store-connect (default) | release-testing | enterprise | development" }, team_id: { type: "string", description: "Apple Developer team ID" }, provisioning_profiles: { type: "object", description: "Map of bundleId -> provisioning profile name (manual signing)" }, signing_style: { type: "string", description: "automatic | manual" }, upload_symbols: { type: "boolean", description: "Include symbols (default true)" }, allow_provisioning_updates: { type: "boolean", description: "Off by default. Lets xcodebuild register devices and create/modify provisioning profiles in the developer account - a change on Apple's side, so it is opt-in" }, timeout_sec: { type: "number", description: "Default 900" } }, required: ["archive_path", "output_dir"] } },
@@ -500,38 +501,29 @@ async function handleIOS(name, args, ctx = {}) {
500
501
  const script = join(__dirname, "ui-tree-dumper.swift");
501
502
  if (!existsSync(script)) return "ui-tree-dumper.swift not found";
502
503
  const depth = num(args.max_depth ?? 10, "max_depth");
503
- const scope = args.scope || null;
504
- const treeJson = run(`swift ${shq(script)} ${depth}`, { timeout: 15000 });
504
+ const treeJson = run(`swift ${shq(script)} ${depth}`, { timeout: 30000 });
505
+ let tree = null;
505
506
  try {
506
- const issues = [];
507
- let totalScanned = 0, totalSkipped = 0;
508
- function auditNode(node, path = "") {
509
- const loc = path ? `${path} > ${node.role}` : node.role;
510
- const w = node.frame?.w || 0, h = node.frame?.h || 0;
511
- const isInteractive = ["AXButton", "AXLink", "AXTextField", "AXTextArea", "AXCheckBox", "AXRadioButton", "AXSlider", "AXSwitch", "AXTab"].includes(node.role);
512
- if (isInteractive) {
513
- // Scope filter: skip elements outside scope
514
- if (scope && node.identifier && !node.identifier.startsWith(scope)) { totalSkipped++; if (node.children) node.children.forEach(c => auditNode(c, loc)); return; }
515
- if (scope && !node.identifier) { /* no identifier = can't scope, still audit */ }
516
- totalScanned++;
517
- if (!node.title && !node.description && !node.value) issues.push({ severity: "critical", issue: "Missing accessibility label", element: loc, identifier: node.identifier, frame: node.frame });
518
- if (!node.identifier) issues.push({ severity: "warning", issue: "Missing accessibility identifier (UI testing)", element: loc });
519
- if (w > 0 && h > 0 && (w < 44 || h < 44)) issues.push({ severity: "important", issue: `Tap target too small: ${w.toFixed(0)}x${h.toFixed(0)}pt (min 44x44)`, element: loc, identifier: node.identifier, frame: node.frame });
520
- }
521
- if (node.children) node.children.forEach(c => auditNode(c, loc));
522
- }
523
- const tree = JSON.parse(treeJson);
524
- auditNode(tree);
525
- return JSON.stringify({ scope: scope || "all", elements_scanned: totalScanned, elements_skipped: totalSkipped, total_issues: issues.length, critical: issues.filter(i => i.severity === "critical").length, important: issues.filter(i => i.severity === "important").length, warning: issues.filter(i => i.severity === "warning").length, issues }, null, 2);
526
- } catch (e) { return `ERROR parsing UI tree: ${e.message}\n\nRaw output:\n${treeJson?.substring(0, 500)}`; }
507
+ tree = JSON.parse(treeJson);
508
+ } catch {
509
+ tree = null;
510
+ }
511
+ const r = auditIosTree({ tree, scope: args.scope || null, rtl: args.rtl === true });
512
+ return JSON.stringify({
513
+ scope: r.scope,
514
+ measurable: r.measurable,
515
+ reason: r.reason,
516
+ elements_scanned: r.elementsScanned,
517
+ elements_skipped: r.elementsSkipped,
518
+ reading_order_ok: r.readingOrderOk ?? null,
519
+ hints_present: r.hintsPresent ?? null,
520
+ total_issues: r.totalIssues,
521
+ critical: r.critical,
522
+ important: r.important,
523
+ warning: r.warning,
524
+ issues: r.issues,
525
+ }, null, 2);
527
526
  }
528
- // `simctl keychain <device> biometric-enroll` / `biometric-match` do not
529
- // exist - `keychain` supports only add-root-cert/add-cert/reset. The only
530
- // available lever is the (undocumented) BiometricKit notification, which is
531
- // posted via notifyutil inside the simulator. notifyutil exits 0 even when
532
- // it cannot set or post the name, so its output has to be inspected: a
533
- // "Failed with code N" line means the notification did not land, and that
534
- // must be reported as a failure rather than as a simulated success.
535
527
  case "ios_biometric": {
536
528
  const d = iosDevice(args.device_id);
537
529
  const action = args.match ? "match" : "nomatch";
@@ -1062,34 +1054,20 @@ async function handleAndroid(name, args, ctx = {}) {
1062
1054
  const f = join(SCREENSHOT_DIR, `a11y_${Date.now()}.xml`);
1063
1055
  run(`adb ${df} pull /sdcard/_mcp_a11y.xml ${shq(f)}`);
1064
1056
  run(`adb ${df} shell rm /sdcard/_mcp_a11y.xml`);
1065
- if (!existsSync(f)) return "ERROR: UI dump failed";
1066
- const xml = readFileSync(f, "utf-8");
1067
- const scope = args.scope || null;
1068
- const issues = [];
1069
- let totalScanned = 0, totalSkipped = 0;
1070
- const nodeRegex = /<node[^>]*>/g;
1071
- let match;
1072
- while ((match = nodeRegex.exec(xml)) !== null) {
1073
- const node = match[0];
1074
- const cls = node.match(/class="([^"]*)"/)?.[1] || "";
1075
- const desc = node.match(/content-desc="([^"]*)"/)?.[1] || "";
1076
- const rid = node.match(/resource-id="([^"]*)"/)?.[1] || "";
1077
- const text = node.match(/text="([^"]*)"/)?.[1] || "";
1078
- const clickable = node.includes('clickable="true"');
1079
- const bounds = node.match(/bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"/);
1080
- if (clickable) {
1081
- if (scope && rid && !rid.includes(scope)) { totalSkipped++; continue; }
1082
- totalScanned++;
1083
- if (!desc && !text) issues.push({ severity: "critical", issue: "Missing contentDescription", element: cls, resourceId: rid });
1084
- if (!rid) issues.push({ severity: "warning", issue: "Missing resource-id (UI testing)", element: cls });
1085
- if (bounds) {
1086
- const w = parseInt(bounds[3]) - parseInt(bounds[1]);
1087
- const h = parseInt(bounds[4]) - parseInt(bounds[2]);
1088
- if (w < 48 || h < 48) issues.push({ severity: "important", issue: `Touch target too small: ${w}x${h}dp (min 48x48)`, element: cls, resourceId: rid });
1089
- }
1090
- }
1091
- }
1092
- return JSON.stringify({ scope: scope || "all", elements_scanned: totalScanned, elements_skipped: totalSkipped, total_issues: issues.length, critical: issues.filter(i => i.severity === "critical").length, important: issues.filter(i => i.severity === "important").length, warning: issues.filter(i => i.severity === "warning").length, issues }, null, 2);
1057
+ const xml = existsSync(f) ? readFileSync(f, "utf-8") : "";
1058
+ const r = auditAndroidDump({ xml, scope: args.scope || null });
1059
+ return JSON.stringify({
1060
+ scope: r.scope,
1061
+ measurable: r.measurable,
1062
+ reason: r.reason,
1063
+ elements_scanned: r.elementsScanned,
1064
+ elements_skipped: r.elementsSkipped,
1065
+ total_issues: r.totalIssues,
1066
+ critical: r.critical,
1067
+ important: r.important,
1068
+ warning: r.warning,
1069
+ issues: r.issues,
1070
+ }, null, 2);
1093
1071
  }
1094
1072
  case "android_launch_time": {
1095
1073
  run(`adb ${df} shell am force-stop ${sanitizeId(args.package_name)}`);
@@ -1557,17 +1535,25 @@ const ISSUE_LIST = {
1557
1535
  },
1558
1536
  };
1559
1537
 
1538
+ // The counts are nullable on purpose. An audit that could not read the tree
1539
+ // reports null, never 0: a count of zero is a measurement, and none was taken.
1540
+ // `measurable` is the field to branch on; the counts are only meaningful when
1541
+ // it is true.
1560
1542
  const ACCESSIBILITY_AUDIT_SCHEMA = {
1561
1543
  type: "object",
1562
- required: ["total_issues", "critical", "important", "warning", "issues"],
1544
+ required: ["measurable", "total_issues", "critical", "important", "warning", "issues"],
1563
1545
  properties: {
1564
1546
  scope: { type: "string" },
1547
+ measurable: { type: "boolean" },
1548
+ reason: { type: ["string", "null"] },
1565
1549
  elements_scanned: { type: "integer" },
1566
1550
  elements_skipped: { type: "integer" },
1567
- total_issues: { type: "integer" },
1568
- critical: { type: "integer" },
1569
- important: { type: "integer" },
1570
- warning: { type: "integer" },
1551
+ reading_order_ok: { type: ["boolean", "null"] },
1552
+ hints_present: { type: ["integer", "null"] },
1553
+ total_issues: { type: ["integer", "null"] },
1554
+ critical: { type: ["integer", "null"] },
1555
+ important: { type: ["integer", "null"] },
1556
+ warning: { type: ["integer", "null"] },
1571
1557
  issues: ISSUE_LIST,
1572
1558
  },
1573
1559
  };
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@mmerterden/multi-agent-toolkit-mcp",
3
- "version": "3.3.0",
3
+ "version": "3.5.0",
4
4
  "description": "MCP server for iOS Simulator, Android Emulator and headless web control. 86 tools: device automation (tap/swipe/type), accessibility audits, visual diff, crash logs, App Store / Play Store pre-submission compliance. Runs standalone over stdio with any MCP client.",
5
5
  "type": "module",
6
6
  "main": "index.js",
@@ -10,7 +10,7 @@
10
10
  },
11
11
  "scripts": {
12
12
  "start": "node index.js",
13
- "test": "node --test tools/design-check/__tests__/design-check.test.mjs tools/design-check/__tests__/plan-determinism.test.mjs tools/ios-app-store-audit/__tests__/app-store-audit.test.mjs tools/ios-testflight/__tests__/testflight.test.mjs tools/ui-inspect/__tests__/ui-inspect.test.mjs tools/crash-logs/__tests__/crash-logs.test.mjs tools/launch-time/__tests__/launch-time.test.mjs tools/memory/__tests__/memory.test.mjs tools/offload/__tests__/offload.test.mjs __tests__/server-tools.test.mjs __tests__/injection.test.mjs",
13
+ "test": "node --test tools/design-check/__tests__/design-check.test.mjs tools/design-check/__tests__/plan-determinism.test.mjs tools/ios-app-store-audit/__tests__/app-store-audit.test.mjs tools/ios-testflight/__tests__/testflight.test.mjs tools/ui-inspect/__tests__/ui-inspect.test.mjs tools/crash-logs/__tests__/crash-logs.test.mjs tools/a11y/__tests__/a11y.test.mjs tools/launch-time/__tests__/launch-time.test.mjs tools/memory/__tests__/memory.test.mjs tools/offload/__tests__/offload.test.mjs __tests__/server-tools.test.mjs __tests__/injection.test.mjs",
14
14
  "gates": "bash scripts/gates.sh"
15
15
  },
16
16
  "keywords": [
@@ -0,0 +1,335 @@
1
+ /**
2
+ * a11y - accessibility audit scoring, split out of index.js so it can be tested.
3
+ *
4
+ * The audits used to return a full clean table for a tree they never read:
5
+ * `elements_scanned: 0, total_issues: 0, critical: 0, important: 0, warning: 0`.
6
+ * On iOS the tree comes from Simulator.app's host accessibility bridge, and that
7
+ * bridge stays empty until the Simulator window has been activated at least
8
+ * once, verified on a live simulator: 1 node before activation, 28 after. So
9
+ * "no issues" and "never looked" were the same answer, and the first is the one
10
+ * a reader believes.
11
+ *
12
+ * Every result therefore carries `measurable` and, when false, a `reason`. A
13
+ * result that could not be measured reports `totalIssues: null`, never 0.
14
+ */
15
+
16
+ const IOS_INTERACTIVE_ROLES = new Set([
17
+ "AXButton",
18
+ "AXLink",
19
+ "AXTextField",
20
+ "AXTextArea",
21
+ "AXCheckBox",
22
+ "AXRadioButton",
23
+ "AXSlider",
24
+ "AXPopUpButton",
25
+ "AXMenuButton",
26
+ "AXSwitch",
27
+ ]);
28
+
29
+ const IOS_MIN_TAP_PT = 44;
30
+ const ANDROID_MIN_TAP_DP = 48;
31
+
32
+ /**
33
+ * An accessibility tree with no children is not a screen without controls, it
34
+ * is a tree that was never populated. Callers must not score it.
35
+ *
36
+ * @param {object|null} tree - parsed ui-tree-dumper output
37
+ * @returns {boolean}
38
+ */
39
+ export function isDegenerateTree(tree) {
40
+ if (!tree || typeof tree !== "object") return true;
41
+ if (tree.error) return true;
42
+ const kids = Array.isArray(tree.children) ? tree.children : [];
43
+ return kids.length === 0;
44
+ }
45
+
46
+ /**
47
+ * The device screen inside Simulator.app's window.
48
+ *
49
+ * The dumped window holds the simulator's own chrome as direct children -
50
+ * hardware buttons (Action, Volume Up, Sleep/Wake), the toolbar, the title
51
+ * text - and the iOS app underneath a single large AXGroup. Verified on a live
52
+ * simulator: seven chrome children beside one 402x873 AXGroup with the app in
53
+ * it. Auditing the window whole reports Apple's 17x35pt Volume button as an
54
+ * app tap-target violation, which is a false positive in exactly the place an
55
+ * accessibility report has to be trusted.
56
+ *
57
+ * @param {object} tree
58
+ * @returns {object|null} the screen subtree, or null when it cannot be found
59
+ */
60
+ export function deviceScreenSubtree(tree) {
61
+ if (!tree || tree.role !== "AXWindow") return tree || null;
62
+ const groups = (tree.children || []).filter((c) => c && c.role === "AXGroup");
63
+ if (groups.length === 0) return null;
64
+ const area = (n) => (n.frame?.w || 0) * (n.frame?.h || 0);
65
+ return groups.reduce((best, c) => (area(c) > area(best) ? c : best), groups[0]);
66
+ }
67
+
68
+ /**
69
+ * Flatten the interactive elements in the order a screen reader would visit
70
+ * them, which is the order the accessibility tree presents them in.
71
+ *
72
+ * @param {object} node
73
+ * @param {Set<string>} roles
74
+ * @returns {object[]}
75
+ */
76
+ function readingSequence(node, roles) {
77
+ const out = [];
78
+ const visit = (n) => {
79
+ if (!n || typeof n !== "object") return;
80
+ if (roles.has(n.role)) out.push(n);
81
+ (n.children || []).forEach(visit);
82
+ };
83
+ visit(node);
84
+ return out;
85
+ }
86
+
87
+ /**
88
+ * Group elements into visual rows.
89
+ *
90
+ * A screen is not a single top-to-bottom column. A home screen is a grid, a
91
+ * toolbar is a row, and comparing the reading order against a naive
92
+ * top-to-bottom sort reports every grid as broken. Two elements share a row
93
+ * when their vertical extents overlap by more than half the shorter one.
94
+ *
95
+ * @param {object[]} els
96
+ * @returns {object[][]} rows, top to bottom, each sorted left to right
97
+ */
98
+ export function visualRows(els) {
99
+ const sorted = [...els].sort((a, b) => (a.frame?.y || 0) - (b.frame?.y || 0));
100
+ const rows = [];
101
+ for (const el of sorted) {
102
+ const y = el.frame?.y || 0;
103
+ const h = el.frame?.h || 0;
104
+ const row = rows.find((r) => {
105
+ const ry = r.top;
106
+ const rh = r.height;
107
+ const overlap = Math.min(y + h, ry + rh) - Math.max(y, ry);
108
+ return overlap > Math.min(h, rh) / 2;
109
+ });
110
+ if (row) {
111
+ row.items.push(el);
112
+ row.top = Math.min(row.top, y);
113
+ row.height = Math.max(row.height, h);
114
+ } else {
115
+ rows.push({ top: y, height: h, items: [el] });
116
+ }
117
+ }
118
+ return rows.map((r) => r.items.sort((a, b) => (a.frame?.x || 0) - (b.frame?.x || 0)));
119
+ }
120
+
121
+ /**
122
+ * Does the reading order follow the visual layout?
123
+ *
124
+ * The check is deliberately coarse: it compares the reading sequence against
125
+ * rows-then-columns, and reports the first element that arrives out of place.
126
+ * A screen can carry a deliberate reading order that differs from the visual
127
+ * one, so this is an `important` finding and never a `critical` one: it is a
128
+ * question worth asking, not a proven defect.
129
+ *
130
+ * @param {object[]} sequence - elements in reading order
131
+ * @param {boolean} rtl - right-to-left layout
132
+ * @returns {{ok: boolean, firstOutOfPlace: object|null, expectedIndex: number|null}}
133
+ */
134
+ export function checkReadingOrder(sequence, rtl = false) {
135
+ if (sequence.length < 2) return { ok: true, firstOutOfPlace: null, expectedIndex: null };
136
+ const rows = visualRows(sequence);
137
+ const expected = rows.flatMap((r) => (rtl ? [...r].reverse() : r));
138
+ for (let i = 0; i < sequence.length; i++) {
139
+ if (sequence[i] !== expected[i]) {
140
+ return { ok: false, firstOutOfPlace: sequence[i], expectedIndex: expected.indexOf(sequence[i]) };
141
+ }
142
+ }
143
+ return { ok: true, firstOutOfPlace: null, expectedIndex: null };
144
+ }
145
+
146
+ /**
147
+ * @param {object} params
148
+ * @param {object|null} params.tree - parsed ui-tree-dumper output
149
+ * @param {string|null} [params.scope] - only audit identifiers with this prefix
150
+ * @returns {{measurable: boolean, reason: string|null, scope: string,
151
+ * elementsScanned: number, elementsSkipped: number,
152
+ * totalIssues: number|null, critical: number|null,
153
+ * important: number|null, warning: number|null, issues: object[]}}
154
+ */
155
+ export function auditIosTree({ tree, scope = null, rtl = false }) {
156
+ if (isDegenerateTree(tree)) {
157
+ return unmeasurable(
158
+ scope,
159
+ tree && tree.error
160
+ ? String(tree.error)
161
+ : "the accessibility tree came back empty; open Simulator.app and bring its window to the front, then retry - the host bridge does not populate until the window has been activated once",
162
+ );
163
+ }
164
+
165
+ const screen = deviceScreenSubtree(tree);
166
+ if (!screen) {
167
+ return unmeasurable(
168
+ scope,
169
+ "the window held no device screen group; the simulator may still be starting up",
170
+ );
171
+ }
172
+
173
+ const issues = [];
174
+ let elementsScanned = 0;
175
+ let elementsSkipped = 0;
176
+ const sequence = [];
177
+
178
+ const visit = (node, path = "") => {
179
+ if (!node || typeof node !== "object") return;
180
+ const loc = path ? `${path} > ${node.role}` : node.role;
181
+ const w = node.frame?.w || 0;
182
+ const h = node.frame?.h || 0;
183
+
184
+ if (IOS_INTERACTIVE_ROLES.has(node.role)) {
185
+ if (scope && node.identifier && !node.identifier.startsWith(scope)) {
186
+ elementsSkipped++;
187
+ (node.children || []).forEach((c) => visit(c, loc));
188
+ return;
189
+ }
190
+ elementsScanned++;
191
+ sequence.push(node);
192
+ // What a screen reader announces the element AS. Without it a control is
193
+ // read out as bare text and the user cannot tell it is actionable.
194
+ if (!node.roleDescription) {
195
+ issues.push({ severity: "important", issue: "No role announced (VoiceOver cannot say what this control is)", element: loc, identifier: node.identifier || null });
196
+ }
197
+ if (!node.title && !node.description && !node.value) {
198
+ issues.push({ severity: "critical", issue: "Missing accessibility label", element: loc, identifier: node.identifier || null });
199
+ }
200
+ if (!node.identifier) {
201
+ issues.push({ severity: "warning", issue: "Missing accessibility identifier (UI testing)", element: loc });
202
+ }
203
+ if (w > 0 && h > 0 && (w < IOS_MIN_TAP_PT || h < IOS_MIN_TAP_PT)) {
204
+ issues.push({ severity: "important", issue: `Tap target too small: ${Math.round(w)}x${Math.round(h)}pt (min ${IOS_MIN_TAP_PT}x${IOS_MIN_TAP_PT})`, element: loc, identifier: node.identifier || null });
205
+ }
206
+ }
207
+ (node.children || []).forEach((c) => visit(c, loc));
208
+ };
209
+ visit(screen);
210
+
211
+ // A populated window whose interactive elements were all filtered out is a
212
+ // scope that matched nothing, not a clean screen.
213
+ if (elementsScanned === 0) {
214
+ return unmeasurable(
215
+ scope,
216
+ scope
217
+ ? `no interactive element matched scope "${scope}"`
218
+ : "the tree was readable but held no interactive elements to audit",
219
+ elementsSkipped,
220
+ );
221
+ }
222
+
223
+ const order = checkReadingOrder(sequence, rtl);
224
+ if (!order.ok) {
225
+ const el = order.firstOutOfPlace;
226
+ issues.push({
227
+ severity: "important",
228
+ issue: `Reading order does not follow the visual layout: this element is read at position ${sequence.indexOf(el) + 1} but sits at position ${order.expectedIndex + 1} on screen`,
229
+ element: el.role,
230
+ identifier: el.identifier || null,
231
+ label: el.title || el.description || el.value || null,
232
+ });
233
+ }
234
+
235
+ return {
236
+ measurable: true,
237
+ reason: null,
238
+ scope: scope || "all",
239
+ elementsScanned,
240
+ elementsSkipped,
241
+ readingOrderOk: order.ok,
242
+ hintsPresent: sequence.filter((n) => n.help).length,
243
+ ...tally(issues),
244
+ };
245
+ }
246
+
247
+ /**
248
+ * @param {object} params
249
+ * @param {string} params.xml - uiautomator dump output
250
+ * @param {string|null} [params.scope] - only audit resource-ids containing this
251
+ * @returns {object} same shape as auditIosTree
252
+ */
253
+ export function auditAndroidDump({ xml, scope = null }) {
254
+ const text = typeof xml === "string" ? xml : "";
255
+ if (!/<node\b/.test(text)) {
256
+ return unmeasurable(scope, "the uiautomator dump held no nodes; the dump failed or the screen was not ready");
257
+ }
258
+
259
+ const issues = [];
260
+ let elementsScanned = 0;
261
+ let elementsSkipped = 0;
262
+
263
+ for (const match of text.matchAll(/<node[^>]*>/g)) {
264
+ const node = match[0];
265
+ const attr = (name) => node.match(new RegExp(`${name}="([^"]*)"`))?.[1] || "";
266
+ if (!node.includes('clickable="true"')) continue;
267
+
268
+ const cls = attr("class");
269
+ const desc = attr("content-desc");
270
+ const rid = attr("resource-id");
271
+ const label = attr("text");
272
+
273
+ if (scope && rid && !rid.includes(scope)) {
274
+ elementsSkipped++;
275
+ continue;
276
+ }
277
+ elementsScanned++;
278
+
279
+ if (!desc && !label) issues.push({ severity: "critical", issue: "Missing contentDescription", element: cls, resourceId: rid || null });
280
+ if (!rid) issues.push({ severity: "warning", issue: "Missing resource-id (UI testing)", element: cls });
281
+
282
+ const bounds = node.match(/bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"/);
283
+ if (bounds) {
284
+ const w = parseInt(bounds[3], 10) - parseInt(bounds[1], 10);
285
+ const h = parseInt(bounds[4], 10) - parseInt(bounds[2], 10);
286
+ if (w < ANDROID_MIN_TAP_DP || h < ANDROID_MIN_TAP_DP) {
287
+ issues.push({ severity: "important", issue: `Touch target too small: ${w}x${h}dp (min ${ANDROID_MIN_TAP_DP}x${ANDROID_MIN_TAP_DP})`, element: cls, resourceId: rid || null });
288
+ }
289
+ }
290
+ }
291
+
292
+ if (elementsScanned === 0) {
293
+ return unmeasurable(
294
+ scope,
295
+ scope ? `no clickable element matched scope "${scope}"` : "the dump was readable but held no clickable elements to audit",
296
+ elementsSkipped,
297
+ );
298
+ }
299
+
300
+ return {
301
+ measurable: true,
302
+ reason: null,
303
+ scope: scope || "all",
304
+ elementsScanned,
305
+ elementsSkipped,
306
+ ...tally(issues),
307
+ };
308
+ }
309
+
310
+ function tally(issues) {
311
+ const by = (s) => issues.filter((i) => i.severity === s).length;
312
+ return {
313
+ totalIssues: issues.length,
314
+ critical: by("critical"),
315
+ important: by("important"),
316
+ warning: by("warning"),
317
+ issues,
318
+ };
319
+ }
320
+
321
+ function unmeasurable(scope, reason, elementsSkipped = 0) {
322
+ return {
323
+ measurable: false,
324
+ reason,
325
+ scope: scope || "all",
326
+ elementsScanned: 0,
327
+ elementsSkipped,
328
+ // null rather than 0: a count of zero is a measurement, and none was taken.
329
+ totalIssues: null,
330
+ critical: null,
331
+ important: null,
332
+ warning: null,
333
+ issues: [],
334
+ };
335
+ }
@@ -9,6 +9,12 @@ import Foundation
9
9
 
10
10
  struct AXNode: Codable {
11
11
  let role: String
12
+ // What a screen reader announces the element AS. Probed on a live simulator:
13
+ // AXRoleDescription carries "button" and friends, AXSubrole carries nothing
14
+ // at all through this bridge, so only the former is read.
15
+ let roleDescription: String?
16
+ // The iOS accessibilityHint bridges to AXHelp ("Double tap to open").
17
+ let help: String?
12
18
  let title: String?
13
19
  let value: String?
14
20
  let description: String?
@@ -53,6 +59,8 @@ func dumpElement(_ element: AXUIElement, depth: Int, maxDepth: Int) -> AXNode? {
53
59
  let value = getString(element, kAXValueAttribute)
54
60
  let desc = getString(element, kAXDescriptionAttribute)
55
61
  let identifier = getString(element, kAXIdentifierAttribute)
62
+ let roleDescription = getString(element, "AXRoleDescription")
63
+ let help = getString(element, "AXHelp")
56
64
  let enabled = getBool(element, kAXEnabledAttribute)
57
65
  let focused = getBool(element, kAXFocusedAttribute)
58
66
  let frame = getFrame(element)
@@ -75,6 +83,8 @@ func dumpElement(_ element: AXUIElement, depth: Int, maxDepth: Int) -> AXNode? {
75
83
 
76
84
  return AXNode(
77
85
  role: role,
86
+ roleDescription: roleDescription,
87
+ help: help,
78
88
  title: title,
79
89
  value: value,
80
90
  description: desc,
@@ -86,10 +96,22 @@ func dumpElement(_ element: AXUIElement, depth: Int, maxDepth: Int) -> AXNode? {
86
96
  )
87
97
  }
88
98
 
99
+ // The host accessibility bridge does not populate the iOS app's elements until
100
+ // the Simulator window has been activated at least once. Measured on a live
101
+ // simulator: 1 node before activation, 28 after, and it stays populated once
102
+ // the window has been backgrounded again. Without this the tree comes back
103
+ // holding only the window, which reads as a screen with no controls.
104
+ func activateSimulator(_ app: NSRunningApplication) {
105
+ if app.isActive { return }
106
+ app.activate(options: [])
107
+ Thread.sleep(forTimeInterval: 1.2)
108
+ }
109
+
89
110
  func findSimulatorWindow() -> AXUIElement? {
90
111
  let apps = NSWorkspace.shared.runningApplications
91
112
  for app in apps {
92
113
  if app.bundleIdentifier == "com.apple.iphonesimulator" {
114
+ activateSimulator(app)
93
115
  let axApp = AXUIElementCreateApplication(app.processIdentifier)
94
116
  var windows: AnyObject?
95
117
  AXUIElementCopyAttributeValue(axApp, kAXWindowsAttribute as CFString, &windows)
@@ -105,7 +127,7 @@ func findSimulatorWindow() -> AXUIElement? {
105
127
  let maxDepth = CommandLine.arguments.count > 1 ? Int(CommandLine.arguments[1]) ?? 10 : 10
106
128
 
107
129
  guard let simWindow = findSimulatorWindow() else {
108
- let error = ["error": "Simulator not running or no window found"]
130
+ let error = ["error": "Simulator.app is not running or has no window. Booting a device with simctl is not enough - the accessibility bridge lives in the Simulator UI app. Run: open -a Simulator"]
109
131
  let data = try! JSONSerialization.data(withJSONObject: error)
110
132
  FileHandle.standardOutput.write(data)
111
133
  exit(1)