@mmerterden/multi-agent-toolkit-mcp 3.4.0 → 3.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +72 -0
- package/README.md +3 -3
- package/README.tr.md +2 -2
- package/index.js +36 -3
- package/package.json +2 -2
- package/tools/a11y/index.js +206 -1
- package/ui-tree-dumper.swift +10 -0
package/CHANGELOG.md
CHANGED
|
@@ -15,6 +15,78 @@ Releases before this file exists are recorded in the git tags and commit history
|
|
|
15
15
|
|
|
16
16
|
---
|
|
17
17
|
|
|
18
|
+
## 3.6.0
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
|
|
22
|
+
- **`ios_accessibility_audit_deep`: Apple's own accessibility audit.** Contrast,
|
|
23
|
+
Dynamic Type and clipped text cannot be reached from an accessibility tree
|
|
24
|
+
dump, whatever you do to the dumper. `XCUIAccessibilityAudit` can see them,
|
|
25
|
+
and it runs inside an XCUITest, so this is a separate tool rather than a mode
|
|
26
|
+
on the existing one: an `xcodebuild test` run, minutes rather than seconds,
|
|
27
|
+
needing a test target the tree-dump audit does not.
|
|
28
|
+
|
|
29
|
+
Verified end to end rather than written from the header. A throwaway SwiftUI
|
|
30
|
+
app with three planted defects was generated, built and driven on a live
|
|
31
|
+
simulator, and Apple's audit returned "Contrast failed", "Contrast nearly
|
|
32
|
+
passed" and "Element has no description". The parser was written against that
|
|
33
|
+
output, and the test fixture is those three messages in the exact result-bundle
|
|
34
|
+
shape `xcresulttool` emits.
|
|
35
|
+
|
|
36
|
+
The findings are classified by audit type. Anything the keyword table does not
|
|
37
|
+
recognise keeps its raw text under `unclassified` rather than being forced into
|
|
38
|
+
a bucket it may not belong to.
|
|
39
|
+
|
|
40
|
+
**The boundary, stated because it cannot be closed from here:** a test that
|
|
41
|
+
never calls `performAccessibilityAudit()` passes, and a passing test looks
|
|
42
|
+
exactly like an audit that found nothing. So the result reports whether the
|
|
43
|
+
named test RAN, the tool description says the test must perform the audit
|
|
44
|
+
itself, and `measurable` is false when the test did not run. What the tool
|
|
45
|
+
cannot do is prove the test called the API.
|
|
46
|
+
|
|
47
|
+
### Note
|
|
48
|
+
|
|
49
|
+
Stage 3 of 4. The Android side is last and stays unverifiable here: this machine
|
|
50
|
+
has no adb and no emulator.
|
|
51
|
+
|
|
52
|
+
## 3.5.0
|
|
53
|
+
|
|
54
|
+
### Added
|
|
55
|
+
|
|
56
|
+
- **`ios_accessibility_audit` checks the reading order.** A screen with perfect
|
|
57
|
+
labels is still unusable if VoiceOver walks it in the wrong order, and nothing
|
|
58
|
+
was looking. The audit compares the reading sequence against the visual layout
|
|
59
|
+
and reports the first element that arrives out of place.
|
|
60
|
+
|
|
61
|
+
The comparison is rows-then-columns, not a naive top-to-bottom sort: a home
|
|
62
|
+
screen is a grid, a toolbar is a row, and sorting by `y` alone calls every
|
|
63
|
+
correct grid broken. Elements share a row when their vertical extents overlap
|
|
64
|
+
by more than half the shorter one, which absorbs the few points of jitter real
|
|
65
|
+
layouts carry. `rtl: true` expects right-to-left within a row.
|
|
66
|
+
|
|
67
|
+
It is an `important` finding and never a `critical` one: a deliberate reading
|
|
68
|
+
order that differs from the visual one is legitimate, so this is a question
|
|
69
|
+
worth asking rather than a proven defect. Verified on a live home screen grid:
|
|
70
|
+
13 elements, reading order correct, no false positive.
|
|
71
|
+
|
|
72
|
+
- **Traits and hints.** The dumper now reads `AXRoleDescription` (what a screen
|
|
73
|
+
reader announces the element AS) and `AXHelp` (where the iOS
|
|
74
|
+
`accessibilityHint` bridges to). A control with no role announcement is read
|
|
75
|
+
out as bare text and the user cannot tell it is actionable, so that is an
|
|
76
|
+
`important` finding. A missing hint is not: hints are not required on every
|
|
77
|
+
element, so the result reports `hints_present` as a count and leaves the
|
|
78
|
+
judgement to the reader.
|
|
79
|
+
|
|
80
|
+
`AXSubrole` was probed first and carries nothing at all through this bridge,
|
|
81
|
+
so it is not read. Guessing at it would have produced a check that never fires.
|
|
82
|
+
|
|
83
|
+
### Note
|
|
84
|
+
|
|
85
|
+
Stage 2 of 4. Contrast, Dynamic Type and clipped text still cannot be reached
|
|
86
|
+
from a tree dump; they need Apple's own `XCUIAccessibilityAudit`, which is an
|
|
87
|
+
XCUITest run rather than a simctl call. The Android side is stage 4 and is
|
|
88
|
+
unverifiable on a machine with no adb.
|
|
89
|
+
|
|
18
90
|
## 3.4.0
|
|
19
91
|
|
|
20
92
|
### Fixed
|
package/README.md
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
|
|
8
8
|
🇹🇷 Türkçe: [README.tr.md](./README.tr.md)
|
|
9
9
|
|
|
10
|
-
**
|
|
10
|
+
**87 tools** for iOS Simulator, Android Emulator, and headless web control. MCP server that lets your AI coding assistant see, interact with, and audit your mobile apps - plus drive browsers, run an 18-rule App Store compliance audit, and orchestrate multi-step batch flows.
|
|
11
11
|
|
|
12
12
|
Distributed on the **public npm registry** - `npx @mmerterden/multi-agent-toolkit-mcp` resolves with no auth, no token, no `~/.npmrc` setup.
|
|
13
13
|
|
|
@@ -19,7 +19,7 @@ That distinction is worth keeping straight. This line once called five hosts "th
|
|
|
19
19
|
- **Device Control** (59 tools) - screenshot, tap, swipe, type, navigate, dark mode, locale, orientation, location, permissions, push notifications, real start/stop screen recording
|
|
20
20
|
- **Memory** (2 tools) - `ios_leaks` runs /usr/bin/leaks against a simulator or host process, snapshot or diff against a saved memory graph; `android_meminfo` reads dumpsys meminfo, snapshot or diff. Both report `measurable:false` rather than a clean result when they could not look
|
|
21
21
|
- **Crash Diagnostics** (2 tools) - `ios_list_crashes` reads the host's DiagnosticReports, `android_list_crashes` dumps the adb crash buffer, both tail-bounded
|
|
22
|
-
- **Accessibility Audit** (
|
|
22
|
+
- **Accessibility Audit** (3 tools) - `ios_accessibility_audit` / `android_accessibility_audit` read the live tree for missing labels, controls a screen reader cannot name, small tap targets, missing identifiers and a reading order that does not follow the visual layout. `ios_accessibility_audit_deep` runs Apple's own XCUIAccessibilityAudit through your XCUITest for the things a tree cannot show: contrast, Dynamic Type, clipped text. All three report `measurable:false` with a reason rather than a clean result when they could not look
|
|
23
23
|
- **Store Compliance** (5 tools) - App Store / Play Store readiness; **18-rule deep `ios_app_store_audit`** cross-references Apple ITMS error codes + App Store Review Guidelines (privacy manifest, required-reason API, Info.plist, code signing, entitlements, embedded SDK, IPv6, debug-tool leak, ...)
|
|
24
24
|
- **Web Automation** (8 tools) - Playwright-powered: goto, click (CSS selectors), type, eval JS, wait for selector, extract text, screenshot. Chromium / WebKit / Firefox engines. **Requires `playwright` peer dependency.**
|
|
25
25
|
- **Design Audit** (6 tools) - mock-mode vs Figma conformance: scenario inventory, mock detection, mock launch, live UI geometry, pixel/geometry/typography compare, and the HTML/PDF report with its coverage gate
|
|
@@ -190,7 +190,7 @@ Run a scripted login flow in one MCP call:
|
|
|
190
190
|
|
|
191
191
|
Returns per-step status so the agent can self-correct on failure.
|
|
192
192
|
|
|
193
|
-
### iOS Tools (
|
|
193
|
+
### iOS Tools (40)
|
|
194
194
|
|
|
195
195
|
| Tool | Description |
|
|
196
196
|
|------|-------------|
|
package/README.tr.md
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
|
|
8
8
|
🇬🇧 English: [README.md](./README.md)
|
|
9
9
|
|
|
10
|
-
iOS Simulator, Android Emulator ve headless web kontrolü için **
|
|
10
|
+
iOS Simulator, Android Emulator ve headless web kontrolü için **87 araç**. AI kodlama asistanının mobil uygulamalarını görmesini, onlarla etkileşime girmesini ve denetlemesini sağlayan bir MCP sunucusu - ayrıca tarayıcıları sürer, 18-kurallık bir App Store uyumluluk denetimi çalıştırır ve çok-adımlı batch akışlarını orkestre eder.
|
|
11
11
|
|
|
12
12
|
**Public npm registry** üzerinden dağıtılır - `npx @mmerterden/multi-agent-toolkit-mcp`, auth'suz, token'sız, `~/.npmrc` ayarı gerekmeden çözülür.
|
|
13
13
|
|
|
@@ -189,7 +189,7 @@ Tek bir MCP çağrısında scriptlenmiş bir login akışı çalıştır:
|
|
|
189
189
|
|
|
190
190
|
Agent'ın hata durumunda kendini düzeltebilmesi için adım-başına durum döner.
|
|
191
191
|
|
|
192
|
-
### iOS Araçları (
|
|
192
|
+
### iOS Araçları (40)
|
|
193
193
|
|
|
194
194
|
| Araç | Açıklama |
|
|
195
195
|
|------|-------------|
|
package/index.js
CHANGED
|
@@ -30,7 +30,7 @@ import {
|
|
|
30
30
|
import { DESIGN_TOOLS, handleDesign } from "./tools/design-check/index.js";
|
|
31
31
|
import { parseLaunchOutput } from "./tools/launch-time/index.js";
|
|
32
32
|
import { parseLeaksOutput, parseMeminfoOutput, diffMeminfo } from "./tools/memory/index.js";
|
|
33
|
-
import { auditIosTree, auditAndroidDump } from "./tools/a11y/index.js";
|
|
33
|
+
import { auditIosTree, auditAndroidDump, parseAuditResults } from "./tools/a11y/index.js";
|
|
34
34
|
import { interactiveElements } from "./tools/ui-inspect/index.js";
|
|
35
35
|
import { selectCrashReports } from "./tools/crash-logs/index.js";
|
|
36
36
|
import {
|
|
@@ -351,7 +351,7 @@ const IOS_TOOLS = [
|
|
|
351
351
|
{ name: "ios_get_app_container", description: "Get iOS app container path", inputSchema: { type: "object", properties: { bundle_id: { type: "string" }, container: { type: "string", enum: ["app", "data", "groups"] }, device_id: { type: "string" } }, required: ["bundle_id"] } },
|
|
352
352
|
{ name: "ios_erase_device", description: "Factory reset iOS simulator", inputSchema: { type: "object", properties: { device_id: { type: "string" } } } },
|
|
353
353
|
{ name: "ios_get_ui_tree", description: "Get iOS accessibility UI tree via macOS AX APIs. Pass `path` to write the raw dump to a file and return only its location - the full tree can be tens of KB.", inputSchema: { type: "object", properties: { max_depth: { type: "number" }, path: { type: "string", description: "Absolute file path to write the raw tree JSON to. The parent directory must already exist." } } } },
|
|
354
|
-
{ name: "ios_accessibility_audit", description: "Audit iOS app accessibility: missing labels,
|
|
354
|
+
{ name: "ios_accessibility_audit", description: "Audit iOS app accessibility on the booted simulator: missing labels, controls a screen reader cannot name, tap targets under 44pt, missing identifiers, and whether the reading order follows the visual layout. Reports measurable:false with a reason rather than a clean result when the tree could not be read - booting a device is not enough, Simulator.app must be running. Scopes to the device screen, never the simulator own chrome.", inputSchema: { type: "object", properties: { max_depth: { type: "number" }, scope: { type: "string", description: "Filter: only audit elements whose identifier starts with this prefix (e.g. 'login_', 'settings_'). Omit to audit all." }, rtl: { type: "boolean", description: "Expect right-to-left reading within a row. Default false." } } } },
|
|
355
355
|
{ name: "ios_biometric", description: "Simulate Face ID / Touch ID on iOS simulator (match or nomatch)", inputSchema: { type: "object", properties: { match: { type: "boolean", description: "true=success, false=failure" }, device_id: { type: "string" } }, required: ["match"] } },
|
|
356
356
|
{ name: "ios_archive_audit", description: "DEPRECATED - use ios_app_store_audit (18-rule deep scan). Lighter 6-check audit kept for backward compatibility; will be removed in the next major.", inputSchema: { type: "object", properties: { archive_path: { type: "string", description: "Path to .xcarchive" } }, required: ["archive_path"] } },
|
|
357
357
|
{ name: "ios_export_ipa", description: "Export a .xcarchive to a signed .ipa via xcodebuild -exportArchive. Generates the exportOptions.plist from the arguments (method defaults to app-store-connect), so callers do not have to hand-maintain one. Returns the .ipa path plus parsed errors; a run that exits 0 without producing an .ipa is reported as a failure. Pair with ios_testflight_validate for the pre-submission gate.", inputSchema: { type: "object", properties: { archive_path: { type: "string", description: "Absolute path to the .xcarchive" }, output_dir: { type: "string", description: "Directory to write the .ipa into" }, method: { type: "string", description: "Export method: app-store-connect (default) | release-testing | enterprise | development" }, team_id: { type: "string", description: "Apple Developer team ID" }, provisioning_profiles: { type: "object", description: "Map of bundleId -> provisioning profile name (manual signing)" }, signing_style: { type: "string", description: "automatic | manual" }, upload_symbols: { type: "boolean", description: "Include symbols (default true)" }, allow_provisioning_updates: { type: "boolean", description: "Off by default. Lets xcodebuild register devices and create/modify provisioning profiles in the developer account - a change on Apple's side, so it is opt-in" }, timeout_sec: { type: "number", description: "Default 900" } }, required: ["archive_path", "output_dir"] } },
|
|
@@ -361,6 +361,7 @@ const IOS_TOOLS = [
|
|
|
361
361
|
{ name: "ios_xcresult", description: "Drill into a previous ios_xcodebuild result by xcresult ID. Modes: summary (counts), errors (file:line + message), warnings, log (last N lines), tests (failed), metrics (XCTMetric performance results as JSON). Use this instead of dumping the whole build log into context.", inputSchema: { type: "object", properties: { id: { type: "string", description: "xcresult ID returned by ios_xcodebuild" }, mode: { type: "string", enum: ["summary", "errors", "warnings", "log", "tests", "metrics"], description: "Default: summary" }, log_lines: { type: "number", description: "Lines of raw log to return when mode=log (default 200)" }, test_id: { type: "string", description: "mode=metrics only: scope to one test case or suite instead of every measured test" } }, required: ["id"] } },
|
|
362
362
|
{ name: "ios_visual_diff", description: "Compare two PNG screenshots. Returns JSON with diff_pct, pass/fail vs threshold, and an optional diff image. Use for snapshot regression checks across light/dark, locale, dynamic type variants.", inputSchema: { type: "object", properties: { baseline: { type: "string", description: "Path to baseline PNG" }, current: { type: "string", description: "Path to current PNG" }, threshold: { type: "number", description: "Per-pixel color threshold 0..1 (default 0.1, lower = stricter)" }, max_diff_pct: { type: "number", description: "Fail if diff exceeds this percent (default 1.0)" }, output: { type: "string", description: "Path to write diff PNG (optional)" } }, required: ["baseline", "current"] } },
|
|
363
363
|
{ name: "ios_leaks", description: "Look for leaked memory in a running simulator (or host) process with /usr/bin/leaks. mode=snapshot reports the current leak count and bytes; mode=diff reports only leaks new since a saved memory graph, which is the shape a regression gate wants. Reports measurable:false when the target lacks get-task-allow rather than reporting it as clean - leaks exits 0 in that case, so an unmeasurable target and a clean one are indistinguishable by exit status. Debug builds are debuggable; Apple-signed apps are not.", inputSchema: { type: "object", properties: { pid: { type: "number", description: "Process id. Either this or bundle_id." }, bundle_id: { type: "string", description: "Bundle id of an app running on the booted simulator; its pid is resolved for you." }, device_id: { type: "string" }, mode: { type: "string", enum: ["snapshot", "diff"], description: "Default: snapshot" }, baseline_graph: { type: "string", description: "mode=diff: path to the memory graph saved by an earlier call" }, output_graph: { type: "string", description: "Save a memory graph here to use as a later baseline" } }, required: [] } },
|
|
364
|
+
{ name: "ios_accessibility_audit_deep", description: "Run Apple's own accessibility audit (XCUIAccessibilityAudit) through an existing XCUITest and report its findings: contrast, Dynamic Type, clipped text, traits, hit regions, element descriptions. These cannot be reached from an accessibility tree dump, which is why this is a separate tool from ios_accessibility_audit and does not replace it. It is an xcodebuild test run, minutes rather than seconds, and the named test MUST itself call performAccessibilityAudit() - a test that does not is indistinguishable from an audit that found nothing, so the result reports whether the test ran and never invents a clean verdict.", inputSchema: { type: "object", properties: { project: { type: "string", description: "Path to the .xcodeproj. Give this or workspace." }, workspace: { type: "string", description: "Path to the .xcworkspace." }, scheme: { type: "string" }, test_identifier: { type: "string", description: "e.g. AuditTest/testAccessibilityAudit(). Omit to read every test that ran." }, device_id: { type: "string", description: "Simulator udid. Defaults to the booted one." } }, required: ["scheme"] } },
|
|
364
365
|
{ name: "ios_list_crashes", description: "List recent crash reports from the host's ~/Library/Logs/DiagnosticReports - where simulator app crashes land. Filter by process name, bound by age and count.", inputSchema: { type: "object", properties: { app: { type: "string", description: "Only reports whose file name (the crashed process) contains this substring" }, since_min: { type: "number", description: "Only reports newer than this many minutes" }, limit: { type: "number", description: "Max reports returned, newest first (default 20)" } } } },
|
|
365
366
|
];
|
|
366
367
|
|
|
@@ -508,13 +509,15 @@ async function handleIOS(name, args, ctx = {}) {
|
|
|
508
509
|
} catch {
|
|
509
510
|
tree = null;
|
|
510
511
|
}
|
|
511
|
-
const r = auditIosTree({ tree, scope: args.scope || null });
|
|
512
|
+
const r = auditIosTree({ tree, scope: args.scope || null, rtl: args.rtl === true });
|
|
512
513
|
return JSON.stringify({
|
|
513
514
|
scope: r.scope,
|
|
514
515
|
measurable: r.measurable,
|
|
515
516
|
reason: r.reason,
|
|
516
517
|
elements_scanned: r.elementsScanned,
|
|
517
518
|
elements_skipped: r.elementsSkipped,
|
|
519
|
+
reading_order_ok: r.readingOrderOk ?? null,
|
|
520
|
+
hints_present: r.hintsPresent ?? null,
|
|
518
521
|
total_issues: r.totalIssues,
|
|
519
522
|
critical: r.critical,
|
|
520
523
|
important: r.important,
|
|
@@ -836,6 +839,34 @@ async function handleIOS(name, args, ctx = {}) {
|
|
|
836
839
|
graph_saved_to: args.output_graph || null,
|
|
837
840
|
}, null, 2);
|
|
838
841
|
}
|
|
842
|
+
case "ios_accessibility_audit_deep": {
|
|
843
|
+
if (!args.project && !args.workspace) return "ERROR: pass project or workspace";
|
|
844
|
+
const container = args.workspace ? `-workspace ${shq(args.workspace)}` : `-project ${shq(args.project)}`;
|
|
845
|
+
const d = iosDevice(args.device_id);
|
|
846
|
+
const bundle = join(SCREENSHOT_DIR, `a11ydeep_${Date.now()}.xcresult`);
|
|
847
|
+
const only = args.test_identifier ? ` -only-testing:${shq(args.test_identifier)}` : "";
|
|
848
|
+
// xcodebuild exits non-zero when the audit finds anything, because each
|
|
849
|
+
// finding is an XCTest failure. That is a result, not a failure to run.
|
|
850
|
+
runCapture(
|
|
851
|
+
`xcodebuild test ${container} -scheme ${shq(args.scheme)} -destination ${shq(`platform=iOS Simulator,id=${d}`)} -resultBundlePath ${shq(bundle)}${only} 2>&1`,
|
|
852
|
+
{ timeout: 900000 },
|
|
853
|
+
);
|
|
854
|
+
if (!existsSync(bundle)) {
|
|
855
|
+
return JSON.stringify({ measurable: false, reason: "xcodebuild produced no result bundle; the build failed before any test ran", test_ran: false, findings: [] }, null, 2);
|
|
856
|
+
}
|
|
857
|
+
const json = runCapture(`xcrun xcresulttool get test-results tests --path ${shq(bundle)} --format json 2>&1`, { timeout: 60000 });
|
|
858
|
+
const r = parseAuditResults(json, args.test_identifier || null);
|
|
859
|
+
return JSON.stringify({
|
|
860
|
+
measurable: r.measurable,
|
|
861
|
+
reason: r.reason,
|
|
862
|
+
test_ran: r.testRan,
|
|
863
|
+
test_result: r.testResult,
|
|
864
|
+
by_type: r.byType,
|
|
865
|
+
total_findings: r.measurable ? r.findings.length : null,
|
|
866
|
+
findings: r.findings,
|
|
867
|
+
result_bundle: bundle,
|
|
868
|
+
}, null, 2);
|
|
869
|
+
}
|
|
839
870
|
case "ios_list_crashes": {
|
|
840
871
|
const dir = join(homedir(), "Library", "Logs", "DiagnosticReports");
|
|
841
872
|
if (!existsSync(dir)) return JSON.stringify({ dir, count: 0, reports: [] }, null, 2);
|
|
@@ -1546,6 +1577,8 @@ const ACCESSIBILITY_AUDIT_SCHEMA = {
|
|
|
1546
1577
|
reason: { type: ["string", "null"] },
|
|
1547
1578
|
elements_scanned: { type: "integer" },
|
|
1548
1579
|
elements_skipped: { type: "integer" },
|
|
1580
|
+
reading_order_ok: { type: ["boolean", "null"] },
|
|
1581
|
+
hints_present: { type: ["integer", "null"] },
|
|
1549
1582
|
total_issues: { type: ["integer", "null"] },
|
|
1550
1583
|
critical: { type: ["integer", "null"] },
|
|
1551
1584
|
important: { type: ["integer", "null"] },
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mmerterden/multi-agent-toolkit-mcp",
|
|
3
|
-
"version": "3.
|
|
4
|
-
"description": "MCP server for iOS Simulator, Android Emulator and headless web control.
|
|
3
|
+
"version": "3.6.0",
|
|
4
|
+
"description": "MCP server for iOS Simulator, Android Emulator and headless web control. 87 tools: device automation (tap/swipe/type), accessibility audits, visual diff, crash logs, App Store / Play Store pre-submission compliance. Runs standalone over stdio with any MCP client.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "index.js",
|
|
7
7
|
"bin": {
|
package/tools/a11y/index.js
CHANGED
|
@@ -65,6 +65,84 @@ export function deviceScreenSubtree(tree) {
|
|
|
65
65
|
return groups.reduce((best, c) => (area(c) > area(best) ? c : best), groups[0]);
|
|
66
66
|
}
|
|
67
67
|
|
|
68
|
+
/**
|
|
69
|
+
* Flatten the interactive elements in the order a screen reader would visit
|
|
70
|
+
* them, which is the order the accessibility tree presents them in.
|
|
71
|
+
*
|
|
72
|
+
* @param {object} node
|
|
73
|
+
* @param {Set<string>} roles
|
|
74
|
+
* @returns {object[]}
|
|
75
|
+
*/
|
|
76
|
+
function readingSequence(node, roles) {
|
|
77
|
+
const out = [];
|
|
78
|
+
const visit = (n) => {
|
|
79
|
+
if (!n || typeof n !== "object") return;
|
|
80
|
+
if (roles.has(n.role)) out.push(n);
|
|
81
|
+
(n.children || []).forEach(visit);
|
|
82
|
+
};
|
|
83
|
+
visit(node);
|
|
84
|
+
return out;
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
/**
|
|
88
|
+
* Group elements into visual rows.
|
|
89
|
+
*
|
|
90
|
+
* A screen is not a single top-to-bottom column. A home screen is a grid, a
|
|
91
|
+
* toolbar is a row, and comparing the reading order against a naive
|
|
92
|
+
* top-to-bottom sort reports every grid as broken. Two elements share a row
|
|
93
|
+
* when their vertical extents overlap by more than half the shorter one.
|
|
94
|
+
*
|
|
95
|
+
* @param {object[]} els
|
|
96
|
+
* @returns {object[][]} rows, top to bottom, each sorted left to right
|
|
97
|
+
*/
|
|
98
|
+
export function visualRows(els) {
|
|
99
|
+
const sorted = [...els].sort((a, b) => (a.frame?.y || 0) - (b.frame?.y || 0));
|
|
100
|
+
const rows = [];
|
|
101
|
+
for (const el of sorted) {
|
|
102
|
+
const y = el.frame?.y || 0;
|
|
103
|
+
const h = el.frame?.h || 0;
|
|
104
|
+
const row = rows.find((r) => {
|
|
105
|
+
const ry = r.top;
|
|
106
|
+
const rh = r.height;
|
|
107
|
+
const overlap = Math.min(y + h, ry + rh) - Math.max(y, ry);
|
|
108
|
+
return overlap > Math.min(h, rh) / 2;
|
|
109
|
+
});
|
|
110
|
+
if (row) {
|
|
111
|
+
row.items.push(el);
|
|
112
|
+
row.top = Math.min(row.top, y);
|
|
113
|
+
row.height = Math.max(row.height, h);
|
|
114
|
+
} else {
|
|
115
|
+
rows.push({ top: y, height: h, items: [el] });
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
return rows.map((r) => r.items.sort((a, b) => (a.frame?.x || 0) - (b.frame?.x || 0)));
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
/**
|
|
122
|
+
* Does the reading order follow the visual layout?
|
|
123
|
+
*
|
|
124
|
+
* The check is deliberately coarse: it compares the reading sequence against
|
|
125
|
+
* rows-then-columns, and reports the first element that arrives out of place.
|
|
126
|
+
* A screen can carry a deliberate reading order that differs from the visual
|
|
127
|
+
* one, so this is an `important` finding and never a `critical` one: it is a
|
|
128
|
+
* question worth asking, not a proven defect.
|
|
129
|
+
*
|
|
130
|
+
* @param {object[]} sequence - elements in reading order
|
|
131
|
+
* @param {boolean} rtl - right-to-left layout
|
|
132
|
+
* @returns {{ok: boolean, firstOutOfPlace: object|null, expectedIndex: number|null}}
|
|
133
|
+
*/
|
|
134
|
+
export function checkReadingOrder(sequence, rtl = false) {
|
|
135
|
+
if (sequence.length < 2) return { ok: true, firstOutOfPlace: null, expectedIndex: null };
|
|
136
|
+
const rows = visualRows(sequence);
|
|
137
|
+
const expected = rows.flatMap((r) => (rtl ? [...r].reverse() : r));
|
|
138
|
+
for (let i = 0; i < sequence.length; i++) {
|
|
139
|
+
if (sequence[i] !== expected[i]) {
|
|
140
|
+
return { ok: false, firstOutOfPlace: sequence[i], expectedIndex: expected.indexOf(sequence[i]) };
|
|
141
|
+
}
|
|
142
|
+
}
|
|
143
|
+
return { ok: true, firstOutOfPlace: null, expectedIndex: null };
|
|
144
|
+
}
|
|
145
|
+
|
|
68
146
|
/**
|
|
69
147
|
* @param {object} params
|
|
70
148
|
* @param {object|null} params.tree - parsed ui-tree-dumper output
|
|
@@ -74,7 +152,7 @@ export function deviceScreenSubtree(tree) {
|
|
|
74
152
|
* totalIssues: number|null, critical: number|null,
|
|
75
153
|
* important: number|null, warning: number|null, issues: object[]}}
|
|
76
154
|
*/
|
|
77
|
-
export function auditIosTree({ tree, scope = null }) {
|
|
155
|
+
export function auditIosTree({ tree, scope = null, rtl = false }) {
|
|
78
156
|
if (isDegenerateTree(tree)) {
|
|
79
157
|
return unmeasurable(
|
|
80
158
|
scope,
|
|
@@ -95,6 +173,7 @@ export function auditIosTree({ tree, scope = null }) {
|
|
|
95
173
|
const issues = [];
|
|
96
174
|
let elementsScanned = 0;
|
|
97
175
|
let elementsSkipped = 0;
|
|
176
|
+
const sequence = [];
|
|
98
177
|
|
|
99
178
|
const visit = (node, path = "") => {
|
|
100
179
|
if (!node || typeof node !== "object") return;
|
|
@@ -109,6 +188,12 @@ export function auditIosTree({ tree, scope = null }) {
|
|
|
109
188
|
return;
|
|
110
189
|
}
|
|
111
190
|
elementsScanned++;
|
|
191
|
+
sequence.push(node);
|
|
192
|
+
// What a screen reader announces the element AS. Without it a control is
|
|
193
|
+
// read out as bare text and the user cannot tell it is actionable.
|
|
194
|
+
if (!node.roleDescription) {
|
|
195
|
+
issues.push({ severity: "important", issue: "No role announced (VoiceOver cannot say what this control is)", element: loc, identifier: node.identifier || null });
|
|
196
|
+
}
|
|
112
197
|
if (!node.title && !node.description && !node.value) {
|
|
113
198
|
issues.push({ severity: "critical", issue: "Missing accessibility label", element: loc, identifier: node.identifier || null });
|
|
114
199
|
}
|
|
@@ -135,12 +220,26 @@ export function auditIosTree({ tree, scope = null }) {
|
|
|
135
220
|
);
|
|
136
221
|
}
|
|
137
222
|
|
|
223
|
+
const order = checkReadingOrder(sequence, rtl);
|
|
224
|
+
if (!order.ok) {
|
|
225
|
+
const el = order.firstOutOfPlace;
|
|
226
|
+
issues.push({
|
|
227
|
+
severity: "important",
|
|
228
|
+
issue: `Reading order does not follow the visual layout: this element is read at position ${sequence.indexOf(el) + 1} but sits at position ${order.expectedIndex + 1} on screen`,
|
|
229
|
+
element: el.role,
|
|
230
|
+
identifier: el.identifier || null,
|
|
231
|
+
label: el.title || el.description || el.value || null,
|
|
232
|
+
});
|
|
233
|
+
}
|
|
234
|
+
|
|
138
235
|
return {
|
|
139
236
|
measurable: true,
|
|
140
237
|
reason: null,
|
|
141
238
|
scope: scope || "all",
|
|
142
239
|
elementsScanned,
|
|
143
240
|
elementsSkipped,
|
|
241
|
+
readingOrderOk: order.ok,
|
|
242
|
+
hintsPresent: sequence.filter((n) => n.help).length,
|
|
144
243
|
...tally(issues),
|
|
145
244
|
};
|
|
146
245
|
}
|
|
@@ -234,3 +333,109 @@ function unmeasurable(scope, reason, elementsSkipped = 0) {
|
|
|
234
333
|
issues: [],
|
|
235
334
|
};
|
|
236
335
|
}
|
|
336
|
+
|
|
337
|
+
// Apple's audit reports its findings as XCTest failure messages. These three
|
|
338
|
+
// were captured from a real run against a deliberately inaccessible app; the
|
|
339
|
+
// rest are classified by keyword and anything unrecognised keeps its raw text
|
|
340
|
+
// rather than being forced into a bucket it may not belong to.
|
|
341
|
+
const AUDIT_TYPE_PATTERNS = [
|
|
342
|
+
[/contrast/i, "contrast"],
|
|
343
|
+
[/has no description|description is|element description/i, "sufficientElementDescription"],
|
|
344
|
+
[/clipped|truncat/i, "textClipped"],
|
|
345
|
+
[/dynamic type|text size/i, "dynamicType"],
|
|
346
|
+
[/hit region|touch target|tap target/i, "hitRegion"],
|
|
347
|
+
[/trait/i, "trait"],
|
|
348
|
+
[/element detection|not detected/i, "elementDetection"],
|
|
349
|
+
[/parent|child/i, "parentChild"],
|
|
350
|
+
[/action/i, "action"],
|
|
351
|
+
];
|
|
352
|
+
|
|
353
|
+
/**
|
|
354
|
+
* Classify one audit failure message.
|
|
355
|
+
*
|
|
356
|
+
* @param {string} message
|
|
357
|
+
* @returns {string} audit type, or "unclassified"
|
|
358
|
+
*/
|
|
359
|
+
export function classifyAuditMessage(message) {
|
|
360
|
+
for (const [re, type] of AUDIT_TYPE_PATTERNS) if (re.test(message)) return type;
|
|
361
|
+
return "unclassified";
|
|
362
|
+
}
|
|
363
|
+
|
|
364
|
+
/**
|
|
365
|
+
* Pull accessibility audit findings out of `xcresulttool get test-results tests`.
|
|
366
|
+
*
|
|
367
|
+
* Shape captured from a real run: testNodes nest plan > bundle > suite > case,
|
|
368
|
+
* and a failing case carries children of nodeType "Failure Message" whose name
|
|
369
|
+
* is "<file>:<line>: <message>".
|
|
370
|
+
*
|
|
371
|
+
* The honest boundary, stated because it cannot be closed from here: a test that
|
|
372
|
+
* never calls performAccessibilityAudit passes, and a passing test is
|
|
373
|
+
* indistinguishable from an audit that found nothing. So the result reports
|
|
374
|
+
* whether the named test RAN, and the caller is told that the test must perform
|
|
375
|
+
* the audit itself. `measurable` is false when the test did not run at all.
|
|
376
|
+
*
|
|
377
|
+
* @param {object|string} results - parsed or raw JSON from xcresulttool
|
|
378
|
+
* @param {string|null} [testIdentifier] - e.g. "AuditTest/testAccessibilityAudit()"
|
|
379
|
+
* @returns {{measurable: boolean, reason: string|null, testRan: boolean,
|
|
380
|
+
* testResult: string|null, findings: object[], byType: object}}
|
|
381
|
+
*/
|
|
382
|
+
export function parseAuditResults(results, testIdentifier = null) {
|
|
383
|
+
let doc = results;
|
|
384
|
+
if (typeof doc === "string") {
|
|
385
|
+
try {
|
|
386
|
+
doc = JSON.parse(doc);
|
|
387
|
+
} catch {
|
|
388
|
+
return { measurable: false, reason: "xcresulttool output was not JSON", testRan: false, testResult: null, findings: [], byType: {} };
|
|
389
|
+
}
|
|
390
|
+
}
|
|
391
|
+
if (!doc || !Array.isArray(doc.testNodes)) {
|
|
392
|
+
return { measurable: false, reason: "no testNodes in the result bundle; the build or the test run failed", testRan: false, testResult: null, findings: [], byType: {} };
|
|
393
|
+
}
|
|
394
|
+
|
|
395
|
+
const cases = [];
|
|
396
|
+
const walk = (n) => {
|
|
397
|
+
if (!n || typeof n !== "object") return;
|
|
398
|
+
if (n.nodeType === "Test Case") cases.push(n);
|
|
399
|
+
(n.children || []).forEach(walk);
|
|
400
|
+
};
|
|
401
|
+
doc.testNodes.forEach(walk);
|
|
402
|
+
|
|
403
|
+
const matched = testIdentifier
|
|
404
|
+
? cases.filter((c) => c.nodeIdentifier === testIdentifier || c.name === testIdentifier)
|
|
405
|
+
: cases;
|
|
406
|
+
|
|
407
|
+
if (matched.length === 0) {
|
|
408
|
+
return {
|
|
409
|
+
measurable: false,
|
|
410
|
+
reason: testIdentifier
|
|
411
|
+
? `no test case named "${testIdentifier}" ran; check the identifier and that the target built`
|
|
412
|
+
: "no test case ran",
|
|
413
|
+
testRan: false,
|
|
414
|
+
testResult: null,
|
|
415
|
+
findings: [],
|
|
416
|
+
byType: {},
|
|
417
|
+
};
|
|
418
|
+
}
|
|
419
|
+
|
|
420
|
+
const findings = [];
|
|
421
|
+
for (const c of matched) {
|
|
422
|
+
for (const child of c.children || []) {
|
|
423
|
+
if (child.nodeType !== "Failure Message") continue;
|
|
424
|
+
const raw = String(child.name || "");
|
|
425
|
+
const message = raw.replace(/^.*?:\d+:\s*/, "").trim();
|
|
426
|
+
findings.push({ test: c.nodeIdentifier || c.name || null, type: classifyAuditMessage(message), message, raw });
|
|
427
|
+
}
|
|
428
|
+
}
|
|
429
|
+
|
|
430
|
+
const byType = {};
|
|
431
|
+
for (const f of findings) byType[f.type] = (byType[f.type] || 0) + 1;
|
|
432
|
+
|
|
433
|
+
return {
|
|
434
|
+
measurable: true,
|
|
435
|
+
reason: null,
|
|
436
|
+
testRan: true,
|
|
437
|
+
testResult: matched[0].result || null,
|
|
438
|
+
findings,
|
|
439
|
+
byType,
|
|
440
|
+
};
|
|
441
|
+
}
|
package/ui-tree-dumper.swift
CHANGED
|
@@ -9,6 +9,12 @@ import Foundation
|
|
|
9
9
|
|
|
10
10
|
struct AXNode: Codable {
|
|
11
11
|
let role: String
|
|
12
|
+
// What a screen reader announces the element AS. Probed on a live simulator:
|
|
13
|
+
// AXRoleDescription carries "button" and friends, AXSubrole carries nothing
|
|
14
|
+
// at all through this bridge, so only the former is read.
|
|
15
|
+
let roleDescription: String?
|
|
16
|
+
// The iOS accessibilityHint bridges to AXHelp ("Double tap to open").
|
|
17
|
+
let help: String?
|
|
12
18
|
let title: String?
|
|
13
19
|
let value: String?
|
|
14
20
|
let description: String?
|
|
@@ -53,6 +59,8 @@ func dumpElement(_ element: AXUIElement, depth: Int, maxDepth: Int) -> AXNode? {
|
|
|
53
59
|
let value = getString(element, kAXValueAttribute)
|
|
54
60
|
let desc = getString(element, kAXDescriptionAttribute)
|
|
55
61
|
let identifier = getString(element, kAXIdentifierAttribute)
|
|
62
|
+
let roleDescription = getString(element, "AXRoleDescription")
|
|
63
|
+
let help = getString(element, "AXHelp")
|
|
56
64
|
let enabled = getBool(element, kAXEnabledAttribute)
|
|
57
65
|
let focused = getBool(element, kAXFocusedAttribute)
|
|
58
66
|
let frame = getFrame(element)
|
|
@@ -75,6 +83,8 @@ func dumpElement(_ element: AXUIElement, depth: Int, maxDepth: Int) -> AXNode? {
|
|
|
75
83
|
|
|
76
84
|
return AXNode(
|
|
77
85
|
role: role,
|
|
86
|
+
roleDescription: roleDescription,
|
|
87
|
+
help: help,
|
|
78
88
|
title: title,
|
|
79
89
|
value: value,
|
|
80
90
|
description: desc,
|