mobile-debug-mcp 0.26.1 → 0.26.3

package/docs/rfcs/006-runtime-action-instrumentation-and-binding-layer.md

@@ -0,0 +1,230 @@
+# RFC 006 — Runtime Action Instrumentation & Binding Layer
+
+## 1. Summary
+
+This RFC defines how the execution model in RFC 005 is mapped onto the current runtime behaviour of the system.
+
+It does not assume a new instrumentation system exists. Instead, it describes how lifecycle semantics are derived from existing execution flows, logs, module behaviour, and lightweight runtime metadata attached to action envelopes.
+
+It specifies:
+- how existing `action_type` values are interpreted under RFC 005 semantics
+- how lifecycle states are inferred from current runtime execution
+- how `src/server` and `src/interact` currently participate in execution
+- how legacy and platform actions are incorporated into the model
+
+This RFC is a runtime binding and normalisation layer over existing implementation behaviour.
+
+---
+
+## 2. Problem Statement
+
+RFC 005 defines a unified execution lifecycle:
+- Resolved
+- Dispatched
+- Pending Verification
+- Verified
+- Failed
+
+However, the current system already contains:
+- a concrete `action_type` execution model
+- execution logic split across `src/server` and `src/interact`
+- platform-specific actions (tap_element, type_text, press_back, start_app, restart_app, scroll_to_element)
+- distributed logging and partial instrumentation within modules
+
+There is no central instrumentation system and no explicit lifecycle emitter.
+Instead, lifecycle meaning is inferred from runtime behaviour and the `lifecycle_state` / `source_module` fields now attached to action envelopes.
+
+This results in:
+- implicit execution state transitions
+- distributed observability signals
+- non-uniform traceability across actions
+
+---
+
+## 3. Design Goals
+
+This layer MUST:
+
+- Map existing runtime behaviour to RFC 005 lifecycle semantics
+- Use existing `action_type` values as the authoritative execution taxonomy
+- Derive lifecycle states from observable runtime transitions
+- Reflect actual module responsibilities (not idealised separation)
+- Work with existing logging and execution hooks
+- Preserve compatibility with all current action implementations
+
+---
+
+## 4. Runtime Execution Flow (Observed)
+
+Current observed execution flow:
+
+UI Request
+→ src/server (routing + validation)
+→ src/interact (execution + platform dispatch)
+→ platform layer
+→ response handling + logs
+→ optional state verification (where available)
+
+Lifecycle states are derived from this flow rather than explicitly emitted.
+
+---
+
+## 5. Action Type Mapping (Current Runtime)
+
+This RFC maps existing `action_type` values to RFC 005 semantics.
+
+| action_type | RFC 005 Semantic Interpretation |
+|------------|---------------------------------|
+| tap | Selection |
+| tap_element | Selection |
+| type_text | Input |
+| press_back | Navigation |
+| start_app | System Action |
+| restart_app | System Action |
+| scroll_to_element | Navigation |
+
+This table reflects the current runtime contract.
+
+---
+
+## 6. Lifecycle State Derivation
+
+Lifecycle states are NOT explicitly emitted. They are inferred as follows:
+
+### 6.1 Resolved
+Inferred when:
+- src/server accepts request
+- action is validated and normalized
+- action_id is assigned (or equivalent identifier exists)
+
+---
+
+### 6.2 Dispatched
+Inferred when:
+- control passes from src/server to src/interact
+- execution call is issued to platform layer
+
+---
+
+### 6.3 Pending Verification
+Inferred when:
+- platform execution returns a result
+- before any UI/state evaluation occurs
+
+---
+
+### 6.4 Verified / Failed
+Inferred when:
+- post-execution evaluation is performed (if available)
+
+Rules:
+- Verified = expected outcome observed in UI/state/log signals
+- Failed = timeout, error, or mismatch in expected outcome
+
+Where no formal verification exists, outcome is derived from best available signals (logs, UI diff, or absence of error).
+
+---
+
+## 7. Instrumentation Reality
+
+There is no central instrumentation layer in the current system.
+
+Instead:
+- src/server emits partial logs during routing and validation
+- src/interact emits execution logs and platform responses
+- platform adapters may emit additional debugging information
+- action envelopes now carry lightweight lifecycle metadata for post-dispatch state and source ownership
+
+Lifecycle traceability is therefore assembled from distributed signals rather than a unified event system.
+
+---
+
+## 8. Module Responsibilities (Observed Behaviour)
+
+### src/server
+- receives action requests
+- performs validation and normalization
+- assigns identifiers where applicable
+- routes actions to src/interact
+- emits partial logs for request lifecycle
+
+---
+
+### src/interact
+- executes platform-specific actions
+- handles retries and fallback behaviours
+- emits execution logs
+- returns execution results
+- may perform lightweight post-processing
+
+---
+
+## 9. Verification Reality
+
+Verification is not a uniform system-wide layer.
+
+It may occur via:
+- UI state comparison (where available)
+- log-based confirmation
+- absence of error signals
+- platform feedback
+
+Verification outcomes are best-effort only where no formal verifier exists, and deterministic where reliable state signals or explicit evaluation paths are available.
+
+---
+
+## 10. Legacy and Special Actions
+
+Actions such as:
+- scroll_to_element
+- start_app
+- restart_app
+- press_back
+
+are fully supported in the runtime.
+
+These actions:
+- may bypass full lifecycle observability
+- may not have explicit verification paths
+- are interpreted using best-effort semantic mapping
+
+---
+
+## 11. Observability Model
+
+Observability is currently distributed across:
+- src/server logs
+- src/interact logs
+- platform debug output
+- action envelope metadata
+
+There is no unified event schema.
+
+Lifecycle reconstruction requires correlation of:
+- action_type
+- timestamps
+- execution boundaries
+- error signals
+
+---
+
+## 12. Relationship to RFC 005
+
+RFC 005 defines the ideal execution lifecycle semantics.
+
+RFC 006 defines how those semantics are interpreted from the existing runtime system.
+
+Together:
+- RFC 005 = conceptual correctness model
+- RFC 006 = runtime behavioural mapping layer
+
+---
+
+## 13. Summary
+
+This RFC ensures:
+- lifecycle semantics can be derived from current runtime behaviour
+- existing action_type contract is preserved as source of truth
+- no assumption of new instrumentation infrastructure is required
+- real module responsibilities are accurately represented
+- observability is understood as distributed rather than centralised

package/docs/rfcs/007-actionability-resolution-and-executable-target-selection.md

@@ -0,0 +1,277 @@
+# RFC 007 — Actionability Resolution and Executable Target Selection
+
+## 1. Summary
+
+This RFC defines how the system resolves which discovered UI element should receive an action before dispatch.
+
+It addresses ambiguity between:
+- visible elements vs actionable elements
+- leaf nodes vs clickable containers
+- semantic targets vs coordinate fallbacks
+- multiple candidate targets with uncertain executability
+
+Goal:
+Improve first-attempt action correctness by resolving the best executable target prior to action dispatch.
+
+This RFC defines the `Resolved` stage semantics referenced in RFC 005 and operationalized by RFC 006.
+It is grounded in the existing element-resolution flow and extends current resolution behavior rather than assuming a wholly new resolver architecture.
+
+---
+
+## 2. Problem Statement
+
+Current interaction failures often arise before execution.
+
+The agent may discover the intended UI concept, but not the correct executable target.
+
+Examples:
+- tapping label text instead of clickable container
+- sliders not surfacing semantic handles
+- generic Compose containers hiding true affordances
+- multiple matching targets without ranking logic
+
+Observed failure modes:
+- false taps
+- submit ambiguity
+- coordinate guessing
+- retry loops
+- brittle fallback behavior
+
+This is a target-resolution problem, not an execution problem.
+
+---
+
+## 3. Design Goals
+
+Resolution MUST:
+- Prefer executable targets over merely visible matches
+- Reduce ambiguous target selection
+- Support confidence-based ranking
+- Build on existing runtime resolution surfaces before introducing new resolution metadata
+- Use structural and semantic resolution signals
+- Minimize coordinate fallback usage
+- Integrate with verification expectations from RFC 005
+
+---
+
+## 4. Actionability Model
+
+Candidate targets are evaluated using actionability signals.
+
+### Structural signals
+- clickable
+- enabled
+- focusable
+- bounds
+- parent action ownership
+
+### Semantic signals
+- control role
+- label association
+- affordance hints
+- selectable or adjustable semantics
+
+### Interaction signals
+- reliable target patterns
+- control-specific heuristics
+- gesture compatibility
+
+---
+
+## 4.1 Current Runtime Resolution Surfaces
+
+This RFC builds on current runtime resolution paths, including:
+- `findElementHandler` for candidate discovery
+- `_resolveActionableAncestor` for executable ancestor promotion
+- `tapElementHandler` for resolved element dispatch
+- `scrollToElementHandler` for scroll-mediated target acquisition
+
+These existing handlers are the current implementation substrate for the Resolved stage.
+This RFC extends and systematizes those behaviors; it does not assume replacement of those paths.
+
+---
+
+## 5. Target Candidate Ranking
+
+When multiple targets match, candidates are ranked.
+
+Illustrative confidence model:
+
+resolution_confidence =
+ interactability_score
+ + semantic_match_score
+ + structural_reliability_score
+
+Highest-confidence executable target is preferred.
+
+The confidence model is illustrative and normative only at the rule-precedence level; implementations may use simpler heuristics while preserving resolution ordering guarantees. Any scoring mechanism is implementation-defined and may not be externally surfaced.
+
+---
+
+## 6. Resolution Rules
+
+### Rule A — Prefer actionable containers over passive leaf nodes
+
+Prefer:
+- clickable container
+
+Over:
+- passive child text nodes
+
+Example:
+Prefer button container over "Generate Session" label node.
+
+---
+
+### Rule B — Prefer semantic controls over coordinate fallbacks
+
+Use semantic control targets whenever possible.
+
+Coordinate fallback only when:
+- no semantic target exists
+- adjustable control semantics absent
+- fallback confidence acceptable
+
+---
+
+### Rule C — Prefer explicit affordance ownership
+
+If child and parent differ:
+prefer the node owning the action handler.
+
+---
+
+## 7. Ambiguity Handling
+
+When multiple plausible targets remain:
+
+System SHOULD:
+- rank candidates
+- expose confidence
+- preserve alternates for fallback reasoning
+
+Low-confidence targets may trigger:
+- guarded execution
+- alternate resolution attempt
+- explicit recovery path
+
+---
+
+## 8. Adjustable Control Resolution
+
+Special handling for:
+- sliders
+- steppers
+- drag controls
+
+Support:
+- adjustable-role recognition
+- control-bound discovery
+- value-aware interaction targeting
+
+This RFC defines target resolution.
+Value-setting behavior remains governed by Adjustable Control Support.
+
+---
+
+## 9. Compose / Custom Control Resolution
+
+Support derived actionability for:
+- merged Compose semantics
+- composite controls
+- inferred interaction contracts
+
+This RFC depends on and strengthens Better Compose / Custom Control Semantics.
+
+---
+
+## 10. Resolution Output Model (Current + Future Extension)
+
+This model is non-normative and represents a progressive enrichment direction rather than a required runtime contract.
+
+Resolution may evolve toward the following enriched output shape. Current runtime implementations may expose only resolved-target output plus limited supporting metadata.
+
+At minimum, current implementations are expected to produce a resolved target. Confidence, alternates, fallback metadata, and reason codes may be introduced incrementally.
+
+Illustrative future-complete shape:
+
+{
+  "resolved_target": "...",
+  "confidence": 0.92,
+  "fallback_available": true,
+  "resolution_reason": "clickable_parent_preferred"
+}
+
+---
+
+## 11. Verification Integration
+
+Resolution is incomplete without verification expectations.
+
+Resolved output should be derived directly from the existing element-resolution flow before adding richer metadata layers.
+
+Resolved target should carry expected post-action signal.
+
+Examples:
+- navigation transition expected
+- menu expected
+- control value change expected
+
+This feeds RFC 005 verification.
+
+---
+
+## 12. Success Metrics
+
+Track:
+- reduced false-tap failures
+- lower retarget retries
+- higher first-attempt action success
+- reduced coordinate fallback usage
+- improved custom control interaction success
+
+---
+
+## 13. Dependencies
+
+Depends on:
+- Stronger State Verification
+- Richer Element Identity
+- Wait and Synchronization Reliability
+
+Strengthens:
+- Adjustable Control Support
+- Better Compose / Custom Control Semantics
+
+---
+
+## 14. Relationship to Other RFCs
+
+RFC 005
+Defines what Resolved means in lifecycle semantics.
+
+RFC 006
+Defines how runtime interprets action execution.
+
+RFC 007
+Defines how a target becomes Resolved.
+Specifically, it formalizes the current discovery → actionable ancestor resolution → dispatch preparation flow already present in runtime handlers.
+
+Together:
+- RFC 005 — action correctness
+- RFC 006 — runtime execution binding
+- RFC 007 — executable target resolution
+
+---
+
+## 15. Summary
+
+This RFC reduces failures caused by acting on the wrong thing, even when the right thing was discovered.
+
+It improves:
+- action precision
+- control reliability
+- Compose interaction robustness
+- agent success with fewer retries
+
+It addresses one of the largest remaining sources of interaction brittleness.

package/docs/specs/mcp-tooling-spec-v1.md

@@ -69,6 +69,8 @@ MUST be returned in this structure:
   action_id: string,
   timestamp: string,
   action_type: string,
+  lifecycle_state?: 'pending_verification' | 'failed',
+  source_module?: 'server' | 'interact',
   target: {
     selector: object,
     resolved: object | null
@@ -87,6 +89,8 @@ Rules:
 
 - `success` is at the top level, not nested
 - `target` contains only selection and resolution context
+- `lifecycle_state` reflects the post-dispatch runtime state
+- `source_module` identifies where the envelope was produced
 - fingerprints represent observed pre/post UI state on a best-effort basis
 - `failure_code` is optional but MUST be used when a structured mapping exists
 

package/docs/tools/interact.md

@@ -36,6 +36,8 @@ Example response:
   "action_id": "tap_1710000000000_1",
   "timestamp": "2026-04-23T08:00:00.000Z",
   "action_type": "tap",
+  "lifecycle_state": "pending_verification",
+  "source_module": "server",
   "target": { "selector": { "x": 100, "y": 200 }, "resolved": null },
   "success": true,
   "ui_fingerprint_before": "fp_before",
@@ -197,7 +199,14 @@ Output:
     "telemetry": { "matchedIndex": 3, "matchedInteractable": true }
   },
   "score": 1.0,
-  "confidence": 1.0
+  "confidence": 1.0,
+  "resolution": {
+    "confidence": 1.0,
+    "reason": "exact_text_match",
+    "fallback_available": false,
+    "matched_count": 1,
+    "alternates": []
+  }
 }
 ```
 
@@ -205,6 +214,7 @@ Notes:
 
 - Best used when no precise selector is available yet.
 - `tapCoordinates` are suitable for `tap` calls.
+- `resolution` explains why the element was selected and may include fallback alternates when the runtime had to promote a parent or nearby control.
 - Prefer `wait_for_ui` when you already know a deterministic selector and want a stable `elementId`.
 
 ---
@@ -333,6 +343,8 @@ Success response:
   "action_id": "tap_element_1710000000000_1",
   "timestamp": "2026-04-23T08:00:00.000Z",
   "action_type": "tap_element",
+  "lifecycle_state": "pending_verification",
+  "source_module": "interact",
   "target": {
     "selector": { "elementId": "el_123" },
     "resolved": {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mobile-debug-mcp",
-  "version": "0.26.1",
+  "version": "0.26.3",
   "description": "MCP server for mobile app debugging (Android + iOS), with focus on security and reliability",
   "type": "module",
   "bin": {