screenforge 0.6.0__tar.gz → 0.7.0__tar.gz

{screenforge-0.6.0 → screenforge-0.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: screenforge
-Version: 0.6.0
+Version: 0.7.0
 Summary: AI-driven cross-platform UI automation engine with test script generation
 License: MIT
 Project-URL: Homepage, https://github.com/jhinzzz/ScreenForge
@@ -51,11 +51,13 @@ Dynamic: license-file
 
 **[中文](https://github.com/jhinzzz/ScreenForge/blob/main/README_CN.md)** | English
 
-> Describe what to test. Watch it happen. Get a pytest script.
+> **Your AI agent runs the test. You keep the pytest file.**
+>
+> Describe the flow in plain language. Watch the agent click through a live site. Walk away with a self-healing pytest script that survives UI changes — and runs in CI. _(Drives real iOS + Android devices too, not just Chrome.)_
 
 ScreenForge is an AI-driven UI automation engine that turns natural language into executable test scripts. Unlike record-and-replay tools, you don't perform the actions yourself — the AI does it for you.
 
-![ScreenForge Demo](https://raw.githubusercontent.com/jhinzzz/ScreenForge/main/docs/assets/demo.gif)
+![ScreenForge Live Mirror — the generated pytest grows beside the live page as the agent runs](https://raw.githubusercontent.com/jhinzzz/ScreenForge/main/docs/assets/hero.gif)
 
 ## Why ScreenForge?
 
@@ -65,6 +67,7 @@ ScreenForge is an AI-driven UI automation engine that turns natural language int
 | Generates replayable test scripts? | Yes | No | No | **Yes (pytest)** |
 | Self-healing when UI changes? | No | No | No | **Yes** |
 | Works as AI Agent tool (MCP)? | No | Yes | No | **Yes** |
+| Drives real iOS / Android devices? | No | No | No | **Yes** |
 
 **Core architecture**: Your AI Agent is the brain (understands requirements, makes decisions). ScreenForge is the hands (executes UI actions, generates code).
 
@@ -107,13 +110,16 @@ Each step: **inspect → decide → act → verify**. The AI decides, ScreenForg
 
 ## Features
 
-- **Cross-platform**: Android (uiautomator2), iOS (wda), Web (Playwright)
-- **Self-healing engine**: When tests break due to UI changes, the engine auto-repairs locators with confidence scoring and AST validation
+- **Keep a real pytest file**: Every run emits a replayable `pytest` script (Allure-instrumented) you can drop straight into CI — not a black-box agent run.
+- **Self-healing engine**: When the UI changes and a locator breaks, the engine auto-repairs it with confidence scoring and AST validation, so your committed test survives.
+- **The agent does the clicking**: You (or your AI agent) describe intent; ScreenForge inspects the DOM, acts, and verifies — you never record by hand.
+- **Real devices, not just browsers**: Drives Android (uiautomator2) and iOS (wda) physical devices over the same protocol — the one thing pure-web agents (Playwright MCP, Browser Use) can't do.
 - **L1/L2 semantic cache**: Same page + same instruction = instant response, no LLM call needed
 - **Visual fallback**: When DOM can't locate elements (Canvas, games), VLM parses screenshots
 - **MCP server**: Any MCP-compatible Agent can drive ScreenForge natively
 - **Structured output**: JSON Lines events + `report/runs/<id>/` artifacts for CI integration
 - **Live Mirror playground**: Watch the generated pytest code grow line-by-line beside a live screenshot as the test runs — `screenforge --playground`. See the [Playground Guide](https://github.com/jhinzzz/ScreenForge/blob/main/docs/playground-guide.md)
+- **Execution-replay review report**: Run any test with `REVIEW_RECORD=1` and get a self-contained, shareable offline `report.html` — a drag-timeline of each step's screenshot + the test source line that triggered it + the DOM tree, plus a `video.gif` filmstrip. Default off, zero overhead. (Web verified; mobile registry/recording seam in place but unverified.)
 
 ## Agent Integration (Claude Code / Cursor / Codex)
 
@@ -206,6 +212,8 @@ If ScreenForge generates tests for your project, add this badge to your README:
 | [MCP Setup (3 min)](https://github.com/jhinzzz/ScreenForge/blob/main/docs/mcp-setup.md) | Connect to Claude Desktop / Cursor / Cline / Claude Code |
 | [Agent Guide](https://github.com/jhinzzz/ScreenForge/blob/main/docs/agent_guide.md) | Integration protocol for AI Agents |
 | [Capability Matrix](https://github.com/jhinzzz/ScreenForge/blob/main/docs/capability-matrix.md) | Supported platforms, actions, and locators |
+| [Architecture Deep-Dive](https://github.com/jhinzzz/ScreenForge/blob/main/docs/architecture-deep-dive.md) | Brain/hands split, semantic cache, self-heal AST gates, hygiene-as-feature |
+| [Examples](https://github.com/jhinzzz/ScreenForge/tree/main/docs/examples) | Real committed workflows + the green pytest they generated |
 | [Playground Guide](https://github.com/jhinzzz/ScreenForge/blob/main/docs/playground-guide.md) | Live Mirror — watch code + screenshots grow as the test runs |
 | [Workflow Examples](https://github.com/jhinzzz/ScreenForge/tree/main/docs/workflows) | YAML workflow templates |
 | [CHANGELOG](https://github.com/jhinzzz/ScreenForge/blob/main/CHANGELOG.md) | Version history |

{screenforge-0.6.0 → screenforge-0.7.0}/README.md

@@ -6,11 +6,13 @@
 
 **[中文](https://github.com/jhinzzz/ScreenForge/blob/main/README_CN.md)** | English
 
-> Describe what to test. Watch it happen. Get a pytest script.
+> **Your AI agent runs the test. You keep the pytest file.**
+>
+> Describe the flow in plain language. Watch the agent click through a live site. Walk away with a self-healing pytest script that survives UI changes — and runs in CI. _(Drives real iOS + Android devices too, not just Chrome.)_
 
 ScreenForge is an AI-driven UI automation engine that turns natural language into executable test scripts. Unlike record-and-replay tools, you don't perform the actions yourself — the AI does it for you.
 
-![ScreenForge Demo](https://raw.githubusercontent.com/jhinzzz/ScreenForge/main/docs/assets/demo.gif)
+![ScreenForge Live Mirror — the generated pytest grows beside the live page as the agent runs](https://raw.githubusercontent.com/jhinzzz/ScreenForge/main/docs/assets/hero.gif)
 
 ## Why ScreenForge?
 
@@ -20,6 +22,7 @@ ScreenForge is an AI-driven UI automation engine that turns natural language int
 | Generates replayable test scripts? | Yes | No | No | **Yes (pytest)** |
 | Self-healing when UI changes? | No | No | No | **Yes** |
 | Works as AI Agent tool (MCP)? | No | Yes | No | **Yes** |
+| Drives real iOS / Android devices? | No | No | No | **Yes** |
 
 **Core architecture**: Your AI Agent is the brain (understands requirements, makes decisions). ScreenForge is the hands (executes UI actions, generates code).
 
@@ -62,13 +65,16 @@ Each step: **inspect → decide → act → verify**. The AI decides, ScreenForg
 
 ## Features
 
-- **Cross-platform**: Android (uiautomator2), iOS (wda), Web (Playwright)
-- **Self-healing engine**: When tests break due to UI changes, the engine auto-repairs locators with confidence scoring and AST validation
+- **Keep a real pytest file**: Every run emits a replayable `pytest` script (Allure-instrumented) you can drop straight into CI — not a black-box agent run.
+- **Self-healing engine**: When the UI changes and a locator breaks, the engine auto-repairs it with confidence scoring and AST validation, so your committed test survives.
+- **The agent does the clicking**: You (or your AI agent) describe intent; ScreenForge inspects the DOM, acts, and verifies — you never record by hand.
+- **Real devices, not just browsers**: Drives Android (uiautomator2) and iOS (wda) physical devices over the same protocol — the one thing pure-web agents (Playwright MCP, Browser Use) can't do.
 - **L1/L2 semantic cache**: Same page + same instruction = instant response, no LLM call needed
 - **Visual fallback**: When DOM can't locate elements (Canvas, games), VLM parses screenshots
 - **MCP server**: Any MCP-compatible Agent can drive ScreenForge natively
 - **Structured output**: JSON Lines events + `report/runs/<id>/` artifacts for CI integration
 - **Live Mirror playground**: Watch the generated pytest code grow line-by-line beside a live screenshot as the test runs — `screenforge --playground`. See the [Playground Guide](https://github.com/jhinzzz/ScreenForge/blob/main/docs/playground-guide.md)
+- **Execution-replay review report**: Run any test with `REVIEW_RECORD=1` and get a self-contained, shareable offline `report.html` — a drag-timeline of each step's screenshot + the test source line that triggered it + the DOM tree, plus a `video.gif` filmstrip. Default off, zero overhead. (Web verified; mobile registry/recording seam in place but unverified.)
 
 ## Agent Integration (Claude Code / Cursor / Codex)
 
@@ -161,6 +167,8 @@ If ScreenForge generates tests for your project, add this badge to your README:
 | [MCP Setup (3 min)](https://github.com/jhinzzz/ScreenForge/blob/main/docs/mcp-setup.md) | Connect to Claude Desktop / Cursor / Cline / Claude Code |
 | [Agent Guide](https://github.com/jhinzzz/ScreenForge/blob/main/docs/agent_guide.md) | Integration protocol for AI Agents |
 | [Capability Matrix](https://github.com/jhinzzz/ScreenForge/blob/main/docs/capability-matrix.md) | Supported platforms, actions, and locators |
+| [Architecture Deep-Dive](https://github.com/jhinzzz/ScreenForge/blob/main/docs/architecture-deep-dive.md) | Brain/hands split, semantic cache, self-heal AST gates, hygiene-as-feature |
+| [Examples](https://github.com/jhinzzz/ScreenForge/tree/main/docs/examples) | Real committed workflows + the green pytest they generated |
 | [Playground Guide](https://github.com/jhinzzz/ScreenForge/blob/main/docs/playground-guide.md) | Live Mirror — watch code + screenshots grow as the test runs |
 | [Workflow Examples](https://github.com/jhinzzz/ScreenForge/tree/main/docs/workflows) | YAML workflow templates |
 | [CHANGELOG](https://github.com/jhinzzz/ScreenForge/blob/main/CHANGELOG.md) | Version history |

screenforge-0.7.0/cli/_version.py

@@ -0,0 +1 @@
+__version__ = "0.7.0"

{screenforge-0.6.0 → screenforge-0.7.0}/cli/dispatch.py

@@ -57,6 +57,7 @@ def _dispatch_execution(
             args,
             output_script_path,
             resume_context,
+            shared_adapter_manager=shared_adapter_manager,
         )
     if args.action and execution_mode == MODE_PLAN_ONLY:
         return run_action_plan_only_mode(

{screenforge-0.6.0 → screenforge-0.7.0}/cli/doctor.py

@@ -373,10 +373,6 @@ def _build_doctor_summary(checks: list[dict]) -> dict:
     }
 
 
-def _build_doctor_remediation_items(checks: list[dict]) -> list[str]:
-    return _build_doctor_summary(checks).get("top_items", [])
-
-
 def _build_doctor_check_failure_message(check: dict) -> str:
     details = []
 

{screenforge-0.6.0 → screenforge-0.7.0}/cli/modes/action.py

@@ -75,6 +75,30 @@ def build_failure_payload(
     return payload
 
 
+def build_success_payload(
+    *,
+    action_name: str,
+    platform: str,
+    ui_tree: dict,
+    current_url: str,
+    output_script: str,
+) -> dict:
+    """Assemble the --action --json success payload.
+
+    Single source of truth shared by the shell --json stdout write and the MCP
+    observation stash, so the two can never drift. Pure: no I/O, no device.
+    """
+    return {
+        "ok": True,
+        "action": action_name,
+        "platform": platform,
+        "ui_tree": ui_tree,
+        "element_count": len(ui_tree.get("ui_elements", []) or []),
+        "output_script": output_script,
+        "current_url": current_url,
+    }
+
+
 def run_action_default_mode(
     args,
     output_script_path: str,
@@ -88,6 +112,7 @@ def run_action_default_mode(
     exit_code = 1
     final_status = "failed"
     final_error = ""
+    final_error_code = ""
     steps_executed = 0
     _emit_run_started(reporter, args, output_script_path, MODE_RUN)
     _apply_resume_summary(reporter, resume_context)
@@ -155,6 +180,9 @@ def run_action_default_mode(
         result = executor.execute_and_record(action_data)
         if not result.get("success"):
             assertion_failed = bool(result.get("assertion_failed"))
+            # Record the executor's code (E0xx) so finalize writes it into
+            # summary.json — assertion verdicts carry none, so this stays "".
+            final_error_code = result.get("error_code", "") or ""
             if assertion_failed:
                 final_error = f"Assertion failed: {action_data['name']}"
             else:
@@ -166,7 +194,7 @@ def run_action_default_mode(
                 action_description=action_data["name"],
             )
             log.error(f"❌ [Action] Failed: {action_data['name']}")
-            if json_mode:
+            if json_mode or shared_adapter_manager is not None:
                 # Distinguish a verification verdict (assertion_failed=true: the
                 # SUT did not meet the assertion — do NOT retry) from a real
                 # engine error. Only engine errors get the did-you-mean
@@ -200,9 +228,14 @@ def run_action_default_mode(
                     ui_tree=ui_tree,
                     current_url=page_url,
                 )
-                json.dump(payload, sys.stdout, ensure_ascii=False)
-                sys.stdout.write("\n")
-                sys.stdout.flush()
+                # Stash for the MCP handler (manager present); write stdout for
+                # shell (--json). MCP keeps stdout clean: stashes but never writes.
+                if shared_adapter_manager is not None:
+                    shared_adapter_manager.set_last_observation(payload)
+                if json_mode:
+                    json.dump(payload, sys.stdout, ensure_ascii=False)
+                    sys.stdout.write("\n")
+                    sys.stdout.flush()
             return 1
 
         history_manager.add_step(result["code_lines"], result["action_description"])
@@ -235,24 +268,28 @@ def run_action_default_mode(
         final_status = "success"
         exit_code = 0
 
-        if json_mode:
+        if json_mode or shared_adapter_manager is not None:
             _ensure_ui_compressors()
             ui_json, _ = _capture_ui_state(args, adapter, reporter, 1)
             try:
                 ui_tree = json.loads(ui_json)
             except (json.JSONDecodeError, TypeError):
                 ui_tree = {}
-            json.dump({
-                "ok": True,
-                "action": action_data["name"],
-                "platform": args.platform,
-                "ui_tree": ui_tree,
-                "element_count": len(ui_tree.get("ui_elements", [])),
-                "output_script": output_script_path,
-                "current_url": current_url(adapter, args.platform),
-            }, sys.stdout, ensure_ascii=False)
-            sys.stdout.write("\n")
-            sys.stdout.flush()
+            payload = build_success_payload(
+                action_name=action_data["name"],
+                platform=args.platform,
+                ui_tree=ui_tree,
+                current_url=current_url(adapter, args.platform),
+                output_script=output_script_path,
+            )
+            # Stash for the MCP handler (manager present); write stdout for shell
+            # (--json). MCP keeps stdout clean: it stashes but never writes.
+            if shared_adapter_manager is not None:
+                shared_adapter_manager.set_last_observation(payload)
+            if json_mode:
+                json.dump(payload, sys.stdout, ensure_ascii=False)
+                sys.stdout.write("\n")
+                sys.stdout.flush()
 
         return 0
     except Exception as e:
@@ -266,6 +303,7 @@ def run_action_default_mode(
             exit_code=exit_code,
             steps_executed=steps_executed,
             last_error=final_error,
+            error_code=final_error_code,
         )
         log.info(f"🏁 Done. Generated script saved to: {output_script_path}")
         if owns_adapter and adapter:

{screenforge-0.6.0 → screenforge-0.7.0}/cli/modes/default.py

@@ -30,6 +30,7 @@ def run_default_mode(
     exit_code = 1
     final_status = "failed"
     final_error = ""
+    final_error_code = ""
     steps_executed = 0
     _emit_run_started(reporter, args, output_script_path, MODE_RUN)
     _apply_resume_summary(reporter, resume_context)
@@ -89,6 +90,9 @@ def run_default_mode(
             status = decision_data.get("status")
             action_data = decision_data.get("result", {})
             last_error = ""
+            # Reset per iteration so a recovered-then-succeeded run finalizes with
+            # error_code="" — only THIS step's failure (set below) should persist.
+            final_error_code = ""
             reporter.emit_event(
                 "decision_received",
                 step=step_count,
@@ -121,6 +125,7 @@ def run_default_mode(
                         )
                     else:
                         final_error = "Final action/assertion failed — task verification did not pass"
+                        final_error_code = result.get("error_code", "") or ""
                         reporter.emit_event(
                             "action_executed",
                             step=step_count,
@@ -193,6 +198,9 @@ def run_default_mode(
                         )
                     else:
                         consecutive_failures += 1
+                        # Retain the executor's code (E0xx) so if the circuit
+                        # breaker trips below, finalize records the real last code.
+                        final_error_code = result.get("error_code", "") or ""
                         action_repr = f"{action_data.get('action')} - {action_data.get('locator_type')}={action_data.get('locator_value')}"
                         last_error = f"Action [{action_repr}] failed — element not found or not interactable on the current page."
                         reporter.emit_event(
@@ -239,6 +247,7 @@ def run_default_mode(
             exit_code=exit_code,
             steps_executed=steps_executed,
             last_error=final_error,
+            error_code=final_error_code,
         )
         log.info(f"🏁 Done. Generated script saved to: {output_script_path}")
         if adapter:

{screenforge-0.6.0 → screenforge-0.7.0}/cli/modes/workflow.py

@@ -1,5 +1,6 @@
 """Workflow execution modes."""
 
+import json
 from pathlib import Path
 
 import cli.shared as _shared
@@ -11,10 +12,14 @@ from cli.reporter import (
     _emit_run_started,
 )
 from cli.shared import (
+    _capture_ui_state,
     _connect_adapter,
     _ensure_executor_runtime,
     _ensure_history_manager,
+    _ensure_ui_compressors,
     _ensure_workflow_loader,
+    _SharedAdapterManager,
+    current_url,
     get_initial_header,
     log,
     save_to_disk,
@@ -242,12 +247,14 @@ def run_workflow_default_mode(
     args,
     output_script_path: str,
     resume_context: dict,
+    shared_adapter_manager: _SharedAdapterManager | None = None,
 ) -> int:
     adapter = None
     reporter = _build_reporter(args, output_script_path, MODE_RUN)
     exit_code = 1
     final_status = "failed"
     final_error = ""
+    final_error_code = ""
     steps_executed = 0
     _emit_run_started(reporter, args, output_script_path, MODE_RUN)
     _apply_resume_summary(reporter, resume_context)
@@ -297,6 +304,7 @@ def run_workflow_default_mode(
 
             steps_executed = index
             action_data = _workflow_step_to_action_data(step, index)
+            last_action_name = action_data["name"]
             reporter.emit_event(
                 "step_started",
                 step=index,
@@ -306,6 +314,9 @@ def run_workflow_default_mode(
             result = executor.execute_and_record(action_data)
             if not result.get("success"):
                 final_error = f"Workflow step failed: {action_data['name']}"
+                # Record the failing step's code (E0xx) for summary.json; assertion
+                # verdicts carry none, so this stays "".
+                final_error_code = result.get("error_code", "") or ""
                 reporter.emit_event(
                     "action_executed",
                     step=index,
@@ -313,6 +324,36 @@ def run_workflow_default_mode(
                     action_description=action_data["name"],
                 )
                 log.error(f"❌ [Workflow] Step failed: {action_data['name']}")
+                # ★ MCP parity: stash the FAILING step's observation (single-point
+                # failure) with its index/name so the agent knows where the batch
+                # stopped. Gated on a manager (MCP/session path) — bare shell pays
+                # nothing. NEVER a per-step array (see design doc, decision §).
+                if shared_adapter_manager is not None:
+                    # Local import dodges an action↔workflow cycle (both import cli.shared).
+                    from cli.modes.action import build_failure_payload
+                    assertion_failed = bool(result.get("assertion_failed"))
+                    page_url = current_url(adapter, args.platform)
+                    fail_ui_tree = {}
+                    if not assertion_failed:
+                        _ensure_ui_compressors()
+                        try:
+                            ui_json, _ = _capture_ui_state(args, adapter, reporter, index)
+                            fail_ui_tree = json.loads(ui_json)
+                        except Exception as e:
+                            log.warning(f"⚠️ [Workflow] Failed to capture post-failure UI state: {e}")
+                            fail_ui_tree = {}
+                    observation = build_failure_payload(
+                        action_name=action_data["name"],
+                        platform=args.platform,
+                        assertion_failed=assertion_failed,
+                        error_code=result.get("error_code", "") or "",
+                        locator_value=action_data.get("locator_value", "") or "",
+                        ui_tree=fail_ui_tree,
+                        current_url=page_url,
+                    )
+                    observation["failed_step_index"] = index
+                    observation["failed_step_name"] = action_data["name"]
+                    shared_adapter_manager.set_last_observation(observation)
                 return 1
 
             history_manager.add_step(
@@ -344,6 +385,30 @@ def run_workflow_default_mode(
             )
         )
         reporter.update_control_summary(executed_steps=executed_steps)
+        # ★ MCP parity: stash ONE live observation — the FINAL step's end-state,
+        # marked with executed_steps so the agent knows it's the end, not step 1.
+        # Gated on a manager; bare shell workflow pays nothing. NEVER per-step.
+        if shared_adapter_manager is not None and executed_steps:
+            # Local import dodges an action↔workflow cycle (both import cli.shared).
+            from cli.modes.action import build_success_payload
+            _ensure_ui_compressors()
+            # steps_executed (raw loop index) is used ONLY for screenshot artifact
+            # naming under --vision; the captured ui_json content is independent of it.
+            try:
+                ui_json, _ = _capture_ui_state(args, adapter, reporter, steps_executed)
+                final_ui_tree = json.loads(ui_json)
+            except Exception as e:
+                log.warning(f"⚠️ [Workflow] Failed to capture final UI state: {e}")
+                final_ui_tree = {}
+            observation = build_success_payload(
+                action_name=last_action_name,
+                platform=args.platform,
+                ui_tree=final_ui_tree,
+                current_url=current_url(adapter, args.platform),
+                output_script=output_script_path,
+            )
+            observation["executed_steps"] = executed_steps
+            shared_adapter_manager.set_last_observation(observation)
         final_status = "success"
         exit_code = 0
         return 0
@@ -360,6 +425,7 @@ def run_workflow_default_mode(
             exit_code=exit_code,
             steps_executed=steps_executed,
             last_error=final_error,
+            error_code=final_error_code,
         )
         log.info(f"🏁 Done. Generated script saved to: {output_script_path}")
         if adapter:

{screenforge-0.6.0 → screenforge-0.7.0}/cli/session.py

@@ -18,10 +18,6 @@ def _session_file(session_id: str) -> str:
     return os.path.join(_SESSION_DIR, f"{session_id}.json")
 
 
-def session_exists(session_id: str) -> bool:
-    return os.path.exists(_session_file(session_id))
-
-
 def load_session(session_id: str) -> dict | None:
     path = _session_file(session_id)
     if not os.path.exists(path):
@@ -125,7 +121,7 @@ def stop_session_recording(session_id: str) -> str:
         pgid = os.getpgid(pid)
         os.killpg(pgid, signal.SIGINT)
         os.waitpid(pid, 0)
-    except (OSError, ChildProcessError):
+    except OSError:
         try:
             os.kill(pid, signal.SIGINT)
             time.sleep(2)

{screenforge-0.6.0 → screenforge-0.7.0}/cli/shared.py

@@ -3,12 +3,6 @@
 import base64
 import time
 
-from common.capabilities import (
-    ACTIONS_REQUIRING_EXTRA_VALUE,
-    GLOBAL_ACTIONS,
-    SUPPORTED_ACTIONS,
-)
-
 
 class _LazyProxy:
     def __init__(self, loader):
@@ -62,9 +56,6 @@ load_workflow_file = None
 WorkflowLoadError = None
 parse_workflow_var_overrides = None
 resolve_workflow_definition = None
-SUPPORTED_INLINE_ACTIONS = SUPPORTED_ACTIONS
-GLOBAL_INLINE_ACTIONS = GLOBAL_ACTIONS
-INLINE_ACTIONS_REQUIRING_EXTRA_VALUE = ACTIONS_REQUIRING_EXTRA_VALUE
 
 
 def _slug_to_test_name(label: str) -> str:
@@ -382,6 +373,18 @@ class _SharedAdapterManager:
         # an inspect_ui and a follow-up `ref @N` action in the same MCP session
         # share state via get_executor() — without any process-global.
         self._executors: dict[str, object] = {}
+        # ONE live post-action observation, stashed by the execute mode
+        # (--action / --workflow) when this manager is present (MCP/session
+        # path only), read once by the MCP handler after dispatch. Cleared on
+        # read so a later execute can't serve a stale prior step's observation.
+        self._last_observation: dict | None = None
+
+    def set_last_observation(self, payload: dict) -> None:
+        self._last_observation = payload
+
+    def take_last_observation(self) -> dict | None:
+        observation, self._last_observation = self._last_observation, None
+        return observation
 
     def get_or_create(self, platform: str, env: str = "dev"):
         if platform in self._adapters:

{screenforge-0.6.0 → screenforge-0.7.0}/cli/tool_protocol_handlers.py

@@ -30,6 +30,31 @@ from common.tool_protocol import (
     load_tool_request_from_stdin,
 )
 
+# Agent-facing fields projected from the live execute observation onto the MCP
+# response — the exact superset of --action --json success + engine_error shapes,
+# plus the single-observation workflow markers. Curated allowlist so it never
+# clobbers the response envelope (ok/operation/mode/exit_code/run_dir/...).
+_OBSERVATION_FIELDS = (
+    "ui_tree",
+    "element_count",
+    "current_url",
+    "output_script",
+    "executed_steps",
+    "result",
+    "assertion_failed",
+    "error_code",
+    "message",
+    "fix",
+    "candidates",
+    "recommended_next_step",
+    "failed_step_index",
+    "failed_step_name",
+)
+
+
+def _project_observation_fields(observation: dict) -> dict:
+    return {key: observation[key] for key in _OBSERVATION_FIELDS if key in observation}
+
 
 class _NullRunReporter:
     def emit_event(self, event: str, **payload) -> None:
@@ -329,11 +354,12 @@ def build_tool_response_payload(request, shared_adapter_manager: _SharedAdapterM
 
     # Minimal MCP-execute enrichment: error_code + fix from the single-source
     # table (NO did-you-mean candidates — this run-report path has no live
-    # ui_elements). NOTE: this stays {} until the autonomous run reporter
-    # propagates the executor's error_code into summary.json; today
-    # run_reporter writes category/stage/last_error but not error_code, so the
-    # `if code:` guard is the honest no-op — never a fabricated code. Wiring it
-    # live is a follow-up (propagate error_code through the run summary).
+    # ui_elements). The execution modes now thread the executor's error_code into
+    # summary.json (run_reporter.finalize(error_code=...)), so this is LIVE for the
+    # --goal autonomous path — the one execute path that produces no observation to
+    # fold below. For action/workflow the live observation overrides this with the
+    # richer engine_error diagnosis. The `if code:` guard keeps it honest: an empty
+    # summary error_code (e.g. assertion verdict) yields {}, never a fabricated code.
     failure_diagnosis = {}
     if exit_code != 0:
         from common.error_codes import lookup
@@ -343,7 +369,7 @@ def build_tool_response_payload(request, shared_adapter_manager: _SharedAdapterM
             msg, fix = lookup(code)
             failure_diagnosis = {"error_code": code, "message": msg, "fix": fix}
 
-    return {
+    response = {
         "ok": exit_code == 0,
         "operation": "execute",
         "mode": execution_mode,
@@ -358,6 +384,34 @@ def build_tool_response_payload(request, shared_adapter_manager: _SharedAdapterM
         "failure_diagnosis": failure_diagnosis,
     }
 
+    # Fold in the live post-action observation (MCP parity with --action --json).
+    # The execute mode stashed it on the manager during dispatch; project its
+    # agent-facing fields ON TOP of the run-report response so live data wins
+    # (e.g. recommended_next_step from the live page beats the run_assets one).
+    # This is what makes failure_diagnosis REAL on MCP — sourced from the live
+    # executor result, not the lossy summary.json. take_* clears the stash.
+    observation = (
+        shared_adapter_manager.take_last_observation()
+        if shared_adapter_manager is not None
+        else None
+    )
+    if observation:
+        response.update(_project_observation_fields(observation))
+        if observation.get("result") == "engine_error":
+            response["failure_diagnosis"] = {
+                "error_code": observation.get("error_code", ""),
+                "message": observation.get("message", ""),
+                "fix": observation.get("fix", ""),
+            }
+        elif observation.get("result") == "assertion_failed":
+            # A failed assertion is a terminal verdict, not a recoverable error.
+            # The bare verdict omits recommended_next_step, so without this the
+            # run_assets-derived value (set above) would survive and bait a retry
+            # on a legitimately-failed assertion. Null it to keep the verdict bare.
+            response["recommended_next_step"] = None
+
+    return response
+
 
 def build_load_run_payload(run_id: str) -> dict:
     run_id = str(run_id).strip()

{screenforge-0.6.0 → screenforge-0.7.0}/common/adapters/base_adapter.py

@@ -17,6 +17,11 @@ class BasePlatformAdapter(ABC):
     def take_screenshot(self) -> bytes:
         pass
 
+    # 录像 seam（诚实边界）：
+    #   - android/ios：start_record/stop_record_and_get_path 落真视频文件（scrcpy / simctl）。
+    #   - web：start_record 是 no-op（CDP attach 无法用 Playwright 录像）；web 的"录像"
+    #     由 review 层用逐操作截图 ffmpeg 拼胶片（review/render.py make_filmstrip）实现，
+    #     不走本 seam。调用方须把返回 "" / None 当作"无视频"，非错误。
     def start_record(self, video_name: str):
         pass
 

{screenforge-0.6.0 → screenforge-0.7.0}/common/ai.py

@@ -9,6 +9,15 @@ from common.logs import log
 from common.progress import ai_status
 
 
+def _strip_json_fences(text: str) -> str:
+    """Unwrap markdown code fences the model may add despite being told not to."""
+    if "```json" in text:
+        return text.split("```json")[1].split("```")[0].strip()
+    if "```" in text:
+        return text.replace("```", "").strip()
+    return text
+
+
 class AIBrain:
     def __init__(self):
         # 实例化文本专属客户端
@@ -251,6 +260,7 @@ class AIBrain:
         """
         封装底层的 LLM 网络调用
         """
+        result_text = ""
         try:
             with ai_status(f"Thinking ({model_name})..."):
                 response = client.chat.completions.create(
@@ -262,16 +272,11 @@ class AIBrain:
                     temperature=0.1,
                 )
 
-            result_text = response.choices[0].message.content.strip()
-
-            if "```json" in result_text:
-                result_text = result_text.split("```json")[1].split("```")[0].strip()
-            elif "```" in result_text:
-                result_text = result_text.replace("```", "").strip()
-
+            result_text = _strip_json_fences(response.choices[0].message.content.strip())
             parsed_json = json.loads(result_text)
             return parsed_json.get("result", {})
 
         except Exception as e:
-            log.error(f"[Error] Model ({model_name}) request or parse failed: {e}")
+            # {} = "retry without cache" per caller contract; log raw text so parse-fail ≠ network-fail.
+            log.error(f"[Error] Model ({model_name}) request or parse failed: {e}\nRaw response: {result_text}")
             return {}