android-midscene-automation 0.1.26 → 0.1.27

package/CHANGELOG.md

@@ -1,5 +1,15 @@
 # 更新记录
 
+## v0.1.27
+
+- `npx android-midscene-automation` 启动时默认使用固定的系统用户数据目录保存脚本、配置、报告和运行态文件，并在启动时尝试迁移旧版启动目录下的数据，降低升级后脚本或配置“不见了”的风险。
+- 新增脚本列表复制和重命名能力，可直接生成“原文件名 Copy”副本并修改脚本名称，便于复用已有录制流程。
+- 新增检测画面变化能力，支持框选设备预览区域或使用当前选中元素作为检测区域，通过“检测画面变化N-开始节点 / 结束节点”对同一分支内的前后截图进行差异对比。
+- 检测画面变化结果写入 Markdown 和 HTML 回放报告，展示检测区域、阈值、最大变化比例、基准帧、对比帧和差异图，HTML 图片支持点击放大查看。
+- 新增空节点，用于在顺序流程中作为无操作占位节点，配合画面变化检测或流程编排使用。
+- 优化流程图节点插入、删除、复制、缩放和判断分支布局，修复节点新增后连线延迟、缩放后节点不可见、删除检测节点后分支错乱、重复添加同一结束节点等问题。
+- 回放报告截图和截图节点改为使用 ADB 截图，避免 Appium `/screenshot` 超时后堵塞 UiAutomator2 命令队列；Appium 请求超时时也会显示更准确的超时提示。
+
 ## v0.1.26
 
 - 修复流程节点复制与粘贴问题，避免 `structuredClone` 失败、粘贴后原节点消失、分支插入位置错误和复制后相对位置偏移。

package/FAQ.md

@@ -1,5 +1,19 @@
 # 常见问题
 
+## 更新后脚本或配置不见了怎么办
+
+通常不是数据被删除，而是旧版本把数据保存到了启动命令所在目录。新版 `npx android-midscene-automation` 会默认使用固定的系统用户数据目录，并在启动时尝试从当前目录自动迁移旧数据。
+
+默认数据目录：
+
+```text
+Windows: %LOCALAPPDATA%\android-midscene-automation
+macOS: ~/Library/Application Support/android-midscene-automation
+Linux: ~/.local/share/android-midscene-automation
+```
+
+如果迁移后仍看不到旧数据，请在之前启动过的目录里查找 `.midscene-app/script-cache.sqlite`，再把 `.midscene-app` 复制到上面的固定数据目录。也可以设置 `ANDROID_MIDSCENE_DATA_ROOT` 指向原来的数据目录继续使用。
+
 ## 同一个 id 定位到错误输入框怎么办
 
 如果账号框和密码框都是 `id/tg_edit`，只按 id 回放可能输入到第一个输入框。

package/README.md

@@ -96,12 +96,16 @@ http://127.0.0.1:5173/
 - `Midscene模型`：执行测试脚本时使用，支持“自定义提供方”和“使用 Codex”。
 - `AI生成脚本模型`：根据测试需求生成脚本时使用。
 
-配置会保存到本地 SQLite：
+使用 `npx android-midscene-automation` 启动时，配置会保存到固定的系统用户数据目录：
 
 ```text
-.midscene-app/script-cache.sqlite
+Windows: %LOCALAPPDATA%\android-midscene-automation\.midscene-app\script-cache.sqlite
+macOS: ~/Library/Application Support/android-midscene-automation/.midscene-app/script-cache.sqlite
+Linux: ~/.local/share/android-midscene-automation/.midscene-app/script-cache.sqlite
 ```
 
+也可以通过 `ANDROID_MIDSCENE_DATA_ROOT` 指定自定义数据目录。使用 `npm run dev` 本地开发时，默认仍写入项目根目录。
+
 旧版 `config.yaml` 或 `config.json` 会在读取后迁移到数据库。当前运行时会优先读取数据库中的模型配置。
 
 ## Android 自动化
@@ -149,7 +153,7 @@ appium
 - Android 预览和操作：`GET /api/android-preview`、`POST /api/android-tap`、`POST /api/android-swipe`、`POST /api/android-keyevent`
 - App 预设：`GET /api/app-presets`、`POST /api/save-app-preset`、`POST /api/delete-app-preset`
 
-后端运行态文件默认都在项目根目录下：
+使用 `npx android-midscene-automation` 启动时，后端运行态文件默认都在系统用户数据目录下；使用 `npm run dev` 本地开发时，默认在项目根目录下。可以通过 `ANDROID_MIDSCENE_DATA_ROOT` 覆盖数据目录：
 
 ```text
 .midscene-app/                 # SQLite 数据库
@@ -1119,6 +1123,20 @@ output/2026-08-21_17-13-42-831-登录流程.html
 
 # 常见问题
 
+## 更新后脚本或配置不见了怎么办
+
+通常不是数据被删除，而是旧版本把数据保存到了启动命令所在目录。新版 `npx android-midscene-automation` 会默认使用固定的系统用户数据目录，并在启动时尝试从当前目录自动迁移旧数据。
+
+默认数据目录：
+
+```text
+Windows: %LOCALAPPDATA%\android-midscene-automation
+macOS: ~/Library/Application Support/android-midscene-automation
+Linux: ~/.local/share/android-midscene-automation
+```
+
+如果迁移后仍看不到旧数据，请在之前启动过的目录里查找 `.midscene-app/script-cache.sqlite`，再把 `.midscene-app` 复制到上面的固定数据目录。也可以设置 `ANDROID_MIDSCENE_DATA_ROOT` 指向原来的数据目录继续使用。
+
 ## 同一个 id 定位到错误输入框怎么办
 
 如果账号框和密码框都是 `id/tg_edit`，只按 id 回放可能输入到第一个输入框。
@@ -1211,6 +1229,16 @@ output/2026-08-21_17-13-42-831-登录流程.html
 
 # 项目更新记录
 
+## v0.1.27
+
+- `npx android-midscene-automation` 启动时默认使用固定的系统用户数据目录保存脚本、配置、报告和运行态文件，并在启动时尝试迁移旧版启动目录下的数据，降低升级后脚本或配置“不见了”的风险。
+- 新增脚本列表复制和重命名能力，可直接生成“原文件名 Copy”副本并修改脚本名称，便于复用已有录制流程。
+- 新增检测画面变化能力，支持框选设备预览区域或使用当前选中元素作为检测区域，通过“检测画面变化N-开始节点 / 结束节点”对同一分支内的前后截图进行差异对比。
+- 检测画面变化结果写入 Markdown 和 HTML 回放报告，展示检测区域、阈值、最大变化比例、基准帧、对比帧和差异图，HTML 图片支持点击放大查看。
+- 新增空节点，用于在顺序流程中作为无操作占位节点，配合画面变化检测或流程编排使用。
+- 优化流程图节点插入、删除、复制、缩放和判断分支布局，修复节点新增后连线延迟、缩放后节点不可见、删除检测节点后分支错乱、重复添加同一结束节点等问题。
+- 回放报告截图和截图节点改为使用 ADB 截图，避免 Appium `/screenshot` 超时后堵塞 UiAutomator2 命令队列；Appium 请求超时时也会显示更准确的超时提示。
+
 ## v0.1.26
 
 - 修复流程节点复制与粘贴问题，避免 `structuredClone` 失败、粘贴后原节点消失、分支插入位置错误和复制后相对位置偏移。

package/bin/android-midscene-automation.js

@@ -1,6 +1,8 @@
 #!/usr/bin/env node
 import { spawn } from 'node:child_process';
 import { createRequire } from 'node:module';
+import fs from 'node:fs';
+import os from 'node:os';
 import path from 'node:path';
 import { fileURLToPath } from 'node:url';
 
@@ -11,6 +13,73 @@ const viteBin = path.join(viteRoot, 'bin', 'vite.js');
 const args = process.argv.slice(2);
 const userRoot = process.cwd();
 const githubUrl = 'https://github.com/oooooooko/android-midscene-automation';
+const appName = 'android-midscene-automation';
+const runtimeEntries = [
+  '.midscene-app',
+  'scripts-output',
+  'midscene_run',
+  '.midscene-generated',
+  'config.json',
+  'config.yaml',
+  'output',
+];
+const runtimeMarkerEntries = ['.midscene-app', 'scripts-output', 'midscene_run', '.midscene-generated'];
+
+function getDefaultDataRoot() {
+  if (process.platform === 'win32') {
+    return path.join(
+      process.env.LOCALAPPDATA || process.env.APPDATA || path.join(os.homedir(), 'AppData', 'Local'),
+      appName,
+    );
+  }
+
+  if (process.platform === 'darwin') {
+    return path.join(os.homedir(), 'Library', 'Application Support', appName);
+  }
+
+  return path.join(process.env.XDG_DATA_HOME || path.join(os.homedir(), '.local', 'share'), appName);
+}
+
+function hasRuntimeData(root) {
+  return runtimeMarkerEntries.some((entry) => fs.existsSync(path.join(root, entry)));
+}
+
+function copyRuntimeEntry(source, target) {
+  const stat = fs.statSync(source);
+  if (stat.isDirectory()) {
+    fs.cpSync(source, target, { recursive: true, force: false, errorOnExist: false });
+    return;
+  }
+
+  if (!fs.existsSync(target)) {
+    fs.copyFileSync(source, target);
+  }
+}
+
+function migrateLegacyDataRoot(sourceRoot, targetRoot) {
+  if (process.env.ANDROID_MIDSCENE_DATA_ROOT || path.resolve(sourceRoot) === path.resolve(targetRoot)) return;
+  if (!hasRuntimeData(sourceRoot)) return;
+
+  try {
+    fs.mkdirSync(targetRoot, { recursive: true });
+    let copied = false;
+    for (const entry of runtimeEntries) {
+      const source = path.join(sourceRoot, entry);
+      if (!fs.existsSync(source)) continue;
+      copyRuntimeEntry(source, path.join(targetRoot, entry));
+      copied = true;
+    }
+    if (copied) {
+      console.log(`已迁移本地数据目录：${sourceRoot} -> ${targetRoot}`);
+    }
+  } catch (error) {
+    console.warn(`迁移本地数据目录失败：${error instanceof Error ? error.message : String(error)}`);
+  }
+}
+
+const dataRoot = process.env.ANDROID_MIDSCENE_DATA_ROOT || getDefaultDataRoot();
+
+migrateLegacyDataRoot(userRoot, dataRoot);
 
 if (!args.includes('--host')) {
   args.unshift('127.0.0.1');
@@ -19,13 +88,14 @@ if (!args.includes('--host')) {
 
 console.log(`\nGitHub 源码 / 二次修改：${githubUrl}`);
 console.log(`问题反馈 / 功能建议：${githubUrl}/issues\n`);
+console.log(`本地数据目录：${dataRoot}\n`);
 
 const child = spawn(process.execPath, [viteBin, ...args], {
   cwd: packageRoot,
   env: {
     ...process.env,
     ANDROID_MIDSCENE_PACKAGE_ROOT: process.env.ANDROID_MIDSCENE_PACKAGE_ROOT || packageRoot,
-    ANDROID_MIDSCENE_DATA_ROOT: process.env.ANDROID_MIDSCENE_DATA_ROOT || userRoot,
+    ANDROID_MIDSCENE_DATA_ROOT: dataRoot,
   },
   stdio: 'inherit',
 });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "android-midscene-automation",
-  "version": "0.1.26",
+  "version": "0.1.27",
   "repository": {
     "type": "git",
     "url": "git+https://github.com/oooooooko/android-midscene-automation.git"
@@ -48,6 +48,8 @@
     "@vue-flow/core": "1.48.2",
     "element-plus": "2.11.1",
     "openai": "5.23.2",
+    "pixelmatch": "7.1.0",
+    "pngjs": "7.0.0",
     "tsx": "4.23.0",
     "typescript": "5.9.3",
     "vite": "7.3.6",
@@ -68,6 +70,7 @@
   "devDependencies": {
     "@playwright/test": "1.61.1",
     "@types/node": "24.13.2",
+    "@types/pngjs": "6.0.5",
     "@types/word-extractor": "1.0.6",
     "@vue/tsconfig": "0.7.0",
     "vue-tsc": "3.3.6"

package/server/appium-recorder/appium-runner.ts

@@ -5,8 +5,9 @@ import { join } from 'node:path';
 import { getAppiumRecordedScript, type AppiumRecordedScriptRecord, type AppiumRecordedStepRecord } from './repository';
 import { appDataPath } from '../paths';
 import { ensureAndroidSdkAvailable, getAdbCommand } from '../android-sdk';
-import { createAppiumReplayReport, type AppiumReplayFrame } from './report';
+import { createAppiumReplayReport, type AppiumReplayFrame, type AppiumReplayVisualCheck } from './report';
 import { startManagedAppiumServer, usesManagedAppiumServer } from './managed-appium';
+import { compareVisualChangeFrames, type VisualChangeRegion } from './visual-change';
 
 type AppiumSessionResponse = {
   value?: {
@@ -34,11 +35,21 @@ class ReplayStoppedError extends Error {
   }
 }
 
+type PendingVisualChangeCheck = {
+  step: AppiumRecordedStepRecord;
+  nodeNumber: number;
+  scriptName: string;
+};
+
 const replayContext = new AsyncLocalStorage<{
   signal?: AbortSignal;
   appiumLog: (line: string) => void;
   serverUrl: string;
+  deviceId: string;
   frames: AppiumReplayFrame[];
+  visualChecks: AppiumReplayVisualCheck[];
+  pendingVisualChecks: PendingVisualChangeCheck[];
+  softFailureCount: number;
 }>();
 
 const appiumServerUrl = () => replayContext.getStore()?.serverUrl || configuredAppiumServerUrl();
@@ -60,6 +71,11 @@ function errorDetail(error: unknown) {
   return error.message;
 }
 
+function isTimeoutError(error: unknown) {
+  const detail = errorDetail(error).toLowerCase();
+  return detail.includes('timeout') || detail.includes('timed out');
+}
+
 function isStaleElementError(error: unknown) {
   const detail = errorDetail(error);
   return detail.includes('stale element reference') || detail.includes('does not exist in DOM anymore');
@@ -174,6 +190,29 @@ function adbText(deviceId: string, args: string[]) {
   });
 }
 
+function adbScreenshotBase64(deviceId: string) {
+  return new Promise<string>((resolve, reject) => {
+    execFile(
+      getAdbCommand(),
+      ['-s', deviceId, 'exec-out', 'screencap', '-p'],
+      { encoding: 'buffer', maxBuffer: 20 * 1024 * 1024, timeout: 8000 },
+      (error, stdout, stderr) => {
+        if (error) {
+          const detail = Buffer.isBuffer(stderr) ? stderr.toString('utf8').trim() : String(stderr || '').trim();
+          reject(new Error(detail || error.message || 'ADB 截图失败'));
+          return;
+        }
+        const imageBuffer = Buffer.isBuffer(stdout) ? stdout : Buffer.from(stdout || '');
+        if (!imageBuffer.length) {
+          reject(new Error('ADB 未返回截图内容'));
+          return;
+        }
+        resolve(imageBuffer.toString('base64'));
+      },
+    );
+  });
+}
+
 async function getCurrentActivity(deviceId: string) {
   const output = await adbText(deviceId, ['shell', 'dumpsys', 'activity', 'activities']);
   const resumedLine = output
@@ -254,6 +293,9 @@ async function appiumRequest<T>(
       throw new ReplayStoppedError();
     }
     context?.appiumLog(`[Appium] 异常：${method} ${path}（${errorDetail(error)}）`);
+    if (isTimeoutError(error)) {
+      throw new Error(`Appium 请求超时 ${requestUrl}，${errorDetail(error)}`);
+    }
     throw new Error(`无法连接 Appium 服务 ${requestUrl}，${errorDetail(error)}`);
   }
 
@@ -291,7 +333,7 @@ function replayFrameSelector(step: AppiumRecordedStepRecord) {
 }
 
 async function captureReplayFrame(
-  sessionId: string,
+  _sessionId: string,
   step: AppiumRecordedStepRecord,
   nodeNumber: number,
   scriptName: string,
@@ -299,7 +341,7 @@ async function captureReplayFrame(
   status: string,
 ) {
   const context = replayContext.getStore();
-  if (!context || !sessionId) return;
+  if (!context || !context.deviceId) return;
   const appendFrame = (imageBase64: string) => {
     context.frames.push({
       sequence: context.frames.length + 1,
@@ -324,14 +366,8 @@ async function captureReplayFrame(
   }
 
   try {
-    const payload = await appiumRequest<AppiumValueResponse<string>>(
-      `/session/${sessionId}/screenshot`,
-      undefined,
-      { ignoreAbort: true, timeoutMs: 5000 },
-    );
-    const imageBase64 = typeof payload.value === 'string' ? payload.value : '';
-    if (!imageBase64) throw new Error('Appium 未返回截图内容');
-    appendFrame(imageBase64);
+    // Report screenshots use ADB so a slow capture cannot block Appium's command queue.
+    appendFrame(await adbScreenshotBase64(context.deviceId));
   } catch (error) {
     context.appiumLog(`[节点 ${nodeNumber}] 截图失败：${errorDetail(error)}`);
   }
@@ -483,16 +519,183 @@ async function waitForElementGone(sessionId: string, step: AppiumRecordedStepRec
   throw new Error(`${step.label} 等待消失超时`);
 }
 
-async function saveScreenshot(sessionId: string) {
-  const payload = await appiumRequest<AppiumValueResponse<string>>(`/session/${sessionId}/screenshot`);
-  if (!payload.value) throw new Error('Appium 未返回截图数据');
+async function saveScreenshot(deviceId: string) {
   const dir = appDataPath('.midscene-app', 'screenshots');
   await mkdir(dir, { recursive: true });
   const file = join(dir, `appium-${Date.now()}.png`);
-  await writeFile(file, Buffer.from(payload.value, 'base64'));
+  await writeFile(file, Buffer.from(await adbScreenshotBase64(deviceId), 'base64'));
 }
 
-async function runStep(sessionId: string, deviceId: string, step: AppiumRecordedStepRecord) {
+type RunStepMeta = {
+  nodeNumber: number;
+  scriptName: string;
+};
+
+type RunStepResult = string | {
+  message?: string;
+  softFailed?: boolean;
+};
+
+function completedStepStatus(step: AppiumRecordedStepRecord, softFailed: boolean) {
+  if (step.type === 'visualChange') {
+    if (step.visualChange?.role === 'start') return '已记录基准帧';
+    if (step.visualChange?.role === 'end') return '已记录对比帧';
+  }
+  return softFailed ? '视觉变化未达预期' : '成功';
+}
+
+function visualChangeNumber(value: unknown, fallback: number, min: number, max = Number.POSITIVE_INFINITY) {
+  const number = Number(value);
+  if (!Number.isFinite(number)) return fallback;
+  return Math.min(max, Math.max(min, number));
+}
+
+function visualChangeRegion(step: AppiumRecordedStepRecord): VisualChangeRegion {
+  const config = step.visualChange;
+  if (!config?.region) throw new Error(`${step.label} 缺少检测区域`);
+  return {
+    x: Math.max(0, Math.round(Number(config.region.x) || 0)),
+    y: Math.max(0, Math.round(Number(config.region.y) || 0)),
+    width: Math.max(1, Math.round(Number(config.region.width) || 1)),
+    height: Math.max(1, Math.round(Number(config.region.height) || 1)),
+  };
+}
+
+function findVisualFrame(
+  frames: AppiumReplayFrame[],
+  nodeId: string,
+  scriptName: string,
+  preferredPhase: AppiumReplayFrame['phase'],
+) {
+  const matches = frames.filter((frame) => frame.nodeId === nodeId && frame.scriptName === scriptName);
+  if (!matches.length) return undefined;
+  const preferred = preferredPhase === 'before'
+    ? matches.find((frame) => frame.phase === preferredPhase)
+    : [...matches].reverse().find((frame) => frame.phase === preferredPhase);
+  return preferred || (preferredPhase === 'before' ? matches[0] : matches[matches.length - 1]);
+}
+
+const EMPTY_PNG_BASE64 = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+/p9sAAAAASUVORK5CYII=';
+
+function registerVisualChangeStep(
+  step: AppiumRecordedStepRecord,
+  meta: RunStepMeta,
+): RunStepResult {
+  const config = step.visualChange;
+  if (!config) throw new Error(`${step.label} 缺少画面变化检测配置`);
+  if (config.role === 'start') {
+    return {
+      message: `已记录画面变化基准点：${config.pairLabel || step.label}`,
+    };
+  }
+  const context = replayContext.getStore();
+  const endStep = {
+    ...step,
+    visualChange: {
+      ...config,
+      endStepId: config.endStepId || step.id,
+    },
+  };
+  context?.pendingVisualChecks.push({ step: endStep, nodeNumber: meta.nodeNumber, scriptName: meta.scriptName });
+  return {
+    message: `已记录画面变化对比点：${config.pairLabel || step.label}`,
+  };
+}
+
+function appendVisualCheckFailure(lines: string[], check: PendingVisualChangeCheck, message: string) {
+  const context = replayContext.getStore();
+  if (context) {
+    const config = check.step.visualChange;
+    context.softFailureCount += 1;
+    context.visualChecks.push({
+      nodeId: check.step.id,
+      nodeNumber: check.nodeNumber,
+      nodeLabel: check.step.label,
+      status: 'failed',
+      message,
+      region: config?.region ? visualChangeRegion(check.step) : { x: 0, y: 0, width: 1, height: 1 },
+      durationMs: 0,
+      intervalMs: 0,
+      sampleCount: 0,
+      changeRatioThreshold: visualChangeNumber(config?.changeRatioThreshold, 2, 0.01),
+      maxChangeRatio: 0,
+      baselineBase64: EMPTY_PNG_BASE64,
+      comparisonBase64: EMPTY_PNG_BASE64,
+      diffBase64: EMPTY_PNG_BASE64,
+      startNodeId: config?.startStepId,
+      endNodeId: config?.endStepId,
+    });
+  }
+  lines.push(`[节点 ${check.nodeNumber}] 视觉变化检测异常：${message}（继续执行）`);
+}
+
+function processPendingVisualChecks(lines: string[]) {
+  const context = replayContext.getStore();
+  if (!context || !context.pendingVisualChecks.length) return;
+
+  // Visual checks compare captured replay frames after the path finishes, so
+  // they never block or mutate the device state for downstream nodes.
+  context.pendingVisualChecks.forEach((check) => {
+    const config = check.step.visualChange;
+    if (!config?.startStepId || !config.endStepId) {
+      appendVisualCheckFailure(lines, check, `${check.step.label} 缺少开始节点或结束节点`);
+      return;
+    }
+    const baselineFrame = findVisualFrame(context.frames, config.startStepId, check.scriptName, 'after');
+    const comparisonFrame = findVisualFrame(context.frames, config.endStepId, check.scriptName, 'after');
+    if (!baselineFrame || !comparisonFrame) {
+      const missing = [
+        baselineFrame ? '' : '开始节点截图',
+        comparisonFrame ? '' : '结束节点截图',
+      ].filter(Boolean).join('、');
+      appendVisualCheckFailure(lines, check, `${check.step.label} 未捕获到${missing}`);
+      return;
+    }
+    let result: ReturnType<typeof compareVisualChangeFrames>;
+    try {
+      result = compareVisualChangeFrames({
+        baselineScreenshot: Buffer.from(baselineFrame.imageBase64, 'base64'),
+        comparisonScreenshot: Buffer.from(comparisonFrame.imageBase64, 'base64'),
+        region: visualChangeRegion(check.step),
+        changeRatioThreshold: visualChangeNumber(config.changeRatioThreshold, 2, 0.01),
+        pixelmatchThreshold: visualChangeNumber(config.pixelmatchThreshold, 0.1, 0, 1),
+      });
+    } catch (error) {
+      appendVisualCheckFailure(lines, check, `${check.step.label} 对比失败：${errorDetail(error)}`);
+      return;
+    }
+    context.visualChecks.push({
+      nodeId: check.step.id,
+      nodeNumber: check.nodeNumber,
+      nodeLabel: check.step.label,
+      status: result.passed ? 'passed' : 'failed',
+      message: result.message,
+      region: result.region,
+      durationMs: result.durationMs,
+      intervalMs: result.intervalMs,
+      sampleCount: result.sampleCount,
+      changeRatioThreshold: result.changeRatioThreshold,
+      maxChangeRatio: result.maxChangeRatio,
+      baselineBase64: result.baselineBase64,
+      comparisonBase64: result.comparisonBase64,
+      diffBase64: result.diffBase64,
+      startNodeId: config.startStepId,
+      startNodeLabel: `${baselineFrame.nodeNumber}. ${baselineFrame.nodeLabel}`,
+      endNodeId: config.endStepId,
+      endNodeLabel: `${comparisonFrame.nodeNumber}. ${comparisonFrame.nodeLabel}`,
+    });
+    if (!result.passed) context.softFailureCount += 1;
+    lines.push(`[节点 ${check.nodeNumber}] 视觉变化检测${result.passed ? '有变化' : '无明显变化'}：${result.message}${result.passed ? '' : '（继续执行）'}`);
+  });
+  context.pendingVisualChecks = [];
+}
+
+async function runStep(
+  sessionId: string,
+  deviceId: string,
+  step: AppiumRecordedStepRecord,
+  meta: RunStepMeta,
+): Promise<RunStepResult | void> {
   throwIfReplayStopped();
   if (step.type === 'delay') {
     await wait(Math.max(0, step.timeoutMs || 1000));
@@ -528,7 +731,15 @@ async function runStep(sessionId: string, deviceId: string, step: AppiumRecorded
   }
 
   if (step.type === 'screenshot') {
-    await saveScreenshot(sessionId);
+    await saveScreenshot(deviceId);
+    return;
+  }
+
+  if (step.type === 'visualChange') {
+    return registerVisualChangeStep(step, meta);
+  }
+
+  if (step.type === 'noop') {
     return;
   }
 
@@ -797,6 +1008,20 @@ async function replayLinkedScript(
   lines.push(`连接脚本完成：${linkedScript.name}`);
 }
 
+function appendRunStepResult(lines: string[], nodeNumber: number, result: RunStepResult | void) {
+  if (!result) return false;
+  if (typeof result === 'string') {
+    if (result) lines.push(`[节点 ${nodeNumber}] 结果：${result}`);
+    return false;
+  }
+  if (result.softFailed) {
+    lines.push(`[节点 ${nodeNumber}] 视觉变化未达预期：${result.message || '未检测到明显变化'}（继续执行）`);
+    return true;
+  }
+  if (result.message) lines.push(`[节点 ${nodeNumber}] 结果：${result.message}`);
+  return false;
+}
+
 async function replayLinearSteps(
   sessionId: string,
   deviceId: string,
@@ -814,14 +1039,18 @@ async function replayLinearSteps(
     lines.push(`[节点 ${index + 1}] 开始：${step.label}`);
     await captureReplayFrame(sessionId, step, index + 1, options.scriptName || '', 'before', '执行前');
     try {
+      let softFailed = false;
       if (step.type === 'runScript') {
         await replayLinkedScript(sessionId, deviceId, step, lines, stack);
       } else {
-        const result = await runStep(sessionId, deviceId, step);
-        if (result) lines.push(`[节点 ${index + 1}] 结果：${result}`);
+        const result = await runStep(sessionId, deviceId, step, {
+          nodeNumber: index + 1,
+          scriptName: options.scriptName || '',
+        });
+        softFailed = appendRunStepResult(lines, index + 1, result);
       }
-      lines.push(`[节点 ${index + 1}] 完成：${step.label}`);
-      await captureReplayFrame(sessionId, step, index + 1, options.scriptName || '', 'after', '成功');
+      lines.push(`[节点 ${index + 1}] 完成：${step.label}${softFailed ? '（存在软失败，已继续）' : ''}`);
+      await captureReplayFrame(sessionId, step, index + 1, options.scriptName || '', 'after', completedStepStatus(step, softFailed));
     } catch (error) {
       if (isReplayStopped(error)) {
         lines.push(`[节点 ${index + 1}] 已终止：${step.label}`);
@@ -915,14 +1144,18 @@ async function replayFlowSteps(
     }
 
     try {
+      let softFailed = false;
       if (step.type === 'runScript') {
         await replayLinkedScript(sessionId, deviceId, step, lines, stack);
       } else {
-        const result = await runStep(sessionId, deviceId, step);
-        if (result) lines.push(`[节点 ${index + 1}] 结果：${result}`);
+        const result = await runStep(sessionId, deviceId, step, {
+          nodeNumber: index + 1,
+          scriptName: options.scriptName || '',
+        });
+        softFailed = appendRunStepResult(lines, index + 1, result);
       }
-      lines.push(`[节点 ${index + 1}] 完成：${step.label}`);
-      await captureReplayFrame(sessionId, step, index + 1, options.scriptName || '', 'after', '成功');
+      lines.push(`[节点 ${index + 1}] 完成：${step.label}${softFailed ? '（存在软失败，已继续）' : ''}`);
+      await captureReplayFrame(sessionId, step, index + 1, options.scriptName || '', 'after', completedStepStatus(step, softFailed));
       index = nextIndexAfterStep(index, step);
     } catch (error) {
       if (isReplayStopped(error)) {
@@ -997,7 +1230,7 @@ async function replayScriptSteps(
     try {
       await replayLinkedScript(sessionId, deviceId, step, lines, stack);
       lines.push(`[节点 ${nodeNumber}] 完成：${step.label}`);
-      await captureReplayFrame(sessionId, step, nodeNumber, options.scriptName || '', 'after', '成功');
+      await captureReplayFrame(sessionId, step, nodeNumber, options.scriptName || '', 'after', completedStepStatus(step, false));
     } catch (error) {
       if (isReplayStopped(error)) {
         lines.push(`[节点 ${nodeNumber}] 已终止：${step.label}`);
@@ -1033,7 +1266,11 @@ export async function replayAppiumScript(
     signal,
     appiumLog: (line) => lines.push(line),
     serverUrl: configuredAppiumServerUrl(),
+    deviceId: targetDeviceId,
     frames: [] as AppiumReplayFrame[],
+    visualChecks: [] as AppiumReplayVisualCheck[],
+    pendingVisualChecks: [] as PendingVisualChangeCheck[],
+    softFailureCount: 0,
   };
   return replayContext.run(context, async () => {
     lines.push(
@@ -1071,8 +1308,14 @@ export async function replayAppiumScript(
       await replayScriptSteps(sessionId, targetDeviceId, script.steps, lines, [script.id], {
         scriptName: script.name,
       });
-      lines.push('回放完成');
-      success = true;
+      processPendingVisualChecks(lines);
+      if (context.softFailureCount > 0) {
+        lines.push(`回放完成，但存在 ${context.softFailureCount} 个视觉变化未达预期节点`);
+        success = false;
+      } else {
+        lines.push('回放完成');
+        success = true;
+      }
     } catch (error) {
       stopped = isReplayStopped(error);
       const detail = errorDetail(error);
@@ -1126,6 +1369,7 @@ export async function replayAppiumScript(
         startedAt,
         completedAt,
         frames: context.frames,
+        visualChecks: context.visualChecks,
       });
       reportPath = report.filePath;
       reportId = report.id;
@@ -1145,6 +1389,7 @@ export async function replayAppiumScript(
       reportId,
       logPath,
       htmlReportPath,
+      softFailureCount: context.softFailureCount,
     };
   });
 }