yunti-browser-runtime 0.1.2 → 0.2.0

package/README.md

@@ -38,18 +38,42 @@ yunti-browser-runtime/
 
 ## 快速启动
 
+推荐作为 MCP 工具接入 Agent，由 Agent 自动启动本地 bridge：
+
 ```bash
-cd yunti-browser-runtime
-npm install
-npm run bridge
+npm install -g yunti-browser-runtime
+yunti-browser-runtime print-config -- --agent codex --human
 ```
 
-安装为 npm 包后，也可以使用 CLI：
+把输出的 MCP 配置加入 Agent 后，加载浏览器扩展并刷新任意 `http` / `https` 页面即可。
+默认不需要填写 token，不需要打开扩展 popup，也不需要保存设置。
+
+需要独立调试 bridge 时，可以手动运行：
 
 ```bash
 yunti-browser-runtime bridge
+```
+
+需要查看本地运行时状态时，可以打开可选控制台：
+
+```bash
+yunti-browser-runtime console
+```
+
+然后访问：
+
+```text
+http://127.0.0.1:48887/console
+```
+
+控制台只展示连接页面、pending/queued 任务、最近工具 activity 和诊断数量等脱敏摘要；
+它不是必选 hub，也不会改变 Agent 的 MCP 使用路径。
+`yunti-browser-runtime doctor` 的人类可读摘要也会打印这个控制台 URL。
+
+其他常用 CLI：
+
+```bash
 yunti-browser-runtime doctor
-yunti-browser-runtime print-config -- --agent codex --human
 yunti-browser-runtime package-extension
 ```
 
@@ -92,19 +116,25 @@ npm run doctor:json
 2. 执行：npm install -g yunti-browser-runtime
 3. 执行：yunti-browser-runtime print-config -- --agent 当前Agent名称 --human
 4. 根据输出，把 MCP server 配置加入当前 Agent 的 MCP 配置。
-5. 执行：yunti-browser-runtime bridge
+5. 告诉我：当前 Agent 启动 MCP server 后会自动启动本地 bridge，一般不需要单独运行 bridge。
 6. 引导我打开 Chrome/Edge 的扩展管理页，开启开发者模式，手动加载扩展目录：
    $(npm root -g)/yunti-browser-runtime/extension
-7. 告诉我扩展里默认 bridge URL 是 http://127.0.0.1:48887，本地默认不需要 token，也不需要保存设置。
+7. 告诉我：扩展默认 bridge URL 是 http://127.0.0.1:48887，本地默认不需要 token，不需要打开 popup，也不需要保存设置。
 8. 打开任意 http/https 页面并刷新。
 9. 执行：yunti-browser-runtime doctor
 10. 如果 doctor 正常，再调用 yunti_list_browser_targets 或 yunti_get_tool_usage_hints 验证你能看到浏览器页面。
 
-注意：Chrome 扩展不能由 npm 静默安装，必须由我手动在浏览器扩展页加载。你要一步步告诉我点哪里、复制什么、检查什么。
+注意：Chrome 扩展不能由 npm 静默安装，必须由我手动在浏览器扩展页加载。扩展加载完成后默认立即可用，不要要求我填写 token、打开 popup 或保存设置；除非 doctor 明确提示 bridge 未运行，才让我单独执行 yunti-browser-runtime bridge。
 ```
 
 ## 安装浏览器扩展
 
+npm 全局安装后，扩展目录通常是：
+
+```bash
+$(npm root -g)/yunti-browser-runtime/extension
+```
+
 开发加载方式：
 
 1. 打开 `chrome://extensions` 或 `edge://extensions`。
@@ -112,11 +142,12 @@ npm run doctor:json
 3. 点击“加载已解压的扩展程序”。
 4. 选择本项目的 `extension/` 目录。
 5. 打开任意 `http` 或 `https` 页面。
-6. 刷新目标页面，确认页面已连接。
+6. 刷新目标页面即可连接。
 
-扩展默认使用 `http://127.0.0.1:48887`，本地安装不需要 token，也不需要保存设置。
-扩展 popup 只用于自定义 bridge URL、页面匹配或可选 token。扩展只负责把页面注册到
-本地 bridge，并执行 Agent 发来的浏览器动作。
+扩展默认使用 `http://127.0.0.1:48887`，本地安装不需要 token，不需要打开 popup，
+也不需要保存设置。扩展 popup 首屏只显示连接状态；自定义 bridge URL、页面匹配或
+可选 token 都收在高级设置里。扩展只负责把页面注册到本地 bridge，并执行 Agent
+发来的浏览器动作。
 
 也可以生成 zip 包用于分发或归档：
 
@@ -125,6 +156,8 @@ npm run package:extension
 ```
 
 输出文件位于 `dist/yunti-browser-runtime-extension-<version>.zip`，zip 内只包含扩展运行所需文件。
+浏览器商店分发前的权限、隐私和素材准备清单见
+[扩展分发准备度](docs/EXTENSION_DISTRIBUTION.md)。
 
 ## 给 Agent 注册 MCP
 
@@ -206,6 +239,7 @@ cp -R skills/yunti-browser-runtime ~/.codex/skills/
 - [工具指南](docs/TOOL_GUIDE.md)
 - [权限和隐私说明](docs/SECURITY.md)
 - [发布运行手册](docs/RELEASE.md)
+- [0.2.0 发布清单](docs/RELEASE_0_2_0_CHECKLIST.md)
 - [发布阻塞处理](docs/PUBLISHING_BLOCKERS.md)
 - [项目初衷](docs/PROJECT_INTENT.md)
 - [项目状态](docs/PROJECT_STATUS.md)
@@ -280,7 +314,10 @@ token。
 
 ## 发布状态与后续事项
 
-- `yunti-browser-runtime@0.1.2` 已发布到官方 npm registry。
-- `0.1.2` 将扩展 popup 的 token 移入高级设置，并补充可复制给 Agent 的安装引导。
+- 当前稳定发布版本：`yunti-browser-runtime@0.1.3`。
+- 当前分支正在收口 `0.2.0` npm/unpacked extension 发布候选版本；`0.2.0`
+  增强了 `yunti_observe_page`、fresh uid 操作闭环、结构化恢复诊断、DOM/diagnostic
+  脱敏和可选本地控制台。
 - 发布后验证命令：`npm run release:verify-published`。
-- 如要上架浏览器扩展商店，需要再次审查 broad host permissions。
+- 浏览器扩展商店版本不阻塞 `0.2.0` npm 发布；如要上架商店，需要沿
+  `docs/EXTENSION_PERMISSION_STRATEGY.md` 另行设计 store-candidate 权限 UX。

package/bin/yunti-browser-runtime.js

@@ -16,6 +16,10 @@ const commands = {
     script: join(rootDir, "mcp", "server.js"),
     env: { YUNTI_BROWSER_BRIDGE_ONLY: "1" },
   },
+  console: {
+    script: join(rootDir, "mcp", "server.js"),
+    env: { YUNTI_BROWSER_BRIDGE_ONLY: "1" },
+  },
   doctor: {
     script: join(rootDir, "scripts", "doctor.js"),
     env: {},
@@ -73,6 +77,7 @@ Usage:
 Commands:
   mcp                Start the stdio MCP server (default)
   bridge             Start only the local HTTP bridge
+  console            Start the bridge and print the optional local console URL
   doctor             Run install and bridge diagnostics
   print-config       Print MCP configuration for an agent
   package-extension  Build the browser extension zip
@@ -80,6 +85,7 @@ Commands:
 Examples:
   yunti-browser-runtime
   yunti-browser-runtime bridge
+  yunti-browser-runtime console
   yunti-browser-runtime doctor
   yunti-browser-runtime print-config -- --agent codex --human
   yunti-browser-runtime package-extension`)

package/docs/ACTION_RESULT_COVERAGE.md

@@ -0,0 +1,41 @@
+# Action Result Coverage
+
+This document is the durable P6.2 action result coverage matrix. It exists so
+future context compaction or thread handoff cannot blur which browser action
+surfaces already expose structured result fields.
+
+Compatibility rule: every action path must keep its historical fields while
+adding structured fields such as `action`, `target`, `ok`, `code`,
+`recoverable`, `recoveryHint`, and `nextStepHint` where useful.
+
+## Coverage Matrix
+
+| Tool | Covered success paths | Covered failure diagnostics | Compatibility fields preserved | Remaining validation |
+| --- | --- | --- | --- | --- |
+| `yunti_click` | uid, coordinate, selector | stale uid, missing uid, selector failure via content result | `clicked`, `uid`, `selector`, `x`, `y`, `method`, `element`, `browserSessionId` | Real-browser observe-click-observe closure pending |
+| `yunti_hover` | uid, coordinate, selector | stale uid, missing uid, selector failure via content result | `hovered`, `uid`, `selector`, `x`, `y`, `method`, `element`, `browserSessionId` | Real-browser hover menu closure pending |
+| `yunti_fill` | uid keyboard, uid select, uid contenteditable, selector | stale uid, missing uid, selector failure, non-editable, hidden, disabled, readonly, option miss, value not applied | `filled`, `uid`, `selector`, `method`, `value`, `valueLength`, `element`, `before`, `after`, `browserSessionId` | Real-browser controlled input closure pending |
+| `yunti_select` | selector value, uid value, uid text | selector failure, uid failure, option miss, disabled option, non-select target | `selected`, `uid`, `selector`, `value`, `text`, `selectedIndex`, `element`, `browserSessionId` | Real-browser async option closure pending |
+| `yunti_fill_form` | aggregate success, partial failure | per-field fill/select failure aggregation | `filled`, `failed`, `results`, `browserSessionId` | Real-browser multi-field form closure pending |
+| `yunti_wait_for` | text, selector, urlContains | timeout | `found`, `text`, `selector`, `condition`, `value`, `waitedMs`, `browserSessionId` | Real-browser async wait closure pending |
+| `yunti_scroll` | document, uid container, coordinate container, coordinate document fallback | uid missing, uid stale, coordinate unavailable, no movement, partial movement, document fallback hint | `scrolled`, `deltaX`, `deltaY`, `target`, `before`, `after`, `moved`, `browserSessionId` | Real-browser nested panel closure pending |
+| `yunti_type_text` | uid, selector | stale uid, missing uid, selector failure via content result | `typed`, `uid`, `selector`, `text`, `textLength`, `method`, `mode`, `element`, `browserSessionId` | Real-browser text editor closure pending |
+| `yunti_press_key` | uid, selector | stale uid, missing uid, selector failure via content result | `pressed`, `uid`, `selector`, `key`, `element`, `valueChanged`, `browserSessionId` | Real-browser shortcut closure pending |
+| `yunti_upload_file` | uid, selector | stale uid, missing uid, selector failure via CDP/content path | `uploaded`, `uid`, `selector`, `fileCount`, `filePaths`, `browserSessionId` | Real-browser file input closure pending |
+| `yunti_drag` | coordinate | coordinate argument validation, CDP dispatch failure via tool error | `dragged`, `from`, `to`, `steps`, `browserSessionId` | Real-browser drag target closure pending |
+
+## Coverage Gate
+
+- Last audited phase: P6.2.7.
+- Required automated check: `npm run check:action-results`.
+- Required release gate: `npm run release:check` runs the coverage check.
+- Real browser closure: Pending until Playwright/Chromium is installed in the
+  validation environment.
+- Real browser command: `YUNTI_E2E=1 npm run test:e2e`.
+
+## Next Coverage Work
+
+P6.2.7 closes the structured action result coverage audit. The next engineering
+slice should not add another manual coverage checklist unless a new action tool
+or result contract is introduced. Prefer moving to real-browser closure
+validation or P6.3 agent workflow contracts.

package/docs/AGENT_WORKFLOW_CONTRACT.md

@@ -0,0 +1,97 @@
+# Agent Workflow Contract
+
+This document is the P6.3.1 default operating contract for agents using Yunti
+Browser Runtime. It is intentionally tool-level and LLM-agnostic: Yunti stays
+the browser operation layer, while the agent remains responsible for task
+planning and user judgment.
+
+## Default Loop
+
+Use this loop for normal page work:
+
+1. Call `yunti_get_tool_usage_hints` when the tool contract is uncertain.
+2. Call `yunti_list_browser_targets` and choose the intended
+   `browserSessionId`.
+3. Call `yunti_observe_page` for the current page state.
+4. Act with a fresh uid when possible.
+5. If the page is loading or changing, call `yunti_wait_for`.
+6. After every wait or action, call `yunti_observe_page` again.
+7. Verify the result using observe, snapshot, evaluate, screenshot, network, or
+   console tools before assuming the task is done.
+8. If a uid/session/action result is stale or recoverable, follow
+   `code`, `recoveryHint`, and `nextStepHint` before retrying.
+
+## Recovery Rules
+
+- Stale route: call `yunti_list_browser_targets` and use the latest
+  `browserSessionId`.
+- Stale uid: call `yunti_observe_page` again and use a fresh uid.
+- Async UI: call `yunti_wait_for`, then `yunti_observe_page`, then continue
+  with a fresh uid.
+- Scroll uncertainty: prefer fresh `scrollableContainers[]` uids, inspect
+  `moved`, `edgeHint`, `partialMovement`, and `recoveryHint`.
+- Fill/select uncertainty: inspect `fillable`, `readOnly`, `disabled`,
+  `fillBlockReason`, `selectedValue`, `selectedText`, and `options[]` before
+  retrying.
+- Timeout: treat `WAIT_TIMEOUT` as a signal to observe and adjust the condition,
+  not proof that the user task failed.
+- Coordinates/selectors: use them as recovery or debugging paths when fresh
+  uids are unavailable.
+
+## Confirmation Boundary
+
+Ask the user for explicit confirmation before:
+
+- submitting forms that change production data;
+- deleting, approving, purchasing, publishing, or sending messages;
+- uploading sensitive files;
+- exposing or copying secrets, credentials, cookies, auth headers, tokens, or
+  private keys;
+- taking an action whose effect cannot be verified from page state.
+
+## Copyable Prompt
+
+```text
+Please operate my browser through Yunti Browser Runtime.
+
+Follow this workflow:
+1. Call yunti_get_tool_usage_hints if tool usage is uncertain.
+2. Call yunti_list_browser_targets and choose the intended browserSessionId.
+3. Use yunti_observe_page before page actions.
+4. Prefer fresh uids from yunti_observe_page for click, hover, fill, select,
+   scroll, type, press, upload, and drag operations.
+5. After each action, verify by observing again or using snapshot, evaluate,
+   screenshot, network, or console tools.
+6. For async rendering, validation, navigation, option loading, or infinite
+   scroll, call yunti_wait_for, then yunti_observe_page, then continue with a
+   fresh uid.
+7. If a result has ok=false, code, recoveryHint, or nextStepHint, follow that
+   guidance before retrying. Do not blindly repeat the same action.
+8. Use selector or coordinate fallback only when fresh uids are unavailable or
+   as an explicit recovery/debugging path.
+9. Before submitting, deleting, approving, purchasing, publishing, uploading
+   sensitive files, or changing production data, ask me for confirmation.
+10. Do not expose raw cookies, passwords, auth headers, tokens, private keys, or
+    other secrets. Remember that DOM observation redaction does not redact
+    screenshots.
+```
+
+## Minimal Use Case Index
+
+The concrete P6.3.2 minimal use cases are maintained in
+[Tool Guide](TOOL_GUIDE.md#minimal-use-cases) and mirrored in the packaged
+skill:
+
+- Click
+- Fill Form
+- Scroll To Find
+- Switch Tab
+- Wait For Async Result
+
+## Non-Goals
+
+- Do not move a built-in LLM loop into Yunti Runtime.
+- Do not replace fine-grained `yunti_*` tools with a single opaque task runner.
+- Do not require a mandatory hub tab or side panel for the default local path.
+- Do not remove CDP, screenshots, network, console, tab control, upload, or
+  diagnostic capabilities while improving page-level workflows.