page-agent-sdk 4.15.0 → 4.16.0

package/README.md

@@ -8,7 +8,7 @@
 
 [![npm](https://img.shields.io/npm/v/page-agent-sdk.svg)](https://www.npmjs.com/package/page-agent-sdk)
 [![license](https://img.shields.io/badge/license-ISC-blue.svg)](https://github.com/whyymj/page-agent-sdk/blob/master/LICENSE)
-[![tests](https://img.shields.io/badge/self%20tests-3519%20asserts-brightgreen.svg)](#self-tests)
+[![tests](https://img.shields.io/badge/self%20tests-3542%20asserts-brightgreen.svg)](#self-tests)
 
 ---
 
@@ -241,7 +241,7 @@ ChatDialog, MessageContent, CodePreview, SkillPanel, DebugDrawer, useChat
 | | `ui` | `boolean \| 'default'` · default `true` | `false` = headless (build UI with `agent.messages`) |
 | | `llm` | `LLMConfig \| BaseChatModel` · **required** | `LLMConfig={provider?,apiKey,baseUrl?,model?,temperature?,maxTokens?}`; `provider` defaults to `'openai'` (OpenAI/DeepSeek-compatible, default DeepSeek); `'anthropic'` dynamic-loads `@langchain/anthropic` for Claude native protocol |
 | | `id` | `string` | Stable id (multi-agent isolation + persistence resume; random+warn if omitted) |
-| | `systemPrompt` | `string` | Agent identity (no hardcoded business; inject via this). Optional — built-in default (JSON operation assistant + `reliableWriteRules`) used if omitted; passing your own fully overrides it. `appendReliableWriteRules` defaults to `true`: auto-appends `reliableWriteRules` with a `---` separator; set `false` to disable |
+| | `systemPrompt` | `string` | Agent identity (no hardcoded business; inject via this). Optional — built-in default is capability-aware (4.16): with data declared = JSON operation assistant + `reliableWriteRules`; with dataOps:false + domInspect = "page content assistant" (write rules not appended); passing your own fully overrides it. `appendReliableWriteRules` defaults to `true`: auto-appends `reliableWriteRules` with a `---` separator (not appended when dataOps:false); set `false` to disable |
 | | `augmentSystem` | `(ctx:{state,data?}) => string \| undefined` | Dynamic system prompt injection hook: called each turn, returns a string injected as a segment based on runtime state/data; return undefined to skip; callback errors degrade to skip (no crash). `ctx.data` is taken from liveData() each turn (auto-syncs after setData), enabling dynamic component descriptions / partial schema hints. Not set = current behavior |
 | **Page data** | `data` | `{schema,bind,description?}` | Single main object: declare zod schema (validation + field descriptions auto-injected into prompt) + bind (reactive/plain object, tools read/write directly, no `window`) + description |
 | | `tools` / `skills` / `memory` | `Tool[]` / `SkillSpec[]` / `string` | Custom tools / skills / AGENTS.md-style directives |
@@ -249,6 +249,7 @@ ChatDialog, MessageContent, CodePreview, SkillPanel, DebugDrawer, useChat
 | | `dialog.autoQuote` | `boolean` | **Selection quoting · silent capture (page-quote, default false)**: when true, opening the drawer or clicking the input area lazily captures the host page's current text selection (outside the dialog) into a removable quote chip, sent with the next message as question context. Privacy opt-in; `sdk.setQuote/clearQuote` work regardless of this toggle. See [usage-guide §6.20](doc/usage-guide.en.md#620-text-selection-quoting--page-qa-page-quote--read_page--pagecontext) |
 | | `dialog.selectionMenu` | `boolean` | **Selection floating menu (page-quote explicit confirm, default false)**: selecting text floats a "❝ Quote to chat" toolbar above the selection; click = attach the quote chip + open the dialog + focus the input; dismiss on outside click / scroll / Esc; composable with autoQuote. See [usage-guide §6.20](doc/usage-guide.en.md#620-text-selection-quoting--page-qa-page-quote--read_page--pagecontext) |
 | | `capabilities.pageContext` | `boolean` | **Page anchor (default false)**: injects the current page title + URL as a pinned system segment each round (pair with `domInspect`'s read_page for page QA). See [usage-guide §6.20](doc/usage-guide.en.md#620-text-selection-quoting--page-qa-page-quote--read_page--pagecontext) |
+| | `screenshot` | `{renderer?}` | **Screenshot config (take_screenshot)**: assembled when domInspect is on AND (multimodal main model \|\| images.describe); default renderer is html-to-image, pass `renderer` when CSP restricts. See [usage-guide §6.21](doc/usage-guide.en.md#621-screenshot-viewing--page-content-analysis-take_screenshot--page-analysis) |
 | **Capability toggles** | `capabilities` | `{planning?,dataOps?,fetch?,skills?,vfs?,summarization?,memory?,subagent?,verify?,focus?}` | Default all on (`verify` default off, opt-in; `focus` = context focus for refining one component, default on); `false` to turn off |
 | | `permissions` | `PermissionRule[]` | Scope whitelist (first-match-wins, default off) |
 | | `humanConfirm` | `boolean` · default `true` | Proactive inquiry (AI asks when uncertain/multi-plan) |
@@ -474,7 +475,7 @@ After `npm run dev`, visit the corresponding page:
 | multi-agent-demo | `/examples/multi-agent-demo/` | Multi-agent parallel + exclusive switch (3 independent agents, drawer hide/show keeps each history) |
 | proxy-demo | `/examples/proxy-demo/` | LLM connection config: proxy to prevent apiKey leakage (browser holds only userToken, proxy injects real key; auto-refresh on expired token; needs `npm run proxy:mock`) + Provider switch (`provider:'anthropic'` for Claude native protocol, streaming + extended thinking) |
 | images-demo | `/examples/images-demo/` | Image input: text-only main model + `images.describe` captioning bypass (captions injected, image never sent; auto direct-send when the main model is multimodal) |
-| docs-demo | `/examples/docs-demo/` | Docs-site integration template: selection-quote questioning (autoQuote dual lazy capture + quote chip) + paginated `read_page` page QA + `pageContext` page anchor; copy into your own docs site |
+| docs-demo | `/examples/docs-demo/` | Docs-site integration template: selection-quote questioning (floating menu + autoQuote lazy capture) + paginated `read_page` page QA + `pageContext` page anchor + `?shot=1` screenshot visual verification; copy into your own docs site |
 
 Framework-agnostic integration: `demo/plain.html` (importmap + esm.sh).
 

package/README.zh-CN.md

@@ -8,7 +8,7 @@
 
 [![npm](https://img.shields.io/npm/v/page-agent-sdk.svg)](https://www.npmjs.com/package/page-agent-sdk)
 [![license](https://img.shields.io/badge/license-ISC-blue.svg)](https://github.com/whyymj/page-agent-sdk/blob/master/LICENSE)
-[![tests](https://img.shields.io/badge/self%20tests-3519%20asserts-brightgreen.svg)](#自测)
+[![tests](https://img.shields.io/badge/self%20tests-3542%20asserts-brightgreen.svg)](#自测)
 
 ---
 
@@ -175,6 +175,7 @@ CDN 零配置：`<script src="https://unpkg.com/page-agent-sdk"></script>` → `
 | 🖼 图片输入 | 对话框内置三入口(📎 选择/拖拽/粘贴截图)→ 压缩闸(长边 ≤1568px/单轮 ≤4 张/超 20MB 拒);主模型多模态(gpt-4o/claude/qwen-vl 查表,或 `llm.vision:true`)→ 图片直发 content parts 零配置;纯文本主模型(deepseek 等)→ 配 `images.describe` 逐图识图转述注入(图不直发);都不配则诚实拒绝不静默丢图;`images.upload` 原图换 https URL(集成方 OSS);持久化只存缩略图 + vfs 引用 | `images: { upload?, describe? }` + `llm.vision` |
 | ❝ 划词引用 | 用户选中页面文字 → 引用 chip 自动挂上(打开抽屉/点输入框双懒捕获)→ 随下一条消息作为提问上下文;`AgentMessage.quote` 消息级字段(content 干净,气泡结构化引用块,LLM 前缀注入,随消息持久化);`sdk.setQuote/clearQuote` 宿主 API(headless 同享);来源自动推导 = 页面 title + 最近在前标题;排队/快捷指令不消费(与图片同口径);隐私 opt-in 默认关;另有 `dialog.selectionMenu` 显式确认形态(划选浮出「❝ 引用到对话」工具条,点击挂引用并打开对话框) | `dialog.autoQuote` / `dialog.selectionMenu` + `sdk.setQuote` |
 | 📖 页面问答 | `read_page` 读当前页正文纯文本(智能定位 article/main/[role=main]/.content,排除 SDK 自身 DOM 与 script/style,`hasMore` 分页续读,大结果自动外存 vfs);`pageContext` 每轮注入当前页 title+URL 锚点 pin 段(跨压缩;子 agent 不继承) | `capabilities: { domInspect: true, pageContext: true }` |
+| 📸 截图查看 | `take_screenshot` 三模式(selector 局部 / fullPage 整页 / 视口);条件注入(domInspect 开 && vision 主模型 \|\| images.describe,不满足 warn 留痕);分层图通道(vision → 工具结果后合成 user 消息 image parts,免疫 trim/offload;纯文本 → describe 转述);压缩闸 ≤1568 + 原图收 vfs;工具步骤行缩略图观察面;`page-analysis` skill(问题分型/探索纪律/回答纪律) | `capabilities: { domInspect: true }` + `llm.vision` 或 `images.describe` + `screenshot.renderer`(可选) |
 
 能力默认开（`verify`/`approval`/`checkpoint` 默认关；**主动征询 `humanConfirm` 默认开**——AI 遇不确定/多方案主动问你、不猜测），可经 `capabilities` 关掉无用的省 token。
 
@@ -237,7 +238,7 @@ ChatDialog, MessageContent, CodePreview, SkillPanel, DebugDrawer, useChat
 | | `ui` | `boolean \| 'default'` · 默认 `true` | `false` = headless（用 `agent.messages` 自建 UI） |
 | | `llm` | `LLMConfig \| BaseChatModel` · **必传** | `LLMConfig={provider?,apiKey,baseUrl?,model?,temperature?,maxTokens?}`；`provider` 缺省 `'openai'`（兼容 OpenAI/DeepSeek 协议，默认接 DeepSeek）；`'anthropic'` 动态加载 `@langchain/anthropic` 走 Claude 原生协议 |
 | | `id` | `string` | 稳定 id（多 agent 隔离 + 持久化恢复；不传随机+warn） |
-| | `systemPrompt` | `string` | Agent 身份(不硬编码业务,靠这注入)。可选——不传用内置默认(JSON 操作助手 + `reliableWriteRules`);传了则完全覆盖。`appendReliableWriteRules` 默认 `true`:自动用 `---` 分隔线追加 reliableWriteRules;设 `false` 关闭 |
+| | `systemPrompt` | `string` | Agent 身份(不硬编码业务,靠这注入)。可选——不传用内置默认(**4.16 能力感知**:有 data 声明 = JSON 操作助手 + `reliableWriteRules`;dataOps:false + domInspect = 「页面内容助手」,不追加写入规则);传了则完全覆盖。`appendReliableWriteRules` 默认 `true`:自动用 `---` 分隔线追加 reliableWriteRules(**dataOps:false 时不追加**);设 `false` 关闭 |
 | | `augmentSystem` | `(ctx:{state,data?}) => string \| undefined` | 动态 system prompt 注入钩子:每轮调,按运行时 state/data 返回字符串作为一段注入;返回 undefined 跳过;回调抛错降级跳过(不崩)。`ctx.data` 每轮从 liveData() 取最新(setData 后自动同步),可据此动态算当前组件说明 / 部分 schema 描述。不配 = 现状行为 |
 | **页面数据** | `data` | `{schema,bind,description?}` | 单主对象:声明 zod schema(校验 + 字段描述自动注入提示词)+ bind(reactive/普通对象,工具直接读写,不挂 window)+ description |
 | | `tools` / `skills` / `memory` | `Tool[]` / `SkillSpec[]` / `string` | 自定义工具 / 技能 / AGENTS.md 风格持久指令 |
@@ -248,6 +249,7 @@ ChatDialog, MessageContent, CodePreview, SkillPanel, DebugDrawer, useChat
 | | `dialog.autoQuote` | `boolean` | **划词引用·静默捕获(page-quote,默认 false)**:true 时打开抽屉/点输入区瞬间懒捕获宿主页面(对话框外)当前选中文本挂「引用 chip」(可删),随下一条消息发给 LLM。隐私 opt-in;`sdk.setQuote/clearQuote` 不受此开关影响。见 [usage-guide §6.20](doc/usage-guide.md#620-划词引用与页面问答page-quote--read_page--pagecontext) |
 | | `dialog.selectionMenu` | `boolean` | **划词浮动菜单(page-quote 显式确认,默认 false)**:划选文字浮出「❝ 引用到对话」工具条,点击 = 挂引用 chip + 打开对话框 + 聚焦输入;点别处/滚动/Esc 消失;与 autoQuote 独立可组合。见 [usage-guide §6.20](doc/usage-guide.md#620-划词引用与页面问答page-quote--read_page--pagecontext) |
 | | `capabilities.pageContext` | `boolean` | **页面锚点(默认 false)**:每轮 system 注入当前页 title+URL(pin 段跨压缩;配合 `domInspect` 的 read_page 读正文答问)。见 [usage-guide §6.20](doc/usage-guide.md#620-划词引用与页面问答page-quote--read_page--pagecontext) |
+| | `screenshot` | `{renderer?}` | **截图配置组(take_screenshot)**:装配条件 = domInspect 开 &&(多模态主模型 \|\| images.describe);默认渲染 html-to-image,CSP 限制时传 `renderer` 自定义。见 [usage-guide §6.21](doc/usage-guide.md#621-截图查看与页面内容分析take_screenshot--page-analysis) |
 | | `permissions` | `PermissionRule[]` | scope 白名单（first-match-wins，默认不启用） |
 | | `humanConfirm` | `boolean` · 默认 `true` | 主动征询（AI 不确定/多方案主动问你，不猜测） |
 | | `approval` | `{tools?,confirm?,timeoutMs?,humanConfirmTool?}` · 默认关 | 被动确认白名单（写操作前弹允许/拒绝） |
@@ -515,7 +517,7 @@ function switchTo(i: number) {
 
 ```bash
 npm test            # 3283 项断言（tsx 源码级，不依赖 LLM）
-npm run test:e2e    # 1048 项集成断言（node 跑构建产物 dist；覆盖各 API/配置项/功能模块/简单与复杂场景：默认 systemPrompt(含能力概述) / 动态注册与 inspect 同步 / inspect(tools/middleware/subagent/verify/mcp/todos/lastCompression/checkpoints 反映配置) / 自定义 tools/middleware/skills/memory 注入 / 运行时动态重配置(setTools/addTool/removeTool/setLlm/setMemory/setSubagents 反映) / switchSession(开/未开) / shareContext 开/关共享独立 / storage 后端+对象配置 / presets 三预设 / checkpoint / 导出项完整(39+ 函数/组件) / 工具函数可用(isQuotaError/estimateTokens/jpEval/searchJson) / source=builtin / mount 边界 / hook 多监听器 / llm 配置 / 错误场景）
+npm run test:e2e    # 1151 项集成断言（node 跑构建产物 dist；覆盖各 API/配置项/功能模块/简单与复杂场景：默认 systemPrompt(含能力概述) / 动态注册与 inspect 同步 / inspect(tools/middleware/subagent/verify/mcp/todos/lastCompression/checkpoints 反映配置) / 自定义 tools/middleware/skills/memory 注入 / 运行时动态重配置(setTools/addTool/removeTool/setLlm/setMemory/setSubagents 反映) / switchSession(开/未开) / shareContext 开/关共享独立 / storage 后端+对象配置 / presets 三预设 / checkpoint / 导出项完整(39+ 函数/组件) / 工具函数可用(isQuotaError/estimateTokens/jpEval/searchJson) / source=builtin / mount 边界 / hook 多监听器 / llm 配置 / 错误场景）
 ```
 
 ## 本地 npm 包测试