page-agent-sdk 3.45.0 → 3.46.0

package/README.md

@@ -8,7 +8,7 @@
 
 [![npm](https://img.shields.io/npm/v/page-agent-sdk.svg)](https://www.npmjs.com/package/page-agent-sdk)
 [![license](https://img.shields.io/badge/license-ISC-blue.svg)](https://github.com/whyymj/page-agent-sdk/blob/master/LICENSE)
-[![tests](https://img.shields.io/badge/self%20tests-2895%20asserts-brightgreen.svg)](#self-tests)
+[![tests](https://img.shields.io/badge/self%20tests-2919%20asserts-brightgreen.svg)](#self-tests)
 
 ---
 
@@ -28,8 +28,9 @@ Starting point for both humans and AI agents (Claude Code / Cursor): find the fe
 | Old bundler (webpack ≤4 / vue-cli 2-3) | [Bundle size & tree-shaking](#bundle-size--tree-shaking) (legacy subpath, es2017 all-inlined) |
 | HTML/code components (AI writes page blocks) | [Capability packs](#createchatsdk-options-cheat-sheet) (`createHtmlSubagent`, auto-registered 3.9+) · `examples/html-page-demo`, `examples/complex-demo` |
 | RAG / MCP tools | [Capability packs](#createchatsdk-options-cheat-sheet) (`createRagSubagent`, `mcp`) · `examples/rag-demo` |
+| Let the AI see images (paste/drop/pick) | [options cheat sheet](#createchatsdk-options-cheat-sheet) (`images` / `llm.vision`) · [usage-guide §6.17](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.en.md#617-image-input-multimodal-direct--captioning-bypass) · `examples/images-demo` |
 | Customize UI (theme / icons / i18n / button labels) | [`DialogConfig` fields](#dialogconfig-fields) · [usage-guide §6.15](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.en.md#615-ui-customization--i18n-icons--theme--language--message-overrides-317321) · `examples/i18n-demo` |
-| Sessions / persistence (IndexedDB) | [options cheat sheet](#createchatsdk-options-cheat-sheet) (`storage`/`session`) · `examples/session-history-demo` |
+| Sessions / persistence (IndexedDB) | [options cheat sheet](#createchatsdk-options-cheat-sheet) (`storage`/`session`) · `examples/page-demo` (`storage:'indexed'` + built-in history dropdown) |
 | Long conversations / big JSON (context & compression) | [usage-guide §6.8](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.en.md) · [context-management doc](https://github.com/whyymj/page-agent-sdk/blob/master/doc/context-management.md) |
 | Events / audit / token usage | [Configuration](#configuration) (`onEvent`/`onAudit`/`sdk.usage`) · [usage-guide §6.9](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.en.md) |
 | Unattended automation / batch / budget | [options cheat sheet](#createchatsdk-options-cheat-sheet) (`capabilities.automation`, `sdk.batch`) · [usage-guide automation section](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.en.md) |
@@ -166,6 +167,7 @@ CDN zero-config: `<script src="https://unpkg.com/page-agent-sdk"></script>` →
 | 🎨 Subagent model/thinking tiering | `createHtmlSubagent({ llm, thinkingMode })`: code-gen subagents get their own stronger model (main stays light for orchestration) + thinking-depth lock (`'deep'` injects thinking params for quality / `'simple'` strips them to save tokens; top-level `subagent.thinkingMode` as global default). LLMConfig construction path only (pre-built instance → warn + no-op); requires a thinking-capable model (deepseek-thinking / claude); `inspect().subagent.subagents` reflects the effective state | `createHtmlSubagent({ llm, thinkingMode })` |
 | ⚡ host actions (2.20+) | Register save/publish/preview etc; SDK auto-generates named tools, agent triggers page ops directly (no `trigger_action` indirection) | `actions` |
 | 🧩 schema tiered disclosure (2.20+) | Large schema → systemPrompt injects top-level overview only (no constraints/no recursion); deep constraints via `schema_data` on demand; small schema unaffected (full) | `schemaHint` |
+| 🖼 image input | Built-in three entry points (📎 pick / drag / paste screenshot) → compression gate (long edge ≤1568px, ≤4 per round, >20MB rejected); multimodal main model (gpt-4o/claude/qwen-vl table hit, or `llm.vision:true`) → images sent directly as content parts, zero config; text-only main model (deepseek etc.) → `images.describe` captions each image into the user context (image never sent); neither → honest rejection, never silently dropped; `images.upload` swaps the original for an https URL (integrator OSS); persistence keeps only a thumbnail + vfs reference | `images: { upload?, describe? }` + `llm.vision` |
 | 📌 cross-compress working memory (2.20+) | Pin recent read/query paths + hashes across compression; no re-fetch, correct optimistic-lock hash | `capabilities.workingMemory` |
 | 🤖 unattended automation (2.20+) | Resource budget guard (`tokenBudget`/`timeBudgetMs`) + fatal-error auto-recovery (`maxAutoRetries`: restore checkpoint + retry) + cross-refresh resume + `sdk.batch(tasks)` batch processing | `capabilities.automation` |
 | 📐 context resilience (2.30+) | Hard floor `contextWindow ≥200K` (rejects <200K models like legacy `deepseek`/`gpt-4o`/`glm-4.5` at startup); three gates (compress/trim/offload) thresholds follow the live window after `setLlm`; reactive retry on `context_length_exceeded` (aggressive trim → single retry, never fails raw); vfs large-result refs protected from LRU eviction + OOM 1.5× fallback; system-prompt budget (25% window, drops non-pinned segments, keeps base/mission/workingMemory) | built-in |
@@ -238,6 +240,7 @@ ChatDialog, MessageContent, CodePreview, SkillPanel, DebugDrawer, useChat
 | | `augmentSystem` | `(ctx:{state,data?}) => string \| undefined` | Dynamic system prompt injection hook: called each turn, returns a string injected as a segment based on runtime state/data; return undefined to skip; callback errors degrade to skip (no crash). `ctx.data` is taken from liveData() each turn (auto-syncs after setData), enabling dynamic component descriptions / partial schema hints. Not set = current behavior |
 | **Page data** | `data` | `{schema,bind,description?}` | Single main object: declare zod schema (validation + field descriptions auto-injected into prompt) + bind (reactive/plain object, tools read/write directly, no `window`) + description |
 | | `tools` / `skills` / `memory` | `Tool[]` / `SkillSpec[]` / `string` | Custom tools / skills / AGENTS.md-style directives |
+| | `images` | `{upload?,describe?,describeTimeoutMs?}` | **Image input (image-input-vision)**: built-in three entry points (📎 pick / drag / paste) → compression gate (long edge ≤1568px, ≤4 per round, >20MB rejected). Multimodal main model (table hit or `llm.vision:true`) → images sent directly as content parts, zero config; text-only main model → configure `describe` to caption each image into the context (image never sent); neither → honest rejection, never silently dropped; `upload` swaps the original for an https URL (integrator OSS). See [usage-guide §6.17](doc/usage-guide.en.md#617-image-input-multimodal-direct--captioning-bypass) |
 | **Capability toggles** | `capabilities` | `{planning?,dataOps?,fetch?,skills?,vfs?,summarization?,memory?,subagent?,verify?,focus?,preferences?}` | Default all on (`verify`/`preferences` default off, opt-in; `focus` = context focus for refining one component, default on; `preferences` = cross-session preference memory); `false` to turn off |
 | | `permissions` | `PermissionRule[]` | Scope whitelist (first-match-wins, default off) |
 | | `humanConfirm` | `boolean` · default `true` | Proactive inquiry (AI asks when uncertain/multi-plan) |
@@ -461,6 +464,7 @@ After `npm run dev`, visit the corresponding page:
 | animation-demo | `/examples/animation-demo/` | ChatDialog enter/collapse/unmount animations + inline/drawer + hide/show |
 | multi-agent-demo | `/examples/multi-agent-demo/` | Multi-agent parallel + exclusive switch (3 independent agents, drawer hide/show keeps each history) |
 | proxy-demo | `/examples/proxy-demo/` | LLM connection config: proxy to prevent apiKey leakage (browser holds only userToken, proxy injects real key; auto-refresh on expired token; needs `npm run proxy:mock`) + Provider switch (`provider:'anthropic'` for Claude native protocol, streaming + extended thinking) |
+| images-demo | `/examples/images-demo/` | Image input: text-only main model + `images.describe` captioning bypass (captions injected, image never sent; auto direct-send when the main model is multimodal) |
 
 Framework-agnostic integration: `demo/plain.html` (importmap + esm.sh).
 
@@ -495,7 +499,7 @@ function switchTo(i: number) {
 ## Self-tests
 
 ```bash
-npm test            # 2895 assertions (tsx, source-level; no LLM dependency)
+npm test            # 2894 assertions (tsx, source-level; no LLM dependency)
 npm run test:e2e    # 943 integration assertions (node, built dist; covers APIs/options/modules/simple&complex scenes: default systemPrompt(capability overview) / dynamic register + inspect sync / inspect(tools/middleware/subagent/verify/mcp/todos/lastCompression/checkpoints reflect config) / custom tools/middleware/skills/memory injection / runtime dynamic reconfiguration(setTools/addTool/removeTool/setLlm/setMemory/setSubagents reflect) / switchSession(on/off) / shareContext on/off sharing/independent / storage backends + object config / presets(3) / checkpoint / exports complete(39+ fns/components) / util fns usable(isQuotaError/estimateTokens/jpEval/searchJson) / source=builtin / mount boundary / hook multi-listener / llm config / hide/show / error scenes)
 ```
 

package/README.zh-CN.md

@@ -8,7 +8,7 @@
 
 [![npm](https://img.shields.io/npm/v/page-agent-sdk.svg)](https://www.npmjs.com/package/page-agent-sdk)
 [![license](https://img.shields.io/badge/license-ISC-blue.svg)](https://github.com/whyymj/page-agent-sdk/blob/master/LICENSE)
-[![tests](https://img.shields.io/badge/self%20tests-2895%20asserts-brightgreen.svg)](#自测)
+[![tests](https://img.shields.io/badge/self%20tests-2919%20asserts-brightgreen.svg)](#自测)
 
 ---
 
@@ -28,8 +28,9 @@
 | 老构建链(webpack ≤4 / vue-cli 2-3) | [体积与按需引入](#体积与按需引入)(legacy 子路径,es2017 全量打包) |
 | HTML/代码组件(AI 生成页面块) | [能力包](#createchatsdk-配置项速查)(`createHtmlSubagent`,3.9+ 自动装配)· `examples/html-page-demo`、`examples/complex-demo` |
 | RAG / MCP 工具 | [能力包](#createchatsdk-配置项速查)(`createRagSubagent`、`mcp`)· `examples/rag-demo` |
+| 让 AI 看图(贴图/截图) | [配置项速查](#createchatsdk-配置项速查)(`images`/`llm.vision`)· [usage-guide §6.17](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.md#617-图片输入多模态直发--识图转述旁路) · `examples/images-demo` |
 | 定制 UI(主题 / 图标 / 国际化 / 按钮文字标签) | [`DialogConfig` 字段表](#dialogconfig-字段) · [usage-guide §6.15](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.md#615-ui-定制与国际化图标--主题--语言--文案覆盖317321) · `examples/i18n-demo` |
-| 会话 / 持久化(IndexedDB) | [配置项速查](#createchatsdk-配置项速查)(`storage`/`session`)· `examples/session-history-demo` |
+| 会话 / 持久化(IndexedDB) | [配置项速查](#createchatsdk-配置项速查)(`storage`/`session`)· `examples/page-demo`(`storage:'indexed'` + 内置历史记录下拉) |
 | 长对话 / 大 JSON(上下文与压缩) | [usage-guide §6.8](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.md) · [context-management 文档](https://github.com/whyymj/page-agent-sdk/blob/master/doc/context-management.md) |
 | 事件 / 审计 / token 用量 | [配置](#配置)(`onEvent`/`onAudit`/`sdk.usage`)· [usage-guide §6.9](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.md) |
 | 无人值守自动化 / 批处理 / 预算 | [配置项速查](#createchatsdk-配置项速查)(`capabilities.automation`、`sdk.batch`)· [usage-guide 自动化节](https://github.com/whyymj/page-agent-sdk/blob/master/doc/usage-guide.md) |
@@ -166,6 +167,7 @@ CDN 零配置：`<script src="https://unpkg.com/page-agent-sdk"></script>` → `
 | 🎯 跨会话用户偏好记忆 | `capabilities.preferences`(**opt-in 默认关**,自动写用户浏览器属行为敏感项):agent 从对话中捕获用户持久偏好 —— 强信号(「记住:…」显式命令,零 LLM)/ 中信号(模式词初筛 + 小 LLM 提炼,核心判定「持久口味 vs 本轮任务指令」)/ 行为推断**不捕获**(宁漏勿误:学错一条假偏好,之后每个会话都带着跑);偏好独立持久化(preferenceStore,IndexedDB,与 storage/skillStorage 同构;同 topic **后说覆盖前说**,FIFO ≤20);每轮经 pin 段注入 system prompt(跨会话/跨压缩生效);`sdk.getPreferences()/removePreference(id)/clearPreferences()` 管理学错条目,DebugDrawer「用户偏好」只读小节可查 | `capabilities: { preferences: true }` + 可选 `preferenceStorage` |
 | 🧭 指令执行力增强 (3.35+) | **完结门禁**:todos 有未完成项却欲纯文本收尾 → 回灌「双出口」反馈续跑(≤2 次),防「拆 3 项做 1 项就收口」的莫名中断;**问句意图守卫**:正则三档启发式逐消息定性问句,命中注入「先答勿做」pin 段(跨压缩存活),防长对话提问被历史拖着误路由成操作(如问「这是啥组件」却去生成代码)。均默认开、零配置、宁漏勿误 | 内置 |
 | 🎨 子 agent 模型/思考分层 | `createHtmlSubagent({ llm, thinkingMode })`:代码生成子 agent 独立强模型(主保持轻量编排)+ 思考深度锁定(`'deep'` 注入思考参数质量优先 / `'simple'` 剥除省 token;顶层 `subagent.thinkingMode` 全局缺省)。仅 LLMConfig 构造路径生效(预构造实例 warn+no-op);需模型支持思考(deepseek thinking 版/claude);`inspect().subagent.subagents` 反射生效状态 | `createHtmlSubagent({ llm, thinkingMode })` |
+| 🖼 图片输入 | 对话框内置三入口(📎 选择/拖拽/粘贴截图)→ 压缩闸(长边 ≤1568px/单轮 ≤4 张/超 20MB 拒);主模型多模态(gpt-4o/claude/qwen-vl 查表,或 `llm.vision:true`)→ 图片直发 content parts 零配置;纯文本主模型(deepseek 等)→ 配 `images.describe` 逐图识图转述注入(图不直发);都不配则诚实拒绝不静默丢图;`images.upload` 原图换 https URL(集成方 OSS);持久化只存缩略图 + vfs 引用 | `images: { upload?, describe? }` + `llm.vision` |
 
 能力默认开（`verify`/`approval`/`checkpoint` 默认关；**主动征询 `humanConfirm` 默认开**——AI 遇不确定/多方案主动问你、不猜测），可经 `capabilities` 关掉无用的省 token。
 
@@ -235,6 +237,7 @@ ChatDialog, MessageContent, CodePreview, SkillPanel, DebugDrawer, useChat
 | **能力开关** | `capabilities` | `{planning?,missionAnchor?,dataOps?,fetch?,skills?,vfs?,summarization?,memory?,workingMemory?,subagent?,verify?,domInspect?,focus?,preferences?}` | 核心默认开（`verify`/`domInspect`/`preferences` 默认关,opt-in;`focus` 上下文聚焦·指定组件精修,默认开;`preferences` 跨会话偏好记忆)；`false` 关掉省 token |
 | | `actions` | `Record<string,{description,run,params?}>` | **(2.18+) 宿主动作**：注册 save_draft/publish 等页面操作 → SDK 自动生成命名 tool 供 agent 触发 |
 | | `schemaHint` | `{maxKeys?,maxChars?}` · 默认 `{15,4000}` | **(2.18+) 大 schema 分层披露阈值**：超则 systemPrompt 只注入顶层概览（不带约束/不递归）,深层约束按需 `schema_data` 查;小 schema 无感（全量） |
+| | `images` | `{upload?,describe?,describeTimeoutMs?}` | **图片输入(image-input-vision)**：对话框内置三入口(📎/拖拽/粘贴)→ 压缩闸(长边≤1568/≤4 张/超 20MB 拒)。主模型多模态(查表或 `llm.vision:true`)→ 图片直发 content parts,零配置;纯文本主模型 → 配 `describe` 逐图识图转述注入(图不直发),都不配则诚实拒绝不静默丢图;`upload` 原图换 https URL(集成方 OSS)。见 [usage-guide §6.17](doc/usage-guide.md#617-图片输入多模态直发--识图转述旁路) |
 | | `permissions` | `PermissionRule[]` | scope 白名单（first-match-wins，默认不启用） |
 | | `humanConfirm` | `boolean` · 默认 `true` | 主动征询（AI 不确定/多方案主动问你，不猜测） |
 | | `approval` | `{tools?,confirm?,timeoutMs?,humanConfirmTool?}` · 默认关 | 被动确认白名单（写操作前弹允许/拒绝） |
@@ -464,6 +467,7 @@ createChatSdk({
 | animation-demo | `/examples/animation-demo/` | ChatDialog 入场/收起/卸载动画 + inline/drawer 模式 + hide/show |
 | multi-agent-demo | `/examples/multi-agent-demo/` | 多 Agent 并行 + 互斥切换（三独立 agent，drawer hide/show 保留各自历史） |
 | proxy-demo | `/examples/proxy-demo/` | LLM 连接配置：代理防 apiKey 泄露（浏览器只持 userToken，代理注入真实 key；含 token 过期自动刷新；需 `npm run proxy:mock`）+ Provider 切换（`provider:'anthropic'` 走 Claude 原生协议，流式 + extended thinking） |
+| images-demo | `/examples/images-demo/` | 图片输入：纯文本主模型 + `images.describe` 识图转述旁路（转述注入、图不直发；主模型多模态时自动直发） |
 
 框架无关集成：`demo/plain.html`（importmap + esm.sh）。
 
@@ -498,7 +502,7 @@ function switchTo(i: number) {
 ## 自测
 
 ```bash
-npm test            # 2895 项断言（tsx 源码级，不依赖 LLM）
+npm test            # 2894 项断言（tsx 源码级，不依赖 LLM）
 npm run test:e2e    # 943 项集成断言（node 跑构建产物 dist；覆盖各 API/配置项/功能模块/简单与复杂场景：默认 systemPrompt(含能力概述) / 动态注册与 inspect 同步 / inspect(tools/middleware/subagent/verify/mcp/todos/lastCompression/checkpoints 反映配置) / 自定义 tools/middleware/skills/memory 注入 / 运行时动态重配置(setTools/addTool/removeTool/setLlm/setMemory/setSubagents 反映) / switchSession(开/未开) / shareContext 开/关共享独立 / storage 后端+对象配置 / presets 三预设 / checkpoint / 导出项完整(39+ 函数/组件) / 工具函数可用(isQuotaError/estimateTokens/jpEval/searchJson) / source=builtin / mount 边界 / hook 多监听器 / llm 配置 / 错误场景）
 ```