deepseek-harness-intelligent-ui 0.2.8 → 0.2.10

package/CHANGELOG.md

@@ -6,6 +6,34 @@ All notable changes to this project are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.2.10]
+
+### Changed
+- Widget policy settings (`cdnAllowlist`, `allowNetwork`, `maxWidgetHeight`, `maxWidgetBytes`, `sendPromptMaxChars`, `openLinkAllowedOrigins`) are now live settings: they can be edited from the official settings forms, including DeepSeek Harness Remote clients, and apply to the next tool call without reloading the plugin.
+- Settings descriptions are available in Simplified Chinese.
+
+### Security
+
+- Separated popup, top-navigation, form and download probes in the sandbox audit. A form check now requires an actual CSP violation and no request; download checks use browser events with a positive control. Cross-origin download links can still issue self-navigation requests even when saving the response is blocked.
+- Widget sandbox boundary audit ([#19](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/issues/19)), with the measured matrix in [docs/widget-csp-audit.md](docs/widget-csp-audit.md) and a regression suite in `npm run e2e:security` covering every resource type and request channel (`img`, `script`, `style`, `font`, `media`, `fetch`, XHR, WebSocket, `EventSource`, `sendBeacon`, `<a ping>`, `iframe`, `object`, form, download, popup, navigation).
+- **Fixed:** host messages now use a document-bound `MessageChannel` established by the bootstrap before generated code. A navigated document cannot receive host RPC results, HTML or theme updates, even if it delays its `load` event indefinitely. A later iframe `load` still removes foreign UI; it is a best-effort visual signal, not the data boundary. Rebuilds rotate credentials and reset lifecycle in a layout effect; stale async replies cannot reach a new channel. Self-navigation itself and data in its initial URL remain possible.
+- Documented that `allowNetwork=false` stops script-visible network access (`fetch`, XHR, WebSocket, `EventSource`, `sendBeacon`, `<a ping>`) but not resource-element requests to an **allowlisted** origin, and that this implementation emits origin-only CSP sources without path restrictions. The `allowNetwork` setting description says so, and an empty `cdnAllowlist` remains the way to block external resource requests, but cannot stop self-navigation.
+
+## [0.2.9]
+
+### Fixed
+
+- `ui_render` and `ui_compose` catalog blocks: the tool schema and description now say that the top-level `root` field is required, and a call without it gets an error that says how to fix it. In real-model runs DeepSeek models often named the first component `"root"` but left out the `root` field, so the first call was refused and had to be retried ([#62](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/pull/62)).
+- Release publication verification: wait up to 10 minutes (30 attempts) for npm registry propagation to avoid false-alarm verification failures ([#56](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/pull/56)).
+- Evaluation routing: guard against missing chart properties and empty block lists, accept local OpenAI-compatible HTTP endpoints via `DEEPSEEK_EVAL_ENDPOINT`, and judge delivery on inline widgets instead of workspace tool calls ([#59](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/pull/59), [#61](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/pull/61)).
+
+### Added
+
+- [docs/presentation-routing.md](docs/presentation-routing.md) records the presentation-routing study for [#58](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/issues/58): rule overlaps, four comparable projects, and a real-model A/B (DeepSeek V4.1 Flash, 3 runs × 39 cases) in which a rewritten decision-order guidance did not route better than the current one (114/117 vs. 111/117 routed correctly), so the current guidance is kept ([#64](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/pull/64)).
+- Real-model evaluation group F: paired presentation-routing cases for [#58](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/issues/58). Each pair keeps the topic and changes only the task, so a prompt cannot score well by always or never drawing UI; the whole answer is judged (explanation, code, interaction, the user's own data), not just the tool name. Categories `no-ui`, `needs-ui` and `update` report the unnecessary-UI rate, the missed-UI rate and follow-up edits separately. API cases also record prompt and completion tokens ([#59](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/pull/59)).
+- Storefront screenshots manifest: `screenshots.json` declares the showcase screenshots and order for plugin markets like `dshmarket`.
+- [CONTRIBUTING.md](CONTRIBUTING.md) guide covering development setup, the committed-bundle rule, test layout, required CI checks, translations, PR automation, and release integrity rules ([#57](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/pull/57)).
+
 ## [0.2.8]
 
 ### Added
@@ -178,7 +206,8 @@ All notable changes to this project are documented here. The format follows
   - The declarative component catalog (`ui_render`) and sandboxed streaming HTML widgets (`ui_render_widget`).
   - The `/ui` command and the guidance system-prompt section.
 
-[Unreleased]: https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/compare/v0.2.8...HEAD
+[Unreleased]: https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/compare/v0.2.9...HEAD
+[0.2.9]: https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/compare/v0.2.8...v0.2.9
 [0.2.8]: https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/compare/v0.2.7...v0.2.8
 [0.2.7]: https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/compare/v0.2.6...v0.2.7
 [0.2.6]: https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/compare/v0.2.5...v0.2.6

package/README.md

@@ -171,6 +171,8 @@ The plugin injects a system-prompt section ([`src/guidance.js`](src/guidance.js)
 - **`ui_render_widget`** for what the catalog cannot draw: 3D, simulations, animations, games, custom tools.
 - **Local first:** anything computable in the browser must update instantly without a message to the model.
 
+Why these rules were kept after a real-model A/B against a rewritten decision order, and how routing is measured: [docs/presentation-routing.md](docs/presentation-routing.md).
+
 `/ui <request>` forces a UI answer.
 
 | Tool | Use it for | Rendering |
@@ -254,8 +256,10 @@ Fields follow the OpenIntelligentUI streaming contract, **in this order**: `titl
 <img src="https://raw.githubusercontent.com/dreamfarer-space/deepseek-harness-intelligent-ui/main/docs/assets/readme/security.svg" alt="Widgets run in an opaque-origin sandboxed iframe under a strict CSP; access to the host page is blocked and only token-checked postMessage calls reach the rate-limited bridge" width="100%">
 
 - **Widget isolation:** `<iframe sandbox="allow-scripts allow-forms">` with `srcdoc` and **no** `allow-same-origin`. The E2E verifies in a browser that `parent.document`, `localStorage` and `document.cookie` throw `SecurityError`, `fetch` is blocked, and `window.origin === "null"`.
-- **CSP:** `default-src 'none'`; scripts, styles, fonts and images only from `cdnAllowlist`; `connect-src 'none'` unless `allowNetwork`; `form-action 'none'`; `base-uri 'none'`.
-- **Messages:** accepted only from the frame's own `contentWindow` with a per-frame random token and a valid shape; forged messages are dropped (verified in E2E).
+- **CSP:** `default-src 'none'`; scripts, styles, fonts and images only from `cdnAllowlist`; `connect-src 'none'` unless `allowNetwork`; `form-action 'none'`; `base-uri 'none'`. A per-widget sandbox boundary audit with the measured matrix is in [docs/widget-csp-audit.md](docs/widget-csp-audit.md) and runs as `npm run e2e:security`.
+- **What `allowNetwork=false` does and does not stop:** it blocks `fetch`, XHR, WebSocket, `EventSource`, `sendBeacon` and `<a ping>`. It does **not** stop `<img>`, `<script src>`, `<link rel=stylesheet>`, `@font-face` or `<video>` from requesting an **allowlisted** origin, and this implementation emits origin-only `cdnAllowlist` sources, without path restrictions — so an allowlisted CDN can still see widget data in a resource URL. Set `cdnAllowlist: []` to block external resource requests; the offline bundle already does. This does not stop a widget navigating its own frame or sending data in that initial navigation URL.
+- **Navigation:** a document-bound MessageChannel prevents a replacement document from receiving host data. A later iframe `load` removes the foreign UI; an unfinished response can delay this visual removal. The initial self-navigation request is still possible. The iframe sandbox blocks top-level navigation, `window.open` and file downloads. A cross-origin download link can still trigger a self-navigation request before the browser blocks saving the response.
+- **Messages:** a one-time handshake validates the iframe source and fresh document token before accepting its transferred port. All later messages use that port and validate token/shape; window RPCs and stale ports are rejected (verified in E2E).
 - **`sendPrompt` limits:** refused for 1 s after mount, at most one per 2 s, capped at `sendPromptMaxChars`, queued as a normal user message.
 - **`openLink`:** https only, optionally restricted to `openLinkAllowedOrigins`.
 - **Catalog:** text renders as React elements, never raw HTML; `Plot` expressions are compiled over a fixed math table with no `eval`, `Function` or property access; new-session launches and host actions need an explicit native click.
@@ -277,7 +281,7 @@ The bundle patch inserts the plugin row automatically. To change settings, add t
 | `enableWidget` | `true` | Register `ui_render_widget` |
 | `injectGuidance` | `true` | Add the "when to use UI" system-prompt section |
 | `cdnAllowlist` | esm.sh, jsdelivr, unpkg | Origins widgets (and Math/Mermaid) may load resources from |
-| `allowNetwork` | `false` | Let widgets `fetch` and use WebSocket to those origins |
+| `allowNetwork` | `false` | Let widgets `fetch` and use WebSocket to those origins. Off still allows resource elements to reach them; use an empty `cdnAllowlist` to block external resource requests (self-navigation is still possible) |
 | `maxWidgetHeight` | `2400` | Maximum frame height (px) |
 | `maxWidgetBytes` | `400000` | Reject larger widgets |
 | `sendPromptMaxChars` | `4000` | Longest message a widget, button, form or choice may send |
@@ -316,11 +320,12 @@ npm run eval:real -- --record && npm run eval:demo  # record and cut the demo vi
 
 | Option | Effect |
 | --- | --- |
-| `--stage mock,api,browser` | Run only some layers (`all` by default); `A`/`B`/`C`/`D` selects API categories |
+| `--stage mock,api,browser` | Run only some layers (`all` by default); `A`–`F` selects API categories (`F`: paired presentation-routing cases, [#58](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/issues/58)) |
 | `--only B3,R-chart` | Run only these cases |
 | `--keep-other-layers` | Keep the layers or cases this run did not execute, each with its own date |
 | `--append` | With `--only`: add the rerun as extra samples; a failure is never replaced by a lucky retry |
 | `--rescore` | Re-judge the last API run from its saved transcripts with the current assertions, no model calls |
+| `--report <file.json>` | Write the report elsewhere, e.g. one file per prompt version to compare a baseline with a candidate |
 | `DEEPSEEK_EVAL_MODEL` | Model id (default `deepseek-flash`) |
 
 The **Real-model eval** workflow runs the same command on demand with the `DEEPSEEK_API_KEY` secret; a full run costs about ¥1.5 with `deepseek-flash`.
@@ -339,6 +344,7 @@ The **Real-model eval** workflow runs the same command on demand with the `DEEPS
 - **Math Plot (C2):** three sliders, every one used in the expression, an `exp(…)·sin/cos(…)` shape.
 - **Three.js and widgets (C1, C3, C4):** `UI.three()` or `import("three")`, animation, lights, start/pause and parameter controls.
 - **Multi-turn edits (D2, D3):** a new column with every value correct and old values kept; bar → line, then a third series with exact values.
+- **Presentation routing (F, not yet in the recorded run):** 21 cases in pairs that keep the topic and change only the task (explain vs. drag a parameter, explain Three.js vs. a rotatable scene here, code to copy vs. a repo change, Markdown table vs. a searchable grid of the user's rows, a formula vs. a derivation with a plot between the steps, a closing Choice vs. a chart after each section), plus a patch that must leave a filtered table untouched, a text-only restriction after a 3D turn and live data without a source. `no-ui` failures are unnecessary UI, `needs-ui` failures missed UI. Details: [#58](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/issues/58).
 
 **Browser stage** (`scripts/eval/browser-stage.mjs`, 11 cases, each in a new conversation): plain text creates no card; chart legend, Q1–Q4 and local hover; table re-sorts without a model call; a form filled and submitted with real mouse and keyboard; Plot slider redraws; Three.js canvas drawn (pixel statistics) and draggable, with "Ask to fix" on failure; stop → preview → Regenerate; two in-place edits; Fast 3G; CDN blocked; 390 px phone width. Console errors are recorded per case, and every screenshot is reviewed by hand.
 

package/README.zh-CN.md

@@ -171,6 +171,8 @@ npx deepseek-harness-intelligent-ui doctor      # 只诊断，不做修改
 - **`ui_render_widget`**：组件目录画不出的东西，例如三维、模拟、动画、游戏、自定义工具。
 - **本地优先**：凡是能在浏览器里算出来的，都必须即时更新，不调用模型。
 
+这些规则为什么在与重写版决策顺序做真实模型 A/B 后被保留，以及路由如何评测，见 [docs/presentation-routing.md](docs/presentation-routing.md)。
+
 `/ui <需求>` 可强制以 UI 作答。
 
 | 工具 | 适用场景 | 渲染方式 |
@@ -254,8 +256,10 @@ npx deepseek-harness-intelligent-ui doctor      # 只诊断，不做修改
 <img src="https://raw.githubusercontent.com/dreamfarer-space/deepseek-harness-intelligent-ui/main/docs/assets/readme/security.zh.svg" alt="组件运行在不透明源沙箱 iframe 中并受严格 CSP 约束；访问宿主页面会被拦截，只有通过 token 校验的 postMessage 才能到达限流的 Bridge" width="100%">
 
 - **隔离：** `<iframe sandbox="allow-scripts allow-forms">` 配合 `srcdoc`，**不加** `allow-same-origin`。E2E 在浏览器中实测：`parent.document`、`localStorage`、`document.cookie` 均抛出 `SecurityError`，`fetch` 被拦截，`window.origin` 为 `"null"`。
-- **CSP：** `default-src 'none'`；脚本、样式、字体、图片只能来自 `cdnAllowlist`；未开启 `allowNetwork` 时 `connect-src 'none'`；`form-action 'none'`、`base-uri 'none'`。
-- **消息校验：** 只接受来自该 iframe 自身 `contentWindow`、带随机 token 且格式合法的消息；伪造消息会被丢弃（E2E 实测）。
+- **CSP：** `default-src 'none'`；脚本、样式、字体、图片只能来自 `cdnAllowlist`；未开启 `allowNetwork` 时 `connect-src 'none'`；`form-action 'none'`、`base-uri 'none'`。逐通道实测矩阵见 [docs/widget-csp-audit.md](docs/widget-csp-audit.md)，并可运行 `npm run e2e:security`。
+- **`allowNetwork=false` 能挡与不能挡什么：** 它能拦截 `fetch`、XHR、WebSocket、`EventSource`、`sendBeacon` 和 `<a ping>`；但**不能**阻止 `<img>`、`<script src>`、`<link rel=stylesheet>`、`@font-face`、`<video>` 向**允许名单内**的来源发起请求，而本实现将 `cdnAllowlist` 转为仅含来源的 CSP 条目，不限制资源路径——允许名单内的 CDN 仍可能在资源 URL 中看到组件数据。要阻止外部资源请求，请设 `cdnAllowlist: []`（离线包已如此）；它仍不能阻止组件自身导航，以及在首次导航 URL 中携带数据。
+- **导航：** 绑定原文档的 MessageChannel 可阻止替换文档接收宿主数据。后续 iframe `load` 会移除外部页面；未完成的响应可延迟这一步界面移除。首次自身导航请求仍可能发生。iframe sandbox 已阻止顶层导航、`window.open` 和文件下载。跨来源下载链接仍可能先触发自身导航请求，再由浏览器阻止保存响应。
+- **消息校验：** 首次握手校验 iframe 来源与每个文档的新 token，再接收其转移的端口。后续消息仅走该端口并校验 token/格式；窗口 RPC 与旧端口均被拒绝（E2E 实测）。
 - **sendPrompt 限制：** 挂载后 1 秒内拒绝，2 秒内最多一次，长度受 `sendPromptMaxChars` 限制，以普通用户消息排队发送。
 - **openLink：** 只允许 https，可用 `openLinkAllowedOrigins` 进一步限制。
 - **组件目录：** 文本作为 React 元素渲染，不作为 HTML；`Plot` 表达式只能调用固定的数学函数，无法使用 `eval`、`Function` 或访问属性；新开会话和宿主动作都要求用户原生点击。
@@ -277,7 +281,7 @@ npx deepseek-harness-intelligent-ui doctor      # 只诊断，不做修改
 | `enableWidget` | `true` | 注册 `ui_render_widget` |
 | `injectGuidance` | `true` | 注入“何时使用 UI”的系统提示 |
 | `cdnAllowlist` | esm.sh、jsdelivr、unpkg | 组件（以及 Math/Mermaid）可加载资源的来源 |
-| `allowNetwork` | `false` | 允许组件对上述来源发起 `fetch` 和 WebSocket |
+| `allowNetwork` | `false` | 允许组件对上述来源发起 `fetch` 和 WebSocket。关闭时资源类元素仍可访问这些来源；要阻止外部资源请求（仍可发生自身导航）请把 `cdnAllowlist` 设为空 |
 | `maxWidgetHeight` | `2400` | iframe 最大高度（px） |
 | `maxWidgetBytes` | `400000` | 超过此大小的组件会被拒绝 |
 | `sendPromptMaxChars` | `4000` | 组件、按钮、表单、选择题单条消息的最大长度 |
@@ -316,11 +320,12 @@ npm run eval:real -- --record && npm run eval:demo  # 录制并剪辑演示视
 
 | 选项 | 作用 |
 | --- | --- |
-| `--stage mock,api,browser` | 只运行部分层（默认 `all`）；`A`/`B`/`C`/`D` 选择 API 类别 |
+| `--stage mock,api,browser` | 只运行部分层（默认 `all`）；`A`–`F` 选择 API 类别（`F`：成对的呈现方式决策用例，[#58](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/issues/58)） |
 | `--only B3,R-chart` | 只运行这些用例 |
 | `--keep-other-layers` | 保留本次没有运行的层或用例，各自保留原来的日期 |
 | `--append` | 配合 `--only`：把重跑结果作为额外样本加入，绝不用一次侥幸成功覆盖失败 |
 | `--rescore` | 用当前断言、基于已保存的对话记录重新判定上一次 API 运行，不调用模型 |
+| `--report <file.json>` | 把报告写到其他文件，例如每个提示词版本一份，用来对比基线与候选提示词 |
 | `DEEPSEEK_EVAL_MODEL` | 模型 id（默认 `deepseek-flash`） |
 
 GitHub 上的 **Real-model eval** 工作流可按需手动触发，使用仓库密钥 `DEEPSEEK_API_KEY` 运行同一条命令；用 `deepseek-flash` 完整跑一遍约花费 ¥1.5。
@@ -339,6 +344,7 @@ GitHub 上的 **Real-model eval** 工作流可按需手动触发，使用仓库
 - **数学 Plot（C2）：** 三个滑块且都出现在表达式中，函数形状为 `exp(…)·sin/cos(…)`。
 - **Three.js 与组件（C1、C3、C4）：** 使用 `UI.three()` 或 `import("three")`，有动画、光照、启动/暂停和参数控件。
 - **多轮修改（D2、D3）：** 新增一列且每个值都正确、旧值保留；柱状 → 折线，再按准确数值加入第三个系列。
+- **呈现方式决策（F，尚未进入已记录的运行）：** 21 个成对用例，主题不变、只改变任务（解释 vs. 拖动参数、解释 Three.js vs. 对话内可旋转场景、可复制代码 vs. 修改仓库、Markdown 表格 vs. 用户数据的可搜索表格、写公式 vs. 推导中插入曲线、结尾 Choice vs. 每节一张图），另有必须保持已筛选表格不变的 patch、3D 回合之后的纯文本限制，以及没有数据源的实时数据请求。`no-ui` 失败表示不必要的 UI，`needs-ui` 失败表示漏掉了 UI。详见 [#58](https://github.com/dreamfarer-space/deepseek-harness-intelligent-ui/issues/58)。
 
 **浏览器阶段**（`scripts/eval/browser-stage.mjs`，11 个用例，每个都在新会话中）：纯文本不创建卡片；图表有图例、Q1–Q4 和本地悬停；数据表重新排序不调用模型；用真实鼠标键盘填写并提交表单；Plot 滑块重绘；Three.js 画布确有内容（像素统计）且可拖动，失败时尝试「让模型修复」；停止 → 预览 → 重新生成；两轮原位修改；Fast 3G；屏蔽 CDN；390 px 手机宽度。每个用例都记录控制台错误，每张截图都经过人工复核。