a2ui-render-in-dsh 0.1.1 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,13 +2,13 @@
2
2
 
3
3
  English | [中文](https://github.com/baihui-ai/a2ui-render-in-dsh/blob/main/README.zh.md)
4
4
 
5
- **A2UI interactive cards for the dsh web UI**: the agent adaptively renders **interactive, visual UI cards** right inside the conversation — quizzes, forms, dropdowns, product cards, ECharts charts, math function plots, Mermaid flowcharts/mind maps, KaTeX formulas, and step-by-step algorithm animations. User interactions flow back to the agent as plain-language messages, closing the loop.
5
+ **A2UI interactive cards for the dsh web UI**: the agent adaptively renders **interactive, visual UI cards** right inside the conversation — quizzes, forms, multi-step wizards, dropdowns, date pickers, product cards, sortable/filterable tables, ECharts charts, China map choropleths, math function plots, Mermaid flowcharts/mind maps, KaTeX formulas, Markdown long-form, image uploads, signature pads, and step-by-step algorithm animations. User interactions flow back to the agent as plain-language messages (images included), cards **stream in progressively** and can be **updated in place**, and everything visible is one click away from the clipboard.
6
6
 
7
7
  The UI protocol is [A2UI v0.9](https://github.com/google/A2UI) (a declarative Agent-to-UI protocol); rendering is powered by Ant Design X's official implementation, [`@ant-design/x-card`](https://www.npmjs.com/package/@ant-design/x-card).
8
8
 
9
- 📸 **[Feature showcase with GIFs → DEMO.md](https://github.com/baihui-ai/a2ui-render-in-dsh/blob/main/DEMO.md)**
9
+ 📸 **[Feature showcase with GIFs → DEMO.md](https://github.com/baihui-ai/a2ui-render-in-dsh/blob/main/DEMO.md)** · 🗺️ **[Scenario × component map](https://github.com/baihui-ai/a2ui-render-in-dsh/blob/main/docs/SCENARIOS.md)**
10
10
 
11
- ![Quiz interaction](https://raw.githubusercontent.com/baihui-ai/a2ui-render-in-dsh/main/docs/demo-quiz.gif)
11
+ ![Quiz interaction](https://raw.githubusercontent.com/baihui-ai/a2ui-render-in-dsh/main/docs/demo-study.gif)
12
12
 
13
13
  ---
14
14
 
@@ -18,21 +18,26 @@ The UI protocol is [A2UI v0.9](https://github.com/google/A2UI) (a declarative Ag
18
18
 
19
19
  ```
20
20
  a2ui-render-in-dsh (one npm package, a dsh bundle)
21
- ├─ Host half lib/index.js cordis plugin registering two agent tools
22
- │ ├─ a2ui_render renders a card (slim ~140-token description, always visible)
21
+ ├─ Host half lib/index.js cordis plugin registering three agent tools
22
+ │ ├─ a2ui_render renders a card (slim ~220-token description, always visible)
23
+ │ ├─ a2ui_update updates an already-rendered card in place (progress,
24
+ │ │ long tasks, live dashboards) via its surfaceId
23
25
  │ └─ a2ui_catalog returns the full component catalog & authoring
24
- │ rules (~975 tokens, loaded on demand, once)
25
- └─ Client half lib/client.js browser bundle registering the a2ui_render toolview
26
+ │ rules (loaded on demand, once per conversation)
27
+ └─ Client half lib/client.js browser bundle registering the toolviews
26
28
  ├─ x-card engine (A2UI command stream, data binding, action resolution)
27
- ├─ component catalog implementations (18 components, themed via --dsw-* tokens)
28
- └─ ECharts 6 / Mermaid 11 / KaTeX (fonts inlined) all bundled — zero external requests
29
+ ├─ component catalog implementations (44 components, themed via --dsw-* tokens)
30
+ ├─ streaming renderer (truncated-JSON repair → cards appear while the model types)
31
+ ├─ session navigator (tasks + full-chat locating, drafts, transcript rebuild)
32
+ └─ ECharts 6 / Mermaid 11 / KaTeX (fonts inlined) / China geoJSON all bundled
33
+ — zero external requests
29
34
  ```
30
35
 
31
36
  ### Key design decisions
32
37
 
33
- **1. Adaptive rendering, decided by the model.** Whether to use UI is entirely the model's call: the tool contract says "only when a card clearly beats prose". Verified both ways — "quiz me" triggers a card, "explain X" stays plain text.
38
+ **1. Proactive rendering, decided by the model — anchored on UI's purposes.** The tool contract frames the judgment as: would a card serve any of UI's four purposes better than prose? **Act** (the user must answer/choose/fill/adjust — render a form), **browse** (the user wants to see or scan data — statistics, rankings, distributions, trends get a chart/table/map, and uncertain data never cancels the card: chart the best known, label the period, caveat in prose), **understand** (structure or notation aids comprehension — math, code, flows, stepwise processes), or **feedback** (long multi-step work gets a progress card advanced via `a2ui_update`). None of the four → prose. Cards always **pair with prose**: the takeaway lives in 1–3 sentences of normal text, the structured content in the card, with no duplication. Grounded in HCI research (Norman's gulfs, the keyhole effect, external cognition) and verified with keyword-free prompts across all purposes plus prose negatives.
34
39
 
35
- **2. Skill-style context design (catalog on demand).** The full component catalog is NOT inlined in the tool description; it lives in a second tool, `a2ui_catalog`. Always-visible cost drops from ~1,200 to ~211 tokens (−82%). The model calls the catalog once before its first card in a conversation and reuses it for later cards; conversations that never draw a card pay nothing. The host validates component names and errors with a "call a2ui_catalog" hint, so the model can't silently guess wrong.
40
+ **2. Skill-style context design (catalog on demand).** The full component catalog is NOT inlined in the tool description; it lives in a second tool, `a2ui_catalog`. The always-visible tool description stays at ~350 tokens for 44 components (the on-demand catalog is ~2.5k, paid once per card-using conversation). The model calls the catalog once before its first card in a conversation and reuses it for later cards; conversations that never draw a card pay nothing. The host validates component names and errors with a "call a2ui_catalog" hint, so the model can't silently guess wrong.
36
41
 
37
42
  **3. Non-blocking answers over the native message path.** Submissions need no custom server channel: the client sends the submission through dsh's own `session.prompt` RPC as an ordinary user message. Messages are **plain language** (button label + chosen values, multi-line for forms) — readable for humans, parseable for the model, no raw JSON in the conversation:
38
43
 
@@ -46,16 +51,30 @@ Tracks: Backend, Data Analysis
46
51
 
47
52
  **5. Let the model do only what it's good at.** Function plotting: the model writes an expression (`tan(x)`); sampling and asymptote breaking are done by a built-in **safe expression evaluator** (whitelist shunting-yard parser, no eval, injection is rejected) — hand-enumerating data points would inevitably fail. Algorithm animations: the model simulates the algorithm into per-step frames (an LLM strength); playback, transitions, and controls belong to the component.
48
53
 
49
- **6. Fully self-contained display stack.** ECharts, Mermaid, and KaTeX (woff2 fonts as data URIs) are all bundled (~5MB, served locally, loaded once) — no CDN, works offline; light/dark theme follows the page automatically.
54
+ **6. Fully self-contained display stack.** ECharts, Mermaid, KaTeX (woff2 fonts as data URIs), and the China province geoJSON are all bundled (~5.5MB, served locally, loaded once) — no CDN, works offline; light/dark theme follows the page automatically.
55
+
56
+ **7. Live cards: streaming in, updating in place.** Cards render progressively while the model is still emitting JSON (a tolerant parser repairs the truncated stream and mounts complete components early), so a big dashboard appears piece by piece instead of after a long pause. And a rendered card is not frozen: `a2ui_update` addresses it by `surfaceId` to patch components or data in place — progress bars that actually move, task cards that fill in results, dashboards that refresh. Updates persist and replay after a page reload.
57
+
58
+ **8. Answers beyond text.** `Upload` (photos) and `Signature` (hand-drawn canvas) send images back through dsh's native prompt channel as real image parts — the model sees the picture, not a placeholder. `Suggestions` renders tappable follow-up chips that send themselves as the next user message. Voice recording is deliberately excluded: dsh's prompt channel carries text + images only.
59
+
60
+ **9. State that outlives the browser.** Drafts auto-save as you type (reload-safe); submissions lock cards with persisted records; and when even localStorage is gone, the client reconstructs submitted-state from the session transcript itself — render calls carry ids, submission messages match back by button label, errored calls excluded. The session navigator surfaces all of it: to-submit/submitted task groups and a full, clickable map of every user message.
50
61
 
51
62
  ## Highlights
52
63
 
53
64
  - 🎯 **Adaptive**: the model chooses text vs. card; verified reliable in both directions
54
- - 🪶 **Context-friendly**: skill-style catalog design, ~211 tokens always-visible
55
- - 💬 **Elegant answers**: plain-language submissions, not raw JSON strings
56
- - 🔒 **Submit-once locking**: forms can't double-submit, records persist; query buttons unaffected
57
- - 📊 **Full visualization family**: ECharts charts/dashboards, function plots, all Mermaid diagram types, KaTeX formulas (matrices are forced into formula rendering), video
58
- - 🎬 **Algorithm animations**: array/bars and grid/matrix forms, auto-detected; auto-plays once per card (component remounts never replay), ↻ manual replay, stepping, progress bar, legend
65
+ - 🪶 **Context-friendly**: skill-style catalog design, ~350 tokens always-visible for 44 components
66
+ - ✅ **Form validation**: `required: true` on any input blocks submission and highlights what's missing
67
+ - 🧭 **Session navigator**: a right-edge drawer with Tasks (to-submit / submitted groups, fill counts, previews, dismissable) and All (every user message in full, click-to-locate with auto "load earlier", live-refreshing); drafts survive reloads and submitted-state survives cache clears (rebuilt from the transcript)
68
+ - 🗜️ **Upload compression**: photos are downscaled client-side (≤1568px, JPEG) before flowing into the prompt
69
+ - 💬 **Elegant answers**: plain-language submissions, not raw JSON strings; photos & signatures return as real images
70
+ - 🔒 **Submit-once locking**: forms can't double-submit, records persist, a "refill" button reopens them; query buttons unaffected
71
+ - ⚡ **Streaming render**: cards appear progressively while the model is still writing the JSON
72
+ - 🔄 **In-place updates**: `a2ui_update` patches a live card by surfaceId — moving progress bars, task cards that finish themselves, refreshing dashboards; survives page reloads
73
+ - 📊 **Full visualization family**: ECharts charts/dashboards, function plots, China map choropleth, all Mermaid diagram types, KaTeX formulas (matrices are forced into formula rendering), image compare slider, video
74
+ - 🎬 **Algorithm animations**: array/bars, grid/matrix, and graph/tree forms, auto-detected; auto-plays once per card (component remounts never replay), ↻ manual replay, stepping, progress bar, legend
75
+ - 🧾 **Interactive tables**: click-to-sort (numeric-aware), filter box, pagination, copy as TSV, CSV export, editable-table input
76
+ - 🧭 **Rich answer kit**: multi-step Wizard, Calendar date picking, drag-free RankList ordering, Suggestions follow-up chips, Upload, Signature
77
+ - 📋 **Quick copy everywhere**: Stat tiles click-copy, tables copy/export, CodeBlock copy, formulas copy their LaTeX, charts download as PNG, Markdown copies its source
59
78
  - ⛶ **Fullscreen zoom**: mind maps/flowcharts/charts/images go fullscreen with fit-to-viewport, wheel zoom + drag pan
60
79
  - 🧱 **Multi-column layout**: Grid for product comparisons and chart dashboards
61
80
  - 🎛️ **Complete input states**: dropdown single/multi select, preselection (dataModel seeds), component- and option-level disabling
@@ -84,21 +103,48 @@ Tracks: Backend, Data Analysis
84
103
  | `Card` | `children, title?` | Bordered group |
85
104
  | `List` | `children, direction?` | List container |
86
105
  | `Divider` | — | Separator |
87
- | `Text` | `text, variant?: h1\|h2\|h3\|body\|caption\|strong` | Text |
106
+ | `Text` | `text, variant?: h1\|h2\|h3\|body\|caption\|strong` | Text; fenced ``` content auto-upgrades to a formatted code block; inline `` `code` `` renders as code chips |
107
+ | `Markdown` | `text` | **Rich long-form**: headings, bold/italic, links, lists, quotes, fenced code, `$...$` math; copy-source button |
88
108
  | `Image` | `url, alt?, width?, height?` | Images (incl. GIF), built-in fullscreen zoom |
89
109
  | `Tag` | `text, color?: blue\|green\|red\|orange\|gray` | Tag/badge |
90
110
  | `Math` | `tex, block?` | **KaTeX** formulas (fonts inlined, zero external requests); matrices/vectors must use this |
91
111
  | `Mermaid` | `code, caption?` | **Mermaid 11**: flowchart/mindmap/sequence/gantt etc., built-in fullscreen zoom |
92
- | `Chart` | `option, height?, functions?, xMin?, xMax?, samples?, yClip?` | **ECharts 6**: data mode (option verbatim) + function-plot mode (expressions sampled automatically, asymptote breaks), built-in fullscreen |
112
+ | `Chart` | `option, height?, functions?, params?, xMin?, xMax?, samples?, yClip?` | **ECharts 6**: data mode (option verbatim) + function-plot mode (expressions sampled automatically, asymptote breaks) + `params` live-bound constants (Slider→curve); fullscreen + PNG download |
113
+ | `Map` | `data, title?, unit?, height?` | **China choropleth**: province-level distribution (sales/users by region), geoJSON bundled |
93
114
  | `Video` | `url, poster?, loop?, muted?, autoplay?` | HTML5 video (mp4/webm) |
94
- | `Anim` | `frames, interval?, height?, autoplay?, labels?` | **Algorithm animation**: bars and grid/matrix forms auto-detected; auto-plays once per card per tab, ↻ replay / pause / step / reset / progress / legend |
115
+ | `ImageCompare` | `before, after` | Drag-divider before/after image comparison |
116
+ | `Anim` | `frames, interval?, height?, autoplay?, labels?` | **Algorithm animation**: bars, grid/matrix, and graph/tree (BFS/DFS, auto circle layout) forms auto-detected; auto-plays once per card per tab, ↻ replay / pause / step / reset / progress / legend |
95
117
  | `Button` | `label, variant?, submit?, action: {event: {name, context?}}` | Sends the submission; `submit` explicitly controls card locking |
96
118
  | `MultipleChoice` | `options, bind, maxAllowedSelections?, disabled?` | Flat multi/single select (`maxAllowedSelections: 1` = single), per-option disable |
97
- | `Select` | `options, bind, label?, placeholder?, multiple?, maxAllowedSelections?, disabled?` | **Dropdown**: single stores a value, multi stores an array; option descriptions/disabling |
119
+ | `Select` | `options, bind, label?, placeholder?, multiple?, maxAllowedSelections?, disabled?` | **Dropdown**: single stores a value, multi stores an array; auto search box on long lists |
98
120
  | `CheckBox` | `label, bind, disabled?` | Boolean toggle |
121
+ | `Slider` | `bind, label?, min?, max?, step?, unit?` | Numeric slider; pairs with Chart `params` for live parameter exploration |
122
+ | `Rate` | `bind, label?, max?` | Star rating |
123
+ | `Calc` | `expr, inputs, out, digits?` | Invisible derived value: live-recomputed expression written back to the data model (calculator engine) |
124
+ | `When` | `value, equals?/includes?/notEmpty?, children` | Conditional container: reveal follow-up fields on selection |
125
+ | `Tabs` | `tabs, children, bind?` | Tab switcher: dataset switching / content grouping |
126
+ | `Table` | `columns, rows, caption?, sortable?, filter?, pageSize?` | **Interactive tables**: click-to-sort (numeric-aware), filter box, pagination, copy TSV / CSV export; dictionary binding switches datasets |
127
+ | `Stat` | `label, value, unit?, trend?, hint?` | KPI tile, click to copy the value; combine in a Grid for metric overviews |
128
+ | `Steps` | `items` | Step list (done/current/pending) |
129
+ | `Progress` | `value, max?, label?` | Progress bar |
130
+ | `Timeline` | `items` | Timeline (history / event review) |
131
+ | `CodeBlock` | `code, language?, title?` | Code with line numbers, light highlighting, copy button — quiz stems included (never inline code in Text) |
132
+ | `Icon` | `name, size?, color?` | 32 built-in stroke icons |
133
+ | `Audio` | `url, title?` | Audio player |
134
+ | `Flashcard` | `front, back` | Tap-to-flip card (vocabulary / recall) |
135
+ | `Countdown` | `to?/seconds?, label?` | Live countdown |
99
136
  | `TextField` | `label?, placeholder?, multiline?, bind, disabled?` | Text input |
137
+ | `Wizard` | `steps, children, submitLabel?` | **Multi-step form**: one pane per step, prev/next + progress built in, final submit sends all collected fields |
138
+ | `Calendar` | `bind, label?, min?, max?, range?` | Month-view date picker; `range: true` picks a start + end date |
139
+ | `RankList` | `items, bind, label?` | Drag (or tap ↑↓) to reorder by priority; submits the ordered list |
140
+ | `EditableTable` | `columns, rows, bind, label?` | User edits cells inline; the whole grid submits |
141
+ | `Upload` | `bind?, label?, max?` | Image picker — chosen photos are sent back to the model as **real images** |
142
+ | `Signature` | `label?` | Handwritten signature pad; the drawing returns as an image |
143
+ | `Suggestions` | `items` | Tappable follow-up chips below an answer; tapping sends that question as the next user message |
100
144
 
101
- Common to inputs: **preselect** by seeding `dataModel` at the bind path; **disable** via component-level `disabled: true` or per-option `disabled`. Data binding: `bind` is a write path WITHOUT a leading slash; display props read live values with `{"path": "/x"}` (WITH a slash).
145
+ **Inline math**: every text position (Text, option labels, table cells, steps, flashcards, animation captions) may embed KaTeX with `$...$` — math-quiz OPTIONS can be formulas. **Reactive bindings**: inputs write the data model and every `{"path"}` binding updates live — slider→curve (Chart `params`), choice→follow-up (When), input→computed result (Calc→Stat), switcher→dataset (Tabs / Table dictionary binding).
146
+
147
+ Common to inputs: **preselect** by seeding `dataModel` at the bind path; **disable** via component-level `disabled: true` or per-option `disabled`; **require** via `required: true` (submission blocks and highlights until filled). `a2ui_update` dataModel changes re-sync bound inputs in place. Data binding: `bind` is a write path WITHOUT a leading slash; display props read live values with `{"path": "/x"}` (WITH a slash).
102
148
 
103
149
  ## Installation
104
150
 
@@ -107,9 +153,24 @@ Common to inputs: **preselect** by seeding `dataModel` at the bind path; **disab
107
153
  | Requirement | Notes |
108
154
  |---|---|
109
155
  | dsh | `@deepseek-ai/dsh` ≥ 0.1.1-rc.1 with an initialized web profile (run `dsh web` once before installing) |
156
+
157
+ Development: `npm test` runs the jsdom interaction suite (host validation + full component/interaction coverage, ~70 assertions).
110
158
  | Node.js | ≥ 20 (with npm) |
111
159
 
112
- ### Option A · From source (recommended for now)
160
+ ### Option A · From npm (recommended)
161
+
162
+ ```sh
163
+ dsh plugin --profile web add a2ui-render-in-dsh
164
+ dsh web
165
+ ```
166
+
167
+ Zero runtime dependencies — two commands and you're done. If your npm mirror hasn't synced the latest version yet, point at the official registry explicitly:
168
+
169
+ ```sh
170
+ dsh plugin --profile web add a2ui-render-in-dsh --registry https://registry.npmjs.org
171
+ ```
172
+
173
+ ### Option B · From source (for developers / hacking on the plugin)
113
174
 
114
175
  ```sh
115
176
  # 1. Clone and build
@@ -117,27 +178,17 @@ git clone https://github.com/baihui-ai/a2ui-render-in-dsh.git
117
178
  cd a2ui-render-in-dsh
118
179
  npm install
119
180
  npm run build
120
- # Build emits two files: lib/index.js (host plugin) + lib/client.js (browser bundle).
181
+ # Build emits lib/index.js (host plugin) + lib/client.js (browser bundle).
121
182
  # lib/ is not committed — you MUST build after cloning or dsh won't find the entry.
122
183
 
123
- # 2. Install into dsh's web profile (link mode)
184
+ # 2. Install into dsh's web profile in link mode
124
185
  dsh plugin --profile web add link:$(pwd)
125
- # Equivalent to pnpm add link:<path> inside the profile; it also appends
126
- # a2ui-render-in-dsh to the profile's dsh.profile.bundles automatically.
186
+ # After code changes: just npm run build + restart dsh web, no reinstall.
127
187
 
128
188
  # 3. Start / restart dsh web
129
189
  dsh web
130
190
  ```
131
191
 
132
- Why link mode: upgrading is just `git pull && npm run build` plus a dsh web restart — no reinstall.
133
-
134
- ### Option B · From npm (once the package is published)
135
-
136
- ```sh
137
- dsh plugin --profile web add a2ui-render-in-dsh
138
- dsh web
139
- ```
140
-
141
192
  ### Verify in three steps
142
193
 
143
194
  ```sh
@@ -156,7 +207,10 @@ curl -s http://127.0.0.1:<port>/ | grep -o "a2ui-render-in-dsh/client.js[^\"]*"
156
207
  ### Upgrade & uninstall
157
208
 
158
209
  ```sh
159
- # upgrade (link install): rebuild after pulling, then restart dsh web
210
+ # upgrade (npm install)
211
+ dsh plugin --profile web update a2ui-render-in-dsh
212
+
213
+ # upgrade (source link install): rebuild after pulling, then restart dsh web
160
214
  git pull && npm run build
161
215
 
162
216
  # uninstall: removes the dependency and the bundles entry, then restart dsh web
package/README.zh.md CHANGED
@@ -2,13 +2,13 @@
2
2
 
3
3
  [English](https://github.com/baihui-ai/a2ui-render-in-dsh/blob/main/README.md) | 中文
4
4
 
5
- **dsh web 的 A2UI 交互卡片插件**:让 Agent 在对话流里自适应地渲染**可交互、可视化的 UI 卡片**——选择题、表单、下拉、商品卡、ECharts 图表、数学函数绘图、Mermaid 流程图/思维导图、KaTeX 公式、算法过程动画——用户的点击/勾选/输入以自然语言消息回传给 Agent,形成完整的交互闭环。
5
+ **dsh web 的 A2UI 交互卡片插件**:让 Agent 在对话流里自适应地渲染**可交互、可视化的 UI 卡片**——选择题、表单、分步向导、下拉、日历选日期、商品卡、可排序/筛选的表格、ECharts 图表、中国地图分布、数学函数绘图、Mermaid 流程图/思维导图、KaTeX 公式、Markdown 长文、图片上传、手写签名、算法过程动画——用户的点击/勾选/输入以自然语言消息回传给 Agent(图片也能回传),卡片**边生成边渲染**、还能**原地更新**,所见内容一键复制。
6
6
 
7
7
  UI 协议基于 [A2UI v0.9](https://github.com/google/A2UI)(Agent-to-UI 声明式界面协议),渲染引擎使用 Ant Design X 官方实现 [`@ant-design/x-card`](https://www.npmjs.com/package/@ant-design/x-card)。
8
8
 
9
- 📸 **[功能示例(含动图)→ DEMO.md](https://github.com/baihui-ai/a2ui-render-in-dsh/blob/main/DEMO.zh.md)**
9
+ 📸 **[功能示例(含动图)→ DEMO.md](https://github.com/baihui-ai/a2ui-render-in-dsh/blob/main/DEMO.zh.md)** · 🗺️ **[场景 × 组件映射](https://github.com/baihui-ai/a2ui-render-in-dsh/blob/main/docs/SCENARIOS.zh.md)**
10
10
 
11
- ![做题交互](https://raw.githubusercontent.com/baihui-ai/a2ui-render-in-dsh/main/docs/demo-quiz.gif)
11
+ ![做题交互](https://raw.githubusercontent.com/baihui-ai/a2ui-render-in-dsh/main/docs/demo-study.gif)
12
12
 
13
13
  ---
14
14
 
@@ -18,20 +18,25 @@ UI 协议基于 [A2UI v0.9](https://github.com/google/A2UI)(Agent-to-UI 声明
18
18
 
19
19
  ```
20
20
  a2ui-render-in-dsh (一个 npm 包,dsh bundle)
21
- ├─ 宿主端 lib/index.js cordis 插件,注册两个 Agent 工具
22
- │ ├─ a2ui_render 渲染卡片(描述仅 ~140 token,常驻)
23
- │ └─ a2ui_catalog 返回完整组件目录与写卡规则(~975 token,按需一次)
24
- └─ 客户端 lib/client.js 浏览器 bundle,注册 a2ui_render 的专属 toolview
21
+ ├─ 宿主端 lib/index.js cordis 插件,注册三个 Agent 工具
22
+ │ ├─ a2ui_render 渲染卡片(描述仅 ~220 token,常驻)
23
+ │ ├─ a2ui_update 按 surfaceId 原地更新已渲染的卡片
24
+ │ │ (进度推进、长任务回填、看板刷新)
25
+ │ └─ a2ui_catalog 返回完整组件目录与写卡规则(按需加载,每会话一次)
26
+ └─ 客户端 lib/client.js 浏览器 bundle,注册各工具的专属 toolview
25
27
  ├─ x-card 引擎(A2UI 命令流、数据绑定、action 解析)
26
- ├─ 组件目录实现(18 个组件,跟随 --dsw-* 主题令牌)
27
- └─ ECharts 6 / Mermaid 11 / KaTeX(字体内联)全部打入 bundle,零外部请求
28
+ ├─ 组件目录实现(44 个组件,跟随 --dsw-* 主题令牌)
29
+ ├─ 流式渲染器(截断 JSON 修复 → 模型边写卡片边出现)
30
+ ├─ 会话导航(任务分组 + 全会话定位 + 草稿 + 会话记录重建)
31
+ └─ ECharts 6 / Mermaid 11 / KaTeX(字体内联)/ 中国省份 geoJSON
32
+ 全部打入 bundle,零外部请求
28
33
  ```
29
34
 
30
35
  ### 关键设计决策
31
36
 
32
- **1. 自适应渲染,由模型判断。** 是否用 UI 完全交给模型:工具契约写明"仅当卡片明显优于纯文本时调用"。实测:「出题考我」触发卡片,「解释一下 X」保持纯文本。
37
+ **1. 主动式渲染,由模型判断——锚定 UI 的目的。** 工具契约把判断框定为:卡片能否在 UI 的四个目的之一上胜过纯文本?**方便操作**(用户要回答/选择/填写/调节——渲染表单)、**方便浏览**(用户想看/扫数据——统计、排名、分布、趋势出图表/表格/地图,数据不确定也不取消卡片:画已知最好的数据、标注时间口径、正文说明局限)、**增强理解**(结构或记号帮助理解——数学、代码、流程、分步过程)、**状态反馈**(长/多步任务先出进度卡,`a2ui_update` 原地推进)。四者都不沾 → 纯文本。卡片始终**与文案搭配**:结论/看点用 1–3 句正文说,结构化内容进卡片,两边不重复。判断框架有 HCI 理论依据(Norman 双鸿沟、钥匙孔效应、外部认知),并用无关键词的自然语言提示词对各目的 + 纯文本反例实测。
33
38
 
34
- **2. skill 式上下文设计(目录按需加载)。** 完整组件目录不内联在工具描述里,而是放进 `a2ui_catalog` 工具:常驻上下文从 ~1200 token 降到 ~211 token(-82%);模型首次画卡前调用一次目录,同会话后续卡片直接复用;不画卡的会话零目录开销。服务端校验未知组件名并报错引导查目录,防止模型跳过目录瞎猜。
39
+ **2. skill 式上下文设计(目录按需加载)。** 完整组件目录不内联在工具描述里,而是放进 `a2ui_catalog` 工具:44 个组件的常驻工具描述约 ~350 token(按需目录约 2.5k,每个用卡会话只付一次);模型首次画卡前调用一次目录,同会话后续卡片直接复用;不画卡的会话零目录开销。服务端校验未知组件名并报错引导查目录,防止模型跳过目录瞎猜。
35
40
 
36
41
  **3. 非阻塞回传,复用原生消息通路。** 卡片提交不需要自定义 server 通道:客户端通过 dsh 自身的 `session.prompt` RPC 把提交内容作为普通用户消息发回会话。消息是**自然语言**(按钮文案 + 所选内容,多字段换行列出),对人可读、对模型可解析,不污染对话观感:
37
42
 
@@ -45,16 +50,30 @@ a2ui-render-in-dsh (一个 npm 包,dsh bundle)
45
50
 
46
51
  **5. 模型只做擅长的事。** 数学函数绘图:模型只写表达式(`tan(x)`),采样、渐近线断线由内置**安全表达式求值器**完成(白名单 shunting-yard 解析,非 eval,注入即拒绝)——让模型手工枚举几百个数据点必然画错。算法动画:模型模拟算法输出逐帧状态(这是 LLM 强项),播放、过渡、控件由组件完成。
47
52
 
48
- **6. 展示组件全部自包含。** ECharts、Mermaid、KaTeX(含 woff2 字体 data-URI)全部打进 bundle(~5MB,本地服务一次加载),无 CDN 依赖、可离线;明暗主题按页面背景亮度自动跟随。
53
+ **6. 展示组件全部自包含。** ECharts、Mermaid、KaTeX(含 woff2 字体 data-URI)、中国省份 geoJSON 全部打进 bundle(~5.5MB,本地服务一次加载),无 CDN 依赖、可离线;明暗主题按页面背景亮度自动跟随。
54
+
55
+ **7. 活的卡片:流式渲染 + 原地更新。** 模型还在输出 JSON 时卡片就开始渲染(容错解析器修复截断的流,完整的组件提前挂载),大卡片逐块出现而不是长时间空白后一次性弹出。渲染完的卡片也不是死的:`a2ui_update` 按 `surfaceId` 原地替换组件或数据——进度条真的会动、任务卡自己填上结果、看板随时刷新;更新持久化,刷新页面后自动重放。
56
+
57
+ **8. 回答不止于文字。** `Upload`(拍照/截图)和 `Signature`(手写画板)走 dsh 原生消息通路把**真实图片**发回给模型——模型看到的是图,不是占位符。`Suggestions` 渲染可点的追问 chips,点一下即作为下一条用户消息发出。录音回传刻意不做:dsh 的 prompt 通道只支持文本 + 图片。
58
+
59
+ **9. 比浏览器更长寿的状态。** 草稿边填边存(刷新不丢);提交锁卡并持久化记录;即使 localStorage 全清,客户端也能从**会话记录本身**重建已提交状态——渲染调用带 id、提交消息按按钮文案回配、失败调用被排除。会话导航把这一切摆到明面:待提交/已提交任务分组 + 每条用户消息的可点击地图。
49
60
 
50
61
  ## 亮点
51
62
 
52
63
  - 🎯 **自适应**:模型自行判断"文本还是卡片",双向实测可靠
53
- - 🪶 **上下文友好**:skill 式目录设计,常驻开销 ~211 token
54
- - 💬 **优雅回传**:自然语言提交消息,非 JSON 裸串
55
- - 🔒 **提交即锁定**:表单防重复提交,记录持久化,查询按钮不受影响
56
- - 📊 **可视化全家桶**:ECharts 图表/仪表盘、函数绘图、Mermaid 全图型、KaTeX 公式(矩阵强制公式化渲染)、视频
57
- - 🎬 **算法动画**:数组/柱状 + 网格/矩阵双形态,自动识别;每卡只自动播一遍(组件重挂载不重播),↻ 手动重播、单步、进度条、图例
64
+ - 🪶 **上下文友好**:skill 式目录设计,44 个组件常驻开销约 ~350 token
65
+ - ✅ **表单校验**:任意输入组件可设 `required: true`——提交被拦截并高亮缺失项
66
+ - 🧭 **会话导航**:右缘抽屉——「任务」页签分组待提交/已提交任务(已填计数、内容预览、可标记无需填写),「全部」页签完整列出每条用户消息并点击定位(自动加载更早、实时同步);草稿刷新不丢,已提交状态清缓存后仍可从会话记录重建
67
+ - 🗜️ **上传压缩**:照片在客户端先压到 ≤1568px JPEG 再进消息,手机原图不再撑爆对话
68
+ - 💬 **优雅回传**:自然语言提交消息,非 JSON 裸串;照片、签名以真实图片回传
69
+ - 🔒 **提交即锁定**:表单防重复提交,记录持久化,"重新填写"可解锁;查询按钮不受影响
70
+ - ⚡ **流式渲染**:模型边写 JSON,卡片边逐块出现
71
+ - 🔄 **原地更新**:`a2ui_update` 按 surfaceId 更新已渲染的卡片——进度条会动、任务卡自己填结果、看板可刷新;刷新页面后重放
72
+ - 📊 **可视化全家桶**:ECharts 图表/仪表盘、函数绘图、中国地图分布、Mermaid 全图型、KaTeX 公式(矩阵强制公式化渲染)、图片对比滑块、视频
73
+ - 🎬 **算法动画**:数组/柱状、网格/矩阵、图/树三形态自动识别;每卡只自动播一遍(组件重挂载不重播),↻ 手动重播、单步、进度条、图例
74
+ - 🧾 **交互表格**:点表头排序(识别数字)、筛选框、分页、复制 TSV、导出 CSV、可编辑表格输入
75
+ - 🧭 **丰富作答组件**:分步向导 Wizard、日历选日期、RankList 排优先级、Suggestions 追问 chips、图片上传、手写签名
76
+ - 📋 **一键复制**:指标块点击复制、表格复制/导出、代码块复制、公式复制 LaTeX 源码、图表下载 PNG、Markdown 复制原文
58
77
  - ⛶ **全屏放大**:思维导图/流程图/图表/图片一键全屏,自动适配视口,滚轮缩放 + 拖拽平移
59
78
  - 🧱 **一行多列**:Grid 布局做商品对比、图表仪表盘
60
79
  - 🎛️ **完整输入态**:下拉单选/多选、预选中(dataModel 初值)、组件级/选项级禁用
@@ -83,21 +102,48 @@ a2ui-render-in-dsh (一个 npm 包,dsh bundle)
83
102
  | `Card` | `children, title?` | 带边框分组 |
84
103
  | `List` | `children, direction?` | 列表容器 |
85
104
  | `Divider` | — | 分隔线 |
86
- | `Text` | `text, variant?: h1\|h2\|h3\|body\|caption\|strong` | 文本 |
105
+ | `Text` | `text, variant?: h1\|h2\|h3\|body\|caption\|strong` | 文本;含 ``` 围栏时自动升级为格式化代码块;行内 `` `code` `` 渲染为代码片 |
106
+ | `Markdown` | `text` | **富文本长文**:标题、加粗/斜体、链接、列表、引用、代码块、`$...$` 公式;带复制原文按钮 |
87
107
  | `Image` | `url, alt?, width?, height?` | 图片(含 GIF),自带全屏放大 |
88
108
  | `Tag` | `text, color?: blue\|green\|red\|orange\|gray` | 标签 |
89
109
  | `Math` | `tex, block?` | **KaTeX** 公式(字体内联零外部请求);矩阵/向量必须走此组件 |
90
110
  | `Mermaid` | `code, caption?` | **Mermaid 11**:流程图/思维导图/时序图/甘特图等,自带全屏放大 |
91
- | `Chart` | `option, height?, functions?, xMin?, xMax?, samples?, yClip?` | **ECharts 6**:数据模式(option 透传)+ 函数绘图模式(表达式自动采样、渐近线断线),自带全屏 |
111
+ | `Chart` | `option, height?, functions?, params?, xMin?, xMax?, samples?, yClip?` | **ECharts 6**:数据模式(option 透传)+ 函数绘图模式(表达式自动采样、渐近线断线)+ `params` 绑定常量(滑杆→曲线联动);全屏 + 下载 PNG |
112
+ | `Map` | `data, title?, unit?, height?` | **中国地图分布**:省级数据热力着色(分区销售/用户分布),geoJSON 内置 |
92
113
  | `Video` | `url, poster?, loop?, muted?, autoplay?` | HTML5 视频(mp4/webm) |
93
- | `Anim` | `frames, interval?, height?, autoplay?, labels?` | **算法动画**:数组/柱状与网格/矩阵双形态自动识别;每卡每页签只自动播一遍,↻ 重播/暂停/单步/重置/进度条/图例 |
114
+ | `ImageCompare` | `before, after` | 拖动分割线的图片前后对比(before/after、A/B) |
115
+ | `Anim` | `frames, interval?, height?, autoplay?, labels?` | **算法动画**:数组/柱状、网格/矩阵、图/树(BFS/DFS,自动圆形布局)三形态自动识别;每卡每页签只自动播一遍,↻ 重播/暂停/单步/重置/进度条/图例 |
94
116
  | `Button` | `label, variant?, submit?, action: {event: {name, context?}}` | 触发回传;`submit` 显式控制是否锁卡 |
95
117
  | `MultipleChoice` | `options, bind, maxAllowedSelections?, disabled?` | 平铺多选/单选(`maxAllowedSelections: 1` 单选),选项级禁用 |
96
- | `Select` | `options, bind, label?, placeholder?, multiple?, maxAllowedSelections?, disabled?` | **下拉选择**:单选存值、多选存数组,选项描述/禁用 |
118
+ | `Select` | `options, bind, label?, placeholder?, multiple?, maxAllowedSelections?, disabled?` | **下拉选择**:单选存值、多选存数组;长列表自动带搜索框 |
97
119
  | `CheckBox` | `label, bind, disabled?` | 布尔勾选 |
120
+ | `Slider` | `bind, label?, min?, max?, step?, unit?` | 数值滑杆;配合 Chart `params` 做参数探索联动 |
121
+ | `Rate` | `bind, label?, max?` | 星级评分 |
122
+ | `Calc` | `expr, inputs, out, digits?` | 隐形派生值:表达式实时重算写回数据模型(计算器引擎) |
123
+ | `When` | `value, equals?/includes?/notEmpty?, children` | 条件容器:选中特定项才显示后续字段 |
124
+ | `Tabs` | `tabs, children, bind?` | 页签:数据集切换 / 内容分组 |
125
+ | `Table` | `columns, rows, caption?, sortable?, filter?, pageSize?` | **交互表格**:点表头排序(识别数字)、筛选框、分页、复制 TSV / 导出 CSV;字典绑定切换数据集 |
126
+ | `Stat` | `label, value, unit?, trend?, hint?` | KPI 指标块,点击复制数值;Grid 组合成速览行 |
127
+ | `Steps` | `items` | 步骤条(done/current/pending) |
128
+ | `Progress` | `value, max?, label?` | 进度条 |
129
+ | `Timeline` | `items` | 时间轴(历程/事件回顾) |
130
+ | `CodeBlock` | `code, language?, title?` | 代码块:行号 + 轻量高亮 + 复制——读代码题的题干代码也走这里(不许塞进 Text) |
131
+ | `Icon` | `name, size?, color?` | 内置 32 个常用线条图标 |
132
+ | `Audio` | `url, title?` | 音频播放器 |
133
+ | `Flashcard` | `front, back` | 点击翻面闪卡(背单词/问答记忆) |
134
+ | `Countdown` | `to?/seconds?, label?` | 实时倒计时 |
98
135
  | `TextField` | `label?, placeholder?, multiline?, bind, disabled?` | 文本输入 |
136
+ | `Wizard` | `steps, children, submitLabel?` | **分步表单**:每步一个面板,内置上一步/下一步 + 进度,最后一步提交全部字段 |
137
+ | `Calendar` | `bind, label?, min?, max?, range?` | 月视图日期选择;`range: true` 选起止两天 |
138
+ | `RankList` | `items, bind, label?` | 拖拽(或点 ↑↓)按优先级排序,提交排好的列表 |
139
+ | `EditableTable` | `columns, rows, bind, label?` | 用户直接改单元格,整表提交 |
140
+ | `Upload` | `bind?, label?, max?` | 图片选择器——所选照片以**真实图片**回传给模型 |
141
+ | `Signature` | `label?` | 手写签名画板,笔迹以图片回传 |
142
+ | `Suggestions` | `items` | 回答下方的可点追问 chips,点一下即作为下一条用户消息发出 |
99
143
 
100
- 输入组件通用:**预选中**在 `dataModel` 给 bind 路径设初值;**禁用**用组件级 `disabled: true` 或选项级 `disabled`。数据绑定:`bind` 为不带前导斜杠的写入路径;展示属性用 `{"path": "/x"}`(带斜杠)读实时值。
144
+ **内联公式**:所有文本位置(Text、选项、表格单元格、步骤、闪卡、动画解说)支持 `$...$` 内嵌 KaTeX——数学选择题的选项可以直接是公式。**响应式联动**:输入组件写数据模型,所有 `{"path"}` 绑定即时更新——滑杆→曲线(Chart `params`)、选择→追问(When)、输入→计算结果(Calc→Stat)、切换→换表(Tabs/Table 字典绑定)。
145
+
146
+ 输入组件通用:**预选中**在 `dataModel` 给 bind 路径设初值;**禁用**用组件级 `disabled: true` 或选项级 `disabled`;**必填**用 `required: true`(未填完提交被拦截并高亮)。`a2ui_update` 改写 dataModel 后,绑定的输入组件会原地同步。数据绑定:`bind` 为不带前导斜杠的写入路径;展示属性用 `{"path": "/x"}`(带斜杠)读实时值。
101
147
 
102
148
  ## 安装
103
149
 
@@ -106,9 +152,24 @@ a2ui-render-in-dsh (一个 npm 包,dsh bundle)
106
152
  | 要求 | 说明 |
107
153
  |---|---|
108
154
  | dsh | `@deepseek-ai/dsh` ≥ 0.1.1-rc.1,且 web profile 已初始化(装插件前先运行过一次 `dsh web`) |
155
+
156
+ 开发:`npm test` 运行 jsdom 交互测试套件(宿主校验 + 组件/交互全覆盖,约 70 条断言)。
109
157
  | Node.js | ≥ 20(含 npm) |
110
158
 
111
- ### 方式 A · 源码安装(当前推荐)
159
+ ### 方式 A · npm 安装(推荐)
160
+
161
+ ```sh
162
+ dsh plugin --profile web add a2ui-render-in-dsh
163
+ dsh web
164
+ ```
165
+
166
+ 零运行时依赖,两条命令装完即用。若你的 npm 镜像尚未同步最新版本,显式指定官方源:
167
+
168
+ ```sh
169
+ dsh plugin --profile web add a2ui-render-in-dsh --registry https://registry.npmjs.org
170
+ ```
171
+
172
+ ### 方式 B · 源码安装(开发者 / 需要改代码时)
112
173
 
113
174
  ```sh
114
175
  # 1. 克隆并构建
@@ -116,27 +177,17 @@ git clone https://github.com/baihui-ai/a2ui-render-in-dsh.git
116
177
  cd a2ui-render-in-dsh
117
178
  npm install
118
179
  npm run build
119
- # 构建产出两个文件:lib/index.js(宿主端插件)+ lib/client.js(浏览器 bundle)
180
+ # 构建产出:lib/index.js(宿主端)+ lib/client.js(浏览器 bundle)
120
181
  # lib/ 不在 git 里,克隆后必须先构建,否则 dsh 启动时找不到入口
121
182
 
122
183
  # 2. 以 link 方式装进 dsh 的 web profile
123
184
  dsh plugin --profile web add link:$(pwd)
124
- # 该命令等价于在 profile 里 pnpm add link:<路径>,并自动把
125
- # a2ui-render-in-dsh 追加到 profile 的 dsh.profile.bundles
185
+ # 改码后只需 npm run build + 重启 dsh web,无需重装
126
186
 
127
187
  # 3. 启动 / 重启 dsh web
128
188
  dsh web
129
189
  ```
130
190
 
131
- link 方式的好处:后续升级只需 `git pull && npm run build` 再重启 dsh web,无需重装。
132
-
133
- ### 方式 B · npm 安装(包发布到 npm 后)
134
-
135
- ```sh
136
- dsh plugin --profile web add a2ui-render-in-dsh
137
- dsh web
138
- ```
139
-
140
191
  ### 三步验证
141
192
 
142
193
  ```sh
@@ -155,7 +206,10 @@ curl -s http://127.0.0.1:<port>/ | grep -o "a2ui-render-in-dsh/client.js[^\"]*"
155
206
  ### 升级与卸载
156
207
 
157
208
  ```sh
158
- # 升级(link 安装):拉新代码重新构建,重启 dsh web 即生效
209
+ # 升级(npm 安装)
210
+ dsh plugin --profile web update a2ui-render-in-dsh
211
+
212
+ # 升级(源码 link 安装):拉新代码重新构建,重启 dsh web 即生效
159
213
  git pull && npm run build
160
214
 
161
215
  # 卸载:从 profile 移除(依赖与 bundles 条目会一并清掉),重启 dsh web