@mengruo/dsh-vision-toolkit 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (216) hide show
  1. package/LICENSE +21 -0
  2. package/README.i18n.yaml +6 -0
  3. package/README.md +356 -0
  4. package/README.zh.md +354 -0
  5. package/assets/community-group-qr.png +0 -0
  6. package/assets/dsh-conversation-artifact.png +0 -0
  7. package/assets/dsh-conversation-image-qa-top.png +0 -0
  8. package/assets/dsh-conversation-image-qa.png +0 -0
  9. package/assets/dsh-conversation-pixel-diff.png +0 -0
  10. package/assets/dsh-conversation-screenshot-debugging-top.png +0 -0
  11. package/assets/dsh-conversation-screenshot-debugging.png +0 -0
  12. package/assets/dsh-conversation-tool-call.png +0 -0
  13. package/assets/dsh-conversation-vision-trace.png +0 -0
  14. package/assets/dsh-view-example.png +0 -0
  15. package/assets/hero-v2.png +0 -0
  16. package/assets/logo_aihubmix.png +0 -0
  17. package/assets/python-bootstrap.json +48 -0
  18. package/assets/skill/SKILL.md +329 -0
  19. package/assets/skill/UPSTREAM.json +71 -0
  20. package/assets/skill/references/gui.md +88 -0
  21. package/assets/skill/references/long-screenshot-ocr.md +77 -0
  22. package/assets/skill/references/restore-graphic.md +84 -0
  23. package/assets/skill/references/restore-structure.md +45 -0
  24. package/assets/skill/references/restore-ui.md +202 -0
  25. package/assets/social-preview.png +0 -0
  26. package/assets/upstream/README.md +18 -0
  27. package/assets/upstream/focus-hint-comparison-1.webp +0 -0
  28. package/assets/upstream/focus-hint-comparison-2.webp +0 -0
  29. package/assets/upstream/infographic-reference.webp +0 -0
  30. package/assets/upstream/infographic-result.webp +0 -0
  31. package/assets/upstream/ui-fast-restore-reference.webp +0 -0
  32. package/assets/upstream/ui-fast-restore-result.webp +0 -0
  33. package/assets/upstream/ui-result.webp +0 -0
  34. package/assets/upstream/ui-sketch.webp +0 -0
  35. package/assets/vision-model-test.png +0 -0
  36. package/assets/vision-settings.png +0 -0
  37. package/assets/wechat-reward.png +0 -0
  38. package/cordis.patch.yml +6 -0
  39. package/docs/aihubmix-gemini-vision.i18n.yaml +6 -0
  40. package/docs/aihubmix-gemini-vision.md +138 -0
  41. package/docs/aihubmix-gemini-vision.zh.md +138 -0
  42. package/docs/assets/aihubmix-api-keys.png +0 -0
  43. package/docs/assets/aihubmix-create-key.png +0 -0
  44. package/docs/assets/aihubmix-free-vision-model.png +0 -0
  45. package/docs/assets/aihubmix-home.png +0 -0
  46. package/docs/assets/aihubmix-sign-up.png +0 -0
  47. package/docs/assets/groq-console-home.png +0 -0
  48. package/docs/assets/groq-console-keys-login.png +0 -0
  49. package/docs/assets/groq-docs-quickstart.png +0 -0
  50. package/docs/assets/groq-docs-qwen3.6.png +0 -0
  51. package/docs/assets/groq-docs-vision.png +0 -0
  52. package/docs/assets/vision-settings.png +0 -0
  53. package/docs/dsh-desktop-install.md +67 -0
  54. package/docs/dsh-desktop-install.zh.md +67 -0
  55. package/docs/groq-qwen3.6-vision.i18n.yaml +6 -0
  56. package/docs/groq-qwen3.6-vision.md +236 -0
  57. package/docs/groq-qwen3.6-vision.zh.md +236 -0
  58. package/docs/python-runtime.i18n.yaml +6 -0
  59. package/docs/python-runtime.md +89 -0
  60. package/docs/python-runtime.zh.md +89 -0
  61. package/docs/requirements-traceability/README.i18n.yaml +6 -0
  62. package/docs/requirements-traceability/README.md +75 -0
  63. package/docs/requirements-traceability/README.zh.md +75 -0
  64. package/examples/ui-restoration/README.i18n.yaml +6 -0
  65. package/examples/ui-restoration/README.md +70 -0
  66. package/examples/ui-restoration/README.zh.md +70 -0
  67. package/examples/ui-restoration/assets/final-heatmap.png +0 -0
  68. package/examples/ui-restoration/assets/final-report.json +83 -0
  69. package/examples/ui-restoration/assets/implementation.png +0 -0
  70. package/examples/ui-restoration/assets/initial-heatmap.png +0 -0
  71. package/examples/ui-restoration/assets/initial-report.json +83 -0
  72. package/examples/ui-restoration/assets/initial.png +0 -0
  73. package/examples/ui-restoration/assets/metrics.json +12 -0
  74. package/examples/ui-restoration/assets/reference.png +0 -0
  75. package/examples/ui-restoration/implementation.html +94 -0
  76. package/examples/ui-restoration/initial.html +57 -0
  77. package/lib/artifact-access.js +369 -0
  78. package/lib/artifact-access.js.map +1 -0
  79. package/lib/artifacts.js +56 -0
  80. package/lib/artifacts.js.map +1 -0
  81. package/lib/client.js +2099 -0
  82. package/lib/client.js.map +1 -0
  83. package/lib/config.js +172 -0
  84. package/lib/config.js.map +1 -0
  85. package/lib/defaults.js +6 -0
  86. package/lib/defaults.js.map +1 -0
  87. package/lib/errors.js +56 -0
  88. package/lib/errors.js.map +1 -0
  89. package/lib/evidence-cache.js +350 -0
  90. package/lib/evidence-cache.js.map +1 -0
  91. package/lib/exposure.js +247 -0
  92. package/lib/exposure.js.map +1 -0
  93. package/lib/image-input-variants.js +882 -0
  94. package/lib/image-input-variants.js.map +1 -0
  95. package/lib/index.js +105 -0
  96. package/lib/index.js.map +1 -0
  97. package/lib/paste-images.js +217 -0
  98. package/lib/paste-images.js.map +1 -0
  99. package/lib/paths.js +354 -0
  100. package/lib/paths.js.map +1 -0
  101. package/lib/plugin-update.js +1003 -0
  102. package/lib/plugin-update.js.map +1 -0
  103. package/lib/runtime-install.js +964 -0
  104. package/lib/runtime-install.js.map +1 -0
  105. package/lib/runtime-manager.js +133 -0
  106. package/lib/runtime-manager.js.map +1 -0
  107. package/lib/runtime.js +1678 -0
  108. package/lib/runtime.js.map +1 -0
  109. package/lib/skill.js +25 -0
  110. package/lib/skill.js.map +1 -0
  111. package/lib/tools.js +549 -0
  112. package/lib/tools.js.map +1 -0
  113. package/lib/types/artifact-access.d.ts +61 -0
  114. package/lib/types/artifact-access.d.ts.map +1 -0
  115. package/lib/types/artifacts.d.ts +42 -0
  116. package/lib/types/artifacts.d.ts.map +1 -0
  117. package/lib/types/client/display-config.d.ts +24 -0
  118. package/lib/types/client/display-config.d.ts.map +1 -0
  119. package/lib/types/client/index.d.ts +350 -0
  120. package/lib/types/client/index.d.ts.map +1 -0
  121. package/lib/types/client/model-variants-hider.d.ts +40 -0
  122. package/lib/types/client/model-variants-hider.d.ts.map +1 -0
  123. package/lib/types/client/paste-images.d.ts +126 -0
  124. package/lib/types/client/paste-images.d.ts.map +1 -0
  125. package/lib/types/config.d.ts +123 -0
  126. package/lib/types/config.d.ts.map +1 -0
  127. package/lib/types/defaults.d.ts +6 -0
  128. package/lib/types/defaults.d.ts.map +1 -0
  129. package/lib/types/errors.d.ts +35 -0
  130. package/lib/types/errors.d.ts.map +1 -0
  131. package/lib/types/evidence-cache.d.ts +91 -0
  132. package/lib/types/evidence-cache.d.ts.map +1 -0
  133. package/lib/types/exposure.d.ts +50 -0
  134. package/lib/types/exposure.d.ts.map +1 -0
  135. package/lib/types/image-input-variants.d.ts +144 -0
  136. package/lib/types/image-input-variants.d.ts.map +1 -0
  137. package/lib/types/index.d.ts +19 -0
  138. package/lib/types/index.d.ts.map +1 -0
  139. package/lib/types/paste-images.d.ts +73 -0
  140. package/lib/types/paste-images.d.ts.map +1 -0
  141. package/lib/types/paths.d.ts +117 -0
  142. package/lib/types/paths.d.ts.map +1 -0
  143. package/lib/types/plugin-update.d.ts +119 -0
  144. package/lib/types/plugin-update.d.ts.map +1 -0
  145. package/lib/types/runtime-install.d.ts +92 -0
  146. package/lib/types/runtime-install.d.ts.map +1 -0
  147. package/lib/types/runtime-manager.d.ts +60 -0
  148. package/lib/types/runtime-manager.d.ts.map +1 -0
  149. package/lib/types/runtime.d.ts +412 -0
  150. package/lib/types/runtime.d.ts.map +1 -0
  151. package/lib/types/skill.d.ts +14 -0
  152. package/lib/types/skill.d.ts.map +1 -0
  153. package/lib/types/tools.d.ts +35 -0
  154. package/lib/types/tools.d.ts.map +1 -0
  155. package/lib/types/upstream.d.ts +233 -0
  156. package/lib/types/upstream.d.ts.map +1 -0
  157. package/lib/types/version.d.ts +15 -0
  158. package/lib/types/version.d.ts.map +1 -0
  159. package/lib/types/web-request.d.ts +11 -0
  160. package/lib/types/web-request.d.ts.map +1 -0
  161. package/lib/types/web.d.ts +118 -0
  162. package/lib/types/web.d.ts.map +1 -0
  163. package/lib/upstream.js +860 -0
  164. package/lib/upstream.js.map +1 -0
  165. package/lib/version.js +18 -0
  166. package/lib/version.js.map +1 -0
  167. package/lib/web-request.js +29 -0
  168. package/lib/web-request.js.map +1 -0
  169. package/lib/web.js +462 -0
  170. package/lib/web.js.map +1 -0
  171. package/package.json +149 -0
  172. package/patches/vision-tools-dsh.patch +970 -0
  173. package/runtime/requirements.lock +3 -0
  174. package/src/artifact-access.ts +386 -0
  175. package/src/artifacts.ts +85 -0
  176. package/src/client/display-config.ts +62 -0
  177. package/src/client/index.tsx +1580 -0
  178. package/src/client/model-variants-hider.ts +159 -0
  179. package/src/client/paste-images.tsx +747 -0
  180. package/src/config.ts +290 -0
  181. package/src/defaults.ts +5 -0
  182. package/src/errors.ts +62 -0
  183. package/src/evidence-cache.ts +418 -0
  184. package/src/exposure.ts +265 -0
  185. package/src/image-input-variants.ts +996 -0
  186. package/src/index.ts +141 -0
  187. package/src/paste-images.ts +286 -0
  188. package/src/paths.ts +383 -0
  189. package/src/plugin-update.ts +1143 -0
  190. package/src/runtime-install.ts +1161 -0
  191. package/src/runtime-manager.ts +174 -0
  192. package/src/runtime.ts +2189 -0
  193. package/src/skill.ts +35 -0
  194. package/src/tools.ts +691 -0
  195. package/src/upstream.ts +1078 -0
  196. package/src/version.ts +37 -0
  197. package/src/web-request.ts +27 -0
  198. package/src/web.ts +594 -0
  199. package/vendor/agent-vision-toolkit/CHANGELOG.md +30 -0
  200. package/vendor/agent-vision-toolkit/LICENSE +21 -0
  201. package/vendor/agent-vision-toolkit/README.md +405 -0
  202. package/vendor/agent-vision-toolkit/UPSTREAM_MANIFEST.json +94 -0
  203. package/vendor/agent-vision-toolkit/bin/crop +90 -0
  204. package/vendor/agent-vision-toolkit/bin/detect +13 -0
  205. package/vendor/agent-vision-toolkit/bin/glance +93 -0
  206. package/vendor/agent-vision-toolkit/bin/ground +13 -0
  207. package/vendor/agent-vision-toolkit/bin/trace +129 -0
  208. package/vendor/agent-vision-toolkit/detect.py +60 -0
  209. package/vendor/agent-vision-toolkit/ground.py +267 -0
  210. package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/dominant_colors.py +224 -0
  211. package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/extract_fg.py +278 -0
  212. package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/html_shot.py +420 -0
  213. package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/long_screenshot_ocr.py +1245 -0
  214. package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/pixel_diff.py +88 -0
  215. package/vendor/agent-vision-toolkit/tests/test_vision_client.py +450 -0
  216. package/vendor/agent-vision-toolkit/vision_client.py +310 -0
@@ -0,0 +1,45 @@
1
+ # Turning a visual diagram into structured code
2
+
3
+ **When to use**: the task is converting a sketch, flowchart, architecture
4
+ diagram, or whiteboard into Mermaid, Graphviz, JSON, or another structured
5
+ representation. The goal is semantic and topological fidelity, not matching
6
+ the source renderer pixel for pixel.
7
+
8
+ ## Steps
9
+
10
+ 1. Run one full-image `vision_glance` pass for the diagram type, reading
11
+ direction, major groups, and visual conventions.
12
+ 2. Use `vision_detect` for the initial node and label inventory. Refine dense
13
+ groups by passing each group's box as `region`.
14
+ 3. Use OCR for labels. Preserve visible spelling, punctuation, and
15
+ abbreviations verbatim; write `[unreadable]` instead of guessing.
16
+ 4. Locate ambiguous arrows, connectors, legends, and group boundaries with
17
+ `vision_ground` and targeted `vision_glance` calls using `region`.
18
+ 5. Build an explicit intermediate inventory before writing the output:
19
+ nodes, labels, groups, edges, directions, and edge labels.
20
+ 6. Generate the Mermaid, Graphviz, or requested structure from that inventory.
21
+
22
+ ## Verify structure, not pixels
23
+
24
+ Render the result, inventory it again, and compare these facts with the
25
+ reference:
26
+
27
+ - node count and label set
28
+ - group membership and nesting
29
+ - edge list and direction
30
+ - edge labels and branch conditions
31
+ - start/end or input/output roles
32
+
33
+ Pixel differences are expected because the target renderer chooses its own
34
+ fonts, spacing, and routing. A missing node or reversed arrow is a failure even
35
+ when the two images look broadly similar; a different curve or gap is not a
36
+ failure when the structure is correct.
37
+
38
+ ## Boundaries
39
+
40
+ - Do not tidy labels, expand abbreviations, or silently repair wording unless
41
+ the user asks for editorial cleanup.
42
+ - Do not infer an edge from proximity alone. If its endpoint or direction is
43
+ unclear, zoom into the connector and mark unresolved facts explicitly.
44
+ - If the user wants a visual SVG copy rather than structured code, follow
45
+ `restore-graphic.md` instead.
@@ -0,0 +1,202 @@
1
+ # Rebuilding a UI from a reference image
2
+
3
+ **When to use**: the task is turning a screenshot or design image into a
4
+ page, component, or application UI in HTML/CSS or the project's existing
5
+ frontend stack. Checking an existing implementation against its reference is
6
+ the same workflow entered at Verify. For an isolated icon, logo, or
7
+ illustration, read `restore-graphic.md`; for Mermaid, Graphviz, or another
8
+ structured diagram, read `restore-structure.md`.
9
+
10
+ ## Choose the restore mode
11
+
12
+ - Use **fast restore mode** when the user asks for a quick, rough,
13
+ approximate, prototype, or first-pass reconstruction, explicitly values
14
+ speed over fidelity, or the reference image itself shows a floating
15
+ speed-intent control such as "快速还原为 HTML" / "快速生成" / "quick
16
+ restore" overlay. Its target is a recognizable screenshot in about
17
+ three minutes when the project already runs.
18
+ - Use the **standard restore workflow** below when the user asks for close,
19
+ precise, pixel-level, or production-ready alignment, or does not opt into a
20
+ faster approximation.
21
+
22
+ ## Fast restore mode: first screenshot in about three minutes
23
+
24
+ Fast mode preserves the page's hierarchy, major regions, visible text, and
25
+ primary state. It deliberately approximates fine spacing, exact colors,
26
+ typography, shadows, decorative details, and icon geometry.
27
+
28
+ ### Hard limits
29
+
30
+ 1. Inspect the existing frontend stack, component library, icon set, source
31
+ assets, and design tokens before writing code, but stop searching as soon as
32
+ a usable local primitive is found.
33
+ 2. Run one full-image `vision_detect` pass. Treat its boxes as layout estimates
34
+ and do not start a region-by-region inventory.
35
+ 3. After that `vision_detect` pass, use at most **six sequential image-inspection
36
+ rounds** through `view_image` (or the host's equivalent built-in viewer) and
37
+ `vision_glance`. Each round may launch up to three independent calls
38
+ concurrently, so the hard ceiling is 18 calls across six rounds. Batch unrelated regions
39
+ or questions into the same round instead of waiting for each result before
40
+ starting the next call. Normally one or two rounds are enough.
41
+ 4. Do not use `vision_trace`, foreground extraction, repeated color sampling,
42
+ iterative `vision_pixel_diff` work, or hand-written SVG in fast mode. Those
43
+ are fidelity tools and will consume the delivery window. Icons and decorative
44
+ marks stay library-based or screenshot-backed; never hand-write SVG code.
45
+
46
+ ### Build the approximation
47
+
48
+ 1. Implement the largest layout regions first, then visible text, primary
49
+ controls, and the most important state. Ignore details that are only visible
50
+ when zoomed in.
51
+ 2. Reuse the project's existing components and CSS tokens. If its frontend or
52
+ icon library contains a reasonably similar component or icon, use it
53
+ directly instead of recreating the reference. Never hand-write SVG in fast
54
+ mode: use an approximate library icon, or extract the original pixels as a
55
+ screenshot-backed asset.
56
+ 3. Use nearby existing palette tokens or visually similar CSS values. Exact
57
+ sampled hex values, gradients, subtle borders, and shadow opacity are out of
58
+ scope unless one of them defines the whole composition.
59
+ 4. Keep text and controls native, selectable, and interactive. Fast mode relaxes
60
+ visual fidelity, not basic UI behavior.
61
+
62
+ ### Render once, fix once, deliver
63
+
64
+ 1. Render the target viewport with `vision_html_screenshot` or the project's
65
+ existing browser setup.
66
+ 2. Inspect the screenshot once. If there is an obvious structural failure such
67
+ as a missing major region, broken wrapping, or a wildly wrong scale, make
68
+ one focused correction and render once more.
69
+ 3. Deliver the screenshot. Stop instead of spending the remaining time on
70
+ small color, icon, font, shadow, radius, or spacing differences.
71
+
72
+ A practical time box is roughly 30 seconds for project inspection plus
73
+ `vision_detect`, 90 seconds for implementation, and the remaining minute for startup,
74
+ rendering, one correction, and screenshot delivery. Dependency installation or
75
+ a project that does not already run may extend that target; do not compensate
76
+ by silently switching back to a long precision loop.
77
+
78
+ ## Standard restore workflow
79
+
80
+ ### Core strategy: code-native UI plus screenshot-backed visuals
81
+
82
+ Do not choose one reconstruction mode for the whole page. Classify each
83
+ element separately. Most finished pages should combine both kinds:
84
+
85
+ | Kind | Build it this way | Typical examples |
86
+ |---|---|---|
87
+ | **Code-native component** | Reuse the project's component library or build it with semantic HTML, CSS, and simple SVG primitives | layout, text, buttons, inputs, cards, tabs, dividers, backgrounds, simple geometric badges |
88
+ | **Screenshot-backed visual** | Extract the original pixels, remove the background when needed, and place the result as an image asset | logos, proprietary icons, illustrations, decorative artwork, textures, complex static marks |
89
+
90
+ Use this decision order for every element:
91
+
92
+ 1. **Reuse an existing component or source asset from the project.** Search
93
+ before recreating anything.
94
+ 2. **Use a code-native component** when the element contains text, accepts
95
+ input, changes state, must respond to layout, or is simple to express with
96
+ the project's normal primitives.
97
+ 3. **Use a screenshot-backed visual** when it is static, visually distinctive,
98
+ expensive to redraw, and no source asset exists.
99
+ 4. Rebuild a vector only when the user requires SVG/editability, the visual
100
+ must scale beyond the screenshot resolution, or extraction cannot isolate
101
+ it cleanly. Follow `restore-graphic.md` for that branch.
102
+
103
+ Never flatten text, controls, or a large layout block into a screenshot. Never
104
+ redraw extractable brand artwork merely to make the implementation look more
105
+ "native". For a complex widget, keep the shell, text, and interaction native;
106
+ use an extracted image only for its static decorative layer.
107
+
108
+ ## Steps
109
+
110
+ ### 1. Preserve the reference and inspect the target project
111
+
112
+ If the reference is in a temporary path, copy it to durable work storage
113
+ before the first tool call. Inspect the existing stack, component library,
114
+ fonts, icons, images, and design tokens before writing replacements.
115
+
116
+ ### 2. Establish the coordinate system
117
+
118
+ Record the reference image dimensions and the target viewport. Screenshots may
119
+ be HiDPI: `vision_detect`, `vision_ground`, and `vision_crop` report image
120
+ pixels, while CSS uses logical pixels. Derive the scale from the actual image and viewport dimensions;
121
+ do not assume a fixed 2x ratio.
122
+
123
+ ### 3. Inventory the UI outside-in
124
+
125
+ 1. Run one full-image `vision_detect` pass for the initial element list and
126
+ boxes.
127
+ 2. Call `vision_detect` with `region` on dense layout blocks; a full-screen pass
128
+ is only a scaffold.
129
+ 3. Use `vision_glance` for hierarchy, component type, visual state, and styling.
130
+ 4. Use OCR for visible text and `vision_ground` for a specific element that
131
+ remains ambiguous.
132
+
133
+ Treat model boxes as approximate handles. Use them to organize the page, but
134
+ do not mistake their final few pixels for measured boundaries.
135
+
136
+ ### 4. Classify before implementing
137
+
138
+ Make a short element list with one of these decisions: reuse existing source,
139
+ code-native, screenshot-backed, or hybrid. This prevents two costly mistakes:
140
+ rebuilding complex artwork from scratch and turning functional UI into static
141
+ image patches.
142
+
143
+ ### 5. Extract screenshot-backed visuals
144
+
145
+ Use a padded crop only to inspect and center the target. Before foreground
146
+ extraction, tighten the final region to the visual's own ink; adjacent text or
147
+ rules inside the region can become foreground components too.
148
+
149
+ ```json
150
+ {"image":"shot.png","region":"X1,Y1,X2,Y2","scale":4,"output":"icon4x.png"}
151
+ {"image":"<icon4x-artifact-path>","output":"icon.clean.png"}
152
+ {"image":"shot.png","region":"X1,Y1,X2,Y2","output":"icon.png"}
153
+ {"image":"shot.png","region":"X1,Y1,X2,Y2","mode":"dark","output":"logo.png"}
154
+ ```
155
+
156
+ Inspect the transparent result before use. Confirm that the whole visual is
157
+ present, no neighboring text or border leaked in, and the alpha edge remains
158
+ clean. When extracting several assets, inspect them together in a contact
159
+ sheet instead of checking them from filenames alone.
160
+
161
+ ### 6. Implement the native frame
162
+
163
+ - Follow the project's existing framework and component patterns.
164
+ - Build content, interaction, responsive layout, borders, shadows, and simple
165
+ shapes natively.
166
+ - Use `vision_dominant_colors` for important background, text, and accent
167
+ values; use `vision_glance` to name a color, not to invent its numeric value.
168
+ - Place extracted assets with explicit logical dimensions. Preserve their
169
+ aspect ratio and avoid baking surrounding whitespace into the asset.
170
+ - Match structure and proportions before tuning small spacing.
171
+
172
+ ## Verify and converge by impact
173
+
174
+ For an existing implementation, start here.
175
+
176
+ 1. Render at the same logical viewport with `vision_html_screenshot` or the
177
+ project's browser test setup.
178
+ 2. Compare the render and reference at the same dimensions. Inspect them side
179
+ by side; use `vision_pixel_diff` to locate differences that are hard to
180
+ spot or explain.
181
+ 3. Fix material discrepancies: missing or wrong content, incorrect hierarchy,
182
+ broken wrapping, visibly wrong alignment or scale, wrong component state,
183
+ distorted assets, and clearly different primary colors.
184
+ 4. Re-render after a meaningful fix. Continue while the comparison exposes a
185
+ user-visible defect; stop when the remaining difference is font
186
+ rasterization, antialiasing, subpixel placement, or another imperceptible
187
+ rendering detail.
188
+
189
+ `vision_pixel_diff` is a locator, not the acceptance target. Do not keep iterating only
190
+ to lower its aggregate percentage, and do not promise zero-difference output
191
+ from a screenshot without a separately scoped browser, font, viewport, and
192
+ rendering environment.
193
+
194
+ ## Boundaries
195
+
196
+ - Do not ship a screenshot of the whole page as the implementation.
197
+ - Do not use screenshot-backed text or controls that must remain selectable,
198
+ accessible, interactive, localized, or responsive.
199
+ - Do not stretch an extracted bitmap beyond the resolution supported by the
200
+ reference. Obtain a source asset or rebuild a vector when it must scale.
201
+ - Always compare against the original reference, never only against an earlier
202
+ version of your own render.
Binary file
@@ -0,0 +1,18 @@
1
+ # Upstream reference images
2
+
3
+ These files are README-sized derivatives of official [`Anionex/agent-vision-toolkit`](https://github.com/Anionex/agent-vision-toolkit) reference assets. They document upstream use cases; this DSH integration does not claim to have rerun or reproduced those results.
4
+
5
+ The source files exist at the packaged runtime pin [`c27d1a300962b553c0884993c575cd3e819465ce`](https://github.com/Anionex/agent-vision-toolkit/tree/c27d1a300962b553c0884993c575cd3e819465ce) and were confirmed byte-identical in upstream commit [`7eafd51e7e62bd14f72627c69f2c11601c508f88`](https://github.com/Anionex/agent-vision-toolkit/tree/7eafd51e7e62bd14f72627c69f2c11601c508f88) on August 11, 2026.
6
+
7
+ | Local derivative | Upstream source |
8
+ |---|---|
9
+ | `infographic-reference.webp` | `assets/infographic-restore-reference.png` |
10
+ | `infographic-result.webp` | `assets/infographic-restore-result.png` |
11
+ | `ui-sketch.webp` | `assets/ui-restore-sketch.png` |
12
+ | `ui-result.webp` | `assets/ui-restore-result.png` |
13
+ | `focus-hint-comparison-1.webp` | `assets/focus-hint-comparison-1.png` |
14
+ | `focus-hint-comparison-2.webp` | `assets/focus-hint-comparison-2.png` |
15
+ | `ui-fast-restore-reference.webp` | `assets/ui-fast-restore-reference.png` |
16
+ | `ui-fast-restore-result.webp` | `assets/ui-fast-restore-result.png` |
17
+
18
+ Each derivative preserves the complete frame, limits the longest edge to 1200 pixels, removes metadata, and uses WebP quality 90 for repository delivery. The originals and visual algorithms remain covered by the upstream [MIT License](https://github.com/Anionex/agent-vision-toolkit/blob/c27d1a300962b553c0884993c575cd3e819465ce/LICENSE).
Binary file
Binary file
Binary file
Binary file
Binary file
@@ -0,0 +1,6 @@
1
+ # dsh-vision-toolkit bundle patch: mounts the plugin into a profile layer stack.
2
+ # Runtime defaults live at the plugin's config boundary. Users override them in
3
+ # their profile patch row with the same id.
4
+ - insert:
5
+ - id: vision-toolkit
6
+ name: '@mengruo/dsh-vision-toolkit'
@@ -0,0 +1,6 @@
1
+ # Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
2
+ # side as of the last confirmed-consistent state. Both languages carry equal authority;
3
+ # after editing either side, bring the other along and re-record with:
4
+ # pnpm run verify-translation-pairing --write dsh-vision-toolkit/docs/aihubmix-gemini-vision.md
5
+ aihubmix-gemini-vision.md: c2958163461843045fc4547e56fbaae78d1b9f72
6
+ aihubmix-gemini-vision.zh.md: dd2ba9246bb921f4986cacfd28e31aaeec4903e3
@@ -0,0 +1,138 @@
1
+ # Get an AIHubMix API key and use free Gemini 3.7 Flash for vision
2
+
3
+ **English** | [中文](aihubmix-gemini-vision.zh.md)
4
+
5
+ This tutorial completes three tasks:
6
+
7
+ 1. Create an AIHubMix account.
8
+ 2. Create and safely store an AIHubMix API key.
9
+ 3. Use the free `gemini-3.7-flash-free` model for image analysis in DSH Vision Toolkit.
10
+
11
+ > As of August 20, 2026, the AIHubMix model page lists `gemini-3.7-flash-free` as a free trial model with image input. Free capacity is limited, may return `429`, and is not guaranteed for production workloads; use the paid `gemini-3.7-flash` route when reliability is required. Models, pricing, and availability can change, so check the [AIHubMix model page](https://aihubmix.com/model/gemini-3.7-flash-free) for current details.
12
+
13
+ ## 1. Open the signup entry and register
14
+
15
+ Open the [Inferera signup entry](https://inferera.com/?aff=5wj6sgx8), which redirects to AIHubMix, then select **Sign up** in the upper-right corner or **Get API Key** on the page.
16
+
17
+ This URL includes the project's referral parameter. You can instead open [Inferera](https://inferera.com/) directly if you prefer not to use a referral link.
18
+
19
+ <p align="center">
20
+ <img src="assets/aihubmix-home.png" width="92%" alt="AIHubMix homepage and signup entry" />
21
+ </p>
22
+
23
+ The signup page supports GitHub, Google, and email. For email signup, enter an email address and password, accept the terms, and complete any verification requested by the page.
24
+
25
+ <p align="center">
26
+ <img src="assets/aihubmix-sign-up.png" width="92%" alt="AIHubMix account signup page" />
27
+ </p>
28
+
29
+ ## 2. Create an API key
30
+
31
+ After signing in, open **Developer → API Keys** in the console sidebar, or go directly to [AIHubMix API Keys](https://console.aihubmix.com/token). The page shows two endpoint hosts:
32
+
33
+ - Default: `https://aihubmix.com`
34
+ - Preferred: `https://api.inferera.com`
35
+
36
+ This guide uses the preferred `https://api.inferera.com/v1` endpoint in Vision Toolkit. If that endpoint does not work from your network, use `https://aihubmix.com/v1` instead.
37
+
38
+ <p align="center">
39
+ <img src="assets/aihubmix-api-keys.png" width="88%" alt="AIHubMix API Keys page with default and preferred base URLs and the Create API key button" />
40
+ </p>
41
+
42
+ Select **Create API key**, then:
43
+
44
+ 1. Enter a recognizable name such as `dsh-vision-toolkit`.
45
+ 2. For an initial trial, turn off unlimited quota and set a small quota, expiration date, model range, or IP restriction so a later paid-model mistake cannot spend without a limit.
46
+ 3. Select **Submit** to create the key.
47
+ 4. Copy the complete `sk-...` value immediately and store it in a password manager or DSH Credential.
48
+
49
+ <p align="center">
50
+ <img src="assets/aihubmix-create-key.png" width="56%" alt="AIHubMix Create API key form" />
51
+ </p>
52
+
53
+ Do not place the complete key in a README, chat transcript, screenshot, Git commit, browser frontend, or public log. Delete and replace a key immediately if it becomes public.
54
+
55
+ ## 3. Select the free vision model
56
+
57
+ Use this exact model ID:
58
+
59
+ ```text
60
+ gemini-3.7-flash-free
61
+ ```
62
+
63
+ The model page lists it as free and shows text, image, audio, video, and PDF input. Vision Toolkit uses its image input and text output.
64
+
65
+ <p align="center">
66
+ <img src="assets/aihubmix-free-vision-model.png" width="92%" alt="AIHubMix Gemini 3.7 Flash free model page showing image input" />
67
+ </p>
68
+
69
+ The free route is for trials and can return `429 Too Many Requests` when capacity is exhausted. For reliable usage, switch the model to `gemini-3.7-flash` and ensure the account has sufficient credit. Optional capabilities such as web search or cache storage may have separate charges; ordinary Vision Toolkit image analysis does not enable them.
70
+
71
+ ## 4. Configure DSH Vision Toolkit
72
+
73
+ 1. Open **Settings → Vision** in DSH Web.
74
+ 2. Enter these values under the vision service:
75
+
76
+ | Field | Value |
77
+ |---|---|
78
+ | API protocol | `OpenAI Chat Completions` |
79
+ | Base URL | `https://api.inferera.com/v1` |
80
+ | Model | `gemini-3.7-flash-free` |
81
+ | API key | Paste the newly created `sk-...` key |
82
+
83
+ 3. Expand **Advanced settings** and change **Credential name** to `AIHUBMIX_API_KEY` instead of reusing the built-in free provider's credential name.
84
+ 4. Select **Save and apply**. The key is stored in DSH Credentials and is not displayed again in full.
85
+ 5. Select **Test vision model**. This action sends the bundled diagnostic image and verifies a real multimodal request; **Test API connection**, which only queries `/models`, is not a substitute.
86
+ 6. After the test succeeds, paste an image into a session and ask a concrete question, for example:
87
+
88
+ ```text
89
+ Transcribe the error in this screenshot exactly, then explain the most likely cause and the repair steps.
90
+ ```
91
+
92
+ The same provider can be stored in a Profile patch. Keep the key value in DSH Credentials rather than YAML:
93
+
94
+ ```yaml
95
+ - id: vision-toolkit
96
+ config:
97
+ provider:
98
+ protocol: openai
99
+ baseUrl: https://api.inferera.com/v1
100
+ model: gemini-3.7-flash-free
101
+ credential: AIHUBMIX_API_KEY
102
+ ```
103
+
104
+ ## 5. Troubleshooting
105
+
106
+ ### `401` or `Unauthorized`
107
+
108
+ - Confirm that you pasted the complete `sk-...` value, not the key name or a masked fragment.
109
+ - Remove extra spaces, line breaks, or an `AIHUBMIX_API_KEY=` prefix from the value.
110
+ - Delete and replace a key that has appeared in a public location.
111
+
112
+ ### `402`, insufficient credit, or an unavailable model
113
+
114
+ - The free trial route must use the full `gemini-3.7-flash-free` model ID.
115
+ - If you switch to `gemini-3.7-flash` or another paid model, add credit from **Credits** in the console.
116
+ - Model availability changes; check the current AIHubMix model page and console.
117
+
118
+ ### `404` or `model not found`
119
+
120
+ - Use exactly `gemini-3.7-flash-free`.
121
+ - Do not send the display name `Gemini 3.7 Flash (free)` as the API model ID.
122
+
123
+ ### `429 Too Many Requests`
124
+
125
+ - Free-model capacity is limited. Wait and retry, or lower concurrency and request frequency.
126
+ - Switch to `gemini-3.7-flash` when stable production service is required.
127
+
128
+ ### Connection timeout or unreachable endpoint
129
+
130
+ - Start with this guide's `https://api.inferera.com/v1` endpoint.
131
+ - If it is unavailable, switch to `https://aihubmix.com/v1` and rerun **Test vision model**.
132
+
133
+ ## Official resources
134
+
135
+ - [Inferera signup entry](https://inferera.com/?aff=5wj6sgx8)
136
+ - [AIHubMix API Keys](https://console.aihubmix.com/token)
137
+ - [AIHubMix documentation](https://docs.aihubmix.com/en)
138
+ - [Gemini 3.7 Flash (free) model page](https://aihubmix.com/model/gemini-3.7-flash-free)
@@ -0,0 +1,138 @@
1
+ # 申请 AIHubMix API Key,并用免费 Gemini 3.7 Flash 识图
2
+
3
+ [English](aihubmix-gemini-vision.md) | **中文**
4
+
5
+ 这篇教程完成三件事:
6
+
7
+ 1. 注册 AIHubMix 账号。
8
+ 2. 创建并安全保存一个 AIHubMix API Key。
9
+ 3. 在 DSH Vision Toolkit 中使用免费的 `gemini-3.7-flash-free` 分析图片。
10
+
11
+ > 截至 2026-08-20,AIHubMix 模型页把 `gemini-3.7-flash-free` 标为支持图像输入的免费试用模型。免费资源有限,可能返回 `429`,不保证生产环境稳定性;生产用途应切换到正式模型 `gemini-3.7-flash`。模型、价格和可用性可能调整,请以 [AIHubMix 模型页](https://aihubmix.com/model/gemini-3.7-flash-free) 为准。
12
+
13
+ ## 1. 打开申请入口并注册
14
+
15
+ 打开 [Inferera 国内申请入口](https://inferera.com/?aff=5wj6sgx8),页面会跳转到 AIHubMix;点击右上角 **注册**,也可以点击页面中的 **立即体验**。
16
+
17
+ 这个入口带有本项目的推荐参数;如果不想使用推荐链接,也可以直接访问 [Inferera](https://inferera.com/)。
18
+
19
+ <p align="center">
20
+ <img src="assets/aihubmix-home.png" width="92%" alt="AIHubMix 中文首页与注册入口" />
21
+ </p>
22
+
23
+ 注册页支持 GitHub、Google 或邮箱。使用邮箱时,填写邮箱和密码、同意服务条款,然后按页面提示完成验证。
24
+
25
+ <p align="center">
26
+ <img src="assets/aihubmix-sign-up.png" width="92%" alt="AIHubMix 中文注册页面" />
27
+ </p>
28
+
29
+ ## 2. 创建 API Key
30
+
31
+ 登录后进入控制台左侧的 **开发 → API Keys**,或直接打开 [AIHubMix API Keys](https://console.aihubmix.com/token)。页面会同时显示两个接口地址:
32
+
33
+ - 默认地址:`https://aihubmix.com`
34
+ - 优选地址:`https://api.inferera.com`
35
+
36
+ 本教程在 Vision Toolkit 中使用优选地址 `https://api.inferera.com/v1`。如果该地址在你的网络中不可用,可以改用 `https://aihubmix.com/v1`。
37
+
38
+ <p align="center">
39
+ <img src="assets/aihubmix-api-keys.png" width="88%" alt="AIHubMix API Keys 页面中的默认地址、优选地址和 Create API key 按钮" />
40
+ </p>
41
+
42
+ 点击 **创建 Key**(英文界面显示 **Create API key**),然后:
43
+
44
+ 1. 在 **名称** 中填写容易辨认的名称,例如 `dsh-vision-toolkit`。
45
+ 2. 首次试用建议关闭无限额度,并设置小额上限、过期时间、模型范围或 IP 限制,避免以后误用付费模型时产生无上限费用。
46
+ 3. 点击 **提交** 创建密钥。
47
+ 4. 立即复制完整的 `sk-...` 密钥,并保存到密码管理器或 DSH Credential 中。
48
+
49
+ <p align="center">
50
+ <img src="assets/aihubmix-create-key.png" width="56%" alt="AIHubMix Create API key 表单" />
51
+ </p>
52
+
53
+ 不要把完整 API Key 放进 README、聊天记录、截图、Git 提交、浏览器前端代码或公开日志。密钥泄露后应立即删除旧 Key 并重新创建。
54
+
55
+ ## 3. 选择免费的视觉模型
56
+
57
+ 本教程使用准确模型 ID:
58
+
59
+ ```text
60
+ gemini-3.7-flash-free
61
+ ```
62
+
63
+ 模型页将它标为免费,并列出文本、图像、音频、视频和 PDF 输入能力。Vision Toolkit 使用其中的图像输入和文本输出。
64
+
65
+ <p align="center">
66
+ <img src="assets/aihubmix-free-vision-model.png" width="92%" alt="AIHubMix Gemini 3.7 Flash free 模型页显示图像输入能力" />
67
+ </p>
68
+
69
+ 免费版本只适合试用,资源紧张时可能返回 `429 Too Many Requests`。需要稳定调用时,把模型改为 `gemini-3.7-flash`,并确认账户余额足够。模型页列出的联网搜索、缓存等可选能力可能单独计费;Vision Toolkit 的普通图片分析不会主动启用这些能力。
70
+
71
+ ## 4. 在 DSH Vision Toolkit 中使用
72
+
73
+ 1. 打开 DSH Web 的 **设置 → 视觉工具**。
74
+ 2. 在“在线视觉服务”中填写:
75
+
76
+ | 字段 | 值 |
77
+ |---|---|
78
+ | API 协议 | `OpenAI Chat Completions` |
79
+ | API 地址 | `https://api.inferera.com/v1` |
80
+ | 模型名称 | `gemini-3.7-flash-free` |
81
+ | API 密钥 | 粘贴刚创建的 `sk-...` 密钥 |
82
+
83
+ 3. 展开 **高级设置**,把 **凭据名称** 改为 `AIHUBMIX_API_KEY`,避免继续使用内置免费服务的凭据名称。
84
+ 4. 点击 **保存设置**。密钥会写入 DSH Credentials,页面以后不会回显完整内容。
85
+ 5. 点击 **测试视觉模型**。这个测试会发送插件自带的诊断图片,验证真实多模态请求;只测试 `/models` 的 **测试 API 连接** 不能代替它。
86
+ 6. 测试成功后,在会话中粘贴图片并直接提问,例如:
87
+
88
+ ```text
89
+ 请先逐字抄出截图中的报错,再判断最可能的原因和修复步骤。
90
+ ```
91
+
92
+ 也可以在 Profile patch 中保存相同配置。密钥值仍应保存在 DSH Credential 中,不要写入 YAML:
93
+
94
+ ```yaml
95
+ - id: vision-toolkit
96
+ config:
97
+ provider:
98
+ protocol: openai
99
+ baseUrl: https://api.inferera.com/v1
100
+ model: gemini-3.7-flash-free
101
+ credential: AIHUBMIX_API_KEY
102
+ ```
103
+
104
+ ## 5. 常见问题
105
+
106
+ ### `401` 或 `Unauthorized`
107
+
108
+ - 确认粘贴的是完整的 `sk-...` 密钥,而不是 Key 名称或被遮盖后的片段。
109
+ - 粘贴的密钥值中不要包含多余空格、换行或 `AIHUBMIX_API_KEY=` 前缀。
110
+ - 如果密钥曾经公开,删除旧 Key 并重新创建。
111
+
112
+ ### `402`、余额不足或模型不可用
113
+
114
+ - 免费试用模型应使用完整 ID `gemini-3.7-flash-free`。
115
+ - 如果改用 `gemini-3.7-flash` 或其他付费模型,请在控制台的 **余额充值** 中补充余额。
116
+ - 模型可用性会变化,请以 AIHubMix 当前模型页和控制台为准。
117
+
118
+ ### `404` 或 `model not found`
119
+
120
+ - 模型名必须准确写成 `gemini-3.7-flash-free`。
121
+ - 不要把显示名称 `Gemini 3.7 Flash (free)` 当作 API 模型 ID。
122
+
123
+ ### `429 Too Many Requests`
124
+
125
+ - 免费模型资源有限,等待后重试,或降低并发和调用频率。
126
+ - 需要稳定生产调用时切换到 `gemini-3.7-flash`。
127
+
128
+ ### 连接超时或无法访问
129
+
130
+ - 先使用本教程推荐的 `https://api.inferera.com/v1`。
131
+ - 如果该地址不可用,改为 `https://aihubmix.com/v1` 后重新执行 **测试视觉模型**。
132
+
133
+ ## 官方资料
134
+
135
+ - [Inferera 国内申请入口](https://inferera.com/?aff=5wj6sgx8)
136
+ - [AIHubMix API Keys](https://console.aihubmix.com/token)
137
+ - [AIHubMix 中文文档](https://docs.aihubmix.com/cn)
138
+ - [Gemini 3.7 Flash (free) 模型页](https://aihubmix.com/model/gemini-3.7-flash-free)
Binary file
Binary file
Binary file
Binary file