@mengruo/dsh-vision-toolkit 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.i18n.yaml +6 -0
- package/README.md +356 -0
- package/README.zh.md +354 -0
- package/assets/community-group-qr.png +0 -0
- package/assets/dsh-conversation-artifact.png +0 -0
- package/assets/dsh-conversation-image-qa-top.png +0 -0
- package/assets/dsh-conversation-image-qa.png +0 -0
- package/assets/dsh-conversation-pixel-diff.png +0 -0
- package/assets/dsh-conversation-screenshot-debugging-top.png +0 -0
- package/assets/dsh-conversation-screenshot-debugging.png +0 -0
- package/assets/dsh-conversation-tool-call.png +0 -0
- package/assets/dsh-conversation-vision-trace.png +0 -0
- package/assets/dsh-view-example.png +0 -0
- package/assets/hero-v2.png +0 -0
- package/assets/logo_aihubmix.png +0 -0
- package/assets/python-bootstrap.json +48 -0
- package/assets/skill/SKILL.md +329 -0
- package/assets/skill/UPSTREAM.json +71 -0
- package/assets/skill/references/gui.md +88 -0
- package/assets/skill/references/long-screenshot-ocr.md +77 -0
- package/assets/skill/references/restore-graphic.md +84 -0
- package/assets/skill/references/restore-structure.md +45 -0
- package/assets/skill/references/restore-ui.md +202 -0
- package/assets/social-preview.png +0 -0
- package/assets/upstream/README.md +18 -0
- package/assets/upstream/focus-hint-comparison-1.webp +0 -0
- package/assets/upstream/focus-hint-comparison-2.webp +0 -0
- package/assets/upstream/infographic-reference.webp +0 -0
- package/assets/upstream/infographic-result.webp +0 -0
- package/assets/upstream/ui-fast-restore-reference.webp +0 -0
- package/assets/upstream/ui-fast-restore-result.webp +0 -0
- package/assets/upstream/ui-result.webp +0 -0
- package/assets/upstream/ui-sketch.webp +0 -0
- package/assets/vision-model-test.png +0 -0
- package/assets/vision-settings.png +0 -0
- package/assets/wechat-reward.png +0 -0
- package/cordis.patch.yml +6 -0
- package/docs/aihubmix-gemini-vision.i18n.yaml +6 -0
- package/docs/aihubmix-gemini-vision.md +138 -0
- package/docs/aihubmix-gemini-vision.zh.md +138 -0
- package/docs/assets/aihubmix-api-keys.png +0 -0
- package/docs/assets/aihubmix-create-key.png +0 -0
- package/docs/assets/aihubmix-free-vision-model.png +0 -0
- package/docs/assets/aihubmix-home.png +0 -0
- package/docs/assets/aihubmix-sign-up.png +0 -0
- package/docs/assets/groq-console-home.png +0 -0
- package/docs/assets/groq-console-keys-login.png +0 -0
- package/docs/assets/groq-docs-quickstart.png +0 -0
- package/docs/assets/groq-docs-qwen3.6.png +0 -0
- package/docs/assets/groq-docs-vision.png +0 -0
- package/docs/assets/vision-settings.png +0 -0
- package/docs/dsh-desktop-install.md +67 -0
- package/docs/dsh-desktop-install.zh.md +67 -0
- package/docs/groq-qwen3.6-vision.i18n.yaml +6 -0
- package/docs/groq-qwen3.6-vision.md +236 -0
- package/docs/groq-qwen3.6-vision.zh.md +236 -0
- package/docs/python-runtime.i18n.yaml +6 -0
- package/docs/python-runtime.md +89 -0
- package/docs/python-runtime.zh.md +89 -0
- package/docs/requirements-traceability/README.i18n.yaml +6 -0
- package/docs/requirements-traceability/README.md +75 -0
- package/docs/requirements-traceability/README.zh.md +75 -0
- package/examples/ui-restoration/README.i18n.yaml +6 -0
- package/examples/ui-restoration/README.md +70 -0
- package/examples/ui-restoration/README.zh.md +70 -0
- package/examples/ui-restoration/assets/final-heatmap.png +0 -0
- package/examples/ui-restoration/assets/final-report.json +83 -0
- package/examples/ui-restoration/assets/implementation.png +0 -0
- package/examples/ui-restoration/assets/initial-heatmap.png +0 -0
- package/examples/ui-restoration/assets/initial-report.json +83 -0
- package/examples/ui-restoration/assets/initial.png +0 -0
- package/examples/ui-restoration/assets/metrics.json +12 -0
- package/examples/ui-restoration/assets/reference.png +0 -0
- package/examples/ui-restoration/implementation.html +94 -0
- package/examples/ui-restoration/initial.html +57 -0
- package/lib/artifact-access.js +369 -0
- package/lib/artifact-access.js.map +1 -0
- package/lib/artifacts.js +56 -0
- package/lib/artifacts.js.map +1 -0
- package/lib/client.js +2099 -0
- package/lib/client.js.map +1 -0
- package/lib/config.js +172 -0
- package/lib/config.js.map +1 -0
- package/lib/defaults.js +6 -0
- package/lib/defaults.js.map +1 -0
- package/lib/errors.js +56 -0
- package/lib/errors.js.map +1 -0
- package/lib/evidence-cache.js +350 -0
- package/lib/evidence-cache.js.map +1 -0
- package/lib/exposure.js +247 -0
- package/lib/exposure.js.map +1 -0
- package/lib/image-input-variants.js +882 -0
- package/lib/image-input-variants.js.map +1 -0
- package/lib/index.js +105 -0
- package/lib/index.js.map +1 -0
- package/lib/paste-images.js +217 -0
- package/lib/paste-images.js.map +1 -0
- package/lib/paths.js +354 -0
- package/lib/paths.js.map +1 -0
- package/lib/plugin-update.js +1003 -0
- package/lib/plugin-update.js.map +1 -0
- package/lib/runtime-install.js +964 -0
- package/lib/runtime-install.js.map +1 -0
- package/lib/runtime-manager.js +133 -0
- package/lib/runtime-manager.js.map +1 -0
- package/lib/runtime.js +1678 -0
- package/lib/runtime.js.map +1 -0
- package/lib/skill.js +25 -0
- package/lib/skill.js.map +1 -0
- package/lib/tools.js +549 -0
- package/lib/tools.js.map +1 -0
- package/lib/types/artifact-access.d.ts +61 -0
- package/lib/types/artifact-access.d.ts.map +1 -0
- package/lib/types/artifacts.d.ts +42 -0
- package/lib/types/artifacts.d.ts.map +1 -0
- package/lib/types/client/display-config.d.ts +24 -0
- package/lib/types/client/display-config.d.ts.map +1 -0
- package/lib/types/client/index.d.ts +350 -0
- package/lib/types/client/index.d.ts.map +1 -0
- package/lib/types/client/model-variants-hider.d.ts +40 -0
- package/lib/types/client/model-variants-hider.d.ts.map +1 -0
- package/lib/types/client/paste-images.d.ts +126 -0
- package/lib/types/client/paste-images.d.ts.map +1 -0
- package/lib/types/config.d.ts +123 -0
- package/lib/types/config.d.ts.map +1 -0
- package/lib/types/defaults.d.ts +6 -0
- package/lib/types/defaults.d.ts.map +1 -0
- package/lib/types/errors.d.ts +35 -0
- package/lib/types/errors.d.ts.map +1 -0
- package/lib/types/evidence-cache.d.ts +91 -0
- package/lib/types/evidence-cache.d.ts.map +1 -0
- package/lib/types/exposure.d.ts +50 -0
- package/lib/types/exposure.d.ts.map +1 -0
- package/lib/types/image-input-variants.d.ts +144 -0
- package/lib/types/image-input-variants.d.ts.map +1 -0
- package/lib/types/index.d.ts +19 -0
- package/lib/types/index.d.ts.map +1 -0
- package/lib/types/paste-images.d.ts +73 -0
- package/lib/types/paste-images.d.ts.map +1 -0
- package/lib/types/paths.d.ts +117 -0
- package/lib/types/paths.d.ts.map +1 -0
- package/lib/types/plugin-update.d.ts +119 -0
- package/lib/types/plugin-update.d.ts.map +1 -0
- package/lib/types/runtime-install.d.ts +92 -0
- package/lib/types/runtime-install.d.ts.map +1 -0
- package/lib/types/runtime-manager.d.ts +60 -0
- package/lib/types/runtime-manager.d.ts.map +1 -0
- package/lib/types/runtime.d.ts +412 -0
- package/lib/types/runtime.d.ts.map +1 -0
- package/lib/types/skill.d.ts +14 -0
- package/lib/types/skill.d.ts.map +1 -0
- package/lib/types/tools.d.ts +35 -0
- package/lib/types/tools.d.ts.map +1 -0
- package/lib/types/upstream.d.ts +233 -0
- package/lib/types/upstream.d.ts.map +1 -0
- package/lib/types/version.d.ts +15 -0
- package/lib/types/version.d.ts.map +1 -0
- package/lib/types/web-request.d.ts +11 -0
- package/lib/types/web-request.d.ts.map +1 -0
- package/lib/types/web.d.ts +118 -0
- package/lib/types/web.d.ts.map +1 -0
- package/lib/upstream.js +860 -0
- package/lib/upstream.js.map +1 -0
- package/lib/version.js +18 -0
- package/lib/version.js.map +1 -0
- package/lib/web-request.js +29 -0
- package/lib/web-request.js.map +1 -0
- package/lib/web.js +462 -0
- package/lib/web.js.map +1 -0
- package/package.json +149 -0
- package/patches/vision-tools-dsh.patch +970 -0
- package/runtime/requirements.lock +3 -0
- package/src/artifact-access.ts +386 -0
- package/src/artifacts.ts +85 -0
- package/src/client/display-config.ts +62 -0
- package/src/client/index.tsx +1580 -0
- package/src/client/model-variants-hider.ts +159 -0
- package/src/client/paste-images.tsx +747 -0
- package/src/config.ts +290 -0
- package/src/defaults.ts +5 -0
- package/src/errors.ts +62 -0
- package/src/evidence-cache.ts +418 -0
- package/src/exposure.ts +265 -0
- package/src/image-input-variants.ts +996 -0
- package/src/index.ts +141 -0
- package/src/paste-images.ts +286 -0
- package/src/paths.ts +383 -0
- package/src/plugin-update.ts +1143 -0
- package/src/runtime-install.ts +1161 -0
- package/src/runtime-manager.ts +174 -0
- package/src/runtime.ts +2189 -0
- package/src/skill.ts +35 -0
- package/src/tools.ts +691 -0
- package/src/upstream.ts +1078 -0
- package/src/version.ts +37 -0
- package/src/web-request.ts +27 -0
- package/src/web.ts +594 -0
- package/vendor/agent-vision-toolkit/CHANGELOG.md +30 -0
- package/vendor/agent-vision-toolkit/LICENSE +21 -0
- package/vendor/agent-vision-toolkit/README.md +405 -0
- package/vendor/agent-vision-toolkit/UPSTREAM_MANIFEST.json +94 -0
- package/vendor/agent-vision-toolkit/bin/crop +90 -0
- package/vendor/agent-vision-toolkit/bin/detect +13 -0
- package/vendor/agent-vision-toolkit/bin/glance +93 -0
- package/vendor/agent-vision-toolkit/bin/ground +13 -0
- package/vendor/agent-vision-toolkit/bin/trace +129 -0
- package/vendor/agent-vision-toolkit/detect.py +60 -0
- package/vendor/agent-vision-toolkit/ground.py +267 -0
- package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/dominant_colors.py +224 -0
- package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/extract_fg.py +278 -0
- package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/html_shot.py +420 -0
- package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/long_screenshot_ocr.py +1245 -0
- package/vendor/agent-vision-toolkit/skills/vision-tools/scripts/pixel_diff.py +88 -0
- package/vendor/agent-vision-toolkit/tests/test_vision_client.py +450 -0
- package/vendor/agent-vision-toolkit/vision_client.py +310 -0
package/src/version.ts
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Plugin and pinned upstream version facts. The upstream snapshot is fixed at
|
|
3
|
+
* build time and recorded in package.json's `dsh.visionToolkit` metadata so
|
|
4
|
+
* the runtime, tool results, and docs all report the same source commit.
|
|
5
|
+
* @module dsh-vision-toolkit/version
|
|
6
|
+
*/
|
|
7
|
+
|
|
8
|
+
import { readFileSync } from 'node:fs'
|
|
9
|
+
|
|
10
|
+
interface VisionToolkitPackageMetadata {
|
|
11
|
+
version: string
|
|
12
|
+
dsh?: {
|
|
13
|
+
visionToolkit?: {
|
|
14
|
+
upstreamRepository?: string
|
|
15
|
+
upstreamVersion?: string
|
|
16
|
+
upstreamCommit?: string
|
|
17
|
+
}
|
|
18
|
+
}
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
const metadata = JSON.parse(
|
|
22
|
+
readFileSync(new URL('../package.json', import.meta.url), 'utf8'),
|
|
23
|
+
) as VisionToolkitPackageMetadata
|
|
24
|
+
|
|
25
|
+
const visionToolkit = metadata.dsh?.visionToolkit
|
|
26
|
+
|
|
27
|
+
/** Plugin package version. */
|
|
28
|
+
export const PLUGIN_VERSION = metadata.version
|
|
29
|
+
|
|
30
|
+
/** Pinned upstream repository URL. */
|
|
31
|
+
export const UPSTREAM_REPOSITORY = visionToolkit?.upstreamRepository ?? 'https://github.com/Anionex/agent-vision-toolkit'
|
|
32
|
+
|
|
33
|
+
/** Pinned upstream release tag. */
|
|
34
|
+
export const UPSTREAM_VERSION = visionToolkit?.upstreamVersion ?? 'v0.1.0+snapshot.8ccf9d7'
|
|
35
|
+
|
|
36
|
+
/** Pinned upstream source commit. */
|
|
37
|
+
export const UPSTREAM_COMMIT = visionToolkit?.upstreamCommit ?? '8ccf9d7d89bc395ad7dd19dde840401a5b84623d'
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
import type { IncomingMessage } from 'node:http'
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* Accept a request only from the DSH Web application's origin. Method-agnostic:
|
|
5
|
+
* the same fence guards state-changing POSTs and policy GETs.
|
|
6
|
+
* @param req - the incoming request whose headers carry the origin evidence.
|
|
7
|
+
* @returns whether the request may be answered.
|
|
8
|
+
*/
|
|
9
|
+
export function sameOriginRequest(req: IncomingMessage): boolean {
|
|
10
|
+
const fetchSite = req.headers['sec-fetch-site']
|
|
11
|
+
if (fetchSite === 'cross-site') return false
|
|
12
|
+
const origin = req.headers.origin
|
|
13
|
+
if (origin === undefined) return fetchSite === 'same-origin' || fetchSite === 'same-site' || fetchSite === 'none'
|
|
14
|
+
const host = req.headers.host
|
|
15
|
+
if (host === undefined) return false
|
|
16
|
+
try {
|
|
17
|
+
const parsed = new URL(origin)
|
|
18
|
+
return (parsed.protocol === 'http:' || parsed.protocol === 'https:') && parsed.host === host
|
|
19
|
+
} catch {
|
|
20
|
+
return false
|
|
21
|
+
}
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
/** Accept state-changing requests only from the DSH Web application's origin. */
|
|
25
|
+
export function sameOriginPost(req: IncomingMessage): boolean {
|
|
26
|
+
return sameOriginRequest(req)
|
|
27
|
+
}
|
package/src/web.ts
ADDED
|
@@ -0,0 +1,594 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Optional Web-profile routes: signed Artifact delivery plus a same-origin
|
|
3
|
+
* Settings/health endpoint. The browser never receives credential values and
|
|
4
|
+
* connection tests run only after an explicit POST action.
|
|
5
|
+
* @module dsh-vision-toolkit/web
|
|
6
|
+
*/
|
|
7
|
+
|
|
8
|
+
import type { IncomingMessage, ServerResponse } from 'node:http'
|
|
9
|
+
import type { Context } from '@deepseek-ai/cordis'
|
|
10
|
+
import type { CredentialInfo, CredentialRef } from '@deepseek-ai/dsh-credentials'
|
|
11
|
+
import { credentialRef } from '@deepseek-ai/dsh-credentials'
|
|
12
|
+
import { SettingsConflictError, type SettingsDescriptor } from '@deepseek-ai/dsh-settings'
|
|
13
|
+
// Type-only import activates the optional webServer Context declaration.
|
|
14
|
+
import type {} from '@deepseek-ai/dsh-host-webserver'
|
|
15
|
+
import { ArtifactAccessController, ARTIFACT_ROUTE_PREFIX } from './artifact-access.ts'
|
|
16
|
+
import {
|
|
17
|
+
PastedImageBackend,
|
|
18
|
+
PASTE_IMAGES_ROUTE,
|
|
19
|
+
PASTE_POLICY_ROUTE,
|
|
20
|
+
type PasteSelectionQuery,
|
|
21
|
+
type PasteVerdict,
|
|
22
|
+
} from './paste-images.ts'
|
|
23
|
+
import {
|
|
24
|
+
resolveConfig,
|
|
25
|
+
isBuiltInFreeVisionProvider,
|
|
26
|
+
VISION_TOOLKIT_SETTINGS_NAMESPACE,
|
|
27
|
+
type ResolvedVisionToolkitConfig,
|
|
28
|
+
type VisionToolkitConfig,
|
|
29
|
+
} from './config.ts'
|
|
30
|
+
import type { VisionToolkitHealthResult } from './runtime.ts'
|
|
31
|
+
import {
|
|
32
|
+
PluginUpdateError,
|
|
33
|
+
VisionToolkitPluginUpdateService,
|
|
34
|
+
type PluginUpdateCapability,
|
|
35
|
+
type PluginUpdateCheck,
|
|
36
|
+
type PluginUpdateResult,
|
|
37
|
+
} from './plugin-update.ts'
|
|
38
|
+
import {
|
|
39
|
+
VisionToolkitRuntimeManager,
|
|
40
|
+
type PreparedRuntimeGeneration,
|
|
41
|
+
type RuntimeManagerStatus,
|
|
42
|
+
} from './runtime-manager.ts'
|
|
43
|
+
import { PLUGIN_VERSION, UPSTREAM_COMMIT, UPSTREAM_REPOSITORY, UPSTREAM_VERSION } from './version.ts'
|
|
44
|
+
import { sameOriginPost, sameOriginRequest } from './web-request.ts'
|
|
45
|
+
|
|
46
|
+
/** Exact route used by the browser Settings page. */
|
|
47
|
+
export const SETTINGS_ROUTE = '/_dsh/vision-toolkit/settings'
|
|
48
|
+
|
|
49
|
+
/** Same-origin route used by the browser client to read display-mode flags. */
|
|
50
|
+
export const DISPLAY_CONFIG_ROUTE = '/_dsh/vision-toolkit/display-config'
|
|
51
|
+
|
|
52
|
+
/** Public Settings snapshot; credential values are deliberately impossible here. */
|
|
53
|
+
export interface VisionToolkitSettingsSnapshot {
|
|
54
|
+
schemaVersion: 1
|
|
55
|
+
writable: boolean
|
|
56
|
+
settings: {
|
|
57
|
+
value: VisionToolkitConfig
|
|
58
|
+
user?: unknown
|
|
59
|
+
base?: unknown
|
|
60
|
+
revision: number
|
|
61
|
+
applies: 'live'
|
|
62
|
+
}
|
|
63
|
+
credential: {
|
|
64
|
+
ref: string
|
|
65
|
+
configured: boolean
|
|
66
|
+
source?: string
|
|
67
|
+
writable: boolean
|
|
68
|
+
}
|
|
69
|
+
runtime: RuntimeManagerStatus
|
|
70
|
+
release: {
|
|
71
|
+
pluginVersion: string
|
|
72
|
+
upstreamRepository: string
|
|
73
|
+
upstreamVersion: string
|
|
74
|
+
upstreamCommit: string
|
|
75
|
+
update: PluginUpdateCapability
|
|
76
|
+
}
|
|
77
|
+
artifactRouteAvailable: boolean
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
interface SaveRequest {
|
|
81
|
+
action: 'save'
|
|
82
|
+
expectedRevision: number
|
|
83
|
+
value: VisionToolkitConfig
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
interface HealthRequest {
|
|
87
|
+
action: 'health'
|
|
88
|
+
testConnection: boolean
|
|
89
|
+
testModel: boolean
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
interface CredentialRequest {
|
|
93
|
+
action: 'credential'
|
|
94
|
+
expectedRevision: number
|
|
95
|
+
ref: CredentialRef
|
|
96
|
+
value: string
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
interface CheckUpdateRequest {
|
|
100
|
+
action: 'check-update'
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
interface ApplyUpdateRequest {
|
|
104
|
+
action: 'apply-update'
|
|
105
|
+
expectedVersion: string
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
type SettingsRequest = SaveRequest | HealthRequest | CredentialRequest | CheckUpdateRequest | ApplyUpdateRequest
|
|
109
|
+
|
|
110
|
+
interface JsonError {
|
|
111
|
+
ok: false
|
|
112
|
+
error: { code: string; message: string }
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
interface JsonSuccess<T> {
|
|
116
|
+
ok: true
|
|
117
|
+
value: T
|
|
118
|
+
}
|
|
119
|
+
|
|
120
|
+
type JsonResponse<T> = JsonSuccess<T> | JsonError
|
|
121
|
+
|
|
122
|
+
/** Minimal runtime-manager face used by the Web route and its tests. */
|
|
123
|
+
export interface WebRuntimeManager {
|
|
124
|
+
readonly ready: boolean
|
|
125
|
+
current(): ReturnType<VisionToolkitRuntimeManager['current']>
|
|
126
|
+
prepareCandidate(raw: VisionToolkitConfig): Promise<PreparedRuntimeGeneration>
|
|
127
|
+
activateCandidate(candidate: PreparedRuntimeGeneration): void
|
|
128
|
+
recordFailure(error: unknown): void
|
|
129
|
+
status(): RuntimeManagerStatus
|
|
130
|
+
}
|
|
131
|
+
|
|
132
|
+
/** Minimal self-update face used by the Web route and its tests. */
|
|
133
|
+
export interface WebPluginUpdater {
|
|
134
|
+
configureWebServer?(host: string, port: number): void
|
|
135
|
+
capability(): Promise<PluginUpdateCapability>
|
|
136
|
+
check(): Promise<PluginUpdateCheck>
|
|
137
|
+
installAndRestart(expectedVersion: string): Promise<PluginUpdateResult>
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
/** Callback invoked when a Settings save makes the first runtime available. */
|
|
141
|
+
export type RuntimeActivated = () => void
|
|
142
|
+
|
|
143
|
+
function isRecord(value: unknown): value is Record<string, unknown> {
|
|
144
|
+
return typeof value === 'object' && value !== null && !Array.isArray(value)
|
|
145
|
+
}
|
|
146
|
+
|
|
147
|
+
class CredentialReferenceConflictError extends Error {}
|
|
148
|
+
|
|
149
|
+
function descriptorOf(ctx: Context): SettingsDescriptor {
|
|
150
|
+
const descriptor = ctx.settings.describe().find(row => row.ns === VISION_TOOLKIT_SETTINGS_NAMESPACE)
|
|
151
|
+
if (descriptor === undefined) throw new Error('vision-toolkit Settings namespace is not registered')
|
|
152
|
+
return descriptor
|
|
153
|
+
}
|
|
154
|
+
|
|
155
|
+
function responseJson<T>(res: ServerResponse, status: number, body: JsonResponse<T>): void {
|
|
156
|
+
const bytes = Buffer.from(JSON.stringify(body))
|
|
157
|
+
res.setHeader('Content-Type', 'application/json; charset=utf-8')
|
|
158
|
+
res.setHeader('Content-Length', String(bytes.length))
|
|
159
|
+
res.setHeader('Cache-Control', 'no-store')
|
|
160
|
+
res.setHeader('X-Content-Type-Options', 'nosniff')
|
|
161
|
+
res.setHeader('Content-Security-Policy', "default-src 'none'; frame-ancestors 'none'")
|
|
162
|
+
res.writeHead(status)
|
|
163
|
+
res.end(bytes)
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
function requestError(res: ServerResponse, status: number, code: string, message: string): void {
|
|
167
|
+
responseJson(res, status, { ok: false, error: { code, message } })
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
async function readJson(req: IncomingMessage, maxBytes = 64 * 1024): Promise<unknown> {
|
|
171
|
+
const contentType = req.headers['content-type']?.split(';', 1)[0]?.trim().toLowerCase()
|
|
172
|
+
if (contentType !== 'application/json') throw new TypeError('Content-Type must be application/json')
|
|
173
|
+
const chunks: Buffer[] = []
|
|
174
|
+
let bytes = 0
|
|
175
|
+
for await (const chunk of req) {
|
|
176
|
+
const part = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk)
|
|
177
|
+
bytes += part.length
|
|
178
|
+
if (bytes > maxBytes) throw new RangeError(`request body exceeds ${maxBytes} bytes`)
|
|
179
|
+
chunks.push(part)
|
|
180
|
+
}
|
|
181
|
+
if (chunks.length === 0) throw new TypeError('request body is empty')
|
|
182
|
+
return JSON.parse(Buffer.concat(chunks).toString('utf8')) as unknown
|
|
183
|
+
}
|
|
184
|
+
|
|
185
|
+
function parseRequest(value: unknown): SettingsRequest {
|
|
186
|
+
if (!isRecord(value) || typeof value.action !== 'string') throw new TypeError('request action is required')
|
|
187
|
+
if (value.action === 'health') {
|
|
188
|
+
if (typeof value.testConnection !== 'boolean') throw new TypeError('health.testConnection must be boolean')
|
|
189
|
+
const testModel = value.testModel === undefined ? false : value.testModel
|
|
190
|
+
if (typeof testModel !== 'boolean') throw new TypeError('health.testModel must be boolean')
|
|
191
|
+
if (testModel && !value.testConnection) throw new TypeError('health.testModel requires health.testConnection')
|
|
192
|
+
return { action: 'health', testConnection: value.testConnection, testModel }
|
|
193
|
+
}
|
|
194
|
+
if (value.action === 'save') {
|
|
195
|
+
if (!Number.isSafeInteger(value.expectedRevision) || (value.expectedRevision as number) < 0) {
|
|
196
|
+
throw new TypeError('save.expectedRevision must be a non-negative integer')
|
|
197
|
+
}
|
|
198
|
+
if (!isRecord(value.value)) throw new TypeError('save.value must be an object')
|
|
199
|
+
return {
|
|
200
|
+
action: 'save',
|
|
201
|
+
expectedRevision: value.expectedRevision as number,
|
|
202
|
+
value: value.value as VisionToolkitConfig,
|
|
203
|
+
}
|
|
204
|
+
}
|
|
205
|
+
if (value.action === 'credential') {
|
|
206
|
+
if (!Number.isSafeInteger(value.expectedRevision) || (value.expectedRevision as number) < 0) {
|
|
207
|
+
throw new TypeError('credential.expectedRevision must be a non-negative integer')
|
|
208
|
+
}
|
|
209
|
+
if (typeof value.ref !== 'string') throw new TypeError('credential.ref must be a string')
|
|
210
|
+
if (typeof value.value !== 'string') throw new TypeError('credential.value must be a string')
|
|
211
|
+
const secret = value.value.trim()
|
|
212
|
+
if (secret.length === 0) throw new TypeError('API key cannot be blank')
|
|
213
|
+
const first = secret[0]
|
|
214
|
+
const quoted = secret.length > 1 && (first === '"' || first === '\'' || first === '`') && secret.endsWith(first)
|
|
215
|
+
const environmentLine = /^[A-Z][A-Z0-9_]*=[^=]/u.test(secret)
|
|
216
|
+
if (quoted || environmentLine || !/^[\x21-\x7E]+$/u.test(secret)) {
|
|
217
|
+
throw new TypeError('paste only the API key, without a variable name, quotes, spaces, or line breaks')
|
|
218
|
+
}
|
|
219
|
+
return {
|
|
220
|
+
action: 'credential',
|
|
221
|
+
expectedRevision: value.expectedRevision as number,
|
|
222
|
+
ref: credentialRef(value.ref),
|
|
223
|
+
value: secret,
|
|
224
|
+
}
|
|
225
|
+
}
|
|
226
|
+
if (value.action === 'check-update') return { action: 'check-update' }
|
|
227
|
+
if (value.action === 'apply-update') {
|
|
228
|
+
if (typeof value.expectedVersion !== 'string' || value.expectedVersion.trim().length === 0) {
|
|
229
|
+
throw new TypeError('apply-update.expectedVersion must be a non-empty string')
|
|
230
|
+
}
|
|
231
|
+
return { action: 'apply-update', expectedVersion: value.expectedVersion.trim() }
|
|
232
|
+
}
|
|
233
|
+
throw new TypeError(`unsupported action: ${value.action}`)
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
function publicMessage(error: unknown): string {
|
|
237
|
+
if (error instanceof Error) return error.message
|
|
238
|
+
return String(error)
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
/** Same-origin Settings and health handler. */
|
|
242
|
+
export class VisionToolkitWebBackend {
|
|
243
|
+
private readonly updater: WebPluginUpdater
|
|
244
|
+
|
|
245
|
+
constructor(
|
|
246
|
+
private readonly ctx: Context,
|
|
247
|
+
private readonly manager: WebRuntimeManager,
|
|
248
|
+
private readonly artifacts: ArtifactAccessController,
|
|
249
|
+
private readonly onRuntimeActivated: RuntimeActivated,
|
|
250
|
+
updater?: WebPluginUpdater,
|
|
251
|
+
) {
|
|
252
|
+
this.updater = updater ?? new VisionToolkitPluginUpdateService(ctx, PLUGIN_VERSION, {
|
|
253
|
+
runtimeReady: () => this.manager.status().ready,
|
|
254
|
+
})
|
|
255
|
+
}
|
|
256
|
+
|
|
257
|
+
/** Supply the active listener address before the Settings route becomes reachable. */
|
|
258
|
+
configureWebServer(host: string, port: number): void {
|
|
259
|
+
this.updater.configureWebServer?.(host, port)
|
|
260
|
+
}
|
|
261
|
+
|
|
262
|
+
private async credential(config: ResolvedVisionToolkitConfig): Promise<CredentialInfo> {
|
|
263
|
+
if (isBuiltInFreeVisionProvider(config.provider)) {
|
|
264
|
+
return { configured: true, source: 'built-in-free', writable: false }
|
|
265
|
+
}
|
|
266
|
+
return this.ctx.credentials.describe(credentialRef(String(config.provider.credential)))
|
|
267
|
+
}
|
|
268
|
+
|
|
269
|
+
/** Build the current settings/runtime/credential snapshot without secrets. */
|
|
270
|
+
async snapshot(): Promise<VisionToolkitSettingsSnapshot> {
|
|
271
|
+
const descriptor = descriptorOf(this.ctx)
|
|
272
|
+
const value = descriptor.value as VisionToolkitConfig
|
|
273
|
+
const resolved = resolveConfig(value)
|
|
274
|
+
const credential = await this.credential(resolved)
|
|
275
|
+
const update = await this.updater.capability()
|
|
276
|
+
return {
|
|
277
|
+
schemaVersion: 1,
|
|
278
|
+
writable: this.ctx.settings.writable,
|
|
279
|
+
settings: {
|
|
280
|
+
value,
|
|
281
|
+
...(descriptor.user === undefined ? {} : { user: descriptor.user }),
|
|
282
|
+
...(descriptor.base === undefined ? {} : { base: descriptor.base }),
|
|
283
|
+
revision: descriptor.revision,
|
|
284
|
+
applies: 'live',
|
|
285
|
+
},
|
|
286
|
+
credential: {
|
|
287
|
+
ref: String(resolved.provider.credential),
|
|
288
|
+
configured: credential.configured,
|
|
289
|
+
...(credential.source === undefined ? {} : { source: credential.source }),
|
|
290
|
+
writable: credential.writable,
|
|
291
|
+
},
|
|
292
|
+
runtime: this.manager.status(),
|
|
293
|
+
release: {
|
|
294
|
+
pluginVersion: PLUGIN_VERSION,
|
|
295
|
+
upstreamRepository: UPSTREAM_REPOSITORY,
|
|
296
|
+
upstreamVersion: UPSTREAM_VERSION,
|
|
297
|
+
upstreamCommit: UPSTREAM_COMMIT,
|
|
298
|
+
update,
|
|
299
|
+
},
|
|
300
|
+
artifactRouteAvailable: this.artifacts.routeAvailable,
|
|
301
|
+
}
|
|
302
|
+
}
|
|
303
|
+
|
|
304
|
+
private async save(request: SaveRequest): Promise<VisionToolkitSettingsSnapshot> {
|
|
305
|
+
if (!this.ctx.settings.writable) throw new Error('settings provider is read-only')
|
|
306
|
+
let candidate: PreparedRuntimeGeneration
|
|
307
|
+
try {
|
|
308
|
+
candidate = await this.manager.prepareCandidate(request.value)
|
|
309
|
+
} catch (error) {
|
|
310
|
+
this.manager.recordFailure(error)
|
|
311
|
+
throw error
|
|
312
|
+
}
|
|
313
|
+
await this.ctx.settings.replace(
|
|
314
|
+
VISION_TOOLKIT_SETTINGS_NAMESPACE,
|
|
315
|
+
request.value as object,
|
|
316
|
+
request.expectedRevision,
|
|
317
|
+
)
|
|
318
|
+
this.manager.activateCandidate(candidate)
|
|
319
|
+
this.onRuntimeActivated()
|
|
320
|
+
return this.snapshot()
|
|
321
|
+
}
|
|
322
|
+
|
|
323
|
+
private async saveCredential(request: CredentialRequest): Promise<VisionToolkitSettingsSnapshot> {
|
|
324
|
+
const descriptor = descriptorOf(this.ctx)
|
|
325
|
+
if (descriptor.revision !== request.expectedRevision) {
|
|
326
|
+
throw new SettingsConflictError(
|
|
327
|
+
VISION_TOOLKIT_SETTINGS_NAMESPACE,
|
|
328
|
+
request.expectedRevision,
|
|
329
|
+
descriptor.revision,
|
|
330
|
+
)
|
|
331
|
+
}
|
|
332
|
+
const resolved = resolveConfig(descriptor.value as VisionToolkitConfig)
|
|
333
|
+
const currentRef = credentialRef(String(resolved.provider.credential))
|
|
334
|
+
if (currentRef !== request.ref) {
|
|
335
|
+
throw new CredentialReferenceConflictError(
|
|
336
|
+
`credential reference changed from "${request.ref}" to "${currentRef}"; reload Settings and try again`,
|
|
337
|
+
)
|
|
338
|
+
}
|
|
339
|
+
if (isBuiltInFreeVisionProvider(resolved.provider)) {
|
|
340
|
+
throw new Error('The built-in free vision provider does not accept a user API key')
|
|
341
|
+
}
|
|
342
|
+
await this.ctx.credentials.set(currentRef, request.value)
|
|
343
|
+
return this.snapshot()
|
|
344
|
+
}
|
|
345
|
+
|
|
346
|
+
private async health(request: HealthRequest, req: IncomingMessage): Promise<VisionToolkitHealthResult> {
|
|
347
|
+
if (!this.manager.ready) throw new Error('runtime is not ready; fix Settings and save a valid configuration first')
|
|
348
|
+
const controller = new AbortController()
|
|
349
|
+
const abort = (): void => { controller.abort() }
|
|
350
|
+
req.once('aborted', abort)
|
|
351
|
+
req.socket.once('close', abort)
|
|
352
|
+
try {
|
|
353
|
+
const runtime = this.manager.current()
|
|
354
|
+
// Use the prepared runtime home instead of the host process cwd.
|
|
355
|
+
return await runtime.health(request.testConnection, {
|
|
356
|
+
signal: controller.signal,
|
|
357
|
+
workspace: runtime.upstreamVersion.runtimeHome,
|
|
358
|
+
sessionId: 'vision-toolkit-settings',
|
|
359
|
+
}, request.testModel)
|
|
360
|
+
} finally {
|
|
361
|
+
req.off('aborted', abort)
|
|
362
|
+
req.socket.off('close', abort)
|
|
363
|
+
}
|
|
364
|
+
}
|
|
365
|
+
|
|
366
|
+
/** Handle the exact Settings route. */
|
|
367
|
+
async handle(req: IncomingMessage, res: ServerResponse): Promise<void> {
|
|
368
|
+
if (req.method === 'GET') {
|
|
369
|
+
try {
|
|
370
|
+
responseJson(res, 200, { ok: true, value: await this.snapshot() })
|
|
371
|
+
} catch (error) {
|
|
372
|
+
this.ctx.logger.warn('dsh-vision-toolkit Settings snapshot failed: %s', publicMessage(error))
|
|
373
|
+
requestError(res, 503, 'settings-unavailable', 'Vision Toolkit Settings are unavailable')
|
|
374
|
+
}
|
|
375
|
+
return
|
|
376
|
+
}
|
|
377
|
+
if (req.method !== 'POST') {
|
|
378
|
+
res.setHeader('Allow', 'GET, POST')
|
|
379
|
+
requestError(res, 405, 'method-not-allowed', 'Use GET or POST')
|
|
380
|
+
return
|
|
381
|
+
}
|
|
382
|
+
if (!sameOriginPost(req)) {
|
|
383
|
+
requestError(res, 403, 'origin-rejected', 'The request must originate from this DSH Web application')
|
|
384
|
+
return
|
|
385
|
+
}
|
|
386
|
+
let parsed: SettingsRequest
|
|
387
|
+
try {
|
|
388
|
+
parsed = parseRequest(await readJson(req))
|
|
389
|
+
} catch (error) {
|
|
390
|
+
requestError(res, error instanceof RangeError ? 413 : 400, 'invalid-request', publicMessage(error))
|
|
391
|
+
return
|
|
392
|
+
}
|
|
393
|
+
try {
|
|
394
|
+
switch (parsed.action) {
|
|
395
|
+
case 'health':
|
|
396
|
+
responseJson(res, 200, { ok: true, value: await this.health(parsed, req) })
|
|
397
|
+
break
|
|
398
|
+
case 'save':
|
|
399
|
+
responseJson(res, 200, { ok: true, value: await this.save(parsed) })
|
|
400
|
+
break
|
|
401
|
+
case 'credential':
|
|
402
|
+
responseJson(res, 200, { ok: true, value: await this.saveCredential(parsed) })
|
|
403
|
+
break
|
|
404
|
+
case 'check-update':
|
|
405
|
+
responseJson(res, 200, { ok: true, value: await this.updater.check() })
|
|
406
|
+
break
|
|
407
|
+
case 'apply-update':
|
|
408
|
+
responseJson(res, 200, { ok: true, value: await this.updater.installAndRestart(parsed.expectedVersion) })
|
|
409
|
+
break
|
|
410
|
+
}
|
|
411
|
+
} catch (error) {
|
|
412
|
+
const settingsConflict = error instanceof SettingsConflictError
|
|
413
|
+
const credentialConflict = error instanceof CredentialReferenceConflictError
|
|
414
|
+
const updateError = error instanceof PluginUpdateError
|
|
415
|
+
const code = settingsConflict
|
|
416
|
+
? 'settings-conflict'
|
|
417
|
+
: credentialConflict
|
|
418
|
+
? 'credential-conflict'
|
|
419
|
+
: updateError
|
|
420
|
+
? error.code
|
|
421
|
+
: parsed.action === 'health'
|
|
422
|
+
? 'health-failed'
|
|
423
|
+
: parsed.action === 'credential'
|
|
424
|
+
? 'credential-rejected'
|
|
425
|
+
: 'settings-rejected'
|
|
426
|
+
const updateConflict = updateError && ['update-in-progress', 'update-stale', 'update-unavailable', 'already-current'].includes(error.code)
|
|
427
|
+
const updateGateway = updateError && error.code === 'update-check-failed'
|
|
428
|
+
const status = settingsConflict || credentialConflict || updateConflict
|
|
429
|
+
? 409
|
|
430
|
+
: parsed.action === 'health'
|
|
431
|
+
? 503
|
|
432
|
+
: updateGateway
|
|
433
|
+
? 502
|
|
434
|
+
: updateError
|
|
435
|
+
? 500
|
|
436
|
+
: 400
|
|
437
|
+
this.ctx.logger.warn('dsh-vision-toolkit Web action=%s failed: %s', parsed.action, publicMessage(error))
|
|
438
|
+
requestError(res, status, code, publicMessage(error))
|
|
439
|
+
}
|
|
440
|
+
}
|
|
441
|
+
}
|
|
442
|
+
|
|
443
|
+
/**
|
|
444
|
+
* Same-origin policy handler for the paste route: whether the browser should
|
|
445
|
+
* take a paste over into workspace paths, or let it flow natively after an
|
|
446
|
+
* optional automatic switch to the image-input variant. The optional `model`
|
|
447
|
+
* query carries the model-selector label the client currently shows; the
|
|
448
|
+
* optional `provider`/`modelId`/`reasoningEffort` queries carry the exact
|
|
449
|
+
* route the client read from the live model catalog, which the resolver
|
|
450
|
+
* prefers (a label alone cannot pick a provider). Unresolvable routes answer
|
|
451
|
+
* native — the safe default.
|
|
452
|
+
* @param resolve - resolves one live Session's paste verdict.
|
|
453
|
+
* @returns the HTTP handler.
|
|
454
|
+
*/
|
|
455
|
+
export function createPastePolicyHandler(
|
|
456
|
+
resolve: (sessionId: string, selection?: PasteSelectionQuery, modelLabel?: string) => Promise<PasteVerdict>,
|
|
457
|
+
): (req: IncomingMessage, res: ServerResponse) => void {
|
|
458
|
+
return (req, res) => {
|
|
459
|
+
void (async () => {
|
|
460
|
+
try {
|
|
461
|
+
if (req.method !== 'GET') {
|
|
462
|
+
requestError(res, 405, 'method-not-allowed', 'Use GET')
|
|
463
|
+
return
|
|
464
|
+
}
|
|
465
|
+
if (!sameOriginRequest(req)) {
|
|
466
|
+
requestError(res, 403, 'origin-rejected', 'The request must originate from this DSH Web application')
|
|
467
|
+
return
|
|
468
|
+
}
|
|
469
|
+
let sessionId: string
|
|
470
|
+
let modelLabel: string | undefined
|
|
471
|
+
let selection: PasteSelectionQuery | undefined
|
|
472
|
+
try {
|
|
473
|
+
const url = new URL(req.url ?? PASTE_POLICY_ROUTE, 'http://dsh.internal')
|
|
474
|
+
const sessions = url.searchParams.getAll('sessionId')
|
|
475
|
+
if (sessions.length !== 1 || sessions[0] === undefined || sessions[0] === '') {
|
|
476
|
+
throw new TypeError('sessionId is required exactly once')
|
|
477
|
+
}
|
|
478
|
+
sessionId = sessions[0]!
|
|
479
|
+
const models = url.searchParams.getAll('model')
|
|
480
|
+
if (models.length > 1) throw new TypeError('model may be given at most once')
|
|
481
|
+
modelLabel = models[0]
|
|
482
|
+
const providers = url.searchParams.getAll('provider')
|
|
483
|
+
if (providers.length > 1) throw new TypeError('provider may be given at most once')
|
|
484
|
+
const modelIds = url.searchParams.getAll('modelId')
|
|
485
|
+
if (modelIds.length > 1) throw new TypeError('modelId may be given at most once')
|
|
486
|
+
const efforts = url.searchParams.getAll('reasoningEffort')
|
|
487
|
+
if (efforts.length > 1) throw new TypeError('reasoningEffort may be given at most once')
|
|
488
|
+
const provider = providers[0]
|
|
489
|
+
const modelId = modelIds[0]
|
|
490
|
+
if (provider !== undefined && modelId !== undefined && provider !== '' && modelId !== '') {
|
|
491
|
+
selection = {
|
|
492
|
+
provider,
|
|
493
|
+
model: modelId,
|
|
494
|
+
...(efforts[0] === undefined || efforts[0] === '' ? {} : { reasoningEffort: efforts[0] }),
|
|
495
|
+
}
|
|
496
|
+
}
|
|
497
|
+
} catch (error) {
|
|
498
|
+
requestError(res, 400, 'invalid-request', publicMessage(error))
|
|
499
|
+
return
|
|
500
|
+
}
|
|
501
|
+
const verdict = await resolve(sessionId, selection, modelLabel)
|
|
502
|
+
responseJson(res, 200, { ok: true, value: verdict })
|
|
503
|
+
} catch (error) {
|
|
504
|
+
requestError(res, 500, 'policy-failed', publicMessage(error))
|
|
505
|
+
}
|
|
506
|
+
})()
|
|
507
|
+
}
|
|
508
|
+
}
|
|
509
|
+
|
|
510
|
+
/**
|
|
511
|
+
* Same-origin display-config handler: exposes whether transparent routing is
|
|
512
|
+
* active. The paste integration uses it to choose its notice text; the model
|
|
513
|
+
* selector hides upstream twins synchronously from DOM display names and does
|
|
514
|
+
* not depend on this route.
|
|
515
|
+
* @param getDisplayConfig - resolves the current display-mode flags.
|
|
516
|
+
* @returns the HTTP handler.
|
|
517
|
+
*/
|
|
518
|
+
export function createDisplayConfigHandler(
|
|
519
|
+
getDisplayConfig: () => { hidden: boolean },
|
|
520
|
+
): (req: IncomingMessage, res: ServerResponse) => void {
|
|
521
|
+
return (req, res) => {
|
|
522
|
+
try {
|
|
523
|
+
if (req.method !== 'GET') {
|
|
524
|
+
requestError(res, 405, 'method-not-allowed', 'Use GET')
|
|
525
|
+
return
|
|
526
|
+
}
|
|
527
|
+
if (!sameOriginRequest(req)) {
|
|
528
|
+
requestError(res, 403, 'origin-rejected', 'The request must originate from this DSH Web application')
|
|
529
|
+
return
|
|
530
|
+
}
|
|
531
|
+
responseJson(res, 200, { ok: true, value: getDisplayConfig() })
|
|
532
|
+
} catch (error) {
|
|
533
|
+
requestError(res, 500, 'display-config-failed', publicMessage(error))
|
|
534
|
+
}
|
|
535
|
+
}
|
|
536
|
+
}
|
|
537
|
+
|
|
538
|
+
/**
|
|
539
|
+
* Attach optional Web routes whenever a webServer service is present.
|
|
540
|
+
* @param ctx - plugin context owning route effects.
|
|
541
|
+
* @param backend - Settings handler.
|
|
542
|
+
* @param artifacts - signed Artifact handler.
|
|
543
|
+
* @param pastedImages - pasted-image workspace handler.
|
|
544
|
+
* @param pastePolicy - paste-policy verdict resolver (sessionId, selection, modelLabel).
|
|
545
|
+
* @param getDisplayConfig - resolves display-mode flags for the browser client.
|
|
546
|
+
*/
|
|
547
|
+
export function installVisionToolkitWeb(
|
|
548
|
+
ctx: Context,
|
|
549
|
+
backend: VisionToolkitWebBackend,
|
|
550
|
+
artifacts: ArtifactAccessController,
|
|
551
|
+
pastedImages: PastedImageBackend,
|
|
552
|
+
pastePolicy: (sessionId: string, selection?: PasteSelectionQuery, modelLabel?: string) => Promise<PasteVerdict>,
|
|
553
|
+
getDisplayConfig: () => { hidden: boolean },
|
|
554
|
+
): void {
|
|
555
|
+
ctx.inject(['webServer'], (webCtx) => {
|
|
556
|
+
webCtx.effect(() => {
|
|
557
|
+
backend.configureWebServer(webCtx.webServer.host, webCtx.webServer.port)
|
|
558
|
+
const detach = artifacts.attachRoute()
|
|
559
|
+
const disposeArtifact = webCtx.webServer.register({
|
|
560
|
+
kind: 'prefix',
|
|
561
|
+
path: ARTIFACT_ROUTE_PREFIX,
|
|
562
|
+
handler: (req, res) => artifacts.handle(req, res),
|
|
563
|
+
})
|
|
564
|
+
const disposeSettings = webCtx.webServer.register({
|
|
565
|
+
kind: 'exact',
|
|
566
|
+
path: SETTINGS_ROUTE,
|
|
567
|
+
handler: (req, res) => backend.handle(req, res),
|
|
568
|
+
})
|
|
569
|
+
const disposePasteImages = webCtx.webServer.register({
|
|
570
|
+
kind: 'exact',
|
|
571
|
+
path: PASTE_IMAGES_ROUTE,
|
|
572
|
+
handler: (req, res) => pastedImages.handle(req, res),
|
|
573
|
+
})
|
|
574
|
+
const disposePastePolicy = webCtx.webServer.register({
|
|
575
|
+
kind: 'exact',
|
|
576
|
+
path: PASTE_POLICY_ROUTE,
|
|
577
|
+
handler: createPastePolicyHandler(pastePolicy),
|
|
578
|
+
})
|
|
579
|
+
const disposeDisplayConfig = webCtx.webServer.register({
|
|
580
|
+
kind: 'exact',
|
|
581
|
+
path: DISPLAY_CONFIG_ROUTE,
|
|
582
|
+
handler: createDisplayConfigHandler(getDisplayConfig),
|
|
583
|
+
})
|
|
584
|
+
return () => {
|
|
585
|
+
disposeDisplayConfig()
|
|
586
|
+
disposePastePolicy()
|
|
587
|
+
disposePasteImages()
|
|
588
|
+
disposeSettings()
|
|
589
|
+
disposeArtifact()
|
|
590
|
+
detach()
|
|
591
|
+
}
|
|
592
|
+
}, 'dsh-vision-toolkit: Web routes')
|
|
593
|
+
})
|
|
594
|
+
}
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable user-facing changes to agent-vision-toolkit are documented in this file.
|
|
4
|
+
|
|
5
|
+
## [Unreleased]
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- Let the shared Python vision client call either Chat Completions or Responses APIs, including optional reasoning effort and explicit `store: false` data handling.
|
|
10
|
+
- Add native Anthropic Messages requests with protocol-specific authentication, image sources, optional thinking control, and text-block response extraction.
|
|
11
|
+
- Rewrite OpenAI Chat Completions `image_url` blocks through the existing vision-description pipeline with a host-neutral channel note.
|
|
12
|
+
|
|
13
|
+
### Fixed
|
|
14
|
+
|
|
15
|
+
- Send a browser-compatible, configurable User-Agent from the shared Python vision client so Cloudflare-backed OpenAI-compatible endpoints do not reject the default `Python-urllib` signature.
|
|
16
|
+
- Honor `Retry-After` and retry Anthropic 529 overload responses.
|
|
17
|
+
- Allow trusted internal endpoints with self-signed certificates or MITM proxies to opt out of TLS certificate verification through `VISION_SSL_VERIFY`, while preserving verification by default.
|
|
18
|
+
|
|
19
|
+
## [0.1.0] - 2026-08-07
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
|
|
23
|
+
- Five vision CLIs — `glance`, `ground`, `detect`, `trace`, and `crop` — plus the `vision-tools` agent skill.
|
|
24
|
+
- Optional seamless integration: a local proxy for Codex and Claude Code, and single-file native extensions for Pi, Oh My Pi, and OpenCode.
|
|
25
|
+
- Pasted-image and tool-fetched image support with task-aware focus hints, parallel multi-image descriptions, per-request caching, and honest failure notes.
|
|
26
|
+
- Vision playbooks for long-screenshot OCR, UI restoration, graphic restoration, structure recovery, and GUI operation.
|
|
27
|
+
- Community contribution, conduct, support, and security policies.
|
|
28
|
+
- Structured issue forms and a pull request template.
|
|
29
|
+
- GitHub funding configuration and continuous integration checks.
|
|
30
|
+
- A bilingual funding policy and sponsorship-use statement.
|