dsh-vision-router 2.1.1 → 2.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/releases/v2.1.2.md +30 -0
- package/lib/agent-request-route-authority.js +94 -0
- package/lib/delegated-call-config.js +27 -0
- package/lib/grounding-coordinate-runtime.js +10 -8
- package/lib/legacy-core-vision-policy-bridge.js +17 -146
- package/lib/runtime-composition.js +30 -8
- package/lib/session-surface-policy.js +59 -32
- package/lib/session-vision-mode-authority.js +254 -0
- package/lib/session-vision-mode-boundary.js +408 -0
- package/lib/twin-image-capability-fallback.js +428 -0
- package/lib/vision-attachment-handle-runtime.js +212 -0
- package/lib/vision-tool-runtime-boundary.js +224 -5
- package/package.json +5 -5
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# v2.1.2
|
|
2
|
+
|
|
3
|
+
DVR 2.1.2 is a reliability hotfix for image-session ownership, delegated vision routing, text-only attachment projection, and composer Vision-mode authority. It also closes CI blind spots that could allow valid regression tests to exist without being executed by the normal release gate.
|
|
4
|
+
|
|
5
|
+
## Fixed
|
|
6
|
+
|
|
7
|
+
- Keeps uploaded image blocks as durable Session facts. Vision Router no longer rewrites the current user image into internal `[attached image: sha256:…]` marker text through the legacy pre-step path. Model-specific projection now stays at adapter/request boundaries instead of leaking into the conversation transcript.
|
|
8
|
+
- Fixes #374 and the wider delegated call-config class: when Vision Router actually hands a request to another provider/model, source-route `maxTokens`, `reasoningEffort`, sampling and stop settings no longer override the target adapter's own defaults. This fixes GLM routes such as `glm-4v-flash` rejecting a Router-supplied 4096-token limit when the configured model cap is 1024.
|
|
9
|
+
- Refines that handoff rule so transparent Vision Router twins keep caller-owned generation settings, while tool, Vision Chain and `agent/request` provider/model handoffs restore target-route authority. Reasoning memory is scoped to the relevant session and route instead of becoming cross-route state.
|
|
10
|
+
- Keeps the `vision_describe` cache as a successful-answer cache. Structured Vision Router failures (`ok: false` with `VISION_*` errors) are no longer retained across turns, so repaired credentials or recovered backends can be retried immediately.
|
|
11
|
+
- Recovers safely when a source adapter is incorrectly catalogued as image-capable. Generated Vision Router twins now recognize explicit runtime image-capability rejection before model-visible output, record bounded negative proof for the exact source route, and re-enter the canonical text-only fallback path instead of repeatedly sending raw pixels to a text-only backend. Generic 4xx/5xx, auth, rate-limit, media-format, cancellation and post-output failures do not trigger this fallback.
|
|
12
|
+
- Resolves DSH text-only projected image handles such as `sha256:deadbeef` back to a canonical attachment only inside the current Session. Unique prefixes resolve, unknown or ambiguous prefixes fail closed, bounded cache eviction cannot erase ambiguity because durable Session history remains authoritative, and attachment-shaped input can no longer fall through into a guessed local filesystem path.
|
|
13
|
+
- Makes the composer `👁 Vision` state the authoritative per-Session permission for Vision Router orchestration. Ordinary text-only and native-multimodal routes no longer receive Router-owned `vision_*` schemas, structured bootstrap, automatic deep-tool mounting or instant visual work while Vision is OFF. Native multimodal pixels remain Host-owned and continue to work normally without Router intervention.
|
|
14
|
+
- Keeps the current Agent step internally stable if the model picker changes while it is running. The new selection is applied at the next real prompt-assembly boundary, while diagnostic/out-of-band prompt renders cannot clear or mutate the active step authority.
|
|
15
|
+
- Keeps Settings `tool=false` as an independent global capability gate. Selecting a Vision Router wrapper cannot re-enable tools that the user explicitly disabled.
|
|
16
|
+
|
|
17
|
+
## Regression and CI hardening
|
|
18
|
+
|
|
19
|
+
- Adds production-shaped coverage for delegated target defaults, transparent wrapper behavior, `agent/request` route handoffs and ordinary non-delegated LLM calls.
|
|
20
|
+
- Adds transcript ownership coverage ensuring accepted user images remain raw Session content and the retired pre-step bridge cannot synthesize attachment markers into durable history.
|
|
21
|
+
- Adds false-multimodal metadata regressions across generated twins, custom wrappers, lazy source registration, route changes and negative-proof isolation.
|
|
22
|
+
- Adds attachment-handle regressions for exact IDs, DSH short projected handles, cache eviction, same-Session prefix collisions, cross-Session isolation, malformed handles and attachment-vs-filesystem dispatch.
|
|
23
|
+
- Adds Session Vision-mode regressions for OFF/ON isolation, native multimodal non-intervention, prompt-assembly visibility, PTC/scoped-tool behavior, mid-step picker changes, diagnostic assemblies, foreign `vision_*` tools and Settings `tool=false`.
|
|
24
|
+
- Makes the default test manifest closed-world: every `tests/**/*.test.js` file must either run in the normal suite or have a documented PR-triggered specialist workflow owner.
|
|
25
|
+
- Verifies specialist workflow ownership against the workflow source so exclusions cannot silently become stale or merely appear in a path trigger without being executed.
|
|
26
|
+
- The final release head passes Node 22/24, DSH rc.6/rc.7/rc.8 contracts, P1/P2/P3 architecture gates, native multimodal cold/process restart coverage, Ubuntu/macOS/Windows host-sharp checks, and CodeQL with no new alerts.
|
|
27
|
+
|
|
28
|
+
## Upgrade
|
|
29
|
+
|
|
30
|
+
Upgrade to 2.1.2 and restart the DSH Web/Desktop process so the new runtime boundaries are loaded. No settings migration is required.
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
import { projectDelegatedCallConfig } from './delegated-call-config.js'
|
|
2
|
+
import { contextWithTwinImageCapabilityFallback } from './twin-image-capability-fallback.js'
|
|
3
|
+
|
|
4
|
+
const routeAuthorityOptionsByContext = new WeakMap()
|
|
5
|
+
|
|
6
|
+
function isObject(value) {
|
|
7
|
+
return value !== null && typeof value === 'object' && !Array.isArray(value)
|
|
8
|
+
}
|
|
9
|
+
|
|
10
|
+
function routeOf(value) {
|
|
11
|
+
if (!isObject(value)) return undefined
|
|
12
|
+
const provider = typeof value.provider === 'string' && value.provider !== '' ? value.provider : undefined
|
|
13
|
+
const model = typeof value.model === 'string' && value.model !== '' ? value.model : undefined
|
|
14
|
+
return provider !== undefined && model !== undefined ? { provider, model } : undefined
|
|
15
|
+
}
|
|
16
|
+
|
|
17
|
+
/**
|
|
18
|
+
* A completed `agent/request` route change is an authority handoff: generation
|
|
19
|
+
* defaults from the source provider/model must not be mistaken for explicit
|
|
20
|
+
* target-model config. Payload/lifecycle fields remain untouched.
|
|
21
|
+
*
|
|
22
|
+
* Fail closed on incomplete route evidence. If either side lacks an exact
|
|
23
|
+
* provider+model tuple, preserve the handler result rather than guessing.
|
|
24
|
+
*/
|
|
25
|
+
export function projectAgentRequestRouteHandoff(sourceConfig, resultConfig) {
|
|
26
|
+
const source = routeOf(sourceConfig)
|
|
27
|
+
const target = routeOf(resultConfig)
|
|
28
|
+
if (source === undefined || target === undefined) return resultConfig
|
|
29
|
+
if (source.provider === target.provider && source.model === target.model) return resultConfig
|
|
30
|
+
return projectDelegatedCallConfig(resultConfig)
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
/**
|
|
34
|
+
* Runtime composition can publish route identities before Core.apply without
|
|
35
|
+
* changing the mature `contextWithAgentRequestRouteAuthority(backendRuntimeCtx)`
|
|
36
|
+
* boundary shape. The WeakMap keeps this per-context and garbage-collectable.
|
|
37
|
+
*/
|
|
38
|
+
export function configureAgentRequestRouteAuthority(ctx, options = {}) {
|
|
39
|
+
if (!isObject(ctx)) return ctx
|
|
40
|
+
routeAuthorityOptionsByContext.set(ctx, {
|
|
41
|
+
wrapperRoute: options.wrapperRoute,
|
|
42
|
+
chainRoute: options.chainRoute,
|
|
43
|
+
logger: options.logger,
|
|
44
|
+
})
|
|
45
|
+
return ctx
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
/**
|
|
49
|
+
* Private Core-facing context. It wraps only `agent/request` handlers and only
|
|
50
|
+
* projects a result when that handler demonstrably changed provider/model.
|
|
51
|
+
* Ordinary request handlers and direct LLM calls retain caller identity.
|
|
52
|
+
*
|
|
53
|
+
* This is also the final Core-facing adapter-registration boundary, so it owns
|
|
54
|
+
* the generated-twin runtime capability correction: if a `<provider>-vision`
|
|
55
|
+
* twin trusted image metadata but the live source explicitly rejects raw image
|
|
56
|
+
* input before producing output, the twin is re-entered once through Core's
|
|
57
|
+
* existing text bridge instead of leaking that 400 to the user. Keeping the
|
|
58
|
+
* fallback here means the deeper prepareCall/ownership layers still observe the
|
|
59
|
+
* final wrapped stream and retain their existing contracts.
|
|
60
|
+
*/
|
|
61
|
+
export function contextWithAgentRequestRouteAuthority(ctx, options) {
|
|
62
|
+
if (!isObject(ctx)) return ctx
|
|
63
|
+
const fallbackOptions = isObject(options)
|
|
64
|
+
? options
|
|
65
|
+
: routeAuthorityOptionsByContext.get(ctx) ?? {}
|
|
66
|
+
const routeAuthorityCtx = new Proxy(ctx, {
|
|
67
|
+
get(target, property) {
|
|
68
|
+
if (property !== 'on') {
|
|
69
|
+
const value = Reflect.get(target, property, target)
|
|
70
|
+
return typeof value === 'function' ? value.bind(target) : value
|
|
71
|
+
}
|
|
72
|
+
const on = Reflect.get(target, property, target)
|
|
73
|
+
if (typeof on !== 'function') return on
|
|
74
|
+
return (event, handler, ...rest) => {
|
|
75
|
+
if (event !== 'agent/request' || typeof handler !== 'function') {
|
|
76
|
+
return on.call(target, event, handler, ...rest)
|
|
77
|
+
}
|
|
78
|
+
return on.call(target, event, async function routeAuthorityAwareRequest(payload, next) {
|
|
79
|
+
let sourceConfig
|
|
80
|
+
const captureNext = typeof next === 'function'
|
|
81
|
+
? async (...args) => {
|
|
82
|
+
const value = await next(...args)
|
|
83
|
+
if (sourceConfig === undefined) sourceConfig = value
|
|
84
|
+
return value
|
|
85
|
+
}
|
|
86
|
+
: next
|
|
87
|
+
const result = await handler.call(this, payload, captureNext)
|
|
88
|
+
return projectAgentRequestRouteHandoff(sourceConfig, result)
|
|
89
|
+
}, ...rest)
|
|
90
|
+
}
|
|
91
|
+
},
|
|
92
|
+
})
|
|
93
|
+
return contextWithTwinImageCapabilityFallback(routeAuthorityCtx, fallbackOptions)
|
|
94
|
+
}
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
const ROUTE_OWNED_CALL_FIELDS = Object.freeze([
|
|
2
|
+
'reasoningEffort',
|
|
3
|
+
'temperature',
|
|
4
|
+
'maxTokens',
|
|
5
|
+
'stop',
|
|
6
|
+
])
|
|
7
|
+
|
|
8
|
+
/**
|
|
9
|
+
* Project an internal Vision Router delegation onto a different provider/model.
|
|
10
|
+
*
|
|
11
|
+
* The delegated route owns its own call defaults and compatibility. Carry the
|
|
12
|
+
* request payload and lifecycle through, but never leak source-route sampling,
|
|
13
|
+
* reasoning or output-limit state into the target adapter. DSH will materialize
|
|
14
|
+
* the target adapter's configured defaults when these fields are absent.
|
|
15
|
+
*
|
|
16
|
+
* Direct transports owned by Vision Router do not use this projection; their
|
|
17
|
+
* wire compatibility remains the Router's responsibility.
|
|
18
|
+
*/
|
|
19
|
+
export function projectDelegatedCallConfig(options) {
|
|
20
|
+
if (!options || typeof options !== 'object' || Array.isArray(options)) return options
|
|
21
|
+
if (!ROUTE_OWNED_CALL_FIELDS.some((field) => Object.hasOwn(options, field))) return options
|
|
22
|
+
const next = { ...options }
|
|
23
|
+
for (const field of ROUTE_OWNED_CALL_FIELDS) delete next[field]
|
|
24
|
+
return next
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
export const delegatedRouteOwnedCallFields = ROUTE_OWNED_CALL_FIELDS
|
|
@@ -6,6 +6,7 @@ import {
|
|
|
6
6
|
createGroundingFrame,
|
|
7
7
|
groundingFrameBoxToSource,
|
|
8
8
|
} from './grounding-coordinate-frame.js'
|
|
9
|
+
import { wrapVisionAttachmentHandleDefinition } from './vision-attachment-handle-runtime.js'
|
|
9
10
|
|
|
10
11
|
const GROUNDING_TOOL_NAMES = new Set(['vision_ground', 'vision_detect'])
|
|
11
12
|
let sharpPromise
|
|
@@ -205,9 +206,10 @@ function wrapGroundingDefinition(ctx, options, def) {
|
|
|
205
206
|
* Give vision_ground / vision_detect one explicit provider-independent
|
|
206
207
|
* coordinate protocol: the model always receives an exact 1000x1000
|
|
207
208
|
* letterboxed raster, while callers always receive Host-canonical source
|
|
208
|
-
* pixels.
|
|
209
|
-
*
|
|
210
|
-
*
|
|
209
|
+
* pixels. The same registration seam also canonicalizes DSH text-only image
|
|
210
|
+
* handles before any Vision Router tool sees them, so short model-facing
|
|
211
|
+
* sha256 prefixes can never fall through into filesystem-path resolution.
|
|
212
|
+
* Core still owns provider selection, fallback, retry and failure semantics.
|
|
211
213
|
*/
|
|
212
214
|
export function contextWithGroundingCoordinateFrame(ctx, options = {}) {
|
|
213
215
|
if (!ctx || (typeof ctx !== 'object' && typeof ctx !== 'function')) return ctx
|
|
@@ -221,11 +223,11 @@ export function contextWithGroundingCoordinateFrame(ctx, options = {}) {
|
|
|
221
223
|
return typeof value === 'function' ? value.bind(target) : value
|
|
222
224
|
}
|
|
223
225
|
const register = Reflect.get(target, property, target)
|
|
224
|
-
return (def, ...rest) =>
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
...rest
|
|
228
|
-
|
|
226
|
+
return (def, ...rest) => {
|
|
227
|
+
const grounded = wrapGroundingDefinition(ctx, options, def)
|
|
228
|
+
const attachmentResolved = wrapVisionAttachmentHandleDefinition(grounded, options)
|
|
229
|
+
return register.call(target, attachmentResolved, ...rest)
|
|
230
|
+
}
|
|
229
231
|
},
|
|
230
232
|
})
|
|
231
233
|
|
|
@@ -1,150 +1,21 @@
|
|
|
1
|
-
import { currentSessionSurfacePolicy } from './session-surface-policy.js'
|
|
2
|
-
import { knownSessionVisionMemory } from './session-vision-state.js'
|
|
3
|
-
|
|
4
|
-
function isObject(value) {
|
|
5
|
-
return value !== null && typeof value === 'object'
|
|
6
|
-
}
|
|
7
|
-
|
|
8
|
-
function collectAttachmentIds(messages) {
|
|
9
|
-
const ids = []
|
|
10
|
-
const seen = new Set()
|
|
11
|
-
const pending = []
|
|
12
|
-
for (const message of messages ?? []) {
|
|
13
|
-
if (message && Array.isArray(message.content)) pending.push(...message.content)
|
|
14
|
-
}
|
|
15
|
-
while (pending.length > 0) {
|
|
16
|
-
const block = pending.pop()
|
|
17
|
-
if (!block || typeof block !== 'object') continue
|
|
18
|
-
if (block.type === 'image') {
|
|
19
|
-
const ref = block.attachment
|
|
20
|
-
const raw = ref && (ref.attachmentId ?? ref.id)
|
|
21
|
-
if (raw !== undefined && raw !== null) {
|
|
22
|
-
const id = String(raw)
|
|
23
|
-
if (id !== '' && !seen.has(id)) {
|
|
24
|
-
seen.add(id)
|
|
25
|
-
ids.push(id)
|
|
26
|
-
}
|
|
27
|
-
}
|
|
28
|
-
}
|
|
29
|
-
if (Array.isArray(block.content)) pending.push(...block.content)
|
|
30
|
-
}
|
|
31
|
-
return ids.reverse()
|
|
32
|
-
}
|
|
33
|
-
|
|
34
|
-
function appendVisionRouterAttachmentHint(payload, decision, config) {
|
|
35
|
-
const policy = currentSessionSurfacePolicy(config)
|
|
36
|
-
if (decision?.kind === 'reject' || policy.ownership !== 'vision-router-owned') {
|
|
37
|
-
return decision
|
|
38
|
-
}
|
|
39
|
-
|
|
40
|
-
// Vision Router-owned wrappers may deliberately preserve raw pixels when the
|
|
41
|
-
// delegated source model is itself multimodal. The provider wire carries the
|
|
42
|
-
// bytes, but not DSH's durable attachmentId, so a model that chooses a
|
|
43
|
-
// precision Vision Router tool would otherwise have to guess an id. Surface
|
|
44
|
-
// the exact current-turn ids as read-only model context while leaving the raw
|
|
45
|
-
// image blocks and the session-scoped lookup fence unchanged.
|
|
46
|
-
const source = Array.isArray(payload?.messages) ? payload.messages : []
|
|
47
|
-
const ids = collectAttachmentIds(source)
|
|
48
|
-
if (ids.length === 0) return decision
|
|
49
|
-
|
|
50
|
-
const baseMessages = Array.isArray(decision?.messages)
|
|
51
|
-
? decision.messages
|
|
52
|
-
: source
|
|
53
|
-
const turn = Number.isInteger(payload?.turn) ? payload.turn : 'current'
|
|
54
|
-
const step = Number.isInteger(payload?.step) ? payload.step : 'step'
|
|
55
|
-
const hintId = `vision-router-attachment-refs-${turn}-${step}`
|
|
56
|
-
if (baseMessages.some((message) => message?.id === hintId)) return decision
|
|
57
|
-
|
|
58
|
-
const quoted = ids.map((id) => `"${id}"`).join(', ')
|
|
59
|
-
const hint = {
|
|
60
|
-
role: 'user',
|
|
61
|
-
id: hintId,
|
|
62
|
-
content: [{
|
|
63
|
-
type: 'text',
|
|
64
|
-
text: `Vision Router attachment references for the image(s) in this step: ${quoted}. If a Vision Router tool requires attachmentIds or an attachment-id image argument, use only these exact ids. Never guess or invent an attachment id.`,
|
|
65
|
-
}],
|
|
66
|
-
source: { kind: 'plugin', plugin: 'dsh-vision-router' },
|
|
67
|
-
}
|
|
68
|
-
const messages = [...baseMessages, hint]
|
|
69
|
-
return isObject(decision)
|
|
70
|
-
? { ...decision, messages }
|
|
71
|
-
: { kind: 'continue', messages }
|
|
72
|
-
}
|
|
73
|
-
|
|
74
|
-
function rewriteTextOnlyDecision(payload, decision, rewriteHistoryImages, config) {
|
|
75
|
-
const policy = currentSessionSurfacePolicy(config)
|
|
76
|
-
if (
|
|
77
|
-
decision?.kind === 'reject' ||
|
|
78
|
-
policy.rewriteCurrentImages !== true ||
|
|
79
|
-
typeof rewriteHistoryImages !== 'function'
|
|
80
|
-
) {
|
|
81
|
-
return decision
|
|
82
|
-
}
|
|
83
|
-
|
|
84
|
-
const source = Array.isArray(decision?.messages)
|
|
85
|
-
? decision.messages
|
|
86
|
-
: Array.isArray(payload?.messages)
|
|
87
|
-
? payload.messages
|
|
88
|
-
: undefined
|
|
89
|
-
if (!source) return decision
|
|
90
|
-
|
|
91
|
-
// Core registers the exact SessionMemoryView while processing this same
|
|
92
|
-
// pre-step. Reuse it here so a text-only fallback preserves cached visual
|
|
93
|
-
// descriptions instead of degrading them back to a generic attachment marker.
|
|
94
|
-
const memory = knownSessionVisionMemory(payload?.agent?.session)
|
|
95
|
-
const rewritten = rewriteHistoryImages(source, memory)
|
|
96
|
-
const messages = rewritten?.messages
|
|
97
|
-
if (!Array.isArray(messages) || messages === source) return decision
|
|
98
|
-
if (isObject(decision)) return { ...decision, messages }
|
|
99
|
-
return { kind: 'continue', messages }
|
|
100
|
-
}
|
|
101
|
-
|
|
102
1
|
/**
|
|
103
|
-
*
|
|
104
|
-
*
|
|
2
|
+
* Retired pre-step compatibility shell.
|
|
3
|
+
*
|
|
4
|
+
* This module remains as a stable internal import while the 2.x composition is
|
|
5
|
+
* still being collapsed, but it deliberately owns no runtime behavior.
|
|
6
|
+
*
|
|
7
|
+
* Historical versions intercepted `agent/pre-step` for two model-only concerns:
|
|
8
|
+
* rewriting text-only image messages into `[attached image: ...]` markers and
|
|
9
|
+
* appending a synthetic attachment-id user message for Vision Router wrappers.
|
|
10
|
+
* DSH persists every admitted pre-step message as `user/message`, so both
|
|
11
|
+
* transforms crossed the durable transcript boundary and could surface in the
|
|
12
|
+
* Web conversation. User-owned image messages must instead remain byte-for-byte
|
|
13
|
+
* Session facts; request-only image projection belongs to the selected adapter.
|
|
105
14
|
*
|
|
106
|
-
*
|
|
107
|
-
*
|
|
108
|
-
*
|
|
109
|
-
* rewrite and surfaces current durable attachment ids for a Vision Router-owned
|
|
110
|
-
* route. Every other context service, Settings object, injected child and config
|
|
111
|
-
* value passes through unchanged.
|
|
15
|
+
* Keep this function identity-only until runtime-composition removes the import
|
|
16
|
+
* in a dedicated closure cleanup. No Settings/config/service identity is
|
|
17
|
+
* impersonated and no `agent/pre-step` listener is installed here.
|
|
112
18
|
*/
|
|
113
|
-
export function installLegacyCoreVisionPolicyBridge(
|
|
114
|
-
ctx,
|
|
115
|
-
config = {},
|
|
116
|
-
{ rewriteHistoryImages } = {},
|
|
117
|
-
) {
|
|
118
|
-
if (!isObject(ctx)) return { ctx, config }
|
|
119
|
-
|
|
120
|
-
const wrappedCtx = new Proxy(ctx, {
|
|
121
|
-
get(target, property) {
|
|
122
|
-
if (property === 'on') {
|
|
123
|
-
const on = Reflect.get(target, property, target)
|
|
124
|
-
if (typeof on !== 'function') return on
|
|
125
|
-
return (event, handler, ...rest) => {
|
|
126
|
-
if (event !== 'agent/pre-step' || typeof handler !== 'function') {
|
|
127
|
-
return on.call(target, event, handler, ...rest)
|
|
128
|
-
}
|
|
129
|
-
return on.call(target, event, async function legacyCoreVisionPolicyPreStep(payload, next) {
|
|
130
|
-
const decision = await handler.call(this, payload, next)
|
|
131
|
-
const rewritten = rewriteTextOnlyDecision(
|
|
132
|
-
payload,
|
|
133
|
-
decision,
|
|
134
|
-
rewriteHistoryImages,
|
|
135
|
-
config,
|
|
136
|
-
)
|
|
137
|
-
return appendVisionRouterAttachmentHint(payload, rewritten, config)
|
|
138
|
-
}, ...rest)
|
|
139
|
-
}
|
|
140
|
-
}
|
|
141
|
-
const value = Reflect.get(target, property, target)
|
|
142
|
-
return typeof value === 'function' ? value.bind(target) : value
|
|
143
|
-
},
|
|
144
|
-
})
|
|
145
|
-
|
|
146
|
-
return {
|
|
147
|
-
ctx: wrappedCtx,
|
|
148
|
-
config,
|
|
149
|
-
}
|
|
19
|
+
export function installLegacyCoreVisionPolicyBridge(ctx, config = {}) {
|
|
20
|
+
return { ctx, config }
|
|
150
21
|
}
|
|
@@ -15,6 +15,7 @@ import { createCoreVisionSurfaceRuntime } from './core-vision-surface.js'
|
|
|
15
15
|
import { installSessionVisionIndexBoundary } from './session-vision-index.js'
|
|
16
16
|
import { createSessionVisionRuntime } from './session-vision-runtime.js'
|
|
17
17
|
import { installLegacyCoreVisionPolicyBridge } from './legacy-core-vision-policy-bridge.js'
|
|
18
|
+
import { installSessionVisionModeBoundary } from './session-vision-mode-boundary.js'
|
|
18
19
|
import { installPiAiBridgeWireCompat } from './pi-ai-bridge-wire-compat.js'
|
|
19
20
|
import { installLiveModelDiscovery } from './live-model-discovery.js'
|
|
20
21
|
import { installVisionModelRegistry } from './vision-model-registry.js'
|
|
@@ -28,6 +29,10 @@ import { installTesseractExecFileCompat } from './tesseract-exec-compat.js'
|
|
|
28
29
|
import { installLocalMutationRouteBoundary } from './web-capability-boundary.js'
|
|
29
30
|
import { installScreenshotSourceBoundary } from './screenshot-source-boundary.js'
|
|
30
31
|
import { installVisionToolRuntimeBoundary } from './vision-tool-runtime-boundary.js'
|
|
32
|
+
import {
|
|
33
|
+
configureAgentRequestRouteAuthority,
|
|
34
|
+
contextWithAgentRequestRouteAuthority,
|
|
35
|
+
} from './agent-request-route-authority.js'
|
|
31
36
|
import { contextWithGroundingCoordinateFrame } from './grounding-coordinate-runtime.js'
|
|
32
37
|
import { installVisionRoutingRuntime } from './vision-routing-runtime.js'
|
|
33
38
|
import { createCapabilityProfileStore } from './vision-capability-probe.js'
|
|
@@ -187,25 +192,32 @@ export function applyVisionRuntimeComposition(ctx, config = {}, core) {
|
|
|
187
192
|
runtimeI18nCore,
|
|
188
193
|
{ logger: logging.logger, index: sessionVisionRuntime.index },
|
|
189
194
|
)
|
|
190
|
-
//
|
|
191
|
-
//
|
|
192
|
-
//
|
|
195
|
+
// The retired bridge stays identity-only. Session mode is a new explicit
|
|
196
|
+
// boundary because it owns a different concern: per-Agent visibility and
|
|
197
|
+
// execution authority derived from DSH's modelSelection projection.
|
|
193
198
|
const legacyCoreCompat = installLegacyCoreVisionPolicyBridge(
|
|
194
199
|
sessionIndexCtx,
|
|
195
200
|
nativeImageCompat.config,
|
|
196
201
|
{ rewriteHistoryImages: core.rewriteHistoryImages },
|
|
197
202
|
)
|
|
203
|
+
const sessionVisionModeCompat = installSessionVisionModeBoundary(
|
|
204
|
+
legacyCoreCompat.ctx,
|
|
205
|
+
legacyCoreCompat.config,
|
|
206
|
+
)
|
|
198
207
|
|
|
199
208
|
// Final structured-flow guard sits closest to core.apply so it sees the
|
|
200
209
|
// actual tool registrations and pre-step listener. The diagnostic observer
|
|
201
210
|
// remains immediately inside it so existing timeout/budget semantics stay
|
|
202
211
|
// unchanged.
|
|
203
212
|
const limitDiagnosticCtx = installVisionLimitDiagnostics(
|
|
204
|
-
|
|
205
|
-
|
|
213
|
+
sessionVisionModeCompat.ctx,
|
|
214
|
+
sessionVisionModeCompat.config,
|
|
206
215
|
logging.logger,
|
|
207
216
|
)
|
|
208
|
-
const structuredCtx = installStructuredFlowHardening(
|
|
217
|
+
const structuredCtx = installStructuredFlowHardening(
|
|
218
|
+
limitDiagnosticCtx,
|
|
219
|
+
sessionVisionModeCompat.config,
|
|
220
|
+
)
|
|
209
221
|
const backgroundProfiling = installBackgroundCapabilityProfiling(
|
|
210
222
|
structuredCtx,
|
|
211
223
|
runtimeConfig,
|
|
@@ -306,6 +318,16 @@ export function applyVisionRuntimeComposition(ctx, config = {}, core) {
|
|
|
306
318
|
})
|
|
307
319
|
installTesseractExecFileCompat(backendRuntimeCtx)
|
|
308
320
|
|
|
321
|
+
// Core owns exactly one agent/request routing hook. Protect the completed
|
|
322
|
+
// provider/model handoff at that event boundary so future route-switch logic
|
|
323
|
+
// cannot accidentally carry source-model call defaults into the target.
|
|
324
|
+
configureAgentRequestRouteAuthority(backendRuntimeCtx, {
|
|
325
|
+
wrapperRoute: runtimeConfig.wrapperRoute,
|
|
326
|
+
chainRoute: runtimeConfig.chainRoute,
|
|
327
|
+
logger: logging.logger,
|
|
328
|
+
})
|
|
329
|
+
const coreRequestAuthorityCtx = contextWithAgentRequestRouteAuthority(backendRuntimeCtx)
|
|
330
|
+
|
|
309
331
|
try {
|
|
310
332
|
const c = hardenedConfig && typeof hardenedConfig === 'object' ? hardenedConfig : {}
|
|
311
333
|
const local = c.localOllama && typeof c.localOllama === 'object' ? c.localOllama : {}
|
|
@@ -327,8 +349,8 @@ export function applyVisionRuntimeComposition(ctx, config = {}, core) {
|
|
|
327
349
|
withVisionCircuitBreakerObserver(
|
|
328
350
|
breakerShadowHealth.capture,
|
|
329
351
|
() => core.apply(
|
|
330
|
-
|
|
331
|
-
|
|
352
|
+
coreRequestAuthorityCtx,
|
|
353
|
+
sessionVisionModeCompat.config,
|
|
332
354
|
{
|
|
333
355
|
sessionVision: sessionVisionRuntime,
|
|
334
356
|
coreVisionSurface: coreVisionSurfaceRuntime,
|
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import { currentSessionVisionPolicy } from './native-image-coexistence.js'
|
|
2
|
+
import { currentSessionVisionModeAuthority } from './session-vision-mode-authority.js'
|
|
2
3
|
|
|
3
4
|
const OWNERSHIP = Object.freeze({
|
|
4
5
|
PLUGIN_OWNED: 'vision-router-owned',
|
|
@@ -22,62 +23,87 @@ function normalizedOwnership(policy) {
|
|
|
22
23
|
* Resolve one immutable, read-only snapshot of the Vision Router capability
|
|
23
24
|
* surface for the current session.
|
|
24
25
|
*
|
|
25
|
-
*
|
|
26
|
-
*
|
|
27
|
-
*
|
|
28
|
-
*
|
|
26
|
+
* Image ownership and Vision-mode authority are intentionally distinct. The
|
|
27
|
+
* native-image policy answers who can consume raw pixels; the mode authority
|
|
28
|
+
* answers whether this Session explicitly selected a Vision Router-owned
|
|
29
|
+
* wrapper/twin for the current step. Only the latter can grant plugin tools or
|
|
30
|
+
* automatic visual work. This prevents a historical image, a native multimodal
|
|
31
|
+
* source model, or a stale request header from silently turning Vision back on.
|
|
32
|
+
*
|
|
33
|
+
* Durable transcript invariant: once a real session policy exists, Core may
|
|
34
|
+
* observe/index image blocks but may never replace a user-owned image message
|
|
35
|
+
* before the Agent loop appends it to the Session log. Image projection belongs
|
|
36
|
+
* at the adapter/request boundary. The historical `rewriteCurrentImages` bit is
|
|
37
|
+
* therefore intentionally ignored even when an older policy producer still
|
|
38
|
+
* exposes it; the compatibility field below is permanently false.
|
|
29
39
|
*/
|
|
30
40
|
export function resolveSessionSurfacePolicy({
|
|
31
41
|
visionPolicy,
|
|
42
|
+
visionModeAuthority,
|
|
32
43
|
config = {},
|
|
33
44
|
schemaBootstrapping = false,
|
|
34
45
|
} = {}) {
|
|
35
46
|
const source = isObject(visionPolicy) ? visionPolicy : undefined
|
|
47
|
+
const authority = isObject(visionModeAuthority) ? visionModeAuthority : undefined
|
|
36
48
|
const value = isObject(config) ? config : {}
|
|
37
49
|
const ownership = normalizedOwnership(source)
|
|
38
|
-
const native = ownership === OWNERSHIP.NATIVE
|
|
39
50
|
const pluginOwned = ownership === OWNERSHIP.PLUGIN_OWNED
|
|
40
|
-
const textOnly = ownership === OWNERSHIP.TEXT_ONLY
|
|
41
51
|
|
|
42
|
-
//
|
|
43
|
-
//
|
|
44
|
-
// explicit
|
|
45
|
-
|
|
46
|
-
const
|
|
47
|
-
|
|
48
|
-
|
|
52
|
+
// During schema bootstrap there is no real Session policy, so global Settings
|
|
53
|
+
// still decide which definitions Core registers. During a real step, an
|
|
54
|
+
// explicit authority snapshot wins; direct helper callers fall back to the
|
|
55
|
+
// already-classified plugin ownership instead of inventing an enabled mode.
|
|
56
|
+
const visionModeEnabled = authority !== undefined
|
|
57
|
+
? authority.enabled === true
|
|
58
|
+
: source === undefined
|
|
59
|
+
? true
|
|
60
|
+
: pluginOwned
|
|
61
|
+
|
|
62
|
+
// No active session policy means no session-specific preservation authority.
|
|
63
|
+
// During a real pre-step, every ownership result — including explicit
|
|
64
|
+
// text-only and unknown — preserves the durable user message unchanged.
|
|
65
|
+
const preserveRawImages = source !== undefined
|
|
66
|
+
const rewriteCurrentImages = false
|
|
67
|
+
const allowStructuredBootstrap = visionModeEnabled
|
|
68
|
+
const allowGenericAutoMount = visionModeEnabled
|
|
49
69
|
|
|
50
70
|
const surface = Object.freeze({
|
|
51
71
|
preserveRawImages,
|
|
52
72
|
rewriteCurrentImages,
|
|
53
|
-
visionTools: value.tool !== false,
|
|
73
|
+
visionTools: value.tool !== false && visionModeEnabled,
|
|
54
74
|
structuredBootstrap:
|
|
55
|
-
value.structuredVisionBootstrap === true &&
|
|
75
|
+
value.structuredVisionBootstrap === true && visionModeEnabled,
|
|
56
76
|
genericAutoMount:
|
|
57
|
-
value.autoActivateOnImage !== false &&
|
|
77
|
+
value.autoActivateOnImage !== false && visionModeEnabled,
|
|
58
78
|
instantDescribe:
|
|
59
|
-
value.instantDescribe !== false &&
|
|
79
|
+
value.instantDescribe !== false && visionModeEnabled,
|
|
60
80
|
})
|
|
61
81
|
|
|
62
82
|
const overrides = {}
|
|
63
|
-
if (schemaBootstrapping === true && value.tool === false)
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
83
|
+
if (schemaBootstrapping === true && source === undefined && value.tool === false) {
|
|
84
|
+
overrides.tool = true
|
|
85
|
+
}
|
|
86
|
+
// Core's historical rewriteImages switch controls an agent/pre-step message
|
|
87
|
+
// transform. Disable it for every real session policy so neither current nor
|
|
88
|
+
// historical user image blocks can be persisted as internal attachment text.
|
|
89
|
+
// Vision Router-owned adapters still perform their private request projection.
|
|
90
|
+
if (source !== undefined && value.rewriteImages !== false) overrides.rewriteImages = false
|
|
91
|
+
|
|
92
|
+
// The composer/model selection is the Session authority. When it is OFF,
|
|
93
|
+
// suppress every automatic/plugin-owned visual surface while preserving the
|
|
94
|
+
// durable image itself. Settings remain unchanged; these are turn-local Core
|
|
95
|
+
// projections only.
|
|
96
|
+
if (source !== undefined && !visionModeEnabled) {
|
|
97
|
+
if (value.tool !== false) overrides.tool = false
|
|
98
|
+
if (value.instantDescribe !== false) overrides.instantDescribe = false
|
|
99
|
+
if (value.autoActivateOnImage !== false) overrides.autoActivateOnImage = false
|
|
100
|
+
if (value.structuredVisionBootstrap !== false) overrides.structuredVisionBootstrap = false
|
|
69
101
|
}
|
|
70
102
|
|
|
71
103
|
return Object.freeze({
|
|
72
104
|
ownership,
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
pluginOwned ||
|
|
76
|
-
textOnly ||
|
|
77
|
-
surface.visionTools ||
|
|
78
|
-
surface.structuredBootstrap ||
|
|
79
|
-
surface.genericAutoMount
|
|
80
|
-
),
|
|
105
|
+
visionModeEnabled,
|
|
106
|
+
participates: source !== undefined,
|
|
81
107
|
preserveRawImages,
|
|
82
108
|
rewriteCurrentImages,
|
|
83
109
|
allowStructuredBootstrap,
|
|
@@ -87,10 +113,11 @@ export function resolveSessionSurfacePolicy({
|
|
|
87
113
|
})
|
|
88
114
|
}
|
|
89
115
|
|
|
90
|
-
/** Read the turn-local ownership
|
|
116
|
+
/** Read the turn-local ownership + mode snapshots and resolve the Core surface. */
|
|
91
117
|
export function currentSessionSurfacePolicy(config = {}, options = {}) {
|
|
92
118
|
return resolveSessionSurfacePolicy({
|
|
93
119
|
visionPolicy: currentSessionVisionPolicy(),
|
|
120
|
+
visionModeAuthority: currentSessionVisionModeAuthority(),
|
|
94
121
|
config,
|
|
95
122
|
schemaBootstrapping: options?.schemaBootstrapping === true,
|
|
96
123
|
})
|