pi-vision-handoff 0.4.0 โ†’ 0.4.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -10,6 +10,8 @@ _Describe images with a vision model you pick, then feed the text to models that
10
10
  [![npm](https://img.shields.io/npm/v/pi-vision-handoff)](https://www.npmjs.com/package/pi-vision-handoff)
11
11
  [![license](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
12
12
 
13
+ <img src="https://raw.githubusercontent.com/monotykamary/pi-vision-handoff/main/assets/vision-handoff.jpg" alt="Vision Handoff picker โ€” an interactive TUI listing every model, vision-capable ones marked with an eye, to choose the describer for text-only models" width="820">
14
+
13
15
  </div>
14
16
 
15
17
  ---
@@ -35,9 +37,9 @@ No `settings.json` touched. No per-provider glue. Pick a describer once and ever
35
37
  ## Features
36
38
 
37
39
  - **๐ŸŽฎ Interactive picker** โ€” `/vision-handoff` opens a TUI listing every model, vision-capable ones first (๐Ÿ‘), to choose your describer.
38
- - **๐Ÿ–ผ๏ธ DataLoader-batched descriptions** โ€” the `read` tools are `load()` callers: N parallel reads coalesce into ONE batched vision call (dispatched after the microtask cascade settles), awaited during the tool-result phase (free time) so the agent's next turn never blocks on the describer. Descriptions are ready before `context` fires, so the swap is a non-blocking cache hit.
40
+ - **๐Ÿ–ผ๏ธ DataLoader-batched descriptions** โ€” the `read` tools are `load()` callers: N parallel reads coalesce into ONE batched vision call (dispatched via `setImmediate` after the poll phase, so reads completing together batch instead of splitting), awaited during the tool-result phase (free time) so the agent's next turn never blocks on the describer. Descriptions are ready before `context` fires, so the swap is a non-blocking cache hit.
39
41
  - **๐Ÿงน Hides pi's "model does not support images" note** โ€” on read results the extension strips pi's `[Current model does not support imagesโ€ฆ]` note from the text block (it's misleading once the handoff delivers the image's content as text), while keeping the image block for kitty inline rendering and `/resume`.
40
- - **๐Ÿ”Œ Provider-agnostic** โ€” uses pi's own model execution machinery (`@earendil-works/pi-ai`'s `complete()`), so it works with any provider/configured model, including custom provider extensions.
42
+ - **๐Ÿ”Œ Provider-agnostic** โ€” uses pi's own model execution machinery (`@earendil-works/pi-ai`'s `completeSimple()`), so it works with any provider/configured model, including custom provider extensions.
41
43
  - **๐Ÿง  Automatic targets** โ€” by default, handoff applies to *every* model that lacks native vision. Opt out with `/vision-handoff auto off`.
42
44
  - **๐Ÿ—‚๏ธ Explicit overrides** โ€” force handoff for specific models (e.g. a weak vision model) with `/vision-handoff add`.
43
45
  - **โšก Pre-warmed at paste-enter** โ€” the moment you press enter, `before_agent_start` scans the prompt for pasted clipboard image temp-file paths (pi writes pasted images to `<tmpdir>/pi-clipboard-<uuid>.<ext>` and inserts the path as text), reads them, and kicks off the ONE batched vision call concurrent with the agent's first response โ€” so by the time the agent reads the files, they're already cache hits.
@@ -137,7 +139,8 @@ pi -e ./vision-handoff.ts
137
139
  The extension implements the **Facebook DataLoader pattern** for image
138
140
  descriptions. The `read` tools are the `load()` callers; a per-image cache
139
141
  memoizes promises; all `load()` calls in the same execution frame coalesce
140
- into ONE batched vision call, dispatched after the microtask cascade settles.
142
+ into ONE batched vision call, dispatched via `setImmediate` after the poll
143
+ phase (so reads completing together batch instead of splitting into N calls).
141
144
 
142
145
  โ†’ before_agent_start
143
146
  โ€ข captures this turn's user prompt (shared by every image description)
@@ -160,10 +163,12 @@ into ONE batched vision call, dispatched after the microtask cascade settles.
160
163
  โ€ข pi runs `read` calls in parallel (Promise.all); each read's
161
164
  tool_result handler calls `loadDescription(img)` for its image
162
165
  blocks and AWAITS the shared batch
163
- โ€ข DataLoader: all `load()` calls in the same microtask frame land in
164
- ONE batch โ†’ ONE `complete()` vision call for the whole read set
165
- (dispatched after the microtask cascade via enqueuePostPromiseJob:
166
- `Promise.resolve().then(() => process.nextTick(dispatch))`)
166
+ โ€ข DataLoader: all `load()` calls in one event-loop poll iteration
167
+ land in ONE batch โ†’ ONE `completeSimple()` vision call for the whole read
168
+ set. `enqueuePostPromiseJob` schedules dispatch via `setImmediate`
169
+ (the check phase, which runs AFTER the whole poll phase โ€” not
170
+ `process.nextTick`, which would drain between the reads' I/O
171
+ callbacks and split them into N single-image calls)
167
172
  โ€ข awaits the shared batch โ€” runs the describer during the tool-result
168
173
  phase (free time: the agent is just waiting for tool results), so
169
174
  the batch is COMPLETE before `context` fires โ†’ `context` is a
@@ -191,24 +196,35 @@ A single frame's image set is described with **one** vision-model call, not
191
196
  one per image. `loadDescription(img)` is synchronous: on a cache miss it
192
197
  pushes the image's key (hash) into the current batch and returns a memoized
193
198
  promise; on a cache hit it returns the existing (in-flight or resolved)
194
- promise. `enqueuePostPromiseJob` schedules dispatch after the microtask
195
- cascade settles, so every `load()` caller in the frame registers its key
196
- before the single vision call fires โ€” exactly DataLoader's `getCurrentBatch`
197
- + `enqueuePostPromiseJob`. The batched call sends every uncached image in a
198
- single user message with per-image `<<<IMAGE k>>> โ€ฆ <<<END>>>` delimiters; the
199
- response is parsed back into per-image descriptions (keyed by
200
- `sha256(mime + base64)` in the per-image cache). If the vision model ignores
201
- the delimiter format and the batched response can't be split, each unparsed
202
- image falls back to its own single-image `complete()` call **in parallel**
203
- (no delimiters to cooperate with) โ€” descriptions still arrive together. Only
204
- when a per-image call itself genuinely fails (auth, timeout, empty) does that
205
- image degrade to `[Image: description unavailable]`; one bad image never
206
- voids the rest.
207
-
208
- Because pi runs parallel `read` tool calls via `Promise.all` and fires the
209
- `tool_result` event inside that `Promise.all` (via `agent.afterToolCall` โ†’
210
- `finalizeExecutedToolCall`), N parallel reads' `load()` calls share ONE
211
- microtask frame โ†’ ONE batch โ†’ ONE vision call, all resolving together.
199
+ promise. `enqueuePostPromiseJob` schedules dispatch via `setImmediate` (the
200
+ check phase, after the whole poll phase AND after the microtask queue
201
+ drains), so every `load()` caller โ€” whether from sync code, a `.then`
202
+ cascade, or a separate I/O callback in the same poll iteration โ€” registers
203
+ its key before the single vision call fires. This is why N parallel reads
204
+ coalesce into one call rather than splitting into N single-image calls:
205
+ `setImmediate` defers past the entire poll phase, whereas DataLoader's
206
+ classic `process.nextTick` would drain between the reads' I/O callbacks and
207
+ fire a dispatch after the first read but before the second. The batched call
208
+ sends every uncached image in a single user message with per-image
209
+ `<<<IMAGE k>>> โ€ฆ <<<END>>>` delimiters; the response is parsed back into
210
+ per-image descriptions (keyed by `sha256(mime + base64)` in the per-image
211
+ cache). If the vision model ignores the delimiter format and the batched
212
+ response can't be split, each unparsed image falls back to its own
213
+ single-image `completeSimple()` call **in parallel** (no delimiters to cooperate
214
+ with) โ€” descriptions still arrive together. Only when a per-image call
215
+ itself genuinely fails (auth, timeout, empty) does that image degrade to
216
+ `[Image: description unavailable]`; one bad image never voids the rest.
217
+
218
+ Because pi runs parallel `read` tool calls via `Promise.all` and fires
219
+ each read's `tool_result` event as that read's I/O completes (poll phase) โ€”
220
+ via `agent.afterToolCall` โ†’ `finalizeExecutedToolCall` โ€” the loader's
221
+ `setImmediate` dispatch defers to the check phase AFTER the whole poll
222
+ iteration, so reads completing together share ONE batch โ†’ ONE vision call,
223
+ all resolving together. Reads completing in separate poll iterations get
224
+ separate calls, but always in parallel, never sequential. (The per-image
225
+ cache also dedupes a duplicate `load()` of the same image in one frame:
226
+ dispatch resolves every callback by hash, so a second load whose first cache
227
+ entry was evicted mid-frame still resolves โ€” it never hangs.)
212
228
 
213
229
  **Failures are never cached.**
214
230
 
@@ -252,7 +268,7 @@ off mid-stream); earlier sections had `<<<END>>>` delimiters and are complete.
252
268
  it before the LLM call), so a slow describer would also make aborting a turn
253
269
  slow โ€” pi has to wait for the transform to return. The hook therefore wires
254
270
  the turn's abort signal (`ctx.signal`, the active run's `AbortController`)
255
- into the describer's `complete()` call, so a user cancel kills the in-flight
271
+ into the describer's `completeSimple()` call, so a user cancel kills the in-flight
256
272
  vision request immediately. A deliberate abort is not warned about (it's
257
273
  not a vision-model failure) and the LLM-bound payload is left untouched since
258
274
  the turn is being torn down anyway.
@@ -266,7 +282,7 @@ the turn is being torn down anyway.
266
282
  | `context` (all messages) | `{ type: "input_image", image_url: "data:โ€ฆ" }` (OpenAI Responses) | detected by shape โ†’ replaced with `{ type: "input_text", text }` |
267
283
  | `context` (all messages) | `{ type: "image", source: { type: "base64", media_type, data } }` (Anthropic Messages) | detected by shape โ†’ replaced with `{ type: "text", text }` |
268
284
 
269
- The describer call itself goes through pi's normal model machinery (`complete()`),
285
+ The describer call itself goes through pi's normal model machinery (`completeSimple()`),
270
286
  **not** the agent event loop โ€” so it never re-triggers `context` (no recursion).
271
287
  The `read` tool result keeps its image block untouched (kitty inline + `/resume`);
272
288
  only the `context`-cloned LLM-bound payload has images swapped for text.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-vision-handoff",
3
- "version": "0.4.0",
3
+ "version": "0.4.2",
4
4
  "description": "Give text-only pi models vision โ€” describe images with a vision model you pick via an interactive picker, then hand off the text description to non-vision models",
5
5
  "type": "module",
6
6
  "author": "Tom X Nguyen",
package/src/dataloader.ts CHANGED
@@ -4,19 +4,23 @@
4
4
  * `loadDescription(img)` returns a memoized Promise for the description and
5
5
  * pushes the image's key (hash) into the CURRENT batch. All `load()` calls in
6
6
  * the same execution frame (+ its microtask cascade) coalesce into ONE batch
7
- * object. Dispatch is scheduled after the microtask queue settles (via
8
- * `enqueuePostPromiseJob`), so every load in the frame lands in the single
9
- * batch before the ONE vision call fires. Each load()'s promise then resolves
10
- * with its description.
7
+ * object. Dispatch is scheduled via `setImmediate` (see `enqueuePostPromiseJob`),
8
+ * so every load in the frame โ€” and every load from a separate I/O callback in
9
+ * the same poll iteration โ€” lands in the single batch before the ONE vision
10
+ * call fires. Each load()'s promise then resolves with its description.
11
11
  *
12
12
  * Mapping: the `read` tools are the `load()` callers. Their `tool_result`
13
- * handler awaits the shared batch โ€” so N parallel reads (pi runs `read` via
14
- * `Promise.all`) block on the SAME single vision call and all resolve together,
15
- * the descriptions landing in the tool results BEFORE the agent's next turn.
16
- * The agent's tool-result wait is free time, so this adds zero latency to the
17
- * critical path. `context` then sees text-described tool results (no image
18
- * blocks to swap); any remaining images (user-attached, custom-injected) are
19
- * cache hits.
13
+ * handler awaits the shared batch โ€” so N parallel reads coalesce into the SAME
14
+ * single vision call and all resolve together, the descriptions landing in the
15
+ * tool results BEFORE the agent's next turn. pi fires each read's `tool_result`
16
+ * as that read's I/O completes (poll phase); the loader's `setImmediate`
17
+ * dispatch defers to the check phase, AFTER the whole poll iteration, so reads
18
+ * completing together (the common case for cached local files) land in ONE
19
+ * batch โ€” and reads completing in separate iterations get separate calls, but
20
+ * always in parallel, never sequential. The agent's tool-result wait is free
21
+ * time, so this adds zero latency to the critical path. `context` then sees
22
+ * text-described tool results (no image blocks to swap); any remaining images
23
+ * (user-attached, custom-injected) are cache hits.
20
24
  *
21
25
  * All mutable state (batch, cache, turn context) lives on the instance โ€” no
22
26
  * module-level globals โ€” and the class implements `Disposable` so a `using`
@@ -43,7 +47,12 @@ export const UNAVAILABLE = `${IMAGE_PLACEHOLDER_PREFIX}description unavailable${
43
47
  interface DescriptionBatch {
44
48
  keys: string[];
45
49
  imgs: ExtractedImage[];
46
- callbacks: { resolve: (v: string) => void; reject: (e: Error) => void }[];
50
+ /** One per `loadDescription()` call. A duplicate load (same hash, but its
51
+ * first cache entry was evicted mid-frame so it couldn't short-circuit on
52
+ * the cache) pushes a second callback for an existing key โ€” so
53
+ * `callbacks.length` can exceed `keys.length`. `dispatchBatch` resolves by
54
+ * hash so every callback is reached. */
55
+ callbacks: { hash: string; resolve: (v: string) => void; reject: (e: Error) => void }[];
47
56
  }
48
57
 
49
58
  /** Engine-provided resolver for the configured vision model. */
@@ -59,16 +68,21 @@ export interface LoaderDeps extends DescriberDeps {
59
68
  resolveVisionModel: VisionModelResolver;
60
69
  }
61
70
 
62
- const resolvedMicrotask = Promise.resolve();
63
-
64
- /** Defer `fn` until after the current microtask cascade settles, so every
65
- * `loadDescription()` in the same frame coalesces into one batch before the
66
- * single vision call fires. Mirrors DataLoader's enqueuePostPromiseJob: a
67
- * Promise Job enqueues a global Job (process.nextTick), guaranteeing dispatch
68
- * runs after "PromiseJobs" ends โ€” after all `load()` callers in the frame
69
- * have registered their keys. */
71
+ /** Defer `fn` to the next check phase (`setImmediate`), so every
72
+ * `loadDescription()` โ€” whether called from sync code, a microtask cascade,
73
+ * or a separate I/O callback in the same poll iteration โ€” coalesces into one
74
+ * batch before the single vision call fires.
75
+ *
76
+ * Why `setImmediate` and not DataLoader's classic `process.nextTick`:
77
+ * nextTick drains between I/O callbacks in the poll phase, so dispatch would
78
+ * fire after the first parallel `read`'s `tool_result` but before the
79
+ * second's โ€” splitting N reads into N single-image calls. `setImmediate` runs
80
+ * in the check phase, AFTER the whole poll phase, so all `tool_result`
81
+ * handlers that fire in one poll iteration land in ONE batch. The check phase
82
+ * also runs after the microtask queue drains, so loads issued from a `.then`
83
+ * cascade (e.g. the clipboard pre-warm) still coalesce. */
70
84
  function enqueuePostPromiseJob(fn: () => void): void {
71
- resolvedMicrotask.then(() => process.nextTick(fn));
85
+ setImmediate(fn);
72
86
  }
73
87
 
74
88
  export class DescriptionLoader implements Disposable {
@@ -119,12 +133,14 @@ export class DescriptionLoader implements Disposable {
119
133
  batch.imgs.push(img);
120
134
  idx = batch.keys.length - 1;
121
135
  } else {
122
- // Same image loaded twice in the frame โ€” share one cache slot, but give
123
- // this caller its own promise.
136
+ // Same image loaded twice in the frame (the first load's cache entry was
137
+ // evicted mid-frame, else the second load would have short-circuited on
138
+ // the cache). Share the one key/image slot but give this caller its own
139
+ // promise โ€” dispatch resolves it by hash alongside the first caller's.
124
140
  batch.imgs[idx] = img;
125
141
  }
126
142
  const promise = new Promise<string>((resolve, reject) => {
127
- batch.callbacks.push({ resolve, reject });
143
+ batch.callbacks.push({ hash, resolve, reject });
128
144
  });
129
145
  const cfg = this.deps.getConfig();
130
146
  if (this.cache.size >= cfg.cacheMax) {
@@ -161,7 +177,14 @@ export class DescriptionLoader implements Disposable {
161
177
  /** Dispatch the current batch: ONE batched `runBatch` vision call for every
162
178
  * key collected this frame, then resolve each load()'s promise with its
163
179
  * description (or {@link UNAVAILABLE} on failure). Failures are evicted from
164
- * the cache so the next turn re-attempts. */
180
+ * the cache so the next turn re-attempts.
181
+ *
182
+ * Results are resolved BY HASH and fanned out to EVERY callback: a batch can
183
+ * hold more callbacks than keys when the same image was loaded twice in one
184
+ * frame (its first cache entry was evicted mid-frame, so the second load
185
+ * pushed a second callback for the same hash). Indexing callbacks by key
186
+ * would skip those duplicates and hang their promises; iterating callbacks
187
+ * and looking up each one's hash fans the one result to all of them. */
165
188
  private async dispatchBatch(): Promise<void> {
166
189
  this.dispatchScheduled = false;
167
190
  const batch = this.batch;
@@ -172,10 +195,8 @@ export class DescriptionLoader implements Disposable {
172
195
  return;
173
196
  }
174
197
  if (this.turnSignal?.aborted) {
175
- for (let i = 0; i < batch.keys.length; i++) {
176
- this.cache.delete(batch.keys[i]);
177
- batch.callbacks[i].resolve(UNAVAILABLE);
178
- }
198
+ for (const key of batch.keys) this.cache.delete(key);
199
+ for (const cb of batch.callbacks) cb.resolve(UNAVAILABLE);
179
200
  return;
180
201
  }
181
202
  this.deps.setLastError(null); // clear before a fresh attempt
@@ -190,19 +211,27 @@ export class DescriptionLoader implements Disposable {
190
211
  this.deps,
191
212
  this.turnSignal,
192
213
  );
214
+ // Build per-hash results, then fan them out to every callback. This reaches
215
+ // duplicate-hash callbacks that a key-indexed loop would have skipped (and
216
+ // left hanging).
217
+ const results = new Map<string, string>();
193
218
  for (let i = 0; i < batch.keys.length; i++) {
194
- const raw = parsed.get(batch.keys[i]);
219
+ const key = batch.keys[i];
220
+ const raw = parsed.get(key);
195
221
  if (raw) {
196
222
  const final = wrapDescription(raw, cfg);
197
- this.cache.set(batch.keys[i], Promise.resolve(final));
198
- batch.callbacks[i].resolve(final);
223
+ results.set(key, final);
224
+ // Cache the resolved value so later loads (this frame or next) hit.
225
+ this.cache.set(key, Promise.resolve(final));
199
226
  } else {
200
227
  // Genuine failure โ€” do NOT cache; next turn re-attempts (and surfaces
201
228
  // the real error). Resolve (not reject) with UNAVAILABLE to match the
202
229
  // graceful-degradation contract and avoid unhandled rejections.
203
- this.cache.delete(batch.keys[i]);
204
- batch.callbacks[i].resolve(UNAVAILABLE);
230
+ this.cache.delete(key);
205
231
  }
206
232
  }
233
+ for (const cb of batch.callbacks) {
234
+ cb.resolve(results.get(cb.hash) ?? UNAVAILABLE);
235
+ }
207
236
  }
208
237
  }
package/src/describer.ts CHANGED
@@ -1,6 +1,14 @@
1
1
  /**
2
- * The vision describer: calls a vision-capable model via pi-ai's `complete()`
3
- * to produce text descriptions of images.
2
+ * The vision describer: calls a vision-capable model via pi-ai's
3
+ * `completeSimple()` to produce text descriptions of images.
4
+ *
5
+ * `completeSimple` (not `complete`) is the path that translates the `reasoning`
6
+ * ThinkingLevel into each provider's `reasoningEffort`/budget. `complete()` โ†’
7
+ * `stream()` reads only the pre-translated `reasoningEffort` field and silently
8
+ * drops a bare `reasoning`, so the describer's thinking setting would be a no-op
9
+ * through `complete()` โ€” the bug this swap fixes. The agent loop, SDK, and
10
+ * compaction all route thinking through `completeSimple`/`streamSimple` for the
11
+ * same reason.
4
12
  *
5
13
  * Two entry points:
6
14
  * - {@link runBatch}: ONE batched call describing N images at once (the
@@ -13,13 +21,13 @@
13
21
  * Resource lifetimes (fetch interceptor, timeout timer, turn-abort wire) are
14
22
  * managed with the `using` keyword via the {@link Disposable} guards in
15
23
  * `dispose.ts`, replacing the manual `try`/`finally` cleanup the old code
16
- * carried. Disposing is lexical and exception-safe: a thrown `complete()`
24
+ * carried. Disposing is lexical and exception-safe: a thrown `completeSimple()`
17
25
  * still tears down the timer, uninstalls the interceptor, and detaches the
18
26
  * abort listener.
19
27
  */
20
28
 
21
29
  import type { Api, ImageContent, Message, Model, TextContent, ThinkingLevel } from "@earendil-works/pi-ai";
22
- import { complete } from "@earendil-works/pi-ai/compat";
30
+ import { completeSimple } from "@earendil-works/pi-ai/compat";
23
31
  import type { ModelRegistry } from "@earendil-works/pi-coding-agent";
24
32
  import {
25
33
  batchUserPrompt,
@@ -52,7 +60,7 @@ import { abortWireGuard, fetchInterceptorGuard, timeoutGuard, type AbortWire } f
52
60
  * model (e.g. 262144 context โˆ’ 8192 reserve = 253952 output budget). */
53
61
  const INPUT_RESERVE_TOKENS = 8192;
54
62
 
55
- /** Resolve the `maxTokens` to pass to `complete()` for a describer call.
63
+ /** Resolve the `maxTokens` to pass to `completeSimple()` for a describer call.
56
64
  *
57
65
  * - A configured `cfg.maxTokens` wins (explicit cost/latency cap), clamped to
58
66
  * fit the context window.
@@ -76,7 +84,7 @@ export function resolveMaxTokens(
76
84
  return Math.max(1, Math.min(requested, cap));
77
85
  }
78
86
 
79
- /** Resolve the `reasoning` (thinking) level to pass to `complete()`, or
87
+ /** Resolve the `reasoning` (thinking) level to pass to `completeSimple()`, or
80
88
  * `undefined` to leave thinking off.
81
89
  *
82
90
  * - Returns `undefined` when thinking is disabled in config.
@@ -108,7 +116,7 @@ export interface DescriberDeps {
108
116
  /** The result of a batched describer call: per-image raw descriptions keyed by hash. */
109
117
  export type BatchResult = Map<string, string>;
110
118
 
111
- /** Describe N images with ONE batched `complete()` call and split the response
119
+ /** Describe N images with ONE batched `completeSimple()` call and split the response
112
120
  * back into per-image descriptions. Returns a map keyed by image hash; an
113
121
  * image whose section failed to parse is omitted (the caller treats omission
114
122
  * as "description unavailable"). On a genuine call failure (auth, abort,
@@ -159,7 +167,7 @@ export async function runBatch(
159
167
  const describeCtx: DescribeContext = { energyReader: undefined };
160
168
  try {
161
169
  const response = await describeAls.run(describeCtx, async () =>
162
- complete(
170
+ completeSimple(
163
171
  visionModel,
164
172
  { systemPrompt, messages: [userMessage] },
165
173
  { apiKey: auth.apiKey, headers: auth.headers, signal: controller.signal, maxTokens, reasoning },
@@ -231,7 +239,7 @@ export async function runBatch(
231
239
  }
232
240
  }
233
241
 
234
- /** Describe a single image with one `complete()` call and return the RAW
242
+ /** Describe a single image with one `completeSimple()` call and return the RAW
235
243
  * description (no envelope, no truncation). Returns null on any genuine
236
244
  * failure (auth, abort/error, empty) so the caller only caches `UNAVAILABLE`
237
245
  * when a real describer attempt failed. */
@@ -273,7 +281,7 @@ export async function describeSingle(
273
281
  const describeCtx: DescribeContext = { energyReader: undefined };
274
282
  try {
275
283
  const response = await describeAls.run(describeCtx, async () =>
276
- complete(
284
+ completeSimple(
277
285
  visionModel,
278
286
  { systemPrompt, messages: [userMessage] },
279
287
  { apiKey: auth.apiKey, headers: auth.headers, signal: controller.signal, maxTokens, reasoning },
package/src/index.ts CHANGED
@@ -38,7 +38,7 @@ export const DEFAULT_USER_PROMPT_PREFIX = "The user's request about this image:
38
38
  export const IMAGE_PLACEHOLDER_PREFIX = "[Image: ";
39
39
  export const IMAGE_PLACEHOLDER_SUFFIX = "]";
40
40
 
41
- /** Marker appended to a description whose `complete()` call ended with
41
+ /** Marker appended to a description whose `completeSimple()` call ended with
42
42
  * `stopReason: "length"` โ€” i.e. the vision model hit a token limit (either the
43
43
  * configured `maxTokens` or the provider's hard output cap) before finishing.
44
44
  * A truncated description is still useful, but the agent must not mistake it
package/src/usage.ts CHANGED
@@ -3,7 +3,7 @@
3
3
  *
4
4
  * One record is produced per REAL describer provider call (cache hits emit
5
5
  * nothing):
6
- * - model + tokens: from complete()'s AssistantMessage.usage
6
+ * - model + tokens: from completeSimple()'s AssistantMessage.usage
7
7
  * - energy + cost + raw MCR/energy/cost payloads: from Neuralwatt SSE comment
8
8
  * lines parsed out of the teed response body (readEnergyFromTee). Present
9
9
  * ONLY when the vision model is a Neuralwatt model โ€” non-Neuralwatt models
@@ -197,7 +197,7 @@ export function buildUsageRecord(
197
197
 
198
198
  // โ”€โ”€ Concurrency-safe fetch interceptor โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
199
199
  //
200
- // The only body-interception point for complete() is globalThis.fetch (pi-ai's
200
+ // The only body-interception point for completeSimple() is globalThis.fetch (pi-ai's
201
201
  // StreamOptions.onResponse exposes headers only, not the body where the
202
202
  // `: energy` SSE comments live). before_agent_start fires several describeImage()
203
203
  // calls fire-and-forget, so a naรฏve save/patch/restore of globalThis.fetch would
package/vision-handoff.ts CHANGED
@@ -210,7 +210,8 @@ export default function (pi: ExtensionAPI) {
210
210
  //
211
211
  // Both sources flow through the same loader: `load()` is synchronous and
212
212
  // memoized, so all images in this frame (attached + clipboard-path)
213
- // coalesce into ONE batch dispatched after the microtask cascade.
213
+ // coalesce into ONE batch dispatched via `setImmediate` after the
214
+ // microtask cascade settles.
214
215
  for (const image of event.images ?? []) {
215
216
  if (!image || image.type !== "image" || !image.data) continue;
216
217
  loader.loadDescription({ data: image.data, mimeType: image.mimeType || "image/png" }).catch(() => {});
@@ -244,12 +245,14 @@ export default function (pi: ExtensionAPI) {
244
245
  // When the agent reads image files, this fires for each read result. It
245
246
  // calls the loader's `loadDescription(img)` for every image block and
246
247
  // AWAITS the shared batch โ€” so N parallel reads (pi runs `read` via
247
- // Promise.all) coalesce into ONE batched vision call (DataLoader: all
248
- // load() calls in the same microtask frame share one batch, dispatched
249
- // after the cascade settles) and all resolve together. The descriptions
250
- // replace the image blocks in the returned `content`, so by the time the
251
- // agent's next turn starts the tool results already carry text โ€” the
252
- // agent never sees raw image blocks it can't process.
248
+ // Promise.all) coalesce into ONE batched vision call: pi fires each read's
249
+ // `tool_result` as its I/O completes (poll phase), and the loader's
250
+ // `setImmediate` dispatch defers to the check phase AFTER the whole poll
251
+ // iteration, so reads completing together land in ONE batch and all resolve
252
+ // together. The descriptions replace the image blocks in the returned
253
+ // `content`, so by the time the agent's next turn starts the tool results
254
+ // already carry text โ€” the agent never sees raw image blocks it can't
255
+ // process.
253
256
  //
254
257
  // Why block here and not in `context`: the tool-result phase is free time
255
258
  // (the agent is just waiting for tool results), so running the describer
@@ -277,9 +280,13 @@ export default function (pi: ExtensionAPI) {
277
280
  }
278
281
 
279
282
  // load() each image โ€” synchronous calls that push into the current batch
280
- // and return memoized promises โ€” then await them all. Parallel reads'
281
- // load() calls land in the SAME batch (same microtask frame), so this is
282
- // ONE vision call for the whole read set, not N. Awaiting here runs the
283
+ // and return memoized promises โ€” then await them all. pi fires each read's
284
+ // `tool_result` event as that read's I/O completes (poll phase); the
285
+ // loader's `setImmediate` dispatch defers to the check phase, AFTER the
286
+ // whole poll iteration, so reads completing together (the common case for
287
+ // cached local files) land in ONE batch โ€” ONE vision call for the whole
288
+ // read set, not N. Reads completing in separate iterations get separate
289
+ // calls, but always in parallel, never sequential. Awaiting here runs the
283
290
  // describer during the tool-result phase (free time โ€” the agent is just
284
291
  // waiting for tool results), so the batch is COMPLETE before `context`
285
292
  // fires, making `context` a non-blocking cache hit instead of a cold miss