pi-multimodal-proxy 1.10.1 → 1.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,124 +1,130 @@
1
- # Changelog
2
-
3
- All notable changes to this project will be documented in this file.
4
-
5
- The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
6
-
7
- ## [1.10.1] - 2026-07-08
8
-
9
- ### Fixed
10
-
11
- - **Source-code files with overloaded media extensions are no longer sent to video models.** The `.ts` extension is mapped to both TypeScript (source code) and MPEG-TS (`video/mp2t`) video. Previously, a file like `store.ts` mentioned in a prompt was matched by the video-path extractor, read as a 200 MB media file, and shipped to the configured video provider (e.g. "Analyzing store.ts via Grok 4.3…"). `readMediaFileWithReason` now sniffs the file contents for extensions whose primary meaning is source code (`.ts`, `.mts`, `.m2ts`): it validates the MPEG-TS sync byte (`0x47`) at the start of the first few 188-byte packets and rejects the file as `not-a-media` when the signature is absent. Genuine MPEG-TS streams continue to work; TypeScript files no longer leak source code to a video model. The `not-a-media` skip message in `vision-proxy.ts` was updated to describe the new content-sniffing behavior.
12
-
13
- ## [1.10.0] - 2026-07-07
14
-
15
- ### Added
16
-
17
- - **Configurable allowed folders** ([#15](https://github.com/pungggi/pi-multimodal-proxy/issues/15)) the file-access allowlist is now a persisted config setting instead of env-var-only. `/multimodal-proxy folders add <path>` (also `remove`, `list`, `reset`) grants media reads from custom absolute folders (`~` is expanded), and `/multimodal-proxy allow-home on|off` is the persisted equivalent of `PI_VISION_PROXY_ALLOW_HOME=1`. Both are stored in `~/.pi/agent/multimodal-proxy.json` alongside the other settings and survive new sessions. Also manageable from the interactive `/multimodal-proxy` menu.
18
- - **New env var**: `PI_VISION_PROXY_ALLOWED_FOLDERS` — platform-delimiter-separated list of absolute folders (`:` on Unix, `;` on Windows); like other env vars it overrides and locks the persisted setting. `PI_VISION_PROXY_ALLOW_HOME` keeps working and now also accepts `0`/`false` to override a persisted `allow-home on`.
19
- - The allowlist applies uniformly to auto-proxied prompt paths (images, video, audio), the `analyze_image` tool, and `/multimodal-proxy describe`. Allowlisted folders are canonicalized via `realpath` before comparison; relative entries are rejected and the list is capped at 100 folders. New tested helpers: `expandLeadingTilde`, `sanitizeAllowedFolders`, `pathAccessFromConfig`, and the `access` parameter on `isPathAllowed` / `readImageFileWithReason` / `readMediaFileWithReason`.
20
- - **Hide-able status line** ([#16](https://github.com/pungggi/pi-multimodal-proxy/issues/16)) — new persisted `statusLine` setting. `/multimodal-proxy status off` hides the steady-state footer status (`multimodal-proxy: fallback → … | video: …`), `/multimodal-proxy status on` restores it, and the interactive config menu gets a matching toggle. The transient analysis progress spinner still shows while a call is in flight and clears when it finishes. New env override: `PI_VISION_PROXY_STATUS_LINE=on|off`.
21
-
22
- ### Fixed
23
-
24
- - Changing any setting via `/multimodal-proxy` refreshed the status line under the stale `vision-proxy` key with an outdated text format, leaving a second, never-updated status entry alongside the real one. The refresh now uses the same `multimodal-proxy` key and steady-state text (including the video model and registry fallback) as session start.
25
- - The interactive `/multimodal-proxy` summary now lists every active env override (previously only shown when mode/model/context was set, hiding overrides like `tool`, `cacheSize`, or `statusLine`), and an invalid env value (e.g. `PI_VISION_PROXY_STATUS_LINE=hidden`) no longer locks the matching subcommand without actually overriding anything.
26
-
27
- ## [1.9.0] - 2026-07-07
28
-
29
- ### Added
30
-
31
- - **Pre-consented providers** ([#14](https://github.com/pungggi/pi-multimodal-proxy/issues/14)) a persisted `allowedProviders` list lets you consent to data egress for chosen providers once, instead of once per session. Providers on the list skip the first-use consent prompt everywhere (auto-proxy, video/audio, the `analyze_image` tool, and `/multimodal-proxy describe`). Manage it with `/multimodal-proxy allowed-providers add|remove <provider>|clear` (or the interactive config menu), or grant-and-persist in one step with `/multimodal-proxy consent always`. The list lives in `~/.pi/agent/multimodal-proxy.json` next to the other persisted settings and is kept out of session-entry configs so per-session config changes can never clobber it.
32
- - **New env var**: `PI_VISION_PROXY_ALLOWED_PROVIDERS` — comma-separated provider ids, overriding the persisted list (a defined-but-empty value disables the list for that shell/project, handy for sensitive repositories).
33
- - Safety semantics: an explicit in-session `/multimodal-proxy consent no` always beats the pre-consent list, and additionally removes the provider from the persisted list so the refusal sticks across sessions. The list only ever matches a specific provider — it is never a blanket grant. Provider ids are validated and canonicalized (`x-ai` → `xai`) on every boundary (file, env, commands).
34
- - New tested helpers in `internal.ts`: `parseProviderList` (comma/whitespace splitting, canonicalization, dedup) and `consentState` (distinguishes "revoked" from "no verdict" so pre-consent can't override a refusal); `hasConsent` gained an optional `allowedProviders` parameter.
35
-
36
- ## [1.8.0] - 2026-07-04
37
-
38
- ### Added
39
-
40
- - **Compaction survival** media knowledge now survives context compaction. Previously, compaction summarized away the user messages carrying image blocks (and the injected video fences), so the `context` handler had nothing left to annotate and the agent lost all knowledge of earlier images/videos — even though the description entries were still persisted in the session. Now, when the active branch contains a compaction entry, the proxy detects which persisted image/video descriptions are no longer visible in context and re-injects them as a **post-compaction recall digest**: truncated description fences keyed by the same stable `image="..."` ids that `analyze_image` recall accepts, placed directly after the compaction summary.
41
- - **Compaction-trigger awareness** (Pi ≥ 0.79.10) — a `session_compact` handler records the compaction's `reason`/`willRetry` metadata. During the **overflow-recovery** window (context hit the hard limit and the turn is retried), the digest switches to lean budgets (200/240 chars per image/video instead of 600/800) so re-injection doesn't contribute to a second overflow; `turn_end` closes the window so later turns get normal budgets again. On older Pi runtimes the fields are absent and the digest simply uses its normal budgets.
42
- - New helpers in `internal.ts`, all covered by unit tests: `findVideoDescriptions` (latest persisted entry per hash), `truncateForDigest` (word-boundary truncation with `… [truncated]` marker), and `buildCompactionDigest` (caps at the 12 most recent images / 4 most recent videos, restates the UNTRUSTED warning, and mentions `analyze_image` recall only when the tool is enabled).
43
- - **`#` image-recall autocomplete** (Pi ≥ 0.79.1) — typing `#` at a token boundary in the interactive editor now suggests images seen earlier in the session (newest first, fuzzy-matched on filename, id, and description as you type). Picking one inserts the image's stable `image="<hash>"` recall id into the prompt, so *"zoom into `#`⇥"* works without copying ids out of fences. Implemented as a stacked autocomplete provider via `ctx.ui.addAutocompleteProvider`: it falls through to the built-in provider when the token matches no image, is a no-op in RPC/print modes and on older Pi versions without the API, and suggests nothing when the proxy is `off`. New tested helpers: `extractRecallToken`, `collectRecallCandidates`, `buildRecallItems` (max 8 items), `parseRecallItemValue`, `applyRecallCompletion`.
44
-
45
- ### Changed
46
-
47
- - **Default vision model bumped to Claude Sonnet 5** (`anthropic/claude-sonnet-5`, in Pi catalogs since 0.80.3), with default-tracking for implicit configs: explicit model choices are now persisted with a `modelExplicit` flag (`/multimodal-proxy model` and `pick`), and only *implicit* model values participate in substitution. An implicit legacy baked-in default (`claude-sonnet-4-5`, which full-config persistence wrote into every config on any settings change) is upgraded to the current default when the registry has it, and the current default falls back to `claude-sonnet-4-5` on older Pi catalogs. Explicit choices (via the flag or `PI_VISION_PROXY_MODEL`) are never rewritten a missing explicit model still surfaces as "Model not found". The registry-resolved model is applied consistently at image analysis, the `analyze_image` tool, `/multimodal-proxy describe`, the status line, and the interactive config menu.
48
- - The `context` handler no longer returns early when the active model supports images natively the post-compaction digest is injected whenever the proxy is not `off` and orphaned descriptions exist, since natively-visioned models also lose compacted-away images. Image-block stripping behavior is unchanged.
49
-
50
- ## [1.7.0] - 2026-06-20
51
-
52
- ### Added
53
-
54
- - **Session image recall** the agent can now re-query an image it saw earlier in the session without a re-attachment or file path. Every `<vision_proxy_description>`, `<vision_proxy_analysis>`, and `<vision_proxy_joint_description>` block already carries an `image="..."` id; passing that id back to `analyze_image` (or `/multimodal-proxy describe`) recalls the original image bytes and re-runs the targeted question or crop against it (e.g. *"zoom into that screenshot from before"*).
55
- - Image bytes are retained **in process memory only** — never written to the session log or disk — in a byte-bounded LRU store (`PI_VISION_PROXY_IMAGE_RECALL_BYTES`, default 64 MB, oldest-first eviction).
56
- - **New env var**: `PI_VISION_PROXY_IMAGE_RECALL_BYTES`.
57
- - **Persistent recall reminder** — when the proxy rewrites earlier images into descriptions, it now restates once per turn (as trusted text, outside the untrusted fence) that those images can be re-queried by id, so the affordance is visible even on turns where no new image was attached.
58
- - **Live progress indicator** — slow image and video/audio analysis now animate a spinner with elapsed seconds on the status line (`multimodal-proxy ⠙ Analyzing image 2/4… (3s)`), restoring the steady-state status when the call completes. No-ops without a UI.
59
- - Unit tests for `parseRecallRef` (bare hash, `sha256:` prefix, `#crop` suffix, case normalization, path rejection), the recall store (round-trip, dedup, byte-budget eviction, oversized-single-image retention, recency bumping), `spinnerFrame`, `formatProgressStatus`, and `RECALL_HINT`.
60
-
61
- ### Changed
62
-
63
- - `analyze_image` now accepts a recall handle (the fence `image="..."` id) as an image reference in addition to a file path; the previous hard rejection of `sha256:` references is removed. The tool schema, tool description, and the injected Vision Proxy system-prompt section document the recall handle.
64
-
65
- ## [1.4.0-beta.1] - 2026-05-03
66
-
67
- ### Added
68
-
69
- - **`analyze_image` tool** agent-facing tool for targeted re-querying of images with multi-form crop support (FR-1.x). Disabled by default during beta; enable with `/vision-proxy tool on`.
70
- - **Three crop forms**: `region` (named areas like `top-right`, `center`), `normalized` (0.0–1.0 fractional coordinates), and `pixels` (absolute pixel coordinates). All resolve to pixel rectangles with clamping and zero-area validation.
71
- - **Image dimension extraction** via `image-size` package. Dimensions and filenames stored in an in-memory `_imageMeta` map populated on first image ingestion.
72
- - **Enhanced fence tags**: `<vision_proxy_description>` now carries `image`, `width`, `height`, `filename`, and `crop_origin` attributes. New `<vision_proxy_analysis>` fence for tool results with optional `grounding_format` attribute.
73
- - **LRU result cache** for `analyze_image` calls, keyed by (image hashes, crop signature, question hash, model).
74
- - **Grounding format registry** with curated Tier 1 defaults (Qwen, Molmo, DeepSeek, InternVL, Gemini). Grounding instructions appended to system prompt per model's native format.
75
- - **New configuration**: `/vision-proxy tool on|off`, `max-images-per-call <n>`, `max-batch <n>`, `cache-size <n>`.
76
- - **New env vars**: `PI_VISION_PROXY_TOOL`, `PI_VISION_PROXY_MAX_IMAGES_PER_CALL`, `PI_VISION_PROXY_MAX_BATCH`, `PI_VISION_PROXY_CACHE_SIZE`, `PI_VISION_PROXY_PHASH_THRESHOLD`.
77
- - **Security**: `fenceUntrusted` now neutralizes all three fence tag types (`description`, `analysis`, `joint_description`).
78
- - **`readImageFileWithReason`** now returns the file's basename in the `filename` field.
79
- - **Telemetry**: `vision_proxy.tool_call` session entries with crop form, latency, cache hit status.
80
- - 112 unit tests covering crop resolution, LRU cache, dimension extraction, fence building, grounding lookups, config backwards compatibility, and all new env var parsing.
81
-
82
- ### Changed
83
-
84
- - `VisionConfig` extended with `tool`, `maxImagesPerCall`, `maxBatch`, `cacheSize`, `pHashSimilarityThreshold`, `groundingModels` fields. Backwards compatible — 1.3.0 config files load unchanged with sensible defaults.
85
- - Version bumped to `1.4.0-beta.1`.
86
- - Added `image-size` as a runtime dependency.
87
-
88
- ## [1.3.0] - 2026-05-01
89
-
90
- ### Added
91
-
92
- - Two-step model picker (`/vision-proxy pick`): provider first, then model. Replaces the single flat list of 400+ models.
93
- - Current provider is shown first with a ★ marker and pre-selected — picker opens directly on the model list, no need to re-select the same provider every time.
94
- - `← Change provider` option inside the model list to switch providers without restarting the picker.
95
- - `🔍 Type to filter models…` option for providers with more than 8 models. Uses fuzzy character-order matching (e.g. `cs4` matches `Claude Sonnet 4.5`). Single matches are auto-selected.
96
- - `fuzzyMatches()` helper exported from `internal.ts` with full test coverage.
97
-
98
- ### Changed
99
-
100
- - Duplicated picker code between `/vision-proxy pick` and the interactive `Model:` row consolidated into a single `pickVisionModel()` function.
101
-
102
- ## [1.2.0] - 2026-05-01
103
-
104
- ### Added
105
-
106
- - `/vision-proxy pick` sub-command. Lists vision-capable models from the registry with friendly names and provider tags via `ctx.ui.select`. Avoids typing canonical ids like `accounts/fireworks/models/kimi-k2p6`.
107
- - Interactive `Model:` row in `/vision-proxy` config now opens the same vision-only picker (was raw text input).
108
- - `friendlyModelLabel(config, registry)` helper. Status line and notifies now display `Kimi K2.6 [fireworks]` instead of `fireworks/accounts/fireworks/models/kimi-k2p6` when the registry knows the model.
109
-
110
- ### Changed
111
-
112
- - "Model not found" error now points to `/vision-proxy pick` instead of `/vision-proxy model`.
113
-
114
- ## [1.1.0] - 2026-05-01
115
-
116
- ### Changed
117
-
118
- - Settings (mode, model, context) now persist across sessions to `~/.pi/agent/vision-proxy.json`. Previously settings were stored only in session entries and lost when starting a new session. Config precedence (highest → lowest): environment variables → session entries → persistent file → defaults.
119
-
120
- ### Added
121
-
122
- - `readPersistentFile()` / `writePersistentFile()` helpers for file-based config storage.
123
- - `fileConfig` parameter on `resolveConfig()` to layer persisted file config between defaults and session entries.
124
- - Tests for persistent file round-trip and layered config resolution.
1
+ # Changelog
2
+
3
+ All notable changes to this project will be documented in this file.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
6
+
7
+ ## [1.11.0] - 2026-07-29
8
+
9
+ ### Added
10
+
11
+ - **Global consent wildcard** `/multimodal-proxy allowed-providers add *` (or `all`) now grants consent for **all providers** globally, so you can consent once and never be prompted again. The wildcard (`*`) appears in the pre-consented list as "* (all providers)" and can be removed with `/multimodal-proxy allowed-providers remove *`. An explicit in-session `consent no` still beats the wildcard and applies only to the revoked provider.
12
+
13
+ ## [1.10.1] - 2026-07-08
14
+
15
+ ### Fixed
16
+
17
+ - **Source-code files with overloaded media extensions are no longer sent to video models.** The `.ts` extension is mapped to both TypeScript (source code) and MPEG-TS (`video/mp2t`) video. Previously, a file like `store.ts` mentioned in a prompt was matched by the video-path extractor, read as a 200 MB media file, and shipped to the configured video provider (e.g. "Analyzing store.ts via Grok 4.3…"). `readMediaFileWithReason` now sniffs the file contents for extensions whose primary meaning is source code (`.ts`, `.mts`, `.m2ts`): it validates the MPEG-TS sync byte (`0x47`) at the start of the first few 188-byte packets and rejects the file as `not-a-media` when the signature is absent. Genuine MPEG-TS streams continue to work; TypeScript files no longer leak source code to a video model. The `not-a-media` skip message in `vision-proxy.ts` was updated to describe the new content-sniffing behavior.
18
+
19
+ ## [1.10.0] - 2026-07-07
20
+
21
+ ### Added
22
+
23
+ - **Configurable allowed folders** ([#15](https://github.com/pungggi/pi-multimodal-proxy/issues/15)) — the file-access allowlist is now a persisted config setting instead of env-var-only. `/multimodal-proxy folders add <path>` (also `remove`, `list`, `reset`) grants media reads from custom absolute folders (`~` is expanded), and `/multimodal-proxy allow-home on|off` is the persisted equivalent of `PI_VISION_PROXY_ALLOW_HOME=1`. Both are stored in `~/.pi/agent/multimodal-proxy.json` alongside the other settings and survive new sessions. Also manageable from the interactive `/multimodal-proxy` menu.
24
+ - **New env var**: `PI_VISION_PROXY_ALLOWED_FOLDERS` platform-delimiter-separated list of absolute folders (`:` on Unix, `;` on Windows); like other env vars it overrides and locks the persisted setting. `PI_VISION_PROXY_ALLOW_HOME` keeps working and now also accepts `0`/`false` to override a persisted `allow-home on`.
25
+ - The allowlist applies uniformly to auto-proxied prompt paths (images, video, audio), the `analyze_image` tool, and `/multimodal-proxy describe`. Allowlisted folders are canonicalized via `realpath` before comparison; relative entries are rejected and the list is capped at 100 folders. New tested helpers: `expandLeadingTilde`, `sanitizeAllowedFolders`, `pathAccessFromConfig`, and the `access` parameter on `isPathAllowed` / `readImageFileWithReason` / `readMediaFileWithReason`.
26
+ - **Hide-able status line** ([#16](https://github.com/pungggi/pi-multimodal-proxy/issues/16)) — new persisted `statusLine` setting. `/multimodal-proxy status off` hides the steady-state footer status (`multimodal-proxy: fallback → … | video: …`), `/multimodal-proxy status on` restores it, and the interactive config menu gets a matching toggle. The transient analysis progress spinner still shows while a call is in flight and clears when it finishes. New env override: `PI_VISION_PROXY_STATUS_LINE=on|off`.
27
+
28
+ ### Fixed
29
+
30
+ - Changing any setting via `/multimodal-proxy` refreshed the status line under the stale `vision-proxy` key with an outdated text format, leaving a second, never-updated status entry alongside the real one. The refresh now uses the same `multimodal-proxy` key and steady-state text (including the video model and registry fallback) as session start.
31
+ - The interactive `/multimodal-proxy` summary now lists every active env override (previously only shown when mode/model/context was set, hiding overrides like `tool`, `cacheSize`, or `statusLine`), and an invalid env value (e.g. `PI_VISION_PROXY_STATUS_LINE=hidden`) no longer locks the matching subcommand without actually overriding anything.
32
+
33
+ ## [1.9.0] - 2026-07-07
34
+
35
+ ### Added
36
+
37
+ - **Pre-consented providers** ([#14](https://github.com/pungggi/pi-multimodal-proxy/issues/14)) — a persisted `allowedProviders` list lets you consent to data egress for chosen providers once, instead of once per session. Providers on the list skip the first-use consent prompt everywhere (auto-proxy, video/audio, the `analyze_image` tool, and `/multimodal-proxy describe`). Manage it with `/multimodal-proxy allowed-providers add|remove <provider>|clear` (or the interactive config menu), or grant-and-persist in one step with `/multimodal-proxy consent always`. The list lives in `~/.pi/agent/multimodal-proxy.json` next to the other persisted settings and is kept out of session-entry configs so per-session config changes can never clobber it.
38
+ - **New env var**: `PI_VISION_PROXY_ALLOWED_PROVIDERS` — comma-separated provider ids, overriding the persisted list (a defined-but-empty value disables the list for that shell/project, handy for sensitive repositories).
39
+ - Safety semantics: an explicit in-session `/multimodal-proxy consent no` always beats the pre-consent list, and additionally removes the provider from the persisted list so the refusal sticks across sessions. The list only ever matches a specific provider — it is never a blanket grant. Provider ids are validated and canonicalized (`x-ai` → `xai`) on every boundary (file, env, commands).
40
+ - New tested helpers in `internal.ts`: `parseProviderList` (comma/whitespace splitting, canonicalization, dedup) and `consentState` (distinguishes "revoked" from "no verdict" so pre-consent can't override a refusal); `hasConsent` gained an optional `allowedProviders` parameter.
41
+
42
+ ## [1.8.0] - 2026-07-04
43
+
44
+ ### Added
45
+
46
+ - **Compaction survival** — media knowledge now survives context compaction. Previously, compaction summarized away the user messages carrying image blocks (and the injected video fences), so the `context` handler had nothing left to annotate and the agent lost all knowledge of earlier images/videos — even though the description entries were still persisted in the session. Now, when the active branch contains a compaction entry, the proxy detects which persisted image/video descriptions are no longer visible in context and re-injects them as a **post-compaction recall digest**: truncated description fences keyed by the same stable `image="..."` ids that `analyze_image` recall accepts, placed directly after the compaction summary.
47
+ - **Compaction-trigger awareness** (Pi 0.79.10) a `session_compact` handler records the compaction's `reason`/`willRetry` metadata. During the **overflow-recovery** window (context hit the hard limit and the turn is retried), the digest switches to lean budgets (200/240 chars per image/video instead of 600/800) so re-injection doesn't contribute to a second overflow; `turn_end` closes the window so later turns get normal budgets again. On older Pi runtimes the fields are absent and the digest simply uses its normal budgets.
48
+ - New helpers in `internal.ts`, all covered by unit tests: `findVideoDescriptions` (latest persisted entry per hash), `truncateForDigest` (word-boundary truncation with `… [truncated]` marker), and `buildCompactionDigest` (caps at the 12 most recent images / 4 most recent videos, restates the UNTRUSTED warning, and mentions `analyze_image` recall only when the tool is enabled).
49
+ - **`#` image-recall autocomplete** (Pi ≥ 0.79.1) — typing `#` at a token boundary in the interactive editor now suggests images seen earlier in the session (newest first, fuzzy-matched on filename, id, and description as you type). Picking one inserts the image's stable `image="<hash>"` recall id into the prompt, so *"zoom into `#`⇥"* works without copying ids out of fences. Implemented as a stacked autocomplete provider via `ctx.ui.addAutocompleteProvider`: it falls through to the built-in provider when the token matches no image, is a no-op in RPC/print modes and on older Pi versions without the API, and suggests nothing when the proxy is `off`. New tested helpers: `extractRecallToken`, `collectRecallCandidates`, `buildRecallItems` (max 8 items), `parseRecallItemValue`, `applyRecallCompletion`.
50
+
51
+ ### Changed
52
+
53
+ - **Default vision model bumped to Claude Sonnet 5** (`anthropic/claude-sonnet-5`, in Pi catalogs since 0.80.3), with default-tracking for implicit configs: explicit model choices are now persisted with a `modelExplicit` flag (`/multimodal-proxy model` and `pick`), and only *implicit* model values participate in substitution. An implicit legacy baked-in default (`claude-sonnet-4-5`, which full-config persistence wrote into every config on any settings change) is upgraded to the current default when the registry has it, and the current default falls back to `claude-sonnet-4-5` on older Pi catalogs. Explicit choices (via the flag or `PI_VISION_PROXY_MODEL`) are never rewritten — a missing explicit model still surfaces as "Model not found". The registry-resolved model is applied consistently at image analysis, the `analyze_image` tool, `/multimodal-proxy describe`, the status line, and the interactive config menu.
54
+ - The `context` handler no longer returns early when the active model supports images natively the post-compaction digest is injected whenever the proxy is not `off` and orphaned descriptions exist, since natively-visioned models also lose compacted-away images. Image-block stripping behavior is unchanged.
55
+
56
+ ## [1.7.0] - 2026-06-20
57
+
58
+ ### Added
59
+
60
+ - **Session image recall** — the agent can now re-query an image it saw earlier in the session without a re-attachment or file path. Every `<vision_proxy_description>`, `<vision_proxy_analysis>`, and `<vision_proxy_joint_description>` block already carries an `image="..."` id; passing that id back to `analyze_image` (or `/multimodal-proxy describe`) recalls the original image bytes and re-runs the targeted question or crop against it (e.g. *"zoom into that screenshot from before"*).
61
+ - Image bytes are retained **in process memory only** — never written to the session log or disk — in a byte-bounded LRU store (`PI_VISION_PROXY_IMAGE_RECALL_BYTES`, default 64 MB, oldest-first eviction).
62
+ - **New env var**: `PI_VISION_PROXY_IMAGE_RECALL_BYTES`.
63
+ - **Persistent recall reminder** when the proxy rewrites earlier images into descriptions, it now restates once per turn (as trusted text, outside the untrusted fence) that those images can be re-queried by id, so the affordance is visible even on turns where no new image was attached.
64
+ - **Live progress indicator** — slow image and video/audio analysis now animate a spinner with elapsed seconds on the status line (`multimodal-proxy ⠙ Analyzing image 2/4… (3s)`), restoring the steady-state status when the call completes. No-ops without a UI.
65
+ - Unit tests for `parseRecallRef` (bare hash, `sha256:` prefix, `#crop` suffix, case normalization, path rejection), the recall store (round-trip, dedup, byte-budget eviction, oversized-single-image retention, recency bumping), `spinnerFrame`, `formatProgressStatus`, and `RECALL_HINT`.
66
+
67
+ ### Changed
68
+
69
+ - `analyze_image` now accepts a recall handle (the fence `image="..."` id) as an image reference in addition to a file path; the previous hard rejection of `sha256:` references is removed. The tool schema, tool description, and the injected Vision Proxy system-prompt section document the recall handle.
70
+
71
+ ## [1.4.0-beta.1] - 2026-05-03
72
+
73
+ ### Added
74
+
75
+ - **`analyze_image` tool** agent-facing tool for targeted re-querying of images with multi-form crop support (FR-1.x). Disabled by default during beta; enable with `/vision-proxy tool on`.
76
+ - **Three crop forms**: `region` (named areas like `top-right`, `center`), `normalized` (0.0–1.0 fractional coordinates), and `pixels` (absolute pixel coordinates). All resolve to pixel rectangles with clamping and zero-area validation.
77
+ - **Image dimension extraction** via `image-size` package. Dimensions and filenames stored in an in-memory `_imageMeta` map populated on first image ingestion.
78
+ - **Enhanced fence tags**: `<vision_proxy_description>` now carries `image`, `width`, `height`, `filename`, and `crop_origin` attributes. New `<vision_proxy_analysis>` fence for tool results with optional `grounding_format` attribute.
79
+ - **LRU result cache** for `analyze_image` calls, keyed by (image hashes, crop signature, question hash, model).
80
+ - **Grounding format registry** with curated Tier 1 defaults (Qwen, Molmo, DeepSeek, InternVL, Gemini). Grounding instructions appended to system prompt per model's native format.
81
+ - **New configuration**: `/vision-proxy tool on|off`, `max-images-per-call <n>`, `max-batch <n>`, `cache-size <n>`.
82
+ - **New env vars**: `PI_VISION_PROXY_TOOL`, `PI_VISION_PROXY_MAX_IMAGES_PER_CALL`, `PI_VISION_PROXY_MAX_BATCH`, `PI_VISION_PROXY_CACHE_SIZE`, `PI_VISION_PROXY_PHASH_THRESHOLD`.
83
+ - **Security**: `fenceUntrusted` now neutralizes all three fence tag types (`description`, `analysis`, `joint_description`).
84
+ - **`readImageFileWithReason`** now returns the file's basename in the `filename` field.
85
+ - **Telemetry**: `vision_proxy.tool_call` session entries with crop form, latency, cache hit status.
86
+ - 112 unit tests covering crop resolution, LRU cache, dimension extraction, fence building, grounding lookups, config backwards compatibility, and all new env var parsing.
87
+
88
+ ### Changed
89
+
90
+ - `VisionConfig` extended with `tool`, `maxImagesPerCall`, `maxBatch`, `cacheSize`, `pHashSimilarityThreshold`, `groundingModels` fields. Backwards compatible — 1.3.0 config files load unchanged with sensible defaults.
91
+ - Version bumped to `1.4.0-beta.1`.
92
+ - Added `image-size` as a runtime dependency.
93
+
94
+ ## [1.3.0] - 2026-05-01
95
+
96
+ ### Added
97
+
98
+ - Two-step model picker (`/vision-proxy pick`): provider first, then model. Replaces the single flat list of 400+ models.
99
+ - Current provider is shown first with a ★ marker and pre-selected — picker opens directly on the model list, no need to re-select the same provider every time.
100
+ - `← Change provider` option inside the model list to switch providers without restarting the picker.
101
+ - `🔍 Type to filter models…` option for providers with more than 8 models. Uses fuzzy character-order matching (e.g. `cs4` matches `Claude Sonnet 4.5`). Single matches are auto-selected.
102
+ - `fuzzyMatches()` helper exported from `internal.ts` with full test coverage.
103
+
104
+ ### Changed
105
+
106
+ - Duplicated picker code between `/vision-proxy pick` and the interactive `Model:` row consolidated into a single `pickVisionModel()` function.
107
+
108
+ ## [1.2.0] - 2026-05-01
109
+
110
+ ### Added
111
+
112
+ - `/vision-proxy pick` sub-command. Lists vision-capable models from the registry with friendly names and provider tags via `ctx.ui.select`. Avoids typing canonical ids like `accounts/fireworks/models/kimi-k2p6`.
113
+ - Interactive `Model:` row in `/vision-proxy` config now opens the same vision-only picker (was raw text input).
114
+ - `friendlyModelLabel(config, registry)` helper. Status line and notifies now display `Kimi K2.6 [fireworks]` instead of `fireworks/accounts/fireworks/models/kimi-k2p6` when the registry knows the model.
115
+
116
+ ### Changed
117
+
118
+ - "Model not found" error now points to `/vision-proxy pick` instead of `/vision-proxy model`.
119
+
120
+ ## [1.1.0] - 2026-05-01
121
+
122
+ ### Changed
123
+
124
+ - Settings (mode, model, context) now persist across sessions to `~/.pi/agent/vision-proxy.json`. Previously settings were stored only in session entries and lost when starting a new session. Config precedence (highest → lowest): environment variables → session entries → persistent file → defaults.
125
+
126
+ ### Added
127
+
128
+ - `readPersistentFile()` / `writePersistentFile()` helpers for file-based config storage.
129
+ - `fileConfig` parameter on `resolveConfig()` to layer persisted file config between defaults and session entries.
130
+ - Tests for persistent file round-trip and layered config resolution.