claude-code-cache-fix 3.2.1 → 3.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.ko.md +32 -0
- package/README.md +55 -1
- package/package.json +6 -1
- package/proxy/extensions/image-strip.mjs +561 -39
- package/proxy/extensions.json +1 -0
- package/proxy/image-resize.mjs +133 -0
package/README.ko.md
CHANGED
|
@@ -254,6 +254,38 @@ export CACHE_FIX_IMAGE_KEEP_LAST=3
|
|
|
254
254
|
|
|
255
255
|
최근 3개 사용자 메시지의 이미지를 유지하고 이전 것은 텍스트 자리 표시자로 대체합니다. `tool_result` 블록만 대상이며, 사용자가 직접 붙여넣은 이미지는 영향받지 않습니다.
|
|
256
256
|
|
|
257
|
+
### 이미지 가드 파이프라인 (v3.3.0)
|
|
258
|
+
|
|
259
|
+
Anthropic의 실제 이미지 규칙을 그대로 반영하는 조건부 파이프라인입니다. 단일 환경 변수로 명시적 활성화:
|
|
260
|
+
|
|
261
|
+
```bash
|
|
262
|
+
export CACHE_FIX_IMAGE_GUARD=1
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
활성화 시 프록시는 다음을 실행합니다:
|
|
266
|
+
|
|
267
|
+
| 패스 | 트리거 | 동작 |
|
|
268
|
+
|------|--------|------|
|
|
269
|
+
| **Pass 0** (레거시) | `CACHE_FIX_IMAGE_KEEP_LAST=N` 설정 | 가장 최근 N개 이외 사용자 메시지의 tool_result 이미지 제거 |
|
|
270
|
+
| **Pass 3** | `CACHE_FIX_IMAGE_PRESERVE_DETAIL=1` AND 긴 변 > 모델 네이티브 캡 | `sharp`를 통해 네이티브 캡(Opus 4.7은 2576px, 그 외는 1568px)으로 Lanczos 리사이즈, 종횡비와 미디어 타입 보존 |
|
|
271
|
+
| **Pass 1** | 긴 변 > 활성 거부 캡 | 제거 후 forensic 자리 표시자로 대체. 활성 캡 = `MAX_DIM` 설정 시 그 값, 아니면 2000px (개수 > 20일 때) 또는 8000px (개수 ≤ 20) |
|
|
272
|
+
| **Pass 2** | 요청 본문이 `CACHE_FIX_IMAGE_REQUEST_SIZE_MAX` (기본 30 MB) 초과 | 예산 이하가 될 때까지 가장 오래된 이미지부터 제거 |
|
|
273
|
+
| **개수 캡** | 잔여 이미지 개수 > `CACHE_FIX_IMAGE_COUNT_MAX` (기본 100) | 캡까지 가장 오래된 이미지 제거 |
|
|
274
|
+
|
|
275
|
+
실행 순서: **Pass 0 → Pass 3 → Pass 1 → Pass 2 → 개수 캡**. 각 패스는 독립적입니다 — Pass 1은 절대 리사이즈하지 않으며, Pass 3는 절대 제거하지 않습니다.
|
|
276
|
+
|
|
277
|
+
#### 선택적 `sharp` 의존성
|
|
278
|
+
|
|
279
|
+
Pass 3는 Lanczos 리사이즈를 위해 [sharp](https://www.npmjs.com/package/sharp)가 필요합니다. **선택적 peer dependency**로 선언되어 있으며, Pass 3를 사용하려면 별도로 설치하십시오:
|
|
280
|
+
|
|
281
|
+
```bash
|
|
282
|
+
npm install sharp
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
`sharp`가 없는 경우 Pass 3는 깨끗하게 건너뛰며 (telemetry에 `library_missing: true`), Pass 1 + Pass 2 + 개수 캡은 정상 실행됩니다.
|
|
286
|
+
|
|
287
|
+
전체 우선순위 매트릭스(레거시 + 신규 환경 변수의 모든 조합) 및 튜닝 가능한 항목은 [README.md](README.md#image-guard-pipeline-v330)를 참조하십시오.
|
|
288
|
+
|
|
257
289
|
## 시스템 프롬프트 재작성 (프리로드 모드, 선택)
|
|
258
290
|
|
|
259
291
|
인터셉터가 Claude Code의 `# Output efficiency` 시스템 프롬프트 섹션을 재작성할 수 있습니다. 기본 비활성화입니다. `CACHE_FIX_OUTPUT_EFFICIENCY_REPLACEMENT`로 활성화하십시오. 세 가지 알려진 프롬프트 변형과 사용법은 [docs/output-efficiency-prompts.md](docs/output-efficiency-prompts.md)를 참조하십시오.
|
package/README.md
CHANGED
|
@@ -334,7 +334,7 @@ export CACHE_FIX_IMAGE_KEEP_LAST=3
|
|
|
334
334
|
|
|
335
335
|
Keeps images in the last 3 user messages, replaces older ones with a text placeholder. Only targets `tool_result` blocks — user-pasted images are never touched.
|
|
336
336
|
|
|
337
|
-
### Oversized-image guard
|
|
337
|
+
### Oversized-image guard (legacy, v3.2.1)
|
|
338
338
|
|
|
339
339
|
```bash
|
|
340
340
|
export CACHE_FIX_IMAGE_MAX_DIM=2000
|
|
@@ -355,6 +355,60 @@ The two compose: with both set, `KEEP_LAST` runs first (drops the count), then `
|
|
|
355
355
|
|
|
356
356
|
Pure-JS PNG and JPEG header parsing — no native deps. Other formats (GIF, WebP, AVIF, BMP) pass through unchanged regardless of dimension. Fail-open: images whose dimensions can't be parsed (truncated header, unsupported format) are kept rather than stripped — better to send a request that might error than to strip a valid image we just couldn't measure.
|
|
357
357
|
|
|
358
|
+
### Image-guard pipeline (v3.3.0)
|
|
359
|
+
|
|
360
|
+
A conditional pipeline that mirrors Anthropic's actual rules. Strictly opt-in via a single env var:
|
|
361
|
+
|
|
362
|
+
```bash
|
|
363
|
+
export CACHE_FIX_IMAGE_GUARD=1
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
When enabled, the proxy runs:
|
|
367
|
+
|
|
368
|
+
| Pass | Trigger | Action |
|
|
369
|
+
|------|---------|--------|
|
|
370
|
+
| **Pass 0** (legacy) | `CACHE_FIX_IMAGE_KEEP_LAST=N` set | Strip tool_result images from user messages older than N most recent |
|
|
371
|
+
| **Pass 3** | `CACHE_FIX_IMAGE_PRESERVE_DETAIL=1` AND image long edge > model native cap | Lanczos resize via `sharp` to native cap (2576 px for Opus 4.7, 1568 px otherwise), preserve aspect ratio and media type |
|
|
372
|
+
| **Pass 1** | image long edge > active rejection cap | Strip and replace with forensic placeholder. Active cap = `MAX_DIM` if set, else 2000 px (when count > 20) or 8000 px (count ≤ 20) |
|
|
373
|
+
| **Pass 2** | request body exceeds `CACHE_FIX_IMAGE_REQUEST_SIZE_MAX` (default 30 MB) | Drop oldest images until under budget |
|
|
374
|
+
| **Count cap** | surviving image count > `CACHE_FIX_IMAGE_COUNT_MAX` (default 100) | Drop oldest images down to the cap |
|
|
375
|
+
|
|
376
|
+
Execution order: **Pass 0 → Pass 3 → Pass 1 → Pass 2 → count cap**. Each pass is independent — Pass 1 never resizes; Pass 3 never strips.
|
|
377
|
+
|
|
378
|
+
#### Optional `sharp` dependency
|
|
379
|
+
|
|
380
|
+
Pass 3 requires [sharp](https://www.npmjs.com/package/sharp) for Lanczos resize. It's declared as an **optional peer dependency** — install separately if you want Pass 3:
|
|
381
|
+
|
|
382
|
+
```bash
|
|
383
|
+
npm install sharp
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
If `sharp` is missing, Pass 3 skips cleanly (telemetry records `library_missing: true`); Pass 1 + Pass 2 + the count cap still run.
|
|
387
|
+
|
|
388
|
+
#### Precedence matrix
|
|
389
|
+
|
|
390
|
+
| Env var combination | Behavior |
|
|
391
|
+
|---|---|
|
|
392
|
+
| Nothing set | No image processing (back-compat default; the extension short-circuits). |
|
|
393
|
+
| `KEEP_LAST=N` only | Existing v3.2.1: count cap on tool_result images in user messages, runs first. No pipeline. |
|
|
394
|
+
| `MAX_DIM=N` only | Existing v3.2.1: hard size cap, strip-only. No pipeline. |
|
|
395
|
+
| `KEEP_LAST=N` + `MAX_DIM=N` | Existing v3.2.1 composition: `KEEP_LAST` runs first (drops count), then `MAX_DIM` runs on survivors (caps size). No pipeline, no Pass 2, no Pass 3. |
|
|
396
|
+
| `IMAGE_GUARD=1` | New pipeline: Pass 1 (conditional cap) + Pass 2 (request-size guard) + image-count cap. |
|
|
397
|
+
| `IMAGE_GUARD=1` + `MAX_DIM=N` | `MAX_DIM` overrides Pass 1's conditional cap (acts as the cap value); Pass 2 still runs. |
|
|
398
|
+
| `IMAGE_GUARD=1` + `PRESERVE_DETAIL=1` | Adds Pass 3 (Lanczos resize via `sharp`). When `sharp` unavailable, falls back to strip behavior. |
|
|
399
|
+
| `IMAGE_GUARD=1` + `KEEP_LAST=N` | `KEEP_LAST` runs first as count cap (Pass 0); pipeline runs on remainder. |
|
|
400
|
+
| `IMAGE_GUARD=1` + `KEEP_LAST=N` + `MAX_DIM=N` | Three-way: `KEEP_LAST` runs first; pipeline runs on remainder, but `MAX_DIM` overrides Pass 1's conditional cap; Pass 2 still runs. |
|
|
401
|
+
| `PRESERVE_DETAIL=1` without `IMAGE_GUARD=1` | Logs warning, treats as no-op. `PRESERVE_DETAIL` is meaningless without the pipeline running. |
|
|
402
|
+
|
|
403
|
+
#### Tunables
|
|
404
|
+
|
|
405
|
+
| Env var | Default | Purpose |
|
|
406
|
+
|---------|---------|---------|
|
|
407
|
+
| `CACHE_FIX_IMAGE_GUARD` | unset | Top-level pipeline gate (`=1` enables). |
|
|
408
|
+
| `CACHE_FIX_IMAGE_PRESERVE_DETAIL` | unset | Enable Pass 3 Lanczos resize via `sharp`. |
|
|
409
|
+
| `CACHE_FIX_IMAGE_REQUEST_SIZE_MAX` | 31457280 (30 MB) | Pass 2 byte budget. 2 MB headroom from Anthropic's 32 MB ceiling. |
|
|
410
|
+
| `CACHE_FIX_IMAGE_COUNT_MAX` | 100 | Hard image-count cap. Set to 600 for legacy Claude 1/2.x/Instant if needed. |
|
|
411
|
+
|
|
358
412
|
## System prompt rewrite (preload mode, optional)
|
|
359
413
|
|
|
360
414
|
The interceptor can rewrite Claude Code's `# Output efficiency` system-prompt section. Disabled by default. Enable with `CACHE_FIX_OUTPUT_EFFICIENCY_REPLACEMENT`. See [docs/output-efficiency-prompts.md](docs/output-efficiency-prompts.md) for the three known prompt variants and usage instructions.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "claude-code-cache-fix",
|
|
3
|
-
"version": "3.
|
|
3
|
+
"version": "3.3.0",
|
|
4
4
|
"description": "Cache optimization proxy and interceptor for Claude Code. Fixes prompt cache bugs, stabilizes prefix, reduces quota burn.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"exports": "./preload.mjs",
|
|
@@ -27,6 +27,11 @@
|
|
|
27
27
|
"dependencies": {
|
|
28
28
|
"hpagent": "^1.2.0"
|
|
29
29
|
},
|
|
30
|
+
"peerDependenciesMeta": {
|
|
31
|
+
"sharp": {
|
|
32
|
+
"optional": true
|
|
33
|
+
}
|
|
34
|
+
},
|
|
30
35
|
"keywords": [
|
|
31
36
|
"claude-code",
|
|
32
37
|
"claude",
|
|
@@ -1,5 +1,26 @@
|
|
|
1
|
+
// image-strip extension — back-compat for v3.2.1 KEEP_LAST/MAX_DIM behavior
|
|
2
|
+
// PLUS the v3.3.0 image-guard pipeline.
|
|
3
|
+
//
|
|
4
|
+
// Activation pattern: `enabled: true` in extensions.json (the extension is
|
|
5
|
+
// always loaded), runtime gates per-feature via env vars. Three independent
|
|
6
|
+
// trigger surfaces:
|
|
7
|
+
//
|
|
8
|
+
// - CACHE_FIX_IMAGE_KEEP_LAST=N → legacy Pass 0 (count cap on
|
|
9
|
+
// tool_result images in user msgs)
|
|
10
|
+
// - CACHE_FIX_IMAGE_MAX_DIM=N → legacy Pass 1 strip-only (back-compat)
|
|
11
|
+
// - CACHE_FIX_IMAGE_GUARD=1 → v3.3.0 pipeline (Pass 1 + 2 + count cap)
|
|
12
|
+
// - CACHE_FIX_IMAGE_GUARD=1 +
|
|
13
|
+
// CACHE_FIX_IMAGE_PRESERVE_DETAIL=1 → adds Pass 3 (Lanczos resize via sharp)
|
|
14
|
+
//
|
|
15
|
+
// Execution order: Pass 0 → Pass 3 → Pass 1 → Pass 2 → image-count cap.
|
|
16
|
+
//
|
|
17
|
+
// See `docs/directives/proxy-image-guard-pipeline.md` for full spec, including
|
|
18
|
+
// the precedence matrix and per-pass trigger/action contracts.
|
|
19
|
+
|
|
1
20
|
import { parseImageDimensions } from "../image-dimensions.mjs";
|
|
21
|
+
import { resizeImageToCap } from "../image-resize.mjs";
|
|
2
22
|
|
|
23
|
+
// --- Legacy v3.2.1 constants (back-compat) ---
|
|
3
24
|
const KEEP_LAST = parseInt(process.env.CACHE_FIX_IMAGE_KEEP_LAST || "0", 10);
|
|
4
25
|
const MAX_DIM = parseInt(process.env.CACHE_FIX_IMAGE_MAX_DIM || "0", 10);
|
|
5
26
|
const PLACEHOLDER = "[image stripped from history — file may still be on disk]";
|
|
@@ -8,6 +29,34 @@ function oversizedPlaceholder(maxDim, w, h) {
|
|
|
8
29
|
return `[image stripped — exceeded ${maxDim}px max dimension (was ${w}x${h}px)]`;
|
|
9
30
|
}
|
|
10
31
|
|
|
32
|
+
// --- v3.3.0 pipeline env helpers (read at call time, not at module load,
|
|
33
|
+
// so per-test isolation works without re-importing the module) ---
|
|
34
|
+
function isImageGuardEnabled() {
|
|
35
|
+
return process.env.CACHE_FIX_IMAGE_GUARD === "1";
|
|
36
|
+
}
|
|
37
|
+
function isPreserveDetailEnabled() {
|
|
38
|
+
return process.env.CACHE_FIX_IMAGE_PRESERVE_DETAIL === "1";
|
|
39
|
+
}
|
|
40
|
+
function getMaxDim() {
|
|
41
|
+
return parseInt(process.env.CACHE_FIX_IMAGE_MAX_DIM || "0", 10);
|
|
42
|
+
}
|
|
43
|
+
function getKeepLast() {
|
|
44
|
+
return parseInt(process.env.CACHE_FIX_IMAGE_KEEP_LAST || "0", 10);
|
|
45
|
+
}
|
|
46
|
+
function getRequestSizeMax() {
|
|
47
|
+
// Default 30 MB (31457280 bytes). 2 MB headroom from Anthropic's 32 MB body limit.
|
|
48
|
+
const v = parseInt(process.env.CACHE_FIX_IMAGE_REQUEST_SIZE_MAX || "31457280", 10);
|
|
49
|
+
return v > 0 ? v : 31457280;
|
|
50
|
+
}
|
|
51
|
+
function getImageCountMax() {
|
|
52
|
+
// Default 100 — single cap covering the only model family in active CC use.
|
|
53
|
+
// Users on legacy Claude 1/2.x/Instant who genuinely need 600 can override.
|
|
54
|
+
const v = parseInt(process.env.CACHE_FIX_IMAGE_COUNT_MAX || "100", 10);
|
|
55
|
+
return v > 0 ? v : 100;
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
// --- Legacy Pass 0: KEEP_LAST tool_result image strip ---
|
|
59
|
+
// (Unchanged from v3.2.1 — only formatting tightened.)
|
|
11
60
|
function stripOldToolResultImages(messages, keepLast) {
|
|
12
61
|
if (!keepLast || keepLast <= 0 || !Array.isArray(messages)) {
|
|
13
62
|
return { messages, stats: null };
|
|
@@ -65,15 +114,9 @@ function stripOldToolResultImages(messages, keepLast) {
|
|
|
65
114
|
return { messages: strippedCount > 0 ? result : messages, stats };
|
|
66
115
|
}
|
|
67
116
|
|
|
68
|
-
//
|
|
69
|
-
//
|
|
70
|
-
//
|
|
71
|
-
// exceeds maxDim. Fail-open: images we can't measure (unsupported format,
|
|
72
|
-
// truncated header) are kept rather than stripped.
|
|
73
|
-
//
|
|
74
|
-
// Stripping by oversize prevents the Anthropic API error:
|
|
75
|
-
// "An image in the conversation exceeds the dimension limit for many-image
|
|
76
|
-
// requests (2000px). Start a new session with fewer images."
|
|
117
|
+
// --- Legacy Pass 1 (strip-only by max dim) ---
|
|
118
|
+
// (Unchanged from v3.2.1.) The new pipeline's Pass 1 is a separate function
|
|
119
|
+
// (`runPass1RejectionCapStrip`) that uses the conditional 2000/8000 logic.
|
|
77
120
|
function stripOversizedImages(messages, maxDim) {
|
|
78
121
|
if (!maxDim || maxDim <= 0 || !Array.isArray(messages)) {
|
|
79
122
|
return { messages, stats: null };
|
|
@@ -87,7 +130,7 @@ function stripOversizedImages(messages, maxDim) {
|
|
|
87
130
|
const src = item.source;
|
|
88
131
|
if (!src || !src.data || !src.media_type) return item;
|
|
89
132
|
const dims = parseImageDimensions(src.media_type, src.data);
|
|
90
|
-
if (!dims) return item;
|
|
133
|
+
if (!dims) return item;
|
|
91
134
|
if (dims.width <= maxDim && dims.height <= maxDim) return item;
|
|
92
135
|
strippedCount++;
|
|
93
136
|
strippedBytes += src.data.length;
|
|
@@ -98,7 +141,6 @@ function stripOversizedImages(messages, maxDim) {
|
|
|
98
141
|
if (!Array.isArray(msg.content)) return msg;
|
|
99
142
|
let mutated = false;
|
|
100
143
|
const newContent = msg.content.map((block) => {
|
|
101
|
-
// Direct image block on a user message
|
|
102
144
|
if (block && block.type === "image") {
|
|
103
145
|
const replaced = maybeStrip(block);
|
|
104
146
|
if (replaced !== block) {
|
|
@@ -107,7 +149,6 @@ function stripOversizedImages(messages, maxDim) {
|
|
|
107
149
|
}
|
|
108
150
|
return block;
|
|
109
151
|
}
|
|
110
|
-
// Image nested inside a tool_result.content array
|
|
111
152
|
if (block && block.type === "tool_result" && Array.isArray(block.content)) {
|
|
112
153
|
let toolMutated = false;
|
|
113
154
|
const newToolContent = block.content.map((item) => {
|
|
@@ -132,49 +173,530 @@ function stripOversizedImages(messages, maxDim) {
|
|
|
132
173
|
return { messages: strippedCount > 0 ? result : messages, stats };
|
|
133
174
|
}
|
|
134
175
|
|
|
135
|
-
|
|
176
|
+
// =============================================================================
|
|
177
|
+
// v3.3.0 pipeline
|
|
178
|
+
// =============================================================================
|
|
179
|
+
|
|
180
|
+
// --- Pure helpers (exported for tests) ---
|
|
181
|
+
|
|
182
|
+
// Pass 1 cap selector. `maxDimOverride > 0` always wins over the conditional
|
|
183
|
+
// rejection cap. Otherwise: count > 20 → 2000 px, else 8000 px.
|
|
184
|
+
function pickPass1Cap(imageCount, maxDimOverride) {
|
|
185
|
+
if (maxDimOverride && maxDimOverride > 0) return maxDimOverride;
|
|
186
|
+
return imageCount > 20 ? 2000 : 8000;
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
// Pass 3 native-cap selector. Only Opus 4.7 has the higher cap.
|
|
190
|
+
function pickPass3NativeCap(modelString) {
|
|
191
|
+
if (typeof modelString === "string" && modelString.startsWith("claude-opus-4-7")) {
|
|
192
|
+
return 2576;
|
|
193
|
+
}
|
|
194
|
+
return 1568;
|
|
195
|
+
}
|
|
196
|
+
|
|
197
|
+
// Anthropic's documented per-image token formula: `width * height / 750`,
|
|
198
|
+
// capped at the model's native token cap. Diagnostic only (no enforcement).
|
|
199
|
+
function estimateImageTokens(width, height, modelTokenCap) {
|
|
200
|
+
if (!width || !height) return 0;
|
|
201
|
+
const raw = Math.ceil((width * height) / 750);
|
|
202
|
+
if (modelTokenCap && raw > modelTokenCap) return modelTokenCap;
|
|
203
|
+
return raw;
|
|
204
|
+
}
|
|
205
|
+
|
|
206
|
+
function nativeTokenCap(modelString) {
|
|
207
|
+
if (typeof modelString === "string" && modelString.startsWith("claude-opus-4-7")) {
|
|
208
|
+
return 4784;
|
|
209
|
+
}
|
|
210
|
+
return 1568;
|
|
211
|
+
}
|
|
212
|
+
|
|
213
|
+
// Walker: enumerate every image in `body.messages`, both user-msg direct content
|
|
214
|
+
// and tool_result.content. Returns a list of `{ msgIdx, blockIdx, itemIdx | null,
|
|
215
|
+
// item }` references. `itemIdx === null` means the image is a direct user-msg
|
|
216
|
+
// content block (not nested in tool_result).
|
|
217
|
+
function walkImages(messages) {
|
|
218
|
+
const out = [];
|
|
219
|
+
if (!Array.isArray(messages)) return out;
|
|
220
|
+
for (let m = 0; m < messages.length; m++) {
|
|
221
|
+
const msg = messages[m];
|
|
222
|
+
if (!Array.isArray(msg.content)) continue;
|
|
223
|
+
for (let b = 0; b < msg.content.length; b++) {
|
|
224
|
+
const block = msg.content[b];
|
|
225
|
+
if (!block) continue;
|
|
226
|
+
if (block.type === "image") {
|
|
227
|
+
out.push({ msgIdx: m, blockIdx: b, itemIdx: null, item: block });
|
|
228
|
+
} else if (block.type === "tool_result" && Array.isArray(block.content)) {
|
|
229
|
+
for (let i = 0; i < block.content.length; i++) {
|
|
230
|
+
const item = block.content[i];
|
|
231
|
+
if (item && item.type === "image") {
|
|
232
|
+
out.push({ msgIdx: m, blockIdx: b, itemIdx: i, item });
|
|
233
|
+
}
|
|
234
|
+
}
|
|
235
|
+
}
|
|
236
|
+
}
|
|
237
|
+
}
|
|
238
|
+
return out;
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
// Mutate a single image at the given (msgIdx, blockIdx, itemIdx) reference,
|
|
242
|
+
// either replacing it with a placeholder text block (strip) or updating its
|
|
243
|
+
// source.data + dims (resize).
|
|
244
|
+
function replaceImageInPlace(messages, ref, replacement) {
|
|
245
|
+
const msg = messages[ref.msgIdx];
|
|
246
|
+
if (!msg || !Array.isArray(msg.content)) return;
|
|
247
|
+
const block = msg.content[ref.blockIdx];
|
|
248
|
+
if (!block) return;
|
|
249
|
+
if (ref.itemIdx === null) {
|
|
250
|
+
msg.content[ref.blockIdx] = replacement;
|
|
251
|
+
} else {
|
|
252
|
+
if (!Array.isArray(block.content)) return;
|
|
253
|
+
block.content[ref.itemIdx] = replacement;
|
|
254
|
+
}
|
|
255
|
+
}
|
|
256
|
+
|
|
257
|
+
// Pure walker for Pass 1: returns the list of image refs whose long edge
|
|
258
|
+
// exceeds `capPx`, alongside their measured dims. Unparseable images are not
|
|
259
|
+
// included (fail-open). Exposed for tests.
|
|
260
|
+
function walkImagesForPass1(messages, capPx) {
|
|
261
|
+
const refs = walkImages(messages);
|
|
262
|
+
const plan = [];
|
|
263
|
+
for (const ref of refs) {
|
|
264
|
+
const src = ref.item.source;
|
|
265
|
+
if (!src || !src.data || !src.media_type) continue;
|
|
266
|
+
const dims = parseImageDimensions(src.media_type, src.data);
|
|
267
|
+
if (!dims) {
|
|
268
|
+
plan.push({ ref, dims: null, action: "skip_unmeasurable" });
|
|
269
|
+
continue;
|
|
270
|
+
}
|
|
271
|
+
const longEdge = Math.max(dims.width, dims.height);
|
|
272
|
+
if (longEdge > capPx) {
|
|
273
|
+
plan.push({ ref, dims, action: "strip" });
|
|
274
|
+
}
|
|
275
|
+
}
|
|
276
|
+
return plan;
|
|
277
|
+
}
|
|
278
|
+
|
|
279
|
+
// Pure walker for Pass 3: returns refs whose long edge exceeds `nativeCapPx`,
|
|
280
|
+
// plus the dims/cap so the caller can do the actual resize.
|
|
281
|
+
function walkImagesForPass3(messages, nativeCapPx) {
|
|
282
|
+
const refs = walkImages(messages);
|
|
283
|
+
const plan = [];
|
|
284
|
+
for (const ref of refs) {
|
|
285
|
+
const src = ref.item.source;
|
|
286
|
+
if (!src || !src.data || !src.media_type) continue;
|
|
287
|
+
const dims = parseImageDimensions(src.media_type, src.data);
|
|
288
|
+
if (!dims) {
|
|
289
|
+
plan.push({ ref, dims: null, action: "skip_unmeasurable" });
|
|
290
|
+
continue;
|
|
291
|
+
}
|
|
292
|
+
const longEdge = Math.max(dims.width, dims.height);
|
|
293
|
+
if (longEdge > nativeCapPx) {
|
|
294
|
+
plan.push({ ref, dims, action: "resize", capPx: nativeCapPx });
|
|
295
|
+
}
|
|
296
|
+
}
|
|
297
|
+
return plan;
|
|
298
|
+
}
|
|
299
|
+
|
|
300
|
+
// Eviction order for Pass 2 / count cap: oldest first (low msgIdx wins),
|
|
301
|
+
// within a message tool_result images are preferred over direct images at the
|
|
302
|
+
// same age. Returns an ordered list of refs to drop.
|
|
303
|
+
function pickEvictionTargets(messages) {
|
|
304
|
+
const refs = walkImages(messages);
|
|
305
|
+
// Stable sort: by msgIdx ascending, then prefer tool_result (itemIdx !== null)
|
|
306
|
+
// over direct (itemIdx === null) at the same msgIdx.
|
|
307
|
+
refs.sort((a, b) => {
|
|
308
|
+
if (a.msgIdx !== b.msgIdx) return a.msgIdx - b.msgIdx;
|
|
309
|
+
const aTool = a.itemIdx !== null ? 0 : 1;
|
|
310
|
+
const bTool = b.itemIdx !== null ? 0 : 1;
|
|
311
|
+
if (aTool !== bTool) return aTool - bTool;
|
|
312
|
+
if (a.blockIdx !== b.blockIdx) return a.blockIdx - b.blockIdx;
|
|
313
|
+
return (a.itemIdx ?? 0) - (b.itemIdx ?? 0);
|
|
314
|
+
});
|
|
315
|
+
return refs;
|
|
316
|
+
}
|
|
317
|
+
|
|
318
|
+
// --- Stats initializer (matches the directive's telemetry surface verbatim) ---
|
|
319
|
+
function initStats() {
|
|
320
|
+
return {
|
|
321
|
+
total_images: 0,
|
|
322
|
+
count_axis_path: "few",
|
|
323
|
+
unsupported_format_count: 0,
|
|
324
|
+
dimension_probe_fail_count: 0,
|
|
325
|
+
resize_attempted: 0,
|
|
326
|
+
resize_succeeded: 0,
|
|
327
|
+
resize_failed: 0,
|
|
328
|
+
library_missing: false,
|
|
329
|
+
images_stripped_pass1: 0,
|
|
330
|
+
images_dropped_for_size: 0,
|
|
331
|
+
images_dropped_for_count_cap: 0,
|
|
332
|
+
|
|
333
|
+
request_bytes_before: 0,
|
|
334
|
+
request_bytes_after: 0,
|
|
335
|
+
request_bytes_headroom: 0,
|
|
336
|
+
image_bytes_total: 0,
|
|
337
|
+
image_bytes_dropped: 0,
|
|
338
|
+
|
|
339
|
+
estimated_image_tokens_total: 0,
|
|
340
|
+
};
|
|
341
|
+
}
|
|
342
|
+
|
|
343
|
+
// --- Pass 3 runtime: native-cap resize via sharp ---
|
|
344
|
+
async function runPass3NativeCapResize(reqCtx, stats) {
|
|
345
|
+
if (stats.library_missing) return;
|
|
346
|
+
|
|
347
|
+
const messages = reqCtx.body.messages;
|
|
348
|
+
const model = reqCtx.body.model;
|
|
349
|
+
const nativeCap = pickPass3NativeCap(model);
|
|
350
|
+
const tokenCap = nativeTokenCap(model);
|
|
351
|
+
|
|
352
|
+
const plan = walkImagesForPass3(messages, nativeCap);
|
|
353
|
+
for (const step of plan) {
|
|
354
|
+
if (step.action === "skip_unmeasurable") {
|
|
355
|
+
// Tracked separately via the unsupported/probe counters at the top-level
|
|
356
|
+
// walker (we double-count if we touch them here too — leave it to the
|
|
357
|
+
// central counter pass).
|
|
358
|
+
continue;
|
|
359
|
+
}
|
|
360
|
+
if (step.action !== "resize") continue;
|
|
361
|
+
|
|
362
|
+
const src = step.ref.item.source;
|
|
363
|
+
stats.resize_attempted++;
|
|
364
|
+
const result = await resizeImageToCap(src.data, src.media_type, step.capPx);
|
|
365
|
+
if (result.ok) {
|
|
366
|
+
stats.resize_succeeded++;
|
|
367
|
+
// Mutate in place: keep the same content block shape, swap data + record dims.
|
|
368
|
+
const newImage = {
|
|
369
|
+
...step.ref.item,
|
|
370
|
+
source: { ...src, data: result.base64 },
|
|
371
|
+
};
|
|
372
|
+
replaceImageInPlace(messages, step.ref, newImage);
|
|
373
|
+
const tokensBefore = estimateImageTokens(step.dims.width, step.dims.height, tokenCap);
|
|
374
|
+
const tokensAfter = estimateImageTokens(result.dims.width, result.dims.height, tokenCap);
|
|
375
|
+
stats.estimated_image_tokens_total += tokensAfter - tokensBefore;
|
|
376
|
+
} else if (result.reason === "library_missing") {
|
|
377
|
+
stats.library_missing = true;
|
|
378
|
+
// Sticky: stop attempting Pass 3 for the remainder of this request.
|
|
379
|
+
// (loadSharp() inside image-resize.mjs is also sticky for the process.)
|
|
380
|
+
return;
|
|
381
|
+
} else {
|
|
382
|
+
stats.resize_failed++;
|
|
383
|
+
// Leave image untouched; Pass 1 will evaluate it against its own cap.
|
|
384
|
+
}
|
|
385
|
+
}
|
|
386
|
+
}
|
|
387
|
+
|
|
388
|
+
// --- Pass 1 runtime: conditional rejection-cap strip ---
|
|
389
|
+
function runPass1RejectionCapStrip(reqCtx, stats, opts) {
|
|
390
|
+
const { maxDimOverride } = opts || {};
|
|
391
|
+
const messages = reqCtx.body.messages;
|
|
392
|
+
|
|
393
|
+
const refs = walkImages(messages);
|
|
394
|
+
const imageCount = refs.length;
|
|
395
|
+
stats.total_images = Math.max(stats.total_images, imageCount);
|
|
396
|
+
stats.count_axis_path = imageCount > 20 ? "many" : "few";
|
|
397
|
+
const cap = pickPass1Cap(imageCount, maxDimOverride);
|
|
398
|
+
|
|
399
|
+
for (const ref of refs) {
|
|
400
|
+
const src = ref.item.source;
|
|
401
|
+
if (!src || !src.data || !src.media_type) continue;
|
|
402
|
+
const dims = parseImageDimensions(src.media_type, src.data);
|
|
403
|
+
if (!dims) {
|
|
404
|
+
// Distinguish unsupported format from probe failure on a known type.
|
|
405
|
+
const mt = (src.media_type || "").toLowerCase();
|
|
406
|
+
if (mt === "image/png" || mt === "image/jpeg" || mt === "image/jpg") {
|
|
407
|
+
stats.dimension_probe_fail_count++;
|
|
408
|
+
} else {
|
|
409
|
+
stats.unsupported_format_count++;
|
|
410
|
+
}
|
|
411
|
+
continue;
|
|
412
|
+
}
|
|
413
|
+
const longEdge = Math.max(dims.width, dims.height);
|
|
414
|
+
if (longEdge > cap) {
|
|
415
|
+
replaceImageInPlace(messages, ref, {
|
|
416
|
+
type: "text",
|
|
417
|
+
text: oversizedPlaceholder(cap, dims.width, dims.height),
|
|
418
|
+
});
|
|
419
|
+
stats.images_stripped_pass1++;
|
|
420
|
+
}
|
|
421
|
+
}
|
|
422
|
+
}
|
|
423
|
+
|
|
424
|
+
// --- Pass 2 runtime: request-size guard ---
|
|
425
|
+
function runPass2RequestSizeGuard(reqCtx, stats) {
|
|
426
|
+
const budget = getRequestSizeMax();
|
|
427
|
+
const before = Buffer.byteLength(JSON.stringify(reqCtx.body));
|
|
428
|
+
if (stats.request_bytes_before === 0) stats.request_bytes_before = before;
|
|
429
|
+
|
|
430
|
+
if (before <= budget) {
|
|
431
|
+
stats.request_bytes_after = before;
|
|
432
|
+
stats.request_bytes_headroom = budget - before;
|
|
433
|
+
return;
|
|
434
|
+
}
|
|
435
|
+
|
|
436
|
+
// Build eviction queue once; drop one at a time, re-measuring after each drop.
|
|
437
|
+
const queue = pickEvictionTargets(reqCtx.body.messages);
|
|
438
|
+
let bytes = before;
|
|
439
|
+
for (const ref of queue) {
|
|
440
|
+
if (bytes <= budget) break;
|
|
441
|
+
const src = ref.item.source;
|
|
442
|
+
const droppedBytes = src && src.data ? src.data.length : 0;
|
|
443
|
+
replaceImageInPlace(reqCtx.body.messages, ref, {
|
|
444
|
+
type: "text",
|
|
445
|
+
text: "[image dropped to fit request-size budget]",
|
|
446
|
+
});
|
|
447
|
+
stats.images_dropped_for_size++;
|
|
448
|
+
stats.image_bytes_dropped += droppedBytes;
|
|
449
|
+
bytes = Buffer.byteLength(JSON.stringify(reqCtx.body));
|
|
450
|
+
}
|
|
451
|
+
stats.request_bytes_after = bytes;
|
|
452
|
+
stats.request_bytes_headroom = budget - bytes;
|
|
453
|
+
// If we exhausted the queue and bytes still exceed budget, the body is
|
|
454
|
+
// over-budget for non-image reasons; the request will fail upstream and we
|
|
455
|
+
// don't address that here. Telemetry already records the final bytes.
|
|
456
|
+
}
|
|
457
|
+
|
|
458
|
+
// --- Hard image-count cap ---
|
|
459
|
+
function runImageCountCap(reqCtx, stats) {
|
|
460
|
+
const cap = getImageCountMax();
|
|
461
|
+
const queue = pickEvictionTargets(reqCtx.body.messages);
|
|
462
|
+
if (queue.length <= cap) return;
|
|
463
|
+
const toDrop = queue.length - cap;
|
|
464
|
+
for (let i = 0; i < toDrop; i++) {
|
|
465
|
+
const ref = queue[i];
|
|
466
|
+
const src = ref.item.source;
|
|
467
|
+
const droppedBytes = src && src.data ? src.data.length : 0;
|
|
468
|
+
replaceImageInPlace(reqCtx.body.messages, ref, {
|
|
469
|
+
type: "text",
|
|
470
|
+
text: "[image dropped — exceeded image-count cap]",
|
|
471
|
+
});
|
|
472
|
+
stats.images_dropped_for_count_cap++;
|
|
473
|
+
stats.image_bytes_dropped += droppedBytes;
|
|
474
|
+
}
|
|
475
|
+
// Recompute request_bytes_after after count-cap evictions so the final
|
|
476
|
+
// telemetry reflects the post-pipeline body. Without this, count-cap-only
|
|
477
|
+
// requests would report unchanged byte totals (Codex review note).
|
|
478
|
+
if (toDrop > 0) {
|
|
479
|
+
const budget = getRequestSizeMax();
|
|
480
|
+
const after = Buffer.byteLength(JSON.stringify(reqCtx.body));
|
|
481
|
+
stats.request_bytes_after = after;
|
|
482
|
+
stats.request_bytes_headroom = budget - after;
|
|
483
|
+
}
|
|
484
|
+
}
|
|
485
|
+
|
|
486
|
+
// --- Telemetry: walk surviving images for byte/token totals ---
|
|
487
|
+
function finalizeTelemetry(reqCtx, stats) {
|
|
488
|
+
const refs = walkImages(reqCtx.body.messages);
|
|
489
|
+
let totalBytes = 0;
|
|
490
|
+
let totalTokens = 0;
|
|
491
|
+
const tokenCap = nativeTokenCap(reqCtx.body.model);
|
|
492
|
+
for (const ref of refs) {
|
|
493
|
+
const src = ref.item.source;
|
|
494
|
+
if (!src || !src.data) continue;
|
|
495
|
+
totalBytes += src.data.length;
|
|
496
|
+
const dims = parseImageDimensions(src.media_type, src.data);
|
|
497
|
+
if (dims) {
|
|
498
|
+
totalTokens += estimateImageTokens(dims.width, dims.height, tokenCap);
|
|
499
|
+
}
|
|
500
|
+
}
|
|
501
|
+
stats.image_bytes_total = totalBytes;
|
|
502
|
+
// estimated_image_tokens_total was decremented by Pass 3 deltas; for the
|
|
503
|
+
// baseline (Pass 3 disabled) recompute from surviving images.
|
|
504
|
+
if (stats.resize_attempted === 0) {
|
|
505
|
+
stats.estimated_image_tokens_total = totalTokens;
|
|
506
|
+
} else {
|
|
507
|
+
// Pass 3 mutated in place; re-measure surviving population to ground-truth.
|
|
508
|
+
stats.estimated_image_tokens_total = totalTokens;
|
|
509
|
+
}
|
|
510
|
+
}
|
|
511
|
+
|
|
512
|
+
// --- Top-level pipeline orchestrator ---
|
|
513
|
+
async function runImageGuard(reqCtx) {
|
|
514
|
+
const stats = initStats();
|
|
515
|
+
const messages = reqCtx.body.messages;
|
|
516
|
+
if (!Array.isArray(messages)) return stats;
|
|
517
|
+
|
|
518
|
+
// Capture initial population count for the summary line.
|
|
519
|
+
stats.total_images = walkImages(messages).length;
|
|
520
|
+
stats.count_axis_path = stats.total_images > 20 ? "many" : "few";
|
|
521
|
+
|
|
522
|
+
const guardOn = isImageGuardEnabled();
|
|
523
|
+
const preserveOn = isPreserveDetailEnabled();
|
|
524
|
+
const maxDimOverride = getMaxDim();
|
|
525
|
+
|
|
526
|
+
// Warn if PRESERVE_DETAIL is set without IMAGE_GUARD (one-time per process).
|
|
527
|
+
if (!guardOn && preserveOn && !_preserveDetailWarned) {
|
|
528
|
+
process.stderr.write(
|
|
529
|
+
"[image-guard] CACHE_FIX_IMAGE_PRESERVE_DETAIL=1 has no effect without CACHE_FIX_IMAGE_GUARD=1\n"
|
|
530
|
+
);
|
|
531
|
+
_preserveDetailWarned = true;
|
|
532
|
+
}
|
|
533
|
+
|
|
534
|
+
// Pass 3: native-cap resize (only when both gates are on)
|
|
535
|
+
if (guardOn && preserveOn) {
|
|
536
|
+
await runPass3NativeCapResize(reqCtx, stats);
|
|
537
|
+
}
|
|
538
|
+
|
|
539
|
+
// Pass 1: rejection-cap strip — runs if IMAGE_GUARD=1 OR legacy MAX_DIM > 0
|
|
540
|
+
if (guardOn || maxDimOverride > 0) {
|
|
541
|
+
runPass1RejectionCapStrip(reqCtx, stats, { maxDimOverride });
|
|
542
|
+
}
|
|
543
|
+
|
|
544
|
+
// Pass 2: request-size guard — IMAGE_GUARD only
|
|
545
|
+
if (guardOn) {
|
|
546
|
+
runPass2RequestSizeGuard(reqCtx, stats);
|
|
547
|
+
}
|
|
548
|
+
|
|
549
|
+
// Hard image-count cap — IMAGE_GUARD only
|
|
550
|
+
if (guardOn) {
|
|
551
|
+
runImageCountCap(reqCtx, stats);
|
|
552
|
+
}
|
|
553
|
+
|
|
554
|
+
// Final telemetry sweep over surviving images
|
|
555
|
+
finalizeTelemetry(reqCtx, stats);
|
|
556
|
+
|
|
557
|
+
return stats;
|
|
558
|
+
}
|
|
559
|
+
|
|
560
|
+
// One-time warning state for PRESERVE_DETAIL-without-GUARD.
|
|
561
|
+
let _preserveDetailWarned = false;
|
|
562
|
+
|
|
563
|
+
// Test hook to reset warning flag.
|
|
564
|
+
function _resetWarningStateForTests() {
|
|
565
|
+
_preserveDetailWarned = false;
|
|
566
|
+
}
|
|
567
|
+
|
|
568
|
+
export {
|
|
569
|
+
// Legacy v3.2.1 exports (kept stable for back-compat tests)
|
|
570
|
+
stripOldToolResultImages,
|
|
571
|
+
stripOversizedImages,
|
|
572
|
+
PLACEHOLDER,
|
|
573
|
+
oversizedPlaceholder,
|
|
574
|
+
// v3.3.0 pipeline pure functions (test seams)
|
|
575
|
+
pickPass1Cap,
|
|
576
|
+
pickPass3NativeCap,
|
|
577
|
+
estimateImageTokens,
|
|
578
|
+
walkImagesForPass1,
|
|
579
|
+
walkImagesForPass3,
|
|
580
|
+
pickEvictionTargets,
|
|
581
|
+
// Orchestrator (used by the extension default export and direct test calls)
|
|
582
|
+
runImageGuard,
|
|
583
|
+
// Test utilities
|
|
584
|
+
_resetWarningStateForTests,
|
|
585
|
+
};
|
|
136
586
|
|
|
137
587
|
export default {
|
|
138
588
|
name: "image-strip",
|
|
139
589
|
description:
|
|
140
|
-
"
|
|
141
|
-
|
|
590
|
+
"v3.2.1 KEEP_LAST/MAX_DIM legacy paths PLUS v3.3.0 image-guard pipeline " +
|
|
591
|
+
"(conditional rejection cap + request-size guard + optional Lanczos resize)",
|
|
592
|
+
enabled: false, // overridden by extensions.json
|
|
142
593
|
order: 150,
|
|
143
594
|
|
|
144
595
|
async onRequest(ctx) {
|
|
145
|
-
const
|
|
146
|
-
const
|
|
147
|
-
|
|
148
|
-
|
|
596
|
+
const guardOn = isImageGuardEnabled();
|
|
597
|
+
const preserveOn = isPreserveDetailEnabled();
|
|
598
|
+
// ctx.meta overrides allow tests to drive the legacy paths without env vars.
|
|
599
|
+
// Pipeline gates (IMAGE_GUARD, PRESERVE_DETAIL) remain env-only — tests that
|
|
600
|
+
// need to flip them set process.env directly.
|
|
601
|
+
const keepLast = parseInt(ctx.meta?.imageKeepLast ?? getKeepLast(), 10);
|
|
602
|
+
const maxDim = parseInt(ctx.meta?.imageMaxDim ?? getMaxDim(), 10);
|
|
149
603
|
|
|
150
|
-
|
|
151
|
-
|
|
604
|
+
// Short-circuit: nothing to do.
|
|
605
|
+
if (!guardOn && keepLast <= 0 && maxDim <= 0) {
|
|
606
|
+
// Surface the PRESERVE_DETAIL-without-GUARD warning even when the
|
|
607
|
+
// pipeline doesn't run otherwise.
|
|
608
|
+
if (preserveOn && !_preserveDetailWarned) {
|
|
609
|
+
process.stderr.write(
|
|
610
|
+
"[image-guard] CACHE_FIX_IMAGE_PRESERVE_DETAIL=1 has no effect without CACHE_FIX_IMAGE_GUARD=1\n"
|
|
611
|
+
);
|
|
612
|
+
_preserveDetailWarned = true;
|
|
613
|
+
}
|
|
614
|
+
return;
|
|
615
|
+
}
|
|
152
616
|
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
617
|
+
if (!ctx.body || !ctx.body.messages) return;
|
|
618
|
+
|
|
619
|
+
// ========== Legacy path (v3.2.1 back-compat) ==========
|
|
620
|
+
// When IMAGE_GUARD=1 is OFF but legacy env vars are set, run the v3.2.1
|
|
621
|
+
// pipeline exactly as before — preserves bug-for-bug compatibility for
|
|
622
|
+
// existing users.
|
|
623
|
+
if (!guardOn) {
|
|
624
|
+
let messages = ctx.body.messages;
|
|
625
|
+
const logParts = [];
|
|
626
|
+
|
|
627
|
+
if (keepLast > 0) {
|
|
628
|
+
const r = stripOldToolResultImages(messages, keepLast);
|
|
629
|
+
if (r.stats) {
|
|
630
|
+
messages = r.messages;
|
|
631
|
+
ctx.meta.imageStripStats = r.stats;
|
|
632
|
+
logParts.push(
|
|
633
|
+
`keep_last: ${r.stats.strippedCount} stripped (~${r.stats.estimatedTokens} tokens saved)`
|
|
634
|
+
);
|
|
635
|
+
}
|
|
636
|
+
}
|
|
637
|
+
|
|
638
|
+
if (maxDim > 0) {
|
|
639
|
+
const r = stripOversizedImages(messages, maxDim);
|
|
640
|
+
if (r.stats) {
|
|
641
|
+
messages = r.messages;
|
|
642
|
+
ctx.meta.imageStripOversizedStats = r.stats;
|
|
643
|
+
logParts.push(
|
|
644
|
+
`max_dim: ${r.stats.strippedCount} oversized stripped ` +
|
|
645
|
+
`(~${r.stats.estimatedTokens} tokens saved)`
|
|
646
|
+
);
|
|
647
|
+
}
|
|
161
648
|
}
|
|
649
|
+
|
|
650
|
+
if (logParts.length > 0) {
|
|
651
|
+
ctx.body.messages = messages;
|
|
652
|
+
process.stderr.write(`[image-strip] ${logParts.join("; ")}\n`);
|
|
653
|
+
}
|
|
654
|
+
return;
|
|
162
655
|
}
|
|
163
656
|
|
|
164
|
-
//
|
|
165
|
-
//
|
|
166
|
-
if (
|
|
167
|
-
const r =
|
|
657
|
+
// ========== v3.3.0 pipeline path ==========
|
|
658
|
+
// KEEP_LAST runs first as Pass 0 (back-compat behavior preserved).
|
|
659
|
+
if (keepLast > 0) {
|
|
660
|
+
const r = stripOldToolResultImages(ctx.body.messages, keepLast);
|
|
168
661
|
if (r.stats) {
|
|
169
|
-
messages = r.messages;
|
|
170
|
-
ctx.meta.
|
|
171
|
-
logParts.push(`max_dim: ${r.stats.strippedCount} oversized stripped (~${r.stats.estimatedTokens} tokens saved)`);
|
|
662
|
+
ctx.body.messages = r.messages;
|
|
663
|
+
ctx.meta.imageStripStats = r.stats;
|
|
172
664
|
}
|
|
173
665
|
}
|
|
174
666
|
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
667
|
+
const stats = await runImageGuard(ctx);
|
|
668
|
+
ctx.meta.imageGuardStats = stats;
|
|
669
|
+
|
|
670
|
+
// Emit summary only if the pipeline actually did anything observable.
|
|
671
|
+
const didSomething =
|
|
672
|
+
stats.images_stripped_pass1 > 0 ||
|
|
673
|
+
stats.images_dropped_for_size > 0 ||
|
|
674
|
+
stats.images_dropped_for_count_cap > 0 ||
|
|
675
|
+
stats.resize_attempted > 0 ||
|
|
676
|
+
stats.resize_succeeded > 0 ||
|
|
677
|
+
stats.unsupported_format_count > 0 ||
|
|
678
|
+
stats.dimension_probe_fail_count > 0;
|
|
679
|
+
if (didSomething) {
|
|
680
|
+
const parts = [];
|
|
681
|
+
if (stats.resize_succeeded > 0) parts.push(`resized=${stats.resize_succeeded}`);
|
|
682
|
+
if (stats.resize_failed > 0) parts.push(`resize_failed=${stats.resize_failed}`);
|
|
683
|
+
if (stats.library_missing) parts.push("sharp=missing");
|
|
684
|
+
if (stats.images_stripped_pass1 > 0) parts.push(`stripped=${stats.images_stripped_pass1}`);
|
|
685
|
+
if (stats.images_dropped_for_size > 0) parts.push(`evicted=${stats.images_dropped_for_size}`);
|
|
686
|
+
if (stats.images_dropped_for_count_cap > 0) {
|
|
687
|
+
parts.push(`count_capped=${stats.images_dropped_for_count_cap}`);
|
|
688
|
+
}
|
|
689
|
+
if (stats.unsupported_format_count > 0) parts.push(`unsupported=${stats.unsupported_format_count}`);
|
|
690
|
+
const summary = parts.join(" ") || "ran";
|
|
691
|
+
const finalImages = stats.total_images
|
|
692
|
+
- stats.images_stripped_pass1
|
|
693
|
+
- stats.images_dropped_for_size
|
|
694
|
+
- stats.images_dropped_for_count_cap;
|
|
695
|
+
process.stderr.write(
|
|
696
|
+
`[image-guard] ${summary} req_bytes=${stats.request_bytes_before}->${stats.request_bytes_after} ` +
|
|
697
|
+
`(headroom=${stats.request_bytes_headroom}) images=${stats.total_images}->${finalImages}\n`
|
|
698
|
+
);
|
|
178
699
|
}
|
|
179
700
|
},
|
|
180
701
|
};
|
|
702
|
+
|
package/proxy/extensions.json
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"fingerprint-strip": { "enabled": true, "order": 100 },
|
|
3
|
+
"image-strip": { "enabled": true, "order": 150 },
|
|
3
4
|
"sort-stabilization": { "enabled": true, "order": 200 },
|
|
4
5
|
"fresh-session-sort": { "enabled": true, "order": 250 },
|
|
5
6
|
"identity-normalization": { "enabled": true, "order": 300 },
|
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
// Lazy `sharp` wrapper for the image-guard pipeline's Pass 3 (native-cap resize).
|
|
2
|
+
//
|
|
3
|
+
// `sharp` is declared as an OPTIONAL peer dependency in package.json. The proxy
|
|
4
|
+
// must run without it. This module:
|
|
5
|
+
//
|
|
6
|
+
// 1. Lazy-imports `sharp` only when first needed (no module-load cost when
|
|
7
|
+
// Pass 3 is disabled or sharp is absent).
|
|
8
|
+
// 2. Caches the import result (success or library-missing failure) so we
|
|
9
|
+
// don't pay the import cost or re-throw repeatedly.
|
|
10
|
+
// 3. Returns a stable `{ ok, reason }` shape the caller can branch on
|
|
11
|
+
// without try/catch around every call.
|
|
12
|
+
//
|
|
13
|
+
// The actual resize uses Lanczos resampling (sharp's default kernel for
|
|
14
|
+
// downscales), preserves aspect ratio, and re-encodes using the SAME media
|
|
15
|
+
// type as the input. No transcoding in v1 — JPEG stays JPEG, PNG stays PNG.
|
|
16
|
+
|
|
17
|
+
let _sharpModule = null; // resolved sharp module (or null if missing)
|
|
18
|
+
let _sharpResolved = false; // have we attempted the import?
|
|
19
|
+
let _sharpMissing = false; // sticky flag — if first import fails, never retry
|
|
20
|
+
|
|
21
|
+
// Reset hook for tests. Not exported in the default surface; tests import by name.
|
|
22
|
+
export function _resetSharpCacheForTests() {
|
|
23
|
+
_sharpModule = null;
|
|
24
|
+
_sharpResolved = false;
|
|
25
|
+
_sharpMissing = false;
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
// Override hook for tests: inject a fake sharp without going through real import.
|
|
29
|
+
// The fake should be callable as `fake(buffer)` returning an object with
|
|
30
|
+
// `.resize()` and `.toBuffer()`/`.toFormat()` methods like real sharp.
|
|
31
|
+
export function _setSharpForTests(fakeSharp) {
|
|
32
|
+
_sharpModule = fakeSharp;
|
|
33
|
+
_sharpResolved = true;
|
|
34
|
+
_sharpMissing = !fakeSharp;
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
async function loadSharp() {
|
|
38
|
+
if (_sharpResolved) {
|
|
39
|
+
return { ok: !_sharpMissing, sharp: _sharpModule };
|
|
40
|
+
}
|
|
41
|
+
try {
|
|
42
|
+
const mod = await import("sharp");
|
|
43
|
+
_sharpModule = mod.default || mod;
|
|
44
|
+
_sharpResolved = true;
|
|
45
|
+
_sharpMissing = false;
|
|
46
|
+
return { ok: true, sharp: _sharpModule };
|
|
47
|
+
} catch (err) {
|
|
48
|
+
_sharpResolved = true;
|
|
49
|
+
_sharpMissing = true;
|
|
50
|
+
_sharpModule = null;
|
|
51
|
+
return { ok: false, sharp: null, err };
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
// Re-encode media type → sharp output format name. Keep symmetric with the
|
|
56
|
+
// dimension probe (PNG + JPEG only in v1).
|
|
57
|
+
function mediaTypeToSharpFormat(mediaType) {
|
|
58
|
+
switch ((mediaType || "").toLowerCase()) {
|
|
59
|
+
case "image/png":
|
|
60
|
+
return "png";
|
|
61
|
+
case "image/jpeg":
|
|
62
|
+
case "image/jpg":
|
|
63
|
+
return "jpeg";
|
|
64
|
+
default:
|
|
65
|
+
return null;
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
// Resize a base64-encoded image to `capPx` on the long edge using Lanczos.
|
|
70
|
+
// Returns:
|
|
71
|
+
// { ok: true, base64, dims: { width, height }, bytes } on success
|
|
72
|
+
// { ok: false, reason: "library_missing" | "unsupported_media_type" | "decode_failed" | "resize_failed" }
|
|
73
|
+
//
|
|
74
|
+
// Caller is expected to:
|
|
75
|
+
// - skip Pass 3 entirely on `library_missing` (sticky for the process)
|
|
76
|
+
// - increment per-image telemetry counters on the other failure modes and
|
|
77
|
+
// leave the original image untouched for Pass 1 to evaluate.
|
|
78
|
+
export async function resizeImageToCap(base64Data, mediaType, capPx) {
|
|
79
|
+
if (!base64Data || typeof base64Data !== "string") {
|
|
80
|
+
return { ok: false, reason: "decode_failed" };
|
|
81
|
+
}
|
|
82
|
+
const format = mediaTypeToSharpFormat(mediaType);
|
|
83
|
+
if (!format) {
|
|
84
|
+
return { ok: false, reason: "unsupported_media_type" };
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
const loaded = await loadSharp();
|
|
88
|
+
if (!loaded.ok) {
|
|
89
|
+
return { ok: false, reason: "library_missing" };
|
|
90
|
+
}
|
|
91
|
+
const sharp = loaded.sharp;
|
|
92
|
+
|
|
93
|
+
let inputBuffer;
|
|
94
|
+
try {
|
|
95
|
+
inputBuffer = Buffer.from(base64Data, "base64");
|
|
96
|
+
} catch {
|
|
97
|
+
return { ok: false, reason: "decode_failed" };
|
|
98
|
+
}
|
|
99
|
+
if (!inputBuffer || inputBuffer.length === 0) {
|
|
100
|
+
return { ok: false, reason: "decode_failed" };
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
try {
|
|
104
|
+
const pipeline = sharp(inputBuffer).resize({
|
|
105
|
+
width: capPx,
|
|
106
|
+
height: capPx,
|
|
107
|
+
fit: "inside", // preserve aspect ratio, neither edge exceeds capPx
|
|
108
|
+
withoutEnlargement: true, // never upscale
|
|
109
|
+
kernel: "lanczos3",
|
|
110
|
+
});
|
|
111
|
+
|
|
112
|
+
// Re-encode using the SAME format as the input. No transcoding in v1.
|
|
113
|
+
const encoded = format === "png"
|
|
114
|
+
? await pipeline.png().toBuffer({ resolveWithObject: true })
|
|
115
|
+
: await pipeline.jpeg().toBuffer({ resolveWithObject: true });
|
|
116
|
+
|
|
117
|
+
const newBase64 = encoded.data.toString("base64");
|
|
118
|
+
return {
|
|
119
|
+
ok: true,
|
|
120
|
+
base64: newBase64,
|
|
121
|
+
dims: { width: encoded.info.width, height: encoded.info.height },
|
|
122
|
+
bytes: encoded.data.length,
|
|
123
|
+
};
|
|
124
|
+
} catch {
|
|
125
|
+
return { ok: false, reason: "resize_failed" };
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
// Tiny helper for tests: returns whether sharp was successfully imported once.
|
|
130
|
+
// Doesn't trigger an import — caller must have invoked resizeImageToCap first.
|
|
131
|
+
export function _sharpStatusForTests() {
|
|
132
|
+
return { resolved: _sharpResolved, missing: _sharpMissing };
|
|
133
|
+
}
|