pi-condense 2.1.1 → 2.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +18 -0
- package/PRUNING.md +32 -0
- package/README.md +15 -3
- package/index.ts +8 -1
- package/package.json +11 -3
- package/src/summarizer-fallback.test.ts +146 -0
- package/src/summarizer-fallback.ts +122 -0
- package/src/summarizer-wiring.test.ts +165 -0
- package/src/summarizer.ts +117 -26
- package/src/types.ts +14 -0
package/CHANGELOG.md
CHANGED
|
@@ -9,6 +9,24 @@ publishes via OIDC trusted publishing. See `.agents/skills/release/SKILL.md`.
|
|
|
9
9
|
|
|
10
10
|
## [Unreleased]
|
|
11
11
|
|
|
12
|
+
## [2.2.0] - 2026-07-06
|
|
13
|
+
|
|
14
|
+
- **Summarizer outage fallback to the session model.** Per-model provider outages (e.g. a cheap `summarizerModel` like Haiku degraded while the session's main model stays healthy) previously stalled pruning for the whole outage - `runSummarization` returned null and the batch retried the same dead model every flush, growing context unbounded. A new sticky in-memory `FallbackController` (`src/summarizer-fallback.ts`) now routes summarization to `ctx.model` on a **transient** failure of the configured model, retrying the failed call once on the session model. Fallback is sticky: while engaged, all calls use the session model until a single probe batch re-tests the configured model after a 10-minute cooldown, then auto-recovers. Trigger is transient-only - auth (pre-flight key failure), unusable (empty/truncated), and abort never trip it. A one-time `warning` fires on enter and an `info` on recovery via `ctx.ui.notify` (UI only, never injected into LLM context). No config key: the target is always `ctx.model`, and the controller is inert when no distinct fallback model exists (`summarizerModel: default` or the resolved model equals `ctx.model`), preserving today's single-attempt behavior byte-for-byte. State is in-memory only (reset on `session_start`, no `context-prune-*` entry).
|
|
15
|
+
|
|
16
|
+
## [2.1.2] - 2026-07-05
|
|
17
|
+
|
|
18
|
+
Branding, funding, and gallery preview. No behavior change.
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
|
|
22
|
+
- **Logo + pi.dev gallery preview.** Repo-root `pi-condense.png` (640x640), shown in the README and wired as `pi.image`.
|
|
23
|
+
- **Buy Me a Coffee funding.** `funding` in `package.json`, `.github/FUNDING.yml`, and a README badge.
|
|
24
|
+
|
|
25
|
+
### Changed
|
|
26
|
+
|
|
27
|
+
- Sharpened `description`; added `context-pruning`, `llm`, `prompt-caching` keywords.
|
|
28
|
+
- README reframed product-first (credit to `championswimmer/pi-context-prune` kept as attribution); fixed a stale `pi-superpowers` -> `pi-gauntlet` reference in a spec doc.
|
|
29
|
+
|
|
12
30
|
## [2.1.1] - 2026-07-04
|
|
13
31
|
|
|
14
32
|
- **`release.yml` posts GitHub Release notes.** A new `release-notes` job (`needs: publish`, `contents: write`) extracts the CHANGELOG section matching the pushed tag with `awk` (skipping `## [Unreleased]`) and publishes it as the GitHub Release body via `gh release create` (falling back to `gh release edit`). No LLM or API key; only `github.token`.
|
package/PRUNING.md
CHANGED
|
@@ -688,6 +688,38 @@ The last attempted prune boundary is persisted as `context-prune-frontier` so `f
|
|
|
688
688
|
- **Live reclaim ratio:** measured once per `pruneMessages` call via `sizeMessages(messages) = JSON.stringify(messages).length`, comparing the input array before pruning to the result after. Estimated tokens = chars / 4. The measurement covers all four reclaim mechanisms in a single point (stub-replace, error-purge, chain-range-prune, thinking-strip); appears on the status line as `│ prune: ON · 92k->14k (-85%) │` once at least one prune has occurred (the `│ … │` wrapper keeps the segment visually isolated in the shared footer, load-order independent).
|
|
689
689
|
- **Live progress for `/pruner now`:** an `aboveEditor` widget shows one row per pending batch with braille spinner, streamed summary-char count, and ✓ / ⚠ status.
|
|
690
690
|
|
|
691
|
+
### Summarizer outage fallback
|
|
692
|
+
|
|
693
|
+
Provider incidents are routinely per-model: a cheap summarizer model (e.g.
|
|
694
|
+
Haiku) can be down while the session's main model (Sonnet/Opus) stays healthy.
|
|
695
|
+
Without a fallback, `runSummarization` returns `null`, the batch is re-queued,
|
|
696
|
+
and the next flush retries the same dead model — pruning stalls for the whole
|
|
697
|
+
outage and context grows unbounded.
|
|
698
|
+
|
|
699
|
+
`src/summarizer-fallback.ts` adds a sticky, in-memory `FallbackController` that
|
|
700
|
+
engages only on **transient** failures of the configured summarizer model and
|
|
701
|
+
only when a distinct fallback model exists (`summarizerModel` != `default` and
|
|
702
|
+
the resolved model differs from `ctx.model`):
|
|
703
|
+
|
|
704
|
+
- **Classification is coarse** (pi-ai surfaces no status code on the throw):
|
|
705
|
+
`auth` (pre-flight key failure) and `unusable` (empty / length-truncated)
|
|
706
|
+
never trip the controller; a `transient` stream error / `stopReason: error`
|
|
707
|
+
does. Aborts propagate unchanged.
|
|
708
|
+
- **Enter:** a transient primary failure is retried once on the session model.
|
|
709
|
+
If that succeeds, the session flips to fallback and a one-time warning fires.
|
|
710
|
+
- **Sticky + probe:** while in fallback, all calls route to the session model.
|
|
711
|
+
After a 10-minute cooldown (`COOLDOWN_MS`, internal, not configurable) one
|
|
712
|
+
batch of the next flush probes the primary; success recovers (info notify),
|
|
713
|
+
failure stays in fallback.
|
|
714
|
+
- **In-memory only:** no `context-prune-*` entry; `reset()` on `session_start`.
|
|
715
|
+
A restart mid-outage re-detects on the next flush.
|
|
716
|
+
|
|
717
|
+
Cost note: the initial detection flush can fire up to N doomed primary calls
|
|
718
|
+
before the outage is known; steady state is 0 doomed calls, plus exactly 1
|
|
719
|
+
probe per 10 minutes. After the primary recovers, summarization keeps running
|
|
720
|
+
on the (often pricier) session model for up to one cooldown before the probe
|
|
721
|
+
switches back.
|
|
722
|
+
|
|
691
723
|
---
|
|
692
724
|
|
|
693
725
|
## Why Summarization Works: Research Evidence
|
package/README.md
CHANGED
|
@@ -1,10 +1,16 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="https://raw.githubusercontent.com/jjuraszek/pi-condense/main/pi-condense.png" alt="pi-condense" width="180">
|
|
3
|
+
</p>
|
|
4
|
+
|
|
1
5
|
# pi-condense
|
|
2
6
|
|
|
3
|
-
A
|
|
7
|
+
[](https://buymeacoffee.com/jjurasszek)
|
|
8
|
+
|
|
9
|
+
A [Pi coding-agent](https://github.com/earendil-works/pi) extension that summarizes completed tool-call batches, replaces raw tool outputs with short stubs in future context, and lets the LLM recover any original via the `context_tree_query` tool.
|
|
4
10
|
|
|
5
11
|
The session JSONL file is never modified — pruning only affects what each *next* request sees.
|
|
6
12
|
|
|
7
|
-
|
|
13
|
+
Adds pre-flush safeguards, agent-message batching, chain compression, and an npm release flow on top of the original approach from [`championswimmer/pi-context-prune`](https://github.com/championswimmer/pi-context-prune).
|
|
8
14
|
|
|
9
15
|
📖 For the algorithm, design rationale, prompt-cache interaction, and the research behind summarization-based context management, see **[PRUNING.md](PRUNING.md)**.
|
|
10
16
|
|
|
@@ -42,7 +48,7 @@ pi -e ~/repos/pi-condense/index.ts
|
|
|
42
48
|
|
|
43
49
|
Pin a specific version with `npm:pi-condense@X.Y.Z`. Upgrade by re-running `pi install`. Remove with `pi remove pi-condense`. Once installed, the extension auto-loads on every `pi` invocation; no flags needed.
|
|
44
50
|
|
|
45
|
-
>
|
|
51
|
+
> See [CHANGELOG.md](CHANGELOG.md) for release history.
|
|
46
52
|
|
|
47
53
|
## Quick start
|
|
48
54
|
|
|
@@ -152,6 +158,8 @@ Single tool results larger than `spillThreshold` chars are written to `<session-
|
|
|
152
158
|
|
|
153
159
|
The `default` setting reuses whatever model you have active in pi — convenient but wasteful, since summary writing doesn't need a top-tier coding model. Picking the smallest/fastest model on your plan saves both latency and cost.
|
|
154
160
|
|
|
161
|
+
If the configured summarizer model suffers a transient outage while your active pi model is healthy, pi-condense automatically falls back to the session model for the duration (with a one-time notice) and probes the configured model back every few minutes — no configuration needed.
|
|
162
|
+
|
|
155
163
|
| Plan | Suggested summarizer |
|
|
156
164
|
|---|---|
|
|
157
165
|
| OpenAI / Codex / Copilot | `openai/gpt-4.1-mini`, `google/gemini-2.5-flash`, `xai/grok-3-fast` |
|
|
@@ -236,6 +244,10 @@ Semantics:
|
|
|
236
244
|
- Content-hash dedup only matches against records already in the indexer (cross-flush). Two identical outputs within the *same* flush are not deduped — both go through the summarizer.
|
|
237
245
|
- The tree browser does not inline original tool outputs — use `context_tree_query` for that.
|
|
238
246
|
|
|
247
|
+
## Support
|
|
248
|
+
|
|
249
|
+
If this saves you tokens, [buy me a coffee](https://buymeacoffee.com/jjurasszek).
|
|
250
|
+
|
|
239
251
|
## References
|
|
240
252
|
|
|
241
253
|
- Anthropic prompt caching: <https://docs.claude.com/en/docs/build-with-claude/prompt-caching>
|
package/index.ts
CHANGED
|
@@ -17,6 +17,7 @@ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
|
17
17
|
import { loadConfig } from "./src/config.js";
|
|
18
18
|
import { captureBatch, captureUnindexedBatchesFromSession, groupBatchesByMode } from "./src/batch-capture.js";
|
|
19
19
|
import { summarizeBatch, summarizeBatches, summarizeRange } from "./src/summarizer.js";
|
|
20
|
+
import { FallbackController } from "./src/summarizer-fallback.js";
|
|
20
21
|
import { ToolCallIndexer } from "./src/indexer.js";
|
|
21
22
|
import { pruneMessages } from "./src/pruner.js";
|
|
22
23
|
import { isProtected } from "./src/protected.js";
|
|
@@ -52,6 +53,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
52
53
|
// Shared stats accumulator — tracks cumulative token/cost stats for summarizer calls
|
|
53
54
|
const statsAccum = new StatsAccumulator();
|
|
54
55
|
|
|
56
|
+
// Session-scoped summarizer outage-fallback controller (in-memory; reset on session_start).
|
|
57
|
+
const fallbackController = new FallbackController();
|
|
58
|
+
|
|
55
59
|
// Shared prune frontier — tracks the last completed prune attempt boundary
|
|
56
60
|
const frontier = new PruneFrontierTracker();
|
|
57
61
|
|
|
@@ -152,7 +156,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
152
156
|
const makeFuseRange = (ctx: any): ((text: string) => Promise<string | null>) | undefined => {
|
|
153
157
|
if (!currentConfig.value.chainCompression.fuseRangeSummary) return undefined;
|
|
154
158
|
return async (text: string) => {
|
|
155
|
-
const r = await summarizeRange(text, currentConfig.value, ctx, {});
|
|
159
|
+
const r = await summarizeRange(text, currentConfig.value, ctx, { controller: fallbackController });
|
|
156
160
|
if (r) {
|
|
157
161
|
statsAccum.add(r.usage);
|
|
158
162
|
statsAccum.addRangesSummarized(1);
|
|
@@ -303,6 +307,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
303
307
|
options.onProgress(i, batches.length, batches[i], "start");
|
|
304
308
|
const r = await summarizeBatch(batches[i], currentConfig.value, ctx, {
|
|
305
309
|
signal: options.signal,
|
|
310
|
+
controller: fallbackController,
|
|
306
311
|
onTextProgress: (receivedChars) => {
|
|
307
312
|
reportBatchTextProgress(i, batches.length, batches[i], receivedChars);
|
|
308
313
|
},
|
|
@@ -326,6 +331,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
326
331
|
reportBatchTextProgress(origIndex, batches.length, batch, receivedChars);
|
|
327
332
|
},
|
|
328
333
|
signal: options.signal,
|
|
334
|
+
controller: fallbackController,
|
|
329
335
|
});
|
|
330
336
|
for (let k = 0; k < nonTrivialIndices.length; k++) {
|
|
331
337
|
results[nonTrivialIndices[k]] = ntResults[k];
|
|
@@ -647,6 +653,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
647
653
|
|
|
648
654
|
// Rebuild stats accumulator from persisted session entries
|
|
649
655
|
statsAccum.reconstructFromSession(ctx);
|
|
656
|
+
fallbackController.reset();
|
|
650
657
|
|
|
651
658
|
// Rebuild prune frontier from persisted session entries
|
|
652
659
|
frontier.reconstructFromSession(ctx);
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-condense",
|
|
3
|
-
"version": "2.
|
|
4
|
-
"description": "Pi extension that summarizes completed tool-call batches, replaces raw outputs with short stubs
|
|
3
|
+
"version": "2.2.0",
|
|
4
|
+
"description": "Pi coding-agent extension that summarizes completed tool-call batches, replaces raw outputs with short stubs, compresses closed tool-call chains, and recovers any original on demand via context_tree_query.",
|
|
5
5
|
"author": "Jacek Juraszek",
|
|
6
6
|
"license": "MIT",
|
|
7
7
|
"type": "module",
|
|
@@ -19,8 +19,11 @@
|
|
|
19
19
|
"pi-coding-agent",
|
|
20
20
|
"context",
|
|
21
21
|
"context-management",
|
|
22
|
+
"context-pruning",
|
|
22
23
|
"pruning",
|
|
23
24
|
"summarization",
|
|
25
|
+
"llm",
|
|
26
|
+
"prompt-caching",
|
|
24
27
|
"ai"
|
|
25
28
|
],
|
|
26
29
|
"engines": {
|
|
@@ -40,7 +43,12 @@
|
|
|
40
43
|
"pi": {
|
|
41
44
|
"extensions": [
|
|
42
45
|
"./index.ts"
|
|
43
|
-
]
|
|
46
|
+
],
|
|
47
|
+
"image": "https://raw.githubusercontent.com/jjuraszek/pi-condense/main/pi-condense.png"
|
|
48
|
+
},
|
|
49
|
+
"funding": {
|
|
50
|
+
"type": "buymeacoffee",
|
|
51
|
+
"url": "https://buymeacoffee.com/jjurasszek"
|
|
44
52
|
},
|
|
45
53
|
"peerDependencies": {
|
|
46
54
|
"@earendil-works/pi-coding-agent": "*",
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
import { describe, it, expect } from "bun:test";
|
|
2
|
+
import { FallbackController, COOLDOWN_MS } from "./summarizer-fallback.js";
|
|
3
|
+
|
|
4
|
+
const HAIKU = { id: "claude-haiku", provider: "anthropic", name: "Haiku" };
|
|
5
|
+
const SONNET = { id: "claude-sonnet", provider: "anthropic", name: "Sonnet" };
|
|
6
|
+
|
|
7
|
+
function clockAt(t: { now: number }) {
|
|
8
|
+
return () => t.now;
|
|
9
|
+
}
|
|
10
|
+
|
|
11
|
+
describe("hasDistinctFallback", () => {
|
|
12
|
+
it("false when models are identical", () => {
|
|
13
|
+
expect(FallbackController.hasDistinctFallback(HAIKU, HAIKU)).toBe(false);
|
|
14
|
+
});
|
|
15
|
+
it("false when either model is missing", () => {
|
|
16
|
+
expect(FallbackController.hasDistinctFallback(undefined, SONNET)).toBe(false);
|
|
17
|
+
expect(FallbackController.hasDistinctFallback(HAIKU, undefined)).toBe(false);
|
|
18
|
+
});
|
|
19
|
+
it("true when id or provider differ", () => {
|
|
20
|
+
expect(FallbackController.hasDistinctFallback(HAIKU, SONNET)).toBe(true);
|
|
21
|
+
expect(
|
|
22
|
+
FallbackController.hasDistinctFallback(
|
|
23
|
+
{ id: "m", provider: "a" },
|
|
24
|
+
{ id: "m", provider: "b" },
|
|
25
|
+
),
|
|
26
|
+
).toBe(true);
|
|
27
|
+
});
|
|
28
|
+
});
|
|
29
|
+
|
|
30
|
+
describe("chooseTarget", () => {
|
|
31
|
+
it("targets primary when not in fallback", () => {
|
|
32
|
+
const c = new FallbackController();
|
|
33
|
+
expect(c.chooseTarget()).toEqual({ target: "primary", wasProbe: false });
|
|
34
|
+
});
|
|
35
|
+
|
|
36
|
+
it("targets fallback while in fallback and before cooldown", () => {
|
|
37
|
+
const t = { now: 1_000_000 };
|
|
38
|
+
const c = new FallbackController(clockAt(t));
|
|
39
|
+
c.onPrimaryFailFallbackOk(false);
|
|
40
|
+
expect(c.inFallback).toBe(true);
|
|
41
|
+
t.now += COOLDOWN_MS - 1;
|
|
42
|
+
expect(c.chooseTarget()).toEqual({ target: "fallback", wasProbe: false });
|
|
43
|
+
});
|
|
44
|
+
|
|
45
|
+
it("elects exactly one probe after cooldown elapses", () => {
|
|
46
|
+
const t = { now: 0 };
|
|
47
|
+
const c = new FallbackController(clockAt(t));
|
|
48
|
+
c.onPrimaryFailFallbackOk(false); // enter, lastProbeAt = 0
|
|
49
|
+
t.now = COOLDOWN_MS;
|
|
50
|
+
expect(c.chooseTarget()).toEqual({ target: "primary", wasProbe: true });
|
|
51
|
+
expect(c.chooseTarget()).toEqual({ target: "fallback", wasProbe: false });
|
|
52
|
+
});
|
|
53
|
+
});
|
|
54
|
+
|
|
55
|
+
describe("enter fallback (single notify across N)", () => {
|
|
56
|
+
it("first rescued transient returns 'enter', rest return 'none'", () => {
|
|
57
|
+
const c = new FallbackController();
|
|
58
|
+
expect(c.onPrimaryFailFallbackOk(false)).toBe("enter");
|
|
59
|
+
expect(c.inFallback).toBe(true);
|
|
60
|
+
expect(c.onPrimaryFailFallbackOk(false)).toBe("none");
|
|
61
|
+
expect(c.onPrimaryFailFallbackOk(false)).toBe("none");
|
|
62
|
+
});
|
|
63
|
+
});
|
|
64
|
+
|
|
65
|
+
describe("recover", () => {
|
|
66
|
+
it("probe success while in fallback returns 'recover' and clears state", () => {
|
|
67
|
+
const c = new FallbackController();
|
|
68
|
+
c.onPrimaryFailFallbackOk(false); // enter
|
|
69
|
+
expect(c.onPrimarySuccess(true)).toBe("recover");
|
|
70
|
+
expect(c.inFallback).toBe(false);
|
|
71
|
+
});
|
|
72
|
+
it("non-probe primary success is a no-op", () => {
|
|
73
|
+
const c = new FallbackController();
|
|
74
|
+
expect(c.onPrimarySuccess(false)).toBe("none");
|
|
75
|
+
expect(c.inFallback).toBe(false);
|
|
76
|
+
});
|
|
77
|
+
});
|
|
78
|
+
|
|
79
|
+
describe("probe transient failure keeps fallback (stay)", () => {
|
|
80
|
+
it("rescued probe returns 'none' and stays in fallback", () => {
|
|
81
|
+
const t = { now: 0 };
|
|
82
|
+
const c = new FallbackController(clockAt(t));
|
|
83
|
+
c.onPrimaryFailFallbackOk(false); // enter
|
|
84
|
+
t.now = COOLDOWN_MS;
|
|
85
|
+
const d = c.chooseTarget(); // claims probe
|
|
86
|
+
expect(d.wasProbe).toBe(true);
|
|
87
|
+
t.now = COOLDOWN_MS + 5;
|
|
88
|
+
expect(c.onPrimaryFailFallbackOk(true)).toBe("none");
|
|
89
|
+
expect(c.inFallback).toBe(true);
|
|
90
|
+
});
|
|
91
|
+
});
|
|
92
|
+
|
|
93
|
+
describe("both-down owes a deferred enter warning", () => {
|
|
94
|
+
it("owed warning fires on the first later fallback success", () => {
|
|
95
|
+
const c = new FallbackController();
|
|
96
|
+
c.onBothDown(); // initial detection both-down: enter silently, owe warning
|
|
97
|
+
expect(c.inFallback).toBe(true);
|
|
98
|
+
expect(c.onFallbackSuccess()).toBe("enter");
|
|
99
|
+
expect(c.onFallbackSuccess()).toBe("none");
|
|
100
|
+
});
|
|
101
|
+
it("both-down while already in fallback does not owe a new warning", () => {
|
|
102
|
+
const c = new FallbackController();
|
|
103
|
+
c.onPrimaryFailFallbackOk(false); // enter (warning already shown)
|
|
104
|
+
c.onBothDown(); // probe both-down while in fallback
|
|
105
|
+
expect(c.onFallbackSuccess()).toBe("none");
|
|
106
|
+
});
|
|
107
|
+
});
|
|
108
|
+
|
|
109
|
+
describe("recover clears an owed enter warning", () => {
|
|
110
|
+
it("a probe recovery after both-down does not later emit a stale enter", () => {
|
|
111
|
+
const t = { now: 0 };
|
|
112
|
+
const c = new FallbackController(clockAt(t));
|
|
113
|
+
c.onBothDown(); // enter silently, owe warning
|
|
114
|
+
t.now = COOLDOWN_MS;
|
|
115
|
+
const d = c.chooseTarget(); // claims probe
|
|
116
|
+
expect(d.wasProbe).toBe(true);
|
|
117
|
+
expect(c.onPrimarySuccess(true)).toBe("recover");
|
|
118
|
+
expect(c.inFallback).toBe(false);
|
|
119
|
+
// owed warning must be cleared by recovery
|
|
120
|
+
expect(c.onFallbackSuccess()).toBe("none");
|
|
121
|
+
});
|
|
122
|
+
});
|
|
123
|
+
|
|
124
|
+
describe("unusable probe does not falsely recover", () => {
|
|
125
|
+
it("no onPrimarySuccess call means inFallback is retained", () => {
|
|
126
|
+
const t = { now: 0 };
|
|
127
|
+
const c = new FallbackController(clockAt(t));
|
|
128
|
+
c.onPrimaryFailFallbackOk(false); // enter
|
|
129
|
+
t.now = COOLDOWN_MS;
|
|
130
|
+
const d = c.chooseTarget();
|
|
131
|
+
expect(d.wasProbe).toBe(true);
|
|
132
|
+
expect(c.inFallback).toBe(true);
|
|
133
|
+
});
|
|
134
|
+
});
|
|
135
|
+
|
|
136
|
+
describe("reset", () => {
|
|
137
|
+
it("clears all state", () => {
|
|
138
|
+
const t = { now: 5 };
|
|
139
|
+
const c = new FallbackController(clockAt(t));
|
|
140
|
+
c.onBothDown();
|
|
141
|
+
c.reset();
|
|
142
|
+
expect(c.inFallback).toBe(false);
|
|
143
|
+
expect(c.onFallbackSuccess()).toBe("none"); // owed cleared
|
|
144
|
+
expect(c.chooseTarget()).toEqual({ target: "primary", wasProbe: false });
|
|
145
|
+
});
|
|
146
|
+
});
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Session-scoped, in-memory state machine for summarizer-model outage fallback.
|
|
3
|
+
*
|
|
4
|
+
* Pure of model IO and notify plumbing: transition methods mutate state and
|
|
5
|
+
* return a transition tag; the caller (runSummarization) performs the LLM runs
|
|
6
|
+
* and emits any notify text. `now()` is injected for deterministic tests.
|
|
7
|
+
*
|
|
8
|
+
* Engaged ONLY on transient (outage-shaped) failures of the configured
|
|
9
|
+
* summarizer model, and only when a distinct fallback model exists. Sticky:
|
|
10
|
+
* once in fallback, all calls route to the session model until a single
|
|
11
|
+
* per-cooldown probe of the primary succeeds. See
|
|
12
|
+
* doc/specs/2026-07-06-summarizer-outage-fallback.md.
|
|
13
|
+
*/
|
|
14
|
+
|
|
15
|
+
/** Re-probe cooldown while in fallback. Internal; deliberately not configurable. */
|
|
16
|
+
export const COOLDOWN_MS = 10 * 60 * 1000;
|
|
17
|
+
|
|
18
|
+
export type FallbackTransition = "enter" | "recover" | "none";
|
|
19
|
+
export type CallTarget = "primary" | "fallback";
|
|
20
|
+
|
|
21
|
+
export interface TargetDecision {
|
|
22
|
+
target: CallTarget;
|
|
23
|
+
wasProbe: boolean;
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
/** Minimal structural view of a pi-ai Model (avoids the generic Api type param). */
|
|
27
|
+
export interface ModelLike {
|
|
28
|
+
id: string;
|
|
29
|
+
provider: string;
|
|
30
|
+
name?: string;
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
export class FallbackController {
|
|
34
|
+
inFallback = false;
|
|
35
|
+
private lastProbeAt = 0;
|
|
36
|
+
private owedEnterWarning = false;
|
|
37
|
+
|
|
38
|
+
constructor(private readonly now: () => number = Date.now) {}
|
|
39
|
+
|
|
40
|
+
reset(): void {
|
|
41
|
+
this.inFallback = false;
|
|
42
|
+
this.lastProbeAt = 0;
|
|
43
|
+
this.owedEnterWarning = false;
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
/**
|
|
47
|
+
* True when primary and the session model are genuinely different. When
|
|
48
|
+
* false the controller must NOT be consulted (behavior identical to today).
|
|
49
|
+
* `Model.provider` is a plain string in pi-ai, not an object.
|
|
50
|
+
*/
|
|
51
|
+
static hasDistinctFallback(
|
|
52
|
+
primary: ModelLike | undefined,
|
|
53
|
+
sessionModel: ModelLike | undefined,
|
|
54
|
+
): boolean {
|
|
55
|
+
if (!primary || !sessionModel) return false;
|
|
56
|
+
return primary.provider !== sessionModel.provider || primary.id !== sessionModel.id;
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
/**
|
|
60
|
+
* Pick the model target for the next call and, if eligible, claim the single
|
|
61
|
+
* per-cooldown probe. The claim is synchronous: the first of N concurrent
|
|
62
|
+
* callers in a flush advances `lastProbeAt`, so siblings see the cooldown as
|
|
63
|
+
* not elapsed and route to the fallback. Call before the first await.
|
|
64
|
+
*/
|
|
65
|
+
chooseTarget(): TargetDecision {
|
|
66
|
+
if (!this.inFallback) return { target: "primary", wasProbe: false };
|
|
67
|
+
if (this.now() - this.lastProbeAt >= COOLDOWN_MS) {
|
|
68
|
+
this.lastProbeAt = this.now();
|
|
69
|
+
return { target: "primary", wasProbe: true };
|
|
70
|
+
}
|
|
71
|
+
return { target: "fallback", wasProbe: false };
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
/** Primary (initial or probe) failed transiently but the fallback retry succeeded. */
|
|
75
|
+
onPrimaryFailFallbackOk(_wasProbe: boolean): FallbackTransition {
|
|
76
|
+
this.lastProbeAt = this.now();
|
|
77
|
+
if (this.owedEnterWarning) {
|
|
78
|
+
this.owedEnterWarning = false;
|
|
79
|
+
this.inFallback = true;
|
|
80
|
+
return "enter";
|
|
81
|
+
}
|
|
82
|
+
if (!this.inFallback) {
|
|
83
|
+
this.inFallback = true;
|
|
84
|
+
return "enter";
|
|
85
|
+
}
|
|
86
|
+
// probe transient, rescued by fallback -> stay, no notify
|
|
87
|
+
return "none";
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
/** Both the primary call and the fallback retry failed transiently. */
|
|
91
|
+
onBothDown(): void {
|
|
92
|
+
this.lastProbeAt = this.now();
|
|
93
|
+
if (!this.inFallback) {
|
|
94
|
+
this.inFallback = true;
|
|
95
|
+
this.owedEnterWarning = true;
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
/** A steady-state fallback call (already in fallback) failed transiently. */
|
|
100
|
+
onFallbackOnlyFail(): void {
|
|
101
|
+
this.lastProbeAt = this.now();
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
/** A primary call succeeded. Recover only when it was the probe. */
|
|
105
|
+
onPrimarySuccess(wasProbe: boolean): FallbackTransition {
|
|
106
|
+
if (wasProbe && this.inFallback) {
|
|
107
|
+
this.inFallback = false;
|
|
108
|
+
this.owedEnterWarning = false;
|
|
109
|
+
return "recover";
|
|
110
|
+
}
|
|
111
|
+
return "none";
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
/** A steady-state fallback call succeeded. Fire the deferred enter warning if owed. */
|
|
115
|
+
onFallbackSuccess(): FallbackTransition {
|
|
116
|
+
if (this.owedEnterWarning) {
|
|
117
|
+
this.owedEnterWarning = false;
|
|
118
|
+
return "enter";
|
|
119
|
+
}
|
|
120
|
+
return "none";
|
|
121
|
+
}
|
|
122
|
+
}
|
|
@@ -0,0 +1,165 @@
|
|
|
1
|
+
import { describe, it, expect, mock } from "bun:test";
|
|
2
|
+
|
|
3
|
+
// Stub pi-ai's `stream` so runSummarization can be exercised without a network
|
|
4
|
+
// call. `streamImpl` is swapped per test to simulate primary/fallback outcomes.
|
|
5
|
+
let streamImpl: (model: any) => any = () => {
|
|
6
|
+
throw new Error("streamImpl not set");
|
|
7
|
+
};
|
|
8
|
+
mock.module("@earendil-works/pi-ai", () => ({
|
|
9
|
+
stream: (model: any) => streamImpl(model),
|
|
10
|
+
}));
|
|
11
|
+
|
|
12
|
+
const { summarizeBatch } = await import("./summarizer.js");
|
|
13
|
+
const { FallbackController } = await import("./summarizer-fallback.js");
|
|
14
|
+
const { DEFAULT_CONFIG } = await import("./types.js");
|
|
15
|
+
|
|
16
|
+
const PRIMARY = { id: "primary-model", provider: "provider-a", name: "Primary" };
|
|
17
|
+
const SESSION = { id: "session-model", provider: "provider-b", name: "Session" };
|
|
18
|
+
|
|
19
|
+
const USAGE = {
|
|
20
|
+
input: 1,
|
|
21
|
+
output: 1,
|
|
22
|
+
cacheRead: 0,
|
|
23
|
+
cacheWrite: 0,
|
|
24
|
+
totalTokens: 2,
|
|
25
|
+
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
|
|
26
|
+
};
|
|
27
|
+
|
|
28
|
+
function okStream(text: string) {
|
|
29
|
+
return {
|
|
30
|
+
async *[Symbol.asyncIterator]() {
|
|
31
|
+
// no events; runOnce only needs .result()
|
|
32
|
+
},
|
|
33
|
+
async result() {
|
|
34
|
+
return { stopReason: "stop", content: [{ type: "text", text }], usage: USAGE };
|
|
35
|
+
},
|
|
36
|
+
};
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
function errStream(message: string) {
|
|
40
|
+
return {
|
|
41
|
+
async *[Symbol.asyncIterator]() {},
|
|
42
|
+
async result() {
|
|
43
|
+
return { stopReason: "error", errorMessage: message, content: [], usage: USAGE };
|
|
44
|
+
},
|
|
45
|
+
};
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
interface Note {
|
|
49
|
+
msg: string;
|
|
50
|
+
level: string;
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
function makeCtx(notes: Note[], sessionModel: any = SESSION, primaryModel: any = PRIMARY) {
|
|
54
|
+
return {
|
|
55
|
+
model: sessionModel,
|
|
56
|
+
modelRegistry: {
|
|
57
|
+
find: () => primaryModel,
|
|
58
|
+
getApiKeyAndHeaders: async () => ({ ok: true, apiKey: "k", headers: {} }),
|
|
59
|
+
},
|
|
60
|
+
ui: { notify: (msg: string, level: string) => notes.push({ msg, level }) },
|
|
61
|
+
} as any;
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
function makeBatch() {
|
|
65
|
+
return {
|
|
66
|
+
turnIndex: 0,
|
|
67
|
+
timestamp: 0,
|
|
68
|
+
assistantText: "",
|
|
69
|
+
toolCalls: [
|
|
70
|
+
{ toolCallId: "t1", toolName: "read", args: {}, resultText: "x".repeat(50), isError: false },
|
|
71
|
+
],
|
|
72
|
+
} as any;
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
const distinctConfig = { ...DEFAULT_CONFIG, summarizerModel: "provider-a/primary-model" };
|
|
76
|
+
|
|
77
|
+
describe("runSummarization wiring — same-model no-op (legacy path)", () => {
|
|
78
|
+
it("summarizerModel=default: transient failure notifies error, returns null, controller untouched", async () => {
|
|
79
|
+
streamImpl = () => errStream("provider overloaded");
|
|
80
|
+
const notes: Note[] = [];
|
|
81
|
+
const ctx = makeCtx(notes);
|
|
82
|
+
const controller = new FallbackController();
|
|
83
|
+
const r = await summarizeBatch(makeBatch(), { ...DEFAULT_CONFIG, summarizerModel: "default" }, ctx, {
|
|
84
|
+
controller,
|
|
85
|
+
});
|
|
86
|
+
expect(r).toBeNull();
|
|
87
|
+
expect(controller.inFallback).toBe(false);
|
|
88
|
+
expect(notes).toHaveLength(1);
|
|
89
|
+
expect(notes[0].level).toBe("error");
|
|
90
|
+
expect(notes[0].msg).toContain("provider overloaded");
|
|
91
|
+
});
|
|
92
|
+
});
|
|
93
|
+
|
|
94
|
+
describe("runSummarization wiring — enter fallback", () => {
|
|
95
|
+
it("primary transient + fallback ok: returns summary, one warning, no error notify, sticky", async () => {
|
|
96
|
+
streamImpl = (model) => (model.id === PRIMARY.id ? errStream("down") : okStream("- fallback summary"));
|
|
97
|
+
const notes: Note[] = [];
|
|
98
|
+
const ctx = makeCtx(notes);
|
|
99
|
+
const controller = new FallbackController();
|
|
100
|
+
const r = await summarizeBatch(makeBatch(), distinctConfig, ctx, { controller });
|
|
101
|
+
expect(r?.summaryText).toBe("- fallback summary");
|
|
102
|
+
expect(controller.inFallback).toBe(true);
|
|
103
|
+
const warnings = notes.filter((n) => n.level === "warning");
|
|
104
|
+
const errors = notes.filter((n) => n.level === "error");
|
|
105
|
+
expect(warnings).toHaveLength(1);
|
|
106
|
+
expect(warnings[0].msg).toContain("Primary");
|
|
107
|
+
expect(warnings[0].msg).toContain("Session");
|
|
108
|
+
expect(errors).toHaveLength(0);
|
|
109
|
+
});
|
|
110
|
+
|
|
111
|
+
it("steady-state after enter routes to the session model only (no primary call, no notify)", async () => {
|
|
112
|
+
const seen: string[] = [];
|
|
113
|
+
streamImpl = (model) => {
|
|
114
|
+
seen.push(model.id);
|
|
115
|
+
return model.id === PRIMARY.id ? errStream("down") : okStream("- ok");
|
|
116
|
+
};
|
|
117
|
+
const notes: Note[] = [];
|
|
118
|
+
const ctx = makeCtx(notes);
|
|
119
|
+
const controller = new FallbackController(); // real clock: cooldown (10m) will not elapse in-test
|
|
120
|
+
await summarizeBatch(makeBatch(), distinctConfig, ctx, { controller }); // enter
|
|
121
|
+
seen.length = 0;
|
|
122
|
+
notes.length = 0;
|
|
123
|
+
const r = await summarizeBatch(makeBatch(), distinctConfig, ctx, { controller }); // steady-state
|
|
124
|
+
expect(r?.summaryText).toBe("- ok");
|
|
125
|
+
expect(seen).toEqual([SESSION.id]); // primary never called again before cooldown
|
|
126
|
+
expect(notes).toHaveLength(0);
|
|
127
|
+
});
|
|
128
|
+
});
|
|
129
|
+
|
|
130
|
+
describe("runSummarization wiring — both-down + deferred warning", () => {
|
|
131
|
+
it("primary + fallback both transient: null, error notify, enters fallback with owed warning", async () => {
|
|
132
|
+
streamImpl = () => errStream("everything down");
|
|
133
|
+
const notes: Note[] = [];
|
|
134
|
+
const ctx = makeCtx(notes);
|
|
135
|
+
const controller = new FallbackController();
|
|
136
|
+
const r = await summarizeBatch(makeBatch(), distinctConfig, ctx, { controller });
|
|
137
|
+
expect(r).toBeNull();
|
|
138
|
+
expect(controller.inFallback).toBe(true);
|
|
139
|
+
const warnings = notes.filter((n) => n.level === "warning");
|
|
140
|
+
const errors = notes.filter((n) => n.level === "error");
|
|
141
|
+
expect(warnings).toHaveLength(0); // warning is owed, not yet fired
|
|
142
|
+
expect(errors).toHaveLength(1);
|
|
143
|
+
|
|
144
|
+
// Next flush: fallback now succeeds -> owed warning fires once.
|
|
145
|
+
streamImpl = (model) => (model.id === PRIMARY.id ? errStream("still down") : okStream("- rescued"));
|
|
146
|
+
notes.length = 0;
|
|
147
|
+
const r2 = await summarizeBatch(makeBatch(), distinctConfig, ctx, { controller });
|
|
148
|
+
expect(r2?.summaryText).toBe("- rescued");
|
|
149
|
+
expect(notes.filter((n) => n.level === "warning")).toHaveLength(1);
|
|
150
|
+
});
|
|
151
|
+
});
|
|
152
|
+
|
|
153
|
+
describe("runSummarization wiring — abort", () => {
|
|
154
|
+
it("re-throws when the signal is already aborted", async () => {
|
|
155
|
+
streamImpl = () => okStream("- never");
|
|
156
|
+
const notes: Note[] = [];
|
|
157
|
+
const ctx = makeCtx(notes);
|
|
158
|
+
const controller = new FallbackController();
|
|
159
|
+
const ac = new AbortController();
|
|
160
|
+
ac.abort();
|
|
161
|
+
await expect(
|
|
162
|
+
summarizeBatch(makeBatch(), distinctConfig, ctx, { controller, signal: ac.signal }),
|
|
163
|
+
).rejects.toThrow();
|
|
164
|
+
});
|
|
165
|
+
});
|
package/src/summarizer.ts
CHANGED
|
@@ -10,6 +10,7 @@ import type {
|
|
|
10
10
|
SummarizeResult,
|
|
11
11
|
} from "./types.js";
|
|
12
12
|
import { serializeBatchForSummarizer } from "./batch-capture.js";
|
|
13
|
+
import { FallbackController, type FallbackTransition } from "./summarizer-fallback.js";
|
|
13
14
|
|
|
14
15
|
const SYSTEM_PROMPT = `You are summarizing a batch of tool calls made by an AI coding assistant.
|
|
15
16
|
For each tool call provide:
|
|
@@ -85,30 +86,38 @@ export function isUsableSummary(llmText: string, stopReason: string): boolean {
|
|
|
85
86
|
return llmText.trim().length > 0 && stopReason !== "length";
|
|
86
87
|
}
|
|
87
88
|
|
|
89
|
+
type RunOutcome =
|
|
90
|
+
| { kind: "ok"; result: SummarizeResult }
|
|
91
|
+
| { kind: "auth"; message: string }
|
|
92
|
+
| { kind: "unusable" }
|
|
93
|
+
| { kind: "transient"; message: string };
|
|
94
|
+
|
|
95
|
+
/** Human label for a model in notify text: prefer name, fall back to provider/id. */
|
|
96
|
+
function modelLabel(model: any): string {
|
|
97
|
+
if (!model) return "unknown model";
|
|
98
|
+
return model.name || `${model.provider}/${model.id}`;
|
|
99
|
+
}
|
|
100
|
+
|
|
88
101
|
/**
|
|
89
|
-
*
|
|
90
|
-
*
|
|
91
|
-
*
|
|
92
|
-
*
|
|
93
|
-
*
|
|
102
|
+
* One summarization attempt against a specific model. Returns a classified
|
|
103
|
+
* outcome instead of throwing (except aborts, which propagate so flushPending
|
|
104
|
+
* can restore state). Auth failure is detected pre-stream and never reaches
|
|
105
|
+
* the fallback path. `unusable` = empty or length-truncated. Everything else
|
|
106
|
+
* that reaches the catch is `transient` (the outage bucket) — pi-ai surfaces
|
|
107
|
+
* no structured status code on the throw, so classification is coarse by design.
|
|
94
108
|
*/
|
|
95
|
-
async function
|
|
109
|
+
async function runOnce(
|
|
110
|
+
model: any,
|
|
96
111
|
userMessage: string,
|
|
97
112
|
config: ContextPruneConfig,
|
|
98
113
|
ctx: ExtensionContext,
|
|
99
114
|
options: SummarizeBatchOptions
|
|
100
|
-
): Promise<
|
|
101
|
-
// Fast-fail if already aborted before we even start.
|
|
102
|
-
if (options.signal?.aborted) throw new Error("summarize: aborted before start");
|
|
103
|
-
|
|
115
|
+
): Promise<RunOutcome> {
|
|
104
116
|
try {
|
|
105
|
-
const model = resolveModel(config, ctx);
|
|
106
|
-
|
|
107
117
|
const auth = await ctx.modelRegistry.getApiKeyAndHeaders(model);
|
|
108
118
|
if (!auth.ok) {
|
|
109
119
|
const authMessage = "error" in auth ? auth.error : "authentication failed";
|
|
110
|
-
|
|
111
|
-
return null;
|
|
120
|
+
return { kind: "auth", message: authMessage };
|
|
112
121
|
}
|
|
113
122
|
|
|
114
123
|
// Pass the abort signal so the underlying fetch is cancelled immediately
|
|
@@ -153,12 +162,12 @@ async function runSummarization(
|
|
|
153
162
|
reportTextProgress(response);
|
|
154
163
|
// stopReason "aborted" means the provider cut the stream short (e.g. signal
|
|
155
164
|
// fired just before the final chunk). Treat identically to the signal check
|
|
156
|
-
// above — throw so
|
|
165
|
+
// above — throw so the catch below can detect options.signal.aborted.
|
|
157
166
|
if (response.stopReason === "aborted") {
|
|
158
167
|
throw new Error("summarize: stream stopped with reason aborted");
|
|
159
168
|
}
|
|
160
169
|
if (response.stopReason === "error") {
|
|
161
|
-
|
|
170
|
+
return { kind: "transient", message: response.errorMessage ?? "Summarizer stopped with reason: error" };
|
|
162
171
|
}
|
|
163
172
|
|
|
164
173
|
const llmText = response.content
|
|
@@ -166,21 +175,101 @@ async function runSummarization(
|
|
|
166
175
|
.map((c: any) => c.text)
|
|
167
176
|
.join("\n");
|
|
168
177
|
|
|
169
|
-
if (!isUsableSummary(llmText, response.stopReason)) return
|
|
178
|
+
if (!isUsableSummary(llmText, response.stopReason)) return { kind: "unusable" };
|
|
170
179
|
|
|
171
|
-
return {
|
|
172
|
-
summaryText: llmText,
|
|
173
|
-
usage: response.usage,
|
|
174
|
-
};
|
|
180
|
+
return { kind: "ok", result: { summaryText: llmText, usage: response.usage } };
|
|
175
181
|
} catch (err: any) {
|
|
176
182
|
// Propagate abort errors upward so flushPending can check signal.aborted
|
|
177
183
|
// and return { ok: false, reason: "aborted" } without showing a UI error.
|
|
178
184
|
if (options.signal?.aborted) throw err;
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
185
|
+
return { kind: "transient", message: err.message };
|
|
186
|
+
}
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
/**
|
|
190
|
+
* Shared LLM-call machinery for both per-batch and range summarization.
|
|
191
|
+
* `userMessage` already embeds the relevant system prompt as leading text
|
|
192
|
+
* (the summarizer is a single-user-message call). Returns the formatted text
|
|
193
|
+
* + usage, or null on failure. Abort errors are re-thrown so flushPending can
|
|
194
|
+
* detect options.signal.aborted and restore state without a UI error.
|
|
195
|
+
*
|
|
196
|
+
* When options.controller is set AND a distinct fallback model exists, a
|
|
197
|
+
* transient failure of the configured summarizer model is retried once on the
|
|
198
|
+
* session model, and the controller stays sticky in fallback until a
|
|
199
|
+
* per-cooldown probe of the primary succeeds.
|
|
200
|
+
*/
|
|
201
|
+
async function runSummarization(
|
|
202
|
+
userMessage: string,
|
|
203
|
+
config: ContextPruneConfig,
|
|
204
|
+
ctx: ExtensionContext,
|
|
205
|
+
options: SummarizeBatchOptions
|
|
206
|
+
): Promise<SummarizeResult | null> {
|
|
207
|
+
// Fast-fail if already aborted before we even start.
|
|
208
|
+
if (options.signal?.aborted) throw new Error("summarize: aborted before start");
|
|
209
|
+
|
|
210
|
+
const primary = resolveModel(config, ctx);
|
|
211
|
+
const controller = options.controller;
|
|
212
|
+
const sessionModel = ctx.model;
|
|
213
|
+
|
|
214
|
+
const notifyError = (msg: string) =>
|
|
215
|
+
ctx.ui.notify(`pruner: summarization failed: ${msg}`, "error");
|
|
216
|
+
|
|
217
|
+
// No controller or no distinct fallback: single attempt, legacy behavior.
|
|
218
|
+
if (!controller || !FallbackController.hasDistinctFallback(primary, sessionModel)) {
|
|
219
|
+
const r = await runOnce(primary, userMessage, config, ctx, options);
|
|
220
|
+
switch (r.kind) {
|
|
221
|
+
case "ok":
|
|
222
|
+
return r.result;
|
|
223
|
+
case "auth":
|
|
224
|
+
case "transient":
|
|
225
|
+
notifyError(r.message);
|
|
226
|
+
return null;
|
|
227
|
+
case "unusable":
|
|
228
|
+
return null;
|
|
229
|
+
}
|
|
230
|
+
}
|
|
231
|
+
|
|
232
|
+
const emit = (t: FallbackTransition) => {
|
|
233
|
+
if (t === "enter") {
|
|
234
|
+
ctx.ui.notify(
|
|
235
|
+
`pi-condense: summarizer model ${modelLabel(primary)} failing, using session model ${modelLabel(sessionModel)} until it recovers`,
|
|
236
|
+
"warning"
|
|
237
|
+
);
|
|
238
|
+
} else if (t === "recover") {
|
|
239
|
+
ctx.ui.notify(`pi-condense: summarizer model ${modelLabel(primary)} recovered`, "info");
|
|
240
|
+
}
|
|
241
|
+
};
|
|
242
|
+
|
|
243
|
+
const decision = controller.chooseTarget();
|
|
244
|
+
const model = decision.target === "primary" ? primary : sessionModel;
|
|
245
|
+
const r = await runOnce(model, userMessage, config, ctx, options);
|
|
246
|
+
|
|
247
|
+
switch (r.kind) {
|
|
248
|
+
case "ok":
|
|
249
|
+
if (decision.target === "primary") emit(controller.onPrimarySuccess(decision.wasProbe));
|
|
250
|
+
else emit(controller.onFallbackSuccess());
|
|
251
|
+
return r.result;
|
|
252
|
+
case "auth":
|
|
253
|
+
notifyError(r.message); // auth never trips the controller
|
|
254
|
+
return null;
|
|
255
|
+
case "unusable":
|
|
256
|
+
return null; // probe unusable => stay (no state change)
|
|
257
|
+
case "transient": {
|
|
258
|
+
if (decision.target === "fallback") {
|
|
259
|
+
controller.onFallbackOnlyFail();
|
|
260
|
+
notifyError(r.message);
|
|
261
|
+
return null;
|
|
262
|
+
}
|
|
263
|
+
// target was primary (initial detection or probe): retry once on the session model.
|
|
264
|
+
const r2 = await runOnce(sessionModel, userMessage, config, ctx, options);
|
|
265
|
+
if (r2.kind === "ok") {
|
|
266
|
+
emit(controller.onPrimaryFailFallbackOk(decision.wasProbe));
|
|
267
|
+
return r2.result; // suppress the legacy error notify — fallback rescued the call
|
|
268
|
+
}
|
|
269
|
+
controller.onBothDown();
|
|
270
|
+
notifyError(r2.kind === "transient" || r2.kind === "auth" ? r2.message : r.message);
|
|
271
|
+
return null;
|
|
272
|
+
}
|
|
184
273
|
}
|
|
185
274
|
}
|
|
186
275
|
|
|
@@ -243,6 +332,7 @@ export async function summarizeBatches(
|
|
|
243
332
|
return [
|
|
244
333
|
await summarizeBatch(batches[0], config, ctx, {
|
|
245
334
|
signal: options.signal,
|
|
335
|
+
controller: options.controller,
|
|
246
336
|
onTextProgress: (receivedChars) => {
|
|
247
337
|
options.onBatchTextProgress?.(0, 1, batches[0], receivedChars);
|
|
248
338
|
},
|
|
@@ -255,6 +345,7 @@ export async function summarizeBatches(
|
|
|
255
345
|
batches.map((batch, index) =>
|
|
256
346
|
summarizeBatch(batch, config, ctx, {
|
|
257
347
|
signal: options.signal,
|
|
348
|
+
controller: options.controller,
|
|
258
349
|
onTextProgress: (receivedChars) => {
|
|
259
350
|
options.onBatchTextProgress?.(index, batches.length, batch, receivedChars);
|
|
260
351
|
},
|
package/src/types.ts
CHANGED
|
@@ -42,6 +42,8 @@
|
|
|
42
42
|
* NOT a hidden side-channel. It makes an explicit LLM call from turn_end.
|
|
43
43
|
*/
|
|
44
44
|
|
|
45
|
+
import type { FallbackController } from "./summarizer-fallback.js";
|
|
46
|
+
|
|
45
47
|
// ── Constants ──────────────────────────────────────────────────────────────
|
|
46
48
|
|
|
47
49
|
/** customType for summary custom_message entries (appear in LLM context) */
|
|
@@ -728,6 +730,12 @@ export interface SummarizeBatchOptions {
|
|
|
728
730
|
* batch is treated as aborted (not a summarizer failure).
|
|
729
731
|
*/
|
|
730
732
|
signal?: AbortSignal;
|
|
733
|
+
/**
|
|
734
|
+
* Session-scoped outage-fallback controller. When present AND a distinct
|
|
735
|
+
* fallback model exists, runSummarization routes/retries via the controller
|
|
736
|
+
* (see src/summarizer-fallback.ts). Absent => today's single-attempt behavior.
|
|
737
|
+
*/
|
|
738
|
+
controller?: FallbackController;
|
|
731
739
|
}
|
|
732
740
|
|
|
733
741
|
/** Options for summarizeBatches() when callers want live per-batch text progress. */
|
|
@@ -739,6 +747,12 @@ export interface SummarizeBatchesOptions {
|
|
|
739
747
|
* When fired, all in-flight stream calls are cancelled.
|
|
740
748
|
*/
|
|
741
749
|
signal?: AbortSignal;
|
|
750
|
+
/**
|
|
751
|
+
* Session-scoped outage-fallback controller. When present AND a distinct
|
|
752
|
+
* fallback model exists, runSummarization routes/retries via the controller
|
|
753
|
+
* (see src/summarizer-fallback.ts). Absent => today's single-attempt behavior.
|
|
754
|
+
*/
|
|
755
|
+
controller?: FallbackController;
|
|
742
756
|
}
|
|
743
757
|
|
|
744
758
|
/**
|