pi-tuck 1.0.14 → 1.0.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -3,6 +3,53 @@
3
3
  All notable changes to pi-tuck. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
4
4
  [Semantic Versioning](https://semver.org/).
5
5
 
6
+ ## [1.0.16] - 2026-11-08
7
+
8
+ ### Fixed
9
+
10
+ - **A long session could lose the task it was working on (1.0.0 to 1.0.15).** A summary quoted only the session's first user turn, up to 12,000 characters, and the
11
+ newest one. Every other user turn competed in the shrunk dialogue, which in a long session clips user turns to a few words and drops the oldest. In a session
12
+ that runs task after task, the current task is an older turn followed by short steps, so it was clipped or dropped while a finished first task stayed whole,
13
+ and agents acted on a days-old task. Short sessions, and the SWE-bench chains behind our benchmarks, keep each task's prompt in the kept tail and never hit it.
14
+ A summary now quotes user turns from both ends: first the opening of every recent answered turn, then the first prompt's opening (up to 2,000 characters),
15
+ then more of the newest turns, then the rest of the first prompt. A turn that does not fit keeps its opening and a locator to the rest. The budget is 15% of
16
+ the room and never less than 12,000 characters. When room is tight the quotes shrink step by step: the extensions first, then the openings, then the first
17
+ prompt alone as before. A turn pi-tuck refused and that never got a reply is not quoted.
18
+
19
+ ### Added
20
+
21
+ - The settings stress test checks that the first prompt and the newest 8 answered prompts keep their openings through every cut, with cells for a run of
22
+ tasks at 32,768, tasks followed by long steps, and a long spec followed by 30 steering turns, at 32,768 and 81,920.
23
+
24
+ ### Known issues
25
+
26
+ - At very small free room (a few thousand tokens or less), quotes give way and an older prompt can still lose its opening: 40 findings in 32 stress cells,
27
+ against 447 in 168 cells on 1.0.15. One sampled cell, with about 1,000 tokens free, loses a short middle prompt that 1.0.15 kept.
28
+ - The known issues listed under 1.0.13 still apply.
29
+
30
+ ### Cost
31
+
32
+ On replay of 17 real single-task sessions, cut rate and cost are unchanged: 7.0 cuts per 100 requests at 131,072 for both builds, and 9.3 at 81,920 (838 cuts
33
+ against 837); billed tokens and request time within 0.1%. Those sessions have a median of one prompt, so the new quotes rarely apply there; their cost in a
34
+ session of hundreds of turns is not measured.
35
+
36
+ ### Release gates
37
+
38
+ - **Shipped without a live gate, as a fix for lost work.** Replay differs from 1.0.15, because summaries now quote more: 11,860 of 13,946 requests on 29
39
+ recordings, with 0 dead tasks, 0 refused requests and every clip read back (7 clips, where 1.0.15 had 28: summaries changed where cuts land). Tests; CI;
40
+ the stress test on Pi 1.2.0 and the newest Pi.
41
+ - The 32k regression bench makes 74 cuts in 480 rounds, up from 67: sessions with a short first prompt now quote their newest turns too.
42
+
43
+ ## [1.0.15] - 2026-11-08
44
+
45
+ ### Fixed
46
+
47
+ - `tuck_search` no longer fails a whole search over one bad `priorResults` entry. An entry that does not match the history, such as a made-up hash or an id that is not an entry's, is now ignored, and the notice says how many were. Before, the search returned "Invalid explicit prior source range", and the agent fell back to word search.
48
+
49
+ ### Release gates
50
+
51
+ - **Shipped without a live gate, because its replay matches 1.0.14:** 13,946 requests on 29 recordings, 0 differ, 0 dead tasks, the same 28 clips. Tests; CI; the stress test on Pi 1.2.0 and the newest Pi, with the same 3 of 274 cells.
52
+
6
53
  ## [1.0.14] - 2026-11-07
7
54
 
8
55
  ### Changed
package/README.md CHANGED
@@ -64,8 +64,8 @@ Then start a new Pi session. That is all. pi-tuck takes over Pi's compaction, bo
64
64
 
65
65
  - Needs Pi 1.2.0 or later (tested on 1.2.0 and 1.4.0), on macOS or Linux. On an older Pi, pi-tuck stays off, Pi's own compaction runs, and pi-tuck says so once. Full features on macOS with Apple silicon; elsewhere, compaction and word recall.
66
66
  - On macOS with Apple silicon, the first session also downloads a local search model in the background (about 390 MB, once). Until it is ready, search works by keyword. See [Semantic search](#semantic-search).
67
- - Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.13`.
68
- - To pin a version, add it: `pi install npm:pi-tuck@1.0.13`.
67
+ - Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.15`.
68
+ - To pin a version, add it: `pi install npm:pi-tuck@1.0.15`.
69
69
 
70
70
  ## How it works
71
71
 
@@ -1 +1 @@
1
- export declare const BUILD_ID = "9457180f73cad9d9bde254328dad70dd2abe8400e373a68e06ad455b5b66f203";
1
+ export declare const BUILD_ID = "e695eee2b3e13753eb135cb61b69783a79d72ac4c1ce08f28304b98cca6b5a49";
@@ -1 +1 @@
1
- export const BUILD_ID = "9457180f73cad9d9bde254328dad70dd2abe8400e373a68e06ad455b5b66f203";
1
+ export const BUILD_ID = "e695eee2b3e13753eb135cb61b69783a79d72ac4c1ce08f28304b98cca6b5a49";
@@ -1,6 +1,6 @@
1
1
  import { type RecallPolicy } from './recall.js';
2
2
  import type { SessionEntry } from '@earendil-works/pi-coding-agent';
3
- export declare const DIAGNOSTIC_BUILD = "9457180f73cad9d9bde254328dad70dd2abe8400e373a68e06ad455b5b66f203";
3
+ export declare const DIAGNOSTIC_BUILD = "e695eee2b3e13753eb135cb61b69783a79d72ac4c1ce08f28304b98cca6b5a49";
4
4
  export interface Receipt {
5
5
  id: string;
6
6
  build: string;
@@ -917,13 +917,10 @@ export class SemanticRecallController {
917
917
  const finalSources = new Map(final.sources.map(source => [source.entry.id, source.sha]));
918
918
  if (snapshot.sources.some(source => finalSources.get(source.entry.id) !== source.sha))
919
919
  throw new Error('Eligible source changed or access revoked before return');
920
- for (const reference of prior) {
921
- const source = snapshot.sources.find(s => s.entry.id === reference.entryId);
922
- if (!Number.isSafeInteger(reference.start) || !Number.isSafeInteger(reference.end) || reference.start > reference.end || !source || reference.sourceSha !== source.sha || reference.end > source.sourceText.length)
923
- throw new Error('Invalid explicit prior source range');
924
- }
925
- const previous = [...prior, ...(words?.hits ?? []).map((hit) => ({ entryId: hit.entryId, start: hit.read.offset, end: hit.read.offset + hit.read.limit, sourceSha: snapshot.sources.find(s => s.entry.id === hit.entryId)?.sha }))];
926
- return { status: 'ok', notice: 'Related original evidence, not current truth or proof that every qualification was found. Scores do not establish authority, event time or supersession. Sources preserve recorded time and original order.' + ` Semantic ranking covers ${covered.conversation.indexedCharacters} of ${covered.conversation.outOfViewCharacters} characters of earlier conversation and ${covered.toolResults.indexedCharacters} of ${covered.toolResults.outOfViewCharacters} characters of earlier tool output (newest first, back to ${covered.toolResults.oldestCoveredTime ?? covered.oldestCoveredTime ?? 'an unknown time'}); use tuck_recall for the rest.`, coverage: { ...this.status(), window: covered }, method: 'embedding', groups: this.assemble(ranked.map(hit => hit.unit), snapshot, previous) };
920
+ // What the caller already read only spares it a repeat. A reference that does not match this history spares nothing, and it must not fail the search.
921
+ const usable = prior.filter(reference => { const source = snapshot.sources.find(s => s.entry.id === reference.entryId); return Number.isSafeInteger(reference.start) && Number.isSafeInteger(reference.end) && reference.start <= reference.end && source !== undefined && reference.sourceSha === source.sha && reference.end <= source.sourceText.length; }), ignored = prior.length - usable.length;
922
+ const previous = [...usable, ...(words?.hits ?? []).map((hit) => ({ entryId: hit.entryId, start: hit.read.offset, end: hit.read.offset + hit.read.limit, sourceSha: snapshot.sources.find(s => s.entry.id === hit.entryId)?.sha }))];
923
+ return { status: 'ok', notice: 'Related original evidence, not current truth or proof that every qualification was found. Scores do not establish authority, event time or supersession. Sources preserve recorded time and original order.' + (ignored ? ` ${ignored} of the priorResults did not match this history and were ignored, so their text may appear again.` : '') + ` Semantic ranking covers ${covered.conversation.indexedCharacters} of ${covered.conversation.outOfViewCharacters} characters of earlier conversation and ${covered.toolResults.indexedCharacters} of ${covered.toolResults.outOfViewCharacters} characters of earlier tool output (newest first, back to ${covered.toolResults.oldestCoveredTime ?? covered.oldestCoveredTime ?? 'an unknown time'}); use tuck_recall for the rest.`, coverage: { ...this.status(), window: covered }, method: 'embedding', groups: this.assemble(ranked.map(hit => hit.unit), snapshot, previous) };
927
924
  }
928
925
  catch (error) {
929
926
  if (error instanceof Error && error.name === 'SemanticAccessError') {
package/dist/src/tuck.js CHANGED
@@ -166,22 +166,75 @@ export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, poli
166
166
  receiptContext.messages.push(entry.message);
167
167
  receiptContext.sourceIndices.push(indices.get(entry.id));
168
168
  }
169
- const mandatory = [];
170
- // The initiating prompt is quoted first, like the current input below: text only, since quoting cannot deliver other blocks.
169
+ // The newest user turns, newest first, then the initiating prompt, share one budget: in a session that runs task after task, the current task is
170
+ // an older user turn followed by short steps, and the dialogue clips user turns oldest first down to a few words, so quoting only the first prompt
171
+ // kept a finished task whole and lost the one being worked on. A turn that does not fit keeps its head and a locator; the first prompt gets what
172
+ // the newest turns leave, so a one-task session, whose later turns are short, still quotes it as before.
171
173
  const initiating = firstUserPosition >= 0 && firstUserPosition < boundary && entries[firstUserPosition].id !== protectedUser?.id ? entries[firstUserPosition] : undefined;
172
- const initiatingContent = initiating?.type === "message" && initiating.message.role === "user" ? initiating.message.content : undefined;
173
- if (initiating && isTextOnly(initiatingContent)) {
174
- const text = textOf(initiatingContent), index = indices.get(initiating.id);
174
+ const quote = (entry, budget, label) => {
175
+ const content = entry.type === "message" && entry.message.role === "user" ? entry.message.content : undefined;
176
+ if (!isTextOnly(content))
177
+ return [];
178
+ const text = textOf(content), index = indices.get(entry.id);
175
179
  if (index === undefined)
176
- throw new Error("Missing initiating user source index");
177
- const quoted = head(text, MAX_TASK_CONTRACT_CHARACTERS), sourceTimes = [{ sourceIndex: index, ...resolveSourceTime(initiating) }];
178
- if (text.trim()) {
179
- mandatory.push({ kind: "excerpt", role: "user", sourceIndex: index, text: quoted, sourceTimes });
180
- if (quoted.length < text.length)
181
- mandatory.push({ kind: "omission", role: "user", sourceIndex: index,
182
- reason: `Initiating prompt continues: characters ${quoted.length}-${text.length} omitted; read them with tuck_recall {"index":${index},"offset":${quoted.length}}`, sourceTimes });
180
+ throw new Error("Missing quoted user source index");
181
+ const quoted = head(text, budget), sourceTimes = [{ sourceIndex: index, ...resolveSourceTime(entry) }];
182
+ if (!text.trim() || !quoted.length)
183
+ return [];
184
+ return [{ kind: "excerpt", role: "user", sourceIndex: index, text: quoted, sourceTimes }, ...(quoted.length < text.length ? [{ kind: "omission", role: "user",
185
+ sourceIndex: index, reason: `${label} continues: characters ${quoted.length}-${text.length} omitted; read them with tuck_recall {"index":${index},"offset":${quoted.length}}`, sourceTimes }] : [])];
186
+ };
187
+ // Both ends are anchored, in this order: the heads of the newest user turns (at most a quarter of the budget), the head of the first prompt (the
188
+ // goal of a long session), then extensions of the newest turns, then the rest of the first prompt. Heads come first so steps after a task can't
189
+ // use up the budget the task turn needs; the first prompt's head comes before any extension so steering turns can't use up the goal's.
190
+ const quotesFor = (budget, headSize, withFirst) => {
191
+ const turns = [];
192
+ // Only answered turns: a prompt that was refused and never got a reply was never taken up, and quoting it would spend the budget for nothing.
193
+ let answered = false;
194
+ for (let at = entries.length - 1; at >= 0; at--) {
195
+ const entry = entries[at];
196
+ if (entry.type === "message" && entry.message.role === "assistant") {
197
+ answered = true;
198
+ continue;
199
+ }
200
+ if (entry.type !== "message" || entry.message.role !== "user")
201
+ continue;
202
+ const replied = answered;
203
+ answered = false;
204
+ if (at >= boundary || !replied || entry.id === protectedUser?.id || entry.id === firstUser?.id || !isTextOnly(entry.message.content))
205
+ continue;
206
+ const text = textOf(entry.message.content);
207
+ if (text.trim())
208
+ turns.push({ entry, text, take: 0 });
183
209
  }
184
- }
210
+ // headSize 0 quotes the first prompt alone, as before any user turn was quoted: the last step before no quote at all.
211
+ if (headSize === 0)
212
+ turns.length = 0;
213
+ let left = budget, heads = Math.floor(budget / 4);
214
+ for (const turn of turns) {
215
+ if (heads <= 0)
216
+ break;
217
+ turn.take = Math.min(turn.text.length, headSize, heads);
218
+ heads -= turn.take;
219
+ left -= turn.take;
220
+ }
221
+ const firstText = withFirst && initiating?.type === "message" && initiating.message.role === "user" && isTextOnly(initiating.message.content) ? textOf(initiating.message.content) : "";
222
+ let firstTake = Math.min(firstText.length, headSize ? Math.round(headSize * 20 / 3) : budget, Math.max(0, left));
223
+ left -= firstTake;
224
+ for (const turn of turns) {
225
+ if (left <= 0)
226
+ break;
227
+ const more = Math.min(turn.text.length - turn.take, left);
228
+ turn.take += more;
229
+ left -= more;
230
+ }
231
+ firstTake += Math.min(firstText.length - firstTake, Math.max(0, left));
232
+ const records = turns.filter(turn => turn.take > 0).reverse().flatMap(turn => quote(turn.entry, turn.take, "User turn"));
233
+ if (firstTake > 0 && initiating)
234
+ records.unshift(...quote(initiating, firstTake, "Initiating prompt"));
235
+ return records;
236
+ };
237
+ const mandatory = [];
185
238
  if (protectedUser?.type === "message" && protectedUser.message.role === "user")
186
239
  mandatory.push({ kind: "excerpt", role: "user", sourceIndex: currentIndex, text: textOf(protectedUser.message.content), sourceTimes: [{ sourceIndex: currentIndex, ...resolveSourceTime(protectedUser) }] });
187
240
  // The file ledger and the action trail of the compacted history are mandatory, so they are charged in base like the quotes above.
@@ -200,22 +253,43 @@ export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, poli
200
253
  };
201
254
  const before = measure(projected.messages);
202
255
  const model = ctx.model;
203
- const floorAt = (keep) => {
256
+ const floorAt = (keep, choice) => {
204
257
  const ledger = [...actionRecords(actions, keep), ...ledgerRecords(items, keep)];
205
- const messages = preview(serializeTranscript([...mandatory, ...ledger]) + DIALOGUE_SUFFIX);
206
- return { ledger, messages, base: measure(messages) };
258
+ const messages = preview(serializeTranscript([...mandatory, ...quotesFor(choice.budget, choice.head, choice.first), ...ledger]) + DIALOGUE_SUFFIX);
259
+ return { ledger, messages, base: measure(messages), choice };
207
260
  };
208
261
  const fits = (base) => Boolean(model) && base + reserveTokens < model.contextWindow && base < before;
209
- // When the whole floor cannot fit, the trail and the ledger keep only their headings and counts; only that can refuse the cut.
210
- let floor = floorAt(Math.max(items.length, actions.length));
211
- if (!fits(floor.base) && (items.length || actions.length))
212
- floor = floorAt(0);
262
+ // When room is tight the newest turns win over the first prompt, which in a long-running session is often a finished task. The floor gives way
263
+ // in this order: the trail and ledger to their headings, the extensions (the budget halved), then the heads, the first prompt's among them
264
+ // (300 characters, 2,000 for the first prompt; then 100 and 667), then everything but the first prompt (as before user turns were quoted),
265
+ // and only then every quote. Only a floor with no quote but the current input can refuse the cut.
266
+ // The budget grows with the room: 15% of it, at least the 12,000 characters a first prompt always had. A long-running session's task can be tens
267
+ // of thousands of characters, and the room is what can afford it.
268
+ const taskBudget = Math.max(MAX_TASK_CONTRACT_CHARACTERS, Math.floor(Math.max(0, (model?.contextWindow ?? 0) - reserveTokens) * charsPerToken * 0.15));
269
+ const budgets = [taskBudget, taskBudget / 2, taskBudget / 4, taskBudget / 8, 1500].map(Math.floor).filter((b, at, all) => b >= 1500 && all.indexOf(b) === at);
270
+ const choices = [{ head: 300, budgets }, { head: 100, budgets: [1500, 600] }]
271
+ .flatMap(({ head, budgets }) => budgets.map(budget => ({ budget, head, first: true }))).concat([{ budget: MAX_TASK_CONTRACT_CHARACTERS, head: 0, first: true }, { budget: 0, head: 0, first: false }]);
272
+ // When even the smallest floor cannot fit, no step of the ladder can: refuse at once instead of building every preview.
273
+ const smallest = floorAt(0, choices[choices.length - 1]);
274
+ let floor = smallest;
275
+ if (fits(smallest.base))
276
+ for (const choice of choices) {
277
+ floor = floorAt(Math.max(items.length, actions.length), choice);
278
+ if (fits(floor.base))
279
+ break;
280
+ if (items.length || actions.length) {
281
+ floor = floorAt(0, choice);
282
+ if (fits(floor.base))
283
+ break;
284
+ }
285
+ }
213
286
  const { ledger, base } = floor;
287
+ const kept = [...mandatory, ...quotesFor(floor.choice.budget, floor.choice.head, floor.choice.first)];
214
288
  if (!model || !fits(base))
215
289
  return { unavailable: "pi-tuck's mandatory context cannot fit or reduce the request budget", capacityRejected: Boolean(model && base + reserveTokens >= model.contextWindow), floorMessages: floor.messages };
216
290
  const maxBriefChars = historicalAllowance(model.contextWindow, reserveTokens, base, before);
217
291
  const compileInput = { messages, sourceIndices, sourceEntryIds: originalCut.selectedIds, sourceTimes: originalCut.selectedIds.map(id => resolveSourceTime(entryById.get(id))), receiptContext,
218
- ranking: { maxBriefChars, mandatorySourceIndices: new Set(mandatory.flatMap(r => r.sourceIndex === undefined ? [] : [r.sourceIndex])) }, };
292
+ ranking: { maxBriefChars, mandatorySourceIndices: new Set(kept.flatMap(r => r.sourceIndex === undefined ? [] : [r.sourceIndex])) }, };
219
293
  // Both views use the same parser, mandatory insertion, native preview and guard, and end with the same closing section.
220
294
  const assemble = (compiled) => {
221
295
  if (typeof compiled !== "string")
@@ -224,7 +298,7 @@ export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, poli
224
298
  if (!split)
225
299
  throw new Error("pi-tuck returned an invalid summary");
226
300
  const records = parseRecords(split.body, split.version);
227
- for (const record of mandatory) {
301
+ for (const record of kept) {
228
302
  const insertion = records.findIndex(row => row.sourceIndex !== undefined && row.sourceIndex > record.sourceIndex);
229
303
  records.splice(insertion < 0 ? records.length : insertion, 0, record);
230
304
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-tuck",
3
- "version": "1.0.14",
3
+ "version": "1.0.16",
4
4
  "description": "Deterministic compaction for Pi with no model call, plus exact recall of earlier history. Search by meaning on Apple silicon.",
5
5
  "keywords": [
6
6
  "pi-package",
package/src/build-id.ts CHANGED
@@ -1 +1 @@
1
- export const BUILD_ID = "9457180f73cad9d9bde254328dad70dd2abe8400e373a68e06ad455b5b66f203";
1
+ export const BUILD_ID = "e695eee2b3e13753eb135cb61b69783a79d72ac4c1ce08f28304b98cca6b5a49";
@@ -362,7 +362,8 @@ export class SemanticRecallController {
362
362
  if(!snapshot.units.length)return{status:'no_matches',notice:'No eligible original source units in this snapshot.',coverage:this.status(),method:'embedding',groups:[]};
363
363
  try{const worker=await this.runtime(snapshot.project);this.recheck(provider,snapshot);const encoded=await worker.call('embed',{texts:[query],kind:'query'},signal);this.operations.queryEmbeddings++;this.recheck(provider,snapshot);const prefix=indexedPrefix(snapshot),stored=snapshot.units.slice(0,prefix),byKey=new Map(stored.map(u=>[u.key,u])),covered=windowCoverage(snapshot,prefix)!;let ranked:{unit:Unit;score:number}[]=[],observed=0;for(let offset=0;offset<stored.length;offset+=SCORE_CHUNK){this.recheck(provider,snapshot);const scored=await this.index('score',snapshot,{keys:stored.slice(offset,offset+SCORE_CHUNK).map(u=>u.key),query:encoded.vectors[0]}) as {ranked:{key:string;score:number}[];observed:number};observed+=scored.observed;ranked.push(...scored.ranked.map(row=>({unit:byKey.get(row.key)!,score:row.score})));ranked.sort((a,b)=>b.score-a.score);ranked=ranked.slice(0,24);if(signal?.aborted)throw abort();await yieldWork();}if(observed!==stored.length)throw new Error('Snapshot index coverage changed');
364
364
  if(signal?.aborted||queryGeneration!==this.generation||!this.enabled)throw abort();this.recheck(provider,snapshot);const final=provider();const finalSources=new Map(final.sources.map(source=>[source.entry.id,source.sha]));if(snapshot.sources.some(source=>finalSources.get(source.entry.id)!==source.sha))throw new Error('Eligible source changed or access revoked before return');
365
- for(const reference of prior){const source=snapshot.sources.find(s=>s.entry.id===reference.entryId);if(!Number.isSafeInteger(reference.start)||!Number.isSafeInteger(reference.end)||reference.start>reference.end||!source||reference.sourceSha!==source.sha||reference.end>source.sourceText.length)throw new Error('Invalid explicit prior source range');}const previous=[...prior,...(words?.hits??[]).map((hit:any)=>({entryId:hit.entryId,start:hit.read.offset,end:hit.read.offset+hit.read.limit,sourceSha:snapshot.sources.find(s=>s.entry.id===hit.entryId)?.sha}))];return{status:'ok',notice:'Related original evidence, not current truth or proof that every qualification was found. Scores do not establish authority, event time or supersession. Sources preserve recorded time and original order.'+` Semantic ranking covers ${covered.conversation.indexedCharacters} of ${covered.conversation.outOfViewCharacters} characters of earlier conversation and ${covered.toolResults.indexedCharacters} of ${covered.toolResults.outOfViewCharacters} characters of earlier tool output (newest first, back to ${covered.toolResults.oldestCoveredTime??covered.oldestCoveredTime??'an unknown time'}); use tuck_recall for the rest.`,coverage:{...this.status(),window:covered},method:'embedding',groups:this.assemble(ranked.map(hit=>hit.unit),snapshot,previous)};
365
+ // What the caller already read only spares it a repeat. A reference that does not match this history spares nothing, and it must not fail the search.
366
+ const usable=prior.filter(reference=>{const source=snapshot.sources.find(s=>s.entry.id===reference.entryId);return Number.isSafeInteger(reference.start)&&Number.isSafeInteger(reference.end)&&reference.start<=reference.end&&source!==undefined&&reference.sourceSha===source.sha&&reference.end<=source.sourceText.length;}),ignored=prior.length-usable.length;const previous=[...usable,...(words?.hits??[]).map((hit:any)=>({entryId:hit.entryId,start:hit.read.offset,end:hit.read.offset+hit.read.limit,sourceSha:snapshot.sources.find(s=>s.entry.id===hit.entryId)?.sha}))];return{status:'ok',notice:'Related original evidence, not current truth or proof that every qualification was found. Scores do not establish authority, event time or supersession. Sources preserve recorded time and original order.'+(ignored?` ${ignored} of the priorResults did not match this history and were ignored, so their text may appear again.`:'')+` Semantic ranking covers ${covered.conversation.indexedCharacters} of ${covered.conversation.outOfViewCharacters} characters of earlier conversation and ${covered.toolResults.indexedCharacters} of ${covered.toolResults.outOfViewCharacters} characters of earlier tool output (newest first, back to ${covered.toolResults.oldestCoveredTime??covered.oldestCoveredTime??'an unknown time'}); use tuck_recall for the rest.`,coverage:{...this.status(),window:covered},method:'embedding',groups:this.assemble(ranked.map(hit=>hit.unit),snapshot,previous)};
366
367
  }catch(error){if(error instanceof Error&&error.name==='SemanticAccessError'){await this.revoked(error);throw error;}if(signal?.aborted||queryGeneration!==this.generation||error instanceof Error&&error.name==='AbortError'){await this.cancelled();throw abort();}this.heal(error);return{status:'unavailable',notice:'Local semantic search failed; ordinary recall remains available. '+(error instanceof Error?error.message:'failure'),coverage:this.status(),groups:[]};}
367
368
  }
368
369
  private assemble(units:Unit[],snapshot:Snapshot,previous:PriorResult[]):SemanticGroup[]{
package/src/tuck.ts CHANGED
@@ -187,20 +187,52 @@ export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEnt
187
187
  for (const entry of entries) if (entry.type === "message") {
188
188
  receiptContext.messages.push(entry.message); receiptContext.sourceIndices.push(indices.get(entry.id));
189
189
  }
190
- const mandatory: TranscriptRecord[] = [];
191
- // The initiating prompt is quoted first, like the current input below: text only, since quoting cannot deliver other blocks.
190
+ // The newest user turns, newest first, then the initiating prompt, share one budget: in a session that runs task after task, the current task is
191
+ // an older user turn followed by short steps, and the dialogue clips user turns oldest first down to a few words, so quoting only the first prompt
192
+ // kept a finished task whole and lost the one being worked on. A turn that does not fit keeps its head and a locator; the first prompt gets what
193
+ // the newest turns leave, so a one-task session, whose later turns are short, still quotes it as before.
192
194
  const initiating = firstUserPosition >= 0 && firstUserPosition < boundary && entries[firstUserPosition]!.id !== protectedUser?.id ? entries[firstUserPosition] : undefined;
193
- const initiatingContent = initiating?.type === "message" && initiating.message.role === "user" ? initiating.message.content : undefined;
194
- if (initiating && isTextOnly(initiatingContent)) {
195
- const text = textOf(initiatingContent!), index = indices.get(initiating.id);
196
- if (index === undefined) throw new Error("Missing initiating user source index");
197
- const quoted = head(text, MAX_TASK_CONTRACT_CHARACTERS), sourceTimes = [{ sourceIndex: index, ...resolveSourceTime(initiating) }];
198
- if (text.trim()) {
199
- mandatory.push({ kind: "excerpt", role: "user", sourceIndex: index, text: quoted, sourceTimes });
200
- if (quoted.length < text.length) mandatory.push({ kind: "omission", role: "user", sourceIndex: index,
201
- reason: `Initiating prompt continues: characters ${quoted.length}-${text.length} omitted; read them with tuck_recall {"index":${index},"offset":${quoted.length}}`, sourceTimes });
195
+ const quote = (entry: SessionEntry, budget: number, label: string): TranscriptRecord[] => {
196
+ const content = entry.type === "message" && entry.message.role === "user" ? entry.message.content : undefined;
197
+ if (!isTextOnly(content)) return [];
198
+ const text = textOf(content!), index = indices.get(entry.id);
199
+ if (index === undefined) throw new Error("Missing quoted user source index");
200
+ const quoted = head(text, budget), sourceTimes = [{ sourceIndex: index, ...resolveSourceTime(entry) }];
201
+ if (!text.trim() || !quoted.length) return [];
202
+ return [{ kind: "excerpt", role: "user", sourceIndex: index, text: quoted, sourceTimes }, ...(quoted.length < text.length ? [{ kind: "omission" as const, role: "user" as const,
203
+ sourceIndex: index, reason: `${label} continues: characters ${quoted.length}-${text.length} omitted; read them with tuck_recall {"index":${index},"offset":${quoted.length}}`, sourceTimes }] : [])];
204
+ };
205
+ // Both ends are anchored, in this order: the heads of the newest user turns (at most a quarter of the budget), the head of the first prompt (the
206
+ // goal of a long session), then extensions of the newest turns, then the rest of the first prompt. Heads come first so steps after a task can't
207
+ // use up the budget the task turn needs; the first prompt's head comes before any extension so steering turns can't use up the goal's.
208
+ const quotesFor = (budget: number, headSize: number, withFirst: boolean): TranscriptRecord[] => {
209
+ const turns: { entry: SessionEntry; text: string; take: number }[] = [];
210
+ // Only answered turns: a prompt that was refused and never got a reply was never taken up, and quoting it would spend the budget for nothing.
211
+ let answered = false;
212
+ for (let at = entries.length - 1; at >= 0; at--) {
213
+ const entry = entries[at]!;
214
+ if (entry.type === "message" && entry.message.role === "assistant") { answered = true; continue; }
215
+ if (entry.type !== "message" || entry.message.role !== "user") continue;
216
+ const replied = answered;
217
+ answered = false;
218
+ if (at >= boundary || !replied || entry.id === protectedUser?.id || entry.id === firstUser?.id || !isTextOnly(entry.message.content)) continue;
219
+ const text = textOf(entry.message.content);
220
+ if (text.trim()) turns.push({ entry, text, take: 0 });
202
221
  }
203
- }
222
+ // headSize 0 quotes the first prompt alone, as before any user turn was quoted: the last step before no quote at all.
223
+ if (headSize === 0) turns.length = 0;
224
+ let left = budget, heads = Math.floor(budget / 4);
225
+ for (const turn of turns) { if (heads <= 0) break; turn.take = Math.min(turn.text.length, headSize, heads); heads -= turn.take; left -= turn.take; }
226
+ const firstText = withFirst && initiating?.type === "message" && initiating.message.role === "user" && isTextOnly(initiating.message.content) ? textOf(initiating.message.content) : "";
227
+ let firstTake = Math.min(firstText.length, headSize ? Math.round(headSize * 20 / 3) : budget, Math.max(0, left));
228
+ left -= firstTake;
229
+ for (const turn of turns) { if (left <= 0) break; const more = Math.min(turn.text.length - turn.take, left); turn.take += more; left -= more; }
230
+ firstTake += Math.min(firstText.length - firstTake, Math.max(0, left));
231
+ const records = turns.filter(turn => turn.take > 0).reverse().flatMap(turn => quote(turn.entry, turn.take, "User turn"));
232
+ if (firstTake > 0 && initiating) records.unshift(...quote(initiating, firstTake, "Initiating prompt"));
233
+ return records;
234
+ };
235
+ const mandatory: TranscriptRecord[] = [];
204
236
  if (protectedUser?.type === "message" && protectedUser.message.role === "user")
205
237
  mandatory.push({ kind: "excerpt", role: "user", sourceIndex: currentIndex, text: textOf(protectedUser.message.content), sourceTimes: [{ sourceIndex: currentIndex!, ...resolveSourceTime(protectedUser) }] });
206
238
  // The file ledger and the action trail of the compacted history are mandatory, so they are charged in base like the quotes above.
@@ -218,21 +250,37 @@ export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEnt
218
250
  };
219
251
  const before = measure(projected.messages);
220
252
  const model = ctx.model;
221
- const floorAt = (keep: number) => {
253
+ const floorAt = (keep: number, choice: { budget: number; head: number; first: boolean }) => {
222
254
  const ledger = [...actionRecords(actions, keep), ...ledgerRecords(items, keep)];
223
- const messages = preview(serializeTranscript([...mandatory, ...ledger]) + DIALOGUE_SUFFIX);
224
- return { ledger, messages, base: measure(messages) };
255
+ const messages = preview(serializeTranscript([...mandatory, ...quotesFor(choice.budget, choice.head, choice.first), ...ledger]) + DIALOGUE_SUFFIX);
256
+ return { ledger, messages, base: measure(messages), choice };
225
257
  };
226
258
  const fits = (base: number) => Boolean(model) && base + reserveTokens < model!.contextWindow && base < before;
227
- // When the whole floor cannot fit, the trail and the ledger keep only their headings and counts; only that can refuse the cut.
228
- let floor = floorAt(Math.max(items.length, actions.length));
229
- if (!fits(floor.base) && (items.length || actions.length)) floor = floorAt(0);
259
+ // When room is tight the newest turns win over the first prompt, which in a long-running session is often a finished task. The floor gives way
260
+ // in this order: the trail and ledger to their headings, the extensions (the budget halved), then the heads, the first prompt's among them
261
+ // (300 characters, 2,000 for the first prompt; then 100 and 667), then everything but the first prompt (as before user turns were quoted),
262
+ // and only then every quote. Only a floor with no quote but the current input can refuse the cut.
263
+ // The budget grows with the room: 15% of it, at least the 12,000 characters a first prompt always had. A long-running session's task can be tens
264
+ // of thousands of characters, and the room is what can afford it.
265
+ const taskBudget = Math.max(MAX_TASK_CONTRACT_CHARACTERS, Math.floor(Math.max(0, (model?.contextWindow ?? 0) - reserveTokens) * charsPerToken * 0.15));
266
+ const budgets = [taskBudget, taskBudget / 2, taskBudget / 4, taskBudget / 8, 1500].map(Math.floor).filter((b, at, all) => b >= 1500 && all.indexOf(b) === at);
267
+ const choices = [{ head: 300, budgets }, { head: 100, budgets: [1500, 600] }]
268
+ .flatMap(({ head, budgets }) => budgets.map(budget => ({ budget, head, first: true }))).concat([{ budget: MAX_TASK_CONTRACT_CHARACTERS, head: 0, first: true }, { budget: 0, head: 0, first: false }]);
269
+ // When even the smallest floor cannot fit, no step of the ladder can: refuse at once instead of building every preview.
270
+ const smallest = floorAt(0, choices[choices.length - 1]!);
271
+ let floor = smallest;
272
+ if (fits(smallest.base)) for (const choice of choices) {
273
+ floor = floorAt(Math.max(items.length, actions.length), choice);
274
+ if (fits(floor.base)) break;
275
+ if (items.length || actions.length) { floor = floorAt(0, choice); if (fits(floor.base)) break; }
276
+ }
230
277
  const { ledger, base } = floor;
278
+ const kept = [...mandatory, ...quotesFor(floor.choice.budget, floor.choice.head, floor.choice.first)];
231
279
  if (!model || !fits(base))
232
280
  return { unavailable: "pi-tuck's mandatory context cannot fit or reduce the request budget", capacityRejected: Boolean(model && base + reserveTokens >= model.contextWindow), floorMessages: floor.messages };
233
281
  const maxBriefChars = historicalAllowance(model.contextWindow, reserveTokens, base, before);
234
282
  const compileInput = { messages, sourceIndices, sourceEntryIds: originalCut.selectedIds, sourceTimes: originalCut.selectedIds.map(id => resolveSourceTime(entryById.get(id)!)), receiptContext,
235
- ranking: { maxBriefChars, mandatorySourceIndices: new Set(mandatory.flatMap(r => r.sourceIndex === undefined ? [] : [r.sourceIndex])) },
283
+ ranking: { maxBriefChars, mandatorySourceIndices: new Set(kept.flatMap(r => r.sourceIndex === undefined ? [] : [r.sourceIndex])) },
236
284
  };
237
285
  // Both views use the same parser, mandatory insertion, native preview and guard, and end with the same closing section.
238
286
  const assemble = (compiled: string) => {
@@ -240,7 +288,7 @@ export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEnt
240
288
  const split = compiled ? splitSummary(compiled) : { body: "", version: 2 as const };
241
289
  if (!split) throw new Error("pi-tuck returned an invalid summary");
242
290
  const records = parseRecords(split.body, split.version);
243
- for (const record of mandatory) {
291
+ for (const record of kept) {
244
292
  const insertion = records.findIndex(row => row.sourceIndex !== undefined && row.sourceIndex > record.sourceIndex!);
245
293
  records.splice(insertion < 0 ? records.length : insertion, 0, record);
246
294
  }