pi-tuck 1.0.5 → 1.0.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -3,6 +3,23 @@
3
3
  All notable changes to pi-tuck. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
4
4
  [Semantic Versioning](https://semver.org/).
5
5
 
6
+ ## [1.0.6] - 2026-11-01
7
+
8
+ ### Changed
9
+
10
+ - Semantic recall embeds on the GPU (Metal) on Apple silicon. The pinned llama.cpp build already includes Metal, but pi-tuck started it on the
11
+ CPU only. On an Apple M5 under heavy load (load average 44-76), a batch of 8 passages took a median 38.6 s on the CPU and 0.69 s on Metal,
12
+ 56 times faster, with nearly the same vectors (97.9% top-10 overlap with the CPU's). Indexing therefore finishes, and `tuck_search` ranks
13
+ by meaning instead of falling back to keywords. Where Metal cannot start (a VM, CI, no GPU), the same server starts on the CPU. The
14
+ installed runtime is kept; each session's index is rebuilt once.
15
+
16
+ ### Fixed
17
+
18
+ - Semantic recall no longer switches itself off on a busy machine. The worker's start (Python, the runtime check, the model load) now has
19
+ its own deadline instead of the 60 s per-call limit, and an embed call's deadline grows with the number of texts it carries, so a slow
20
+ machine slows indexing down instead of making it fail. A worker that still fails is retried in the same session and recovers once the
21
+ machine is free.
22
+
6
23
  ## [1.0.5] - 2026-10-31
7
24
 
8
25
  ### Fixed
package/README.md CHANGED
@@ -58,8 +58,8 @@ Then start a new Pi session. That is all. pi-tuck takes over Pi's compaction, bo
58
58
 
59
59
  - Needs Pi 1.1 or later (tested on 1.1.0 and 1.4.0), on macOS or Linux.
60
60
  - On macOS with Apple silicon, the first session also downloads a local search model in the background (about 390 MB, once). Until it is ready, search works by keyword. See [Semantic search](#semantic-search).
61
- - Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.5`.
62
- - To pin a version, add it: `pi install npm:pi-tuck@1.0.5`.
61
+ - Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.6`.
62
+ - To pin a version, add it: `pi install npm:pi-tuck@1.0.6`.
63
63
 
64
64
  ## How it works
65
65
 
@@ -120,7 +120,7 @@ Each row says what was measured, under which conditions, and where the data is.
120
120
  | Answers after cutting at 60% instead of 100% | No more losses than the limit on all 4 sets. Losses / wins over n questions: 128k shared 3 / 7 of 99; 128k asymmetric 3 / 1 of 67; 32k shared 5 / 3 of 77; 32k asymmetric 9 / 5 of 343. In the 32k asymmetric set the losses lean against 60% (test_result 4 / 2, error_hit 3 / 1); it is the set to watch | Paired replay, same 5 sessions, DeepSeek flash as agent and judge. Limit, set in advance: losses minus wins at most max(2, 5% of n). Asymmetric = evidence the base keeps native and the variant has summarised | [`rnd/replay32/RESULT-lean.md`](rnd/replay32/RESULT-lean.md) |
121
121
  | 1.0.2 changes (plain-text summary, shrinking dialogue view, time on user turns only) against 1.0.1 | No more losses than the limit on all 4 sets. Losses / wins: 128k 1 / 2 of 128; 128k time questions 0 / 0 of 48; 32k 4 / 2 of 127; 32k asymmetric 0 / 2 of 56 | Paired replay, 5 recorded live sessions, DeepSeek flash as agent and judge. 11 of 146 answer sessions hit a 15-minute wall limit and drop out of the pairing | [`rnd/replay32/RESULT-lean.md`](rnd/replay32/RESULT-lean.md), [`CHANGELOG.md`](CHANGELOG.md) |
122
122
  | Summary size, plain-text format against the 1.0.1 JSON format | About 30% shorter where the agent writes little, about 14% shorter where it writes a lot | Same dialogue | [`CHANGELOG.md`](CHANGELOG.md) 1.0.2 |
123
- | Small-window fix | A session that writes little prose cut 174 times in 300 tool rounds at 32k with a 16k reserve (1.0.1 code). With the fix, a test pins that no cut frees under a tenth of the room | Real planner on synthetic sessions | [`test/cut-yield.test.ts`](test/cut-yield.test.ts), which pins that every cut frees at least a tenth of the room |
123
+ | Small-window fix | A session that writes little prose cut 174 times in 300 tool rounds at 32k with a 16k reserve (1.0.1 code). With the fix, a test pins that no cut frees under a tenth of the room | Real planner on synthetic sessions | [`CHANGELOG.md`](CHANGELOG.md) 1.0.2 (the 174 cuts); [`test/cut-yield.test.ts`](test/cut-yield.test.ts), which pins that every cut frees at least a tenth of the room |
124
124
  | Stall fix: a long session's cut | A cut at 3,888 entries took 72 s in 1.0.0 and takes 174 ms since 1.0.1. Since 1.0.1 the cost grows linearly with the session | One machine, the same synthetic session for both numbers (little prose, 128k window, reserve 16,384, turn 644) | [`CHANGELOG.md`](CHANGELOG.md) 1.0.1 |
125
125
 
126
126
  Notes:
@@ -1 +1 @@
1
- export declare const BUILD_ID = "6c391c76284110aae5bb375c04b2147d2fab17de3cb427448e8a869f5ad1e659";
1
+ export declare const BUILD_ID = "d2b2e6b0e594b5dd41e9771c43f4ed0355d8fd048c4e247541199f3b812751d3";
@@ -1 +1 @@
1
- export const BUILD_ID = "6c391c76284110aae5bb375c04b2147d2fab17de3cb427448e8a869f5ad1e659";
1
+ export const BUILD_ID = "d2b2e6b0e594b5dd41e9771c43f4ed0355d8fd048c4e247541199f3b812751d3";
@@ -1,6 +1,6 @@
1
1
  import { type RecallPolicy } from './recall.js';
2
2
  import type { SessionEntry } from '@earendil-works/pi-coding-agent';
3
- export declare const DIAGNOSTIC_BUILD = "6c391c76284110aae5bb375c04b2147d2fab17de3cb427448e8a869f5ad1e659";
3
+ export declare const DIAGNOSTIC_BUILD = "d2b2e6b0e594b5dd41e9771c43f4ed0355d8fd048c4e247541199f3b812751d3";
4
4
  export interface Receipt {
5
5
  id: string;
6
6
  build: string;
@@ -110,6 +110,16 @@ export declare const HEAL_DELAYS_MS: readonly number[];
110
110
  export declare const HEAL_STEADY_MS = 600000;
111
111
  /** A session that has not searched for this long gives its worker back (the index stays on disk). Enabling starts nothing; a search starts the worker again. */
112
112
  export declare const IDLE_STOP_MS = 600000;
113
+ /** A worker call's deadline once the worker has answered a call after `ready`: index, embed and tokenize on a loaded model. */
114
+ export declare const CALL_DEADLINE_MS = 60000;
115
+ /** What an `embed` call adds to its deadline for each text it carries. A batch of 8 passages on a starved CPU took longer than the per-call
116
+ * deadline, and the worker failed when it should have slowed down; a query (one text) still fails fast when the server hangs. The worker
117
+ * scales its native request timeout by the same amount (semantic-worker.py), so that limit is not reached first. */
118
+ export declare const EMBED_TEXT_MS = 30000;
119
+ /** The deadline for a worker's start: Python start, `ready` (which verifies every runtime file) and the first call after it, which loads the
120
+ * model and starts its native server. On a busy machine that takes minutes; held to the per-call deadline, the worker never started and
121
+ * semantic recall went unavailable. It stays above the worker's own native start limits (120 s on Metal, then 400 s on the CPU, in semantic-worker.py). */
122
+ export declare const START_DEADLINE_MS = 600000;
113
123
  /** The most units one index window holds. The worker's SQLite store stops at 8,008 rows (`PRAGMA max_page_count=8192` in scripts/semantic-worker.py), so a cap below that keeps a window of short units from filling the store and sticking in `unavailable`. Raise it only together with the store. */
114
124
  export declare const MAX_INDEX_UNITS = 8000;
115
125
  /** The wait before the next automatic attempt, given how many were already armed since the last success. */
@@ -168,6 +178,9 @@ export declare class SemanticRecallController {
168
178
  healDelaysMs?: readonly number[];
169
179
  healSteadyMs?: number;
170
180
  idleStopMs?: number;
181
+ startDeadlineMs?: number;
182
+ callDeadlineMs?: number;
183
+ embedTextMs?: number;
171
184
  });
172
185
  /** Be told when `status().state` or `lastError` changes: once per change, after the synchronous work that caused it. Returns the unsubscribe. A listener that throws is ignored. */
173
186
  onStateChange(listener: (state: SemanticState, reason: string | null) => void): () => void;
@@ -169,6 +169,16 @@ export const HEAL_DELAYS_MS = [5000, 30000, 120000];
169
169
  export const HEAL_STEADY_MS = 600000;
170
170
  /** A session that has not searched for this long gives its worker back (the index stays on disk). Enabling starts nothing; a search starts the worker again. */
171
171
  export const IDLE_STOP_MS = 600000;
172
+ /** A worker call's deadline once the worker has answered a call after `ready`: index, embed and tokenize on a loaded model. */
173
+ export const CALL_DEADLINE_MS = 60000;
174
+ /** What an `embed` call adds to its deadline for each text it carries. A batch of 8 passages on a starved CPU took longer than the per-call
175
+ * deadline, and the worker failed when it should have slowed down; a query (one text) still fails fast when the server hangs. The worker
176
+ * scales its native request timeout by the same amount (semantic-worker.py), so that limit is not reached first. */
177
+ export const EMBED_TEXT_MS = 30000;
178
+ /** The deadline for a worker's start: Python start, `ready` (which verifies every runtime file) and the first call after it, which loads the
179
+ * model and starts its native server. On a busy machine that takes minutes; held to the per-call deadline, the worker never started and
180
+ * semantic recall went unavailable. It stays above the worker's own native start limits (120 s on Metal, then 400 s on the CPU, in semantic-worker.py). */
181
+ export const START_DEADLINE_MS = 600000;
172
182
  /** The most units one index window holds. The worker's SQLite store stops at 8,008 rows (`PRAGMA max_page_count=8192` in scripts/semantic-worker.py), so a cap below that keeps a window of short units from filling the store and sticking in `unavailable`. Raise it only together with the store. */
173
183
  export const MAX_INDEX_UNITS = 8000;
174
184
  /** Keys per worker `score` request: a multiple of the worker's 128-key sub-batches, aligned from the start of the covered prefix. A key is a SHA-256 hex digest (64 characters, see `hash`), so 1024 keys and the 768-float query make a request of about 90 KiB, far under both the 512 KiB design bound and the worker's 1 MiB line limit. */
@@ -183,6 +193,7 @@ class LocalWorker {
183
193
  project;
184
194
  cleanupFailure;
185
195
  onLoss;
196
+ deadlines;
186
197
  nativeHelper = false;
187
198
  jobId;
188
199
  helperReceipt;
@@ -202,11 +213,13 @@ class LocalWorker {
202
213
  stopped = false;
203
214
  lost = false;
204
215
  closing;
205
- constructor(config, project, cleanupFailure, onLoss = () => { }) {
216
+ loaded = false;
217
+ constructor(config, project, cleanupFailure, onLoss = () => { }, deadlines = { startMs: START_DEADLINE_MS, callMs: CALL_DEADLINE_MS, embedTextMs: EMBED_TEXT_MS }) {
206
218
  this.config = config;
207
219
  this.project = project;
208
220
  this.cleanupFailure = cleanupFailure;
209
221
  this.onLoss = onLoss;
222
+ this.deadlines = deadlines;
210
223
  }
211
224
  closeInBackground() { void this.close().catch(error => this.cleanupFailure(error)); }
212
225
  // The worker stopped without the controller asking: report it once, then join it like any other background close.
@@ -303,7 +316,8 @@ class LocalWorker {
303
316
  const id = ++this.next, body = JSON.stringify({ id, op, ...input }) + '\n';
304
317
  if (Buffer.byteLength(body) > 1048576)
305
318
  throw new Error('Local worker input bound');
306
- return new Promise((resolveReply, reject) => { const onAbort = () => { this.closeInBackground(); reject(abort()); }; signal?.addEventListener('abort', onAbort, { once: true }); const finish = (callback, value) => { signal?.removeEventListener('abort', onAbort); callback(value); }; const timer = setTimeout(() => { this.lose('Local runtime deadline'); finish(reject, new WorkerFailure('Local runtime deadline')); }, 60000); this.pending.set(id, { resolve: value => finish(resolveReply, value), reject: error => finish(reject, error), timer }); if (this.nativeHelper) {
319
+ return new Promise((resolveReply, reject) => { const onAbort = () => { this.closeInBackground(); reject(abort()); }; signal?.addEventListener('abort', onAbort, { once: true }); const finish = (callback, value) => { signal?.removeEventListener('abort', onAbort); callback(value); }; const timer = setTimeout(() => { this.lose('Local runtime deadline'); finish(reject, new WorkerFailure('Local runtime deadline')); }, this.loaded ? this.deadlines.callMs + (op === 'embed' ? this.deadlines.embedTextMs * (input.texts?.length ?? 0) : 0) : this.deadlines.startMs); this.pending.set(id, { resolve: value => { if (op !== 'ready')
320
+ this.loaded = true; finish(resolveReply, value); }, reject: error => finish(reject, error), timer }); if (this.nativeHelper) {
307
321
  const control = { op: 'input', input: Buffer.from(body).toString('base64') };
308
322
  this.nativeInputs.push({ id, control: JSON.stringify(control) + '\n' });
309
323
  this.pumpNativeInput();
@@ -620,7 +634,7 @@ export class SemanticRecallController {
620
634
  if (!this.worker) {
621
635
  const worker = new LocalWorker(this.config, project, error => { if (this.worker !== worker)
622
636
  return; this.cleanupError = error instanceof Error ? error.message : 'Semantic cleanup failed'; this.error = this.cleanupError; this.enabled = false; this.generation++; }, reason => { if (this.worker === worker)
623
- this.heal(new WorkerFailure(reason)); });
637
+ this.heal(new WorkerFailure(reason)); }, { startMs: this.timing.startDeadlineMs ?? START_DEADLINE_MS, callMs: this.timing.callDeadlineMs ?? CALL_DEADLINE_MS, embedTextMs: this.timing.embedTextMs ?? EMBED_TEXT_MS });
624
638
  this.worker = worker;
625
639
  }
626
640
  if (!this.identity) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-tuck",
3
- "version": "1.0.5",
3
+ "version": "1.0.6",
4
4
  "description": "Deterministic compaction for Pi: compact summaries that keep the dialogue and the files touched, plus history recall by keyword and by meaning, all local.",
5
5
  "keywords": [
6
6
  "pi-package",
@@ -12,12 +12,13 @@ WINDOWS=os.name=='nt';encoder_port=None;encoder_token=None
12
12
  class LocalHTTP(http.client.HTTPConnection):
13
13
  def connect(self):
14
14
  self.sock=socket.socket(socket.AF_UNIX,socket.SOCK_STREAM);self.sock.settimeout(self.timeout);self.sock.connect('encoder.sock')
15
- def request(path,data=None):
15
+ # A native request's timeout: 55 s, plus 30 s per text it embeds (the host's EMBED_TEXT_MS), so a starved CPU slows a batch down and does not fail it.
16
+ def request(path,data=None,texts=0):
16
17
  if path not in ('/health','/props','/v1/embeddings'):raise Refusal('native_path_failure')
17
18
  if WINDOWS:
18
19
  if encoder_port is None or not encoder_token:raise Refusal('native_transport_unavailable')
19
- connection=http.client.HTTPConnection('127.0.0.1',encoder_port,timeout=55)
20
- else:connection=LocalHTTP('localhost',timeout=55)
20
+ connection=http.client.HTTPConnection('127.0.0.1',encoder_port,timeout=55+30*texts)
21
+ else:connection=LocalHTTP('localhost',timeout=55+30*texts)
21
22
  try:
22
23
  headers={'Content-Type':'application/json'}
23
24
  if WINDOWS:headers['Authorization']='Bearer '+encoder_token
@@ -26,6 +27,7 @@ def request(path,data=None):
26
27
  if len(payload)>16*1024*1024:raise Refusal('native_response_bound')
27
28
  if response.status!=200:raise Refusal('native_request_failure')
28
29
  return json.loads(payload)
30
+ except TimeoutError:raise Refusal('native_request_failure')
29
31
  finally:connection.close()
30
32
  def native_executable():
31
33
  executable=ROOT/('native/llama-server.exe' if WINDOWS else 'native/llama-server')
@@ -90,18 +92,27 @@ def load():
90
92
  # Pinned common/arg.cpp supports LLAMA_API_KEY; server-http.cpp requires Bearer
91
93
  # on /props and embeddings, but /health is public. No token in process argv.
92
94
  env['LLAMA_API_KEY']=encoder_token
93
- server=subprocess.Popen([str(executable),'-m',str(ROOT/'model.gguf'),'--embedding','--pooling','mean','--embd-normalize','2','--ctx-size','8192','--batch-size','8192','--ubatch-size','8192','--parallel','1','--n-gpu-layers','0','--cache-type-k','f32','--cache-type-v','f32','--flash-attn','off','--threads','4',*transport,'--no-webui','--log-disable'],stdin=subprocess.DEVNULL,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,env=env)
94
- started=time.monotonic()
95
- while True:
96
- if server.poll() is not None:raise Refusal('native_start_failure')
97
- try:
98
- if request('/health').get('status')=='ok':
99
- if WINDOWS:request('/props') # Public health alone is not authenticated readiness.
100
- if server.poll() is not None:raise Refusal('native_start_failure')
101
- break
102
- except (OSError,Refusal):pass
103
- if time.monotonic()-started>50:raise Refusal('native_start_timeout')
104
- time.sleep(.02)
95
+ def launch(gpu_layers,limit):
96
+ """The server, or None when it exits or is not healthy within `limit` seconds (it is then stopped)."""
97
+ child=subprocess.Popen([str(executable),'-m',str(ROOT/'model.gguf'),'--embedding','--pooling','mean','--embd-normalize','2','--ctx-size','8192','--batch-size','8192','--ubatch-size','8192','--parallel','1','--n-gpu-layers',str(gpu_layers),'--cache-type-k','f32','--cache-type-v','f32','--flash-attn','off','--threads','4',*transport,'--no-webui','--log-disable'],stdin=subprocess.DEVNULL,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL,env=env)
98
+ started=time.monotonic()
99
+ while child.poll() is None and time.monotonic()-started<=limit:
100
+ try:
101
+ if request('/health').get('status')=='ok':
102
+ if WINDOWS:request('/props') # Public health alone is not authenticated readiness.
103
+ if child.poll() is None:return child
104
+ except (OSError,Refusal):pass
105
+ time.sleep(.02)
106
+ if child.poll() is None:
107
+ child.kill();child.wait()
108
+ return None
109
+ # Metal first: on Apple silicon it embeds a batch about 56 times faster than the CPU (0.7 s against 39 s for 8 passages at load 44-76),
110
+ # with vectors of cosine 0.9998 or more to the CPU's. Where Metal cannot start (a VM, CI, no GPU) the server exits or never becomes
111
+ # healthy, and the same server starts on the CPU. A busy machine can take minutes to start it; both limits stay under the host's start
112
+ # deadline (START_DEADLINE_MS, 600 s). Windows has no Metal.
113
+ server=None if WINDOWS else launch(99,120)
114
+ if server is None:server=launch(0,400)
115
+ if server is None:raise Refusal('native_start_failure')
105
116
  return server
106
117
  def run(req):
107
118
  op=req['op']
@@ -132,7 +143,7 @@ def run(req):
132
143
  if len(a)>8:raise Refusal('batch_limit')
133
144
  ids=[enc(framed(t,kind)) for t in a]
134
145
  if not a:return {'vectors':[],'identity':IDENTITY}
135
- load();out=request('/v1/embeddings',{'input':ids,'encoding_format':'float','truncate':False})['data'];out.sort(key=lambda row:row['index']);vectors=[row['embedding'] for row in out]
146
+ load();out=request('/v1/embeddings',{'input':ids,'encoding_format':'float','truncate':False},len(ids))['data'];out.sort(key=lambda row:row['index']);vectors=[row['embedding'] for row in out]
136
147
  if len(vectors)!=len(a) or any(len(v)!=768 or any(not math.isfinite(x) for x in v) or abs(sum(x*x for x in v)**.5-1)>1e-5 for v in vectors):raise Refusal('invalid_vector')
137
148
  return {'vectors':vectors,'identity':IDENTITY,'nativePID':server.pid}
138
149
  if op=='index':return index(req)
package/src/build-id.ts CHANGED
@@ -1 +1 @@
1
- export const BUILD_ID = "6c391c76284110aae5bb375c04b2147d2fab17de3cb427448e8a869f5ad1e659";
1
+ export const BUILD_ID = "d2b2e6b0e594b5dd41e9771c43f4ed0355d8fd048c4e247541199f3b812751d3";
@@ -102,6 +102,16 @@ export const HEAL_DELAYS_MS:readonly number[]=[5000,30000,120000];
102
102
  export const HEAL_STEADY_MS=600000;
103
103
  /** A session that has not searched for this long gives its worker back (the index stays on disk). Enabling starts nothing; a search starts the worker again. */
104
104
  export const IDLE_STOP_MS=600000;
105
+ /** A worker call's deadline once the worker has answered a call after `ready`: index, embed and tokenize on a loaded model. */
106
+ export const CALL_DEADLINE_MS=60000;
107
+ /** What an `embed` call adds to its deadline for each text it carries. A batch of 8 passages on a starved CPU took longer than the per-call
108
+ * deadline, and the worker failed when it should have slowed down; a query (one text) still fails fast when the server hangs. The worker
109
+ * scales its native request timeout by the same amount (semantic-worker.py), so that limit is not reached first. */
110
+ export const EMBED_TEXT_MS=30000;
111
+ /** The deadline for a worker's start: Python start, `ready` (which verifies every runtime file) and the first call after it, which loads the
112
+ * model and starts its native server. On a busy machine that takes minutes; held to the per-call deadline, the worker never started and
113
+ * semantic recall went unavailable. It stays above the worker's own native start limits (120 s on Metal, then 400 s on the CPU, in semantic-worker.py). */
114
+ export const START_DEADLINE_MS=600000;
105
115
  /** The most units one index window holds. The worker's SQLite store stops at 8,008 rows (`PRAGMA max_page_count=8192` in scripts/semantic-worker.py), so a cap below that keeps a window of short units from filling the store and sticking in `unavailable`. Raise it only together with the store. */
106
116
  export const MAX_INDEX_UNITS=8000;
107
117
  /** Keys per worker `score` request: a multiple of the worker's 128-key sub-batches, aligned from the start of the covered prefix. A key is a SHA-256 hex digest (64 characters, see `hash`), so 1024 keys and the 768-float query make a request of about 90 KiB, far under both the 512 KiB design bound and the worker's 1 MiB line limit. */
@@ -118,8 +128,8 @@ class LocalWorker {
118
128
  // This leaves room for the previous native write to finish plus one queued
119
129
  // control, even when the child replies before that writer goroutine retires.
120
130
  private nativeInputs:{id:number;control:string}[]=[];private nativeInputId?:number;
121
- private child?:ChildProcessWithoutNullStreams;private scratch?:string;private starting?:Promise<void>;private exited?:Promise<void>;private buffer='';private next=0;private pending=new Map<number,{resolve:(value:any)=>void;reject:(error:Error)=>void;timer:ReturnType<typeof setTimeout>}>();private stopped=false;private lost=false;private closing?:Promise<void>;
122
- constructor(private config:SemanticRecallConfig, private project:string, private cleanupFailure:(error:unknown)=>void, private onLoss:(reason:string)=>void=()=>{}){}
131
+ private child?:ChildProcessWithoutNullStreams;private scratch?:string;private starting?:Promise<void>;private exited?:Promise<void>;private buffer='';private next=0;private pending=new Map<number,{resolve:(value:any)=>void;reject:(error:Error)=>void;timer:ReturnType<typeof setTimeout>}>();private stopped=false;private lost=false;private closing?:Promise<void>;private loaded=false;
132
+ constructor(private config:SemanticRecallConfig, private project:string, private cleanupFailure:(error:unknown)=>void, private onLoss:(reason:string)=>void=()=>{}, private deadlines:{startMs:number;callMs:number;embedTextMs:number}={startMs:START_DEADLINE_MS,callMs:CALL_DEADLINE_MS,embedTextMs:EMBED_TEXT_MS}){}
123
133
  private closeInBackground(){void this.close().catch(error=>this.cleanupFailure(error));}
124
134
  // The worker stopped without the controller asking: report it once, then join it like any other background close.
125
135
  private lose(reason:string){if(!this.stopped&&!this.lost){this.lost=true;this.onLoss(reason);}this.closeInBackground();}
@@ -152,7 +162,7 @@ class LocalWorker {
152
162
  if(signal?.aborted)throw abort();if(this.stopped)throw new WorkerFailure('Local worker closed');
153
163
  await this.start();if(this.stopped||signal?.aborted)throw this.lost&&!signal?.aborted?new WorkerFailure('Local worker stopped'):abort();if(this.pending.size>=2)throw new Error('Local semantic concurrency bound');
154
164
  const id=++this.next,body=JSON.stringify({id,op,...input})+'\n';if(Buffer.byteLength(body)>1048576)throw new Error('Local worker input bound');
155
- return new Promise((resolveReply,reject)=>{const onAbort=()=>{this.closeInBackground();reject(abort());};signal?.addEventListener('abort',onAbort,{once:true});const finish=(callback:(value:any)=>void,value:any)=>{signal?.removeEventListener('abort',onAbort);callback(value);};const timer=setTimeout(()=>{this.lose('Local runtime deadline');finish(reject,new WorkerFailure('Local runtime deadline'));},60000);this.pending.set(id,{resolve:value=>finish(resolveReply,value),reject:error=>finish(reject,error),timer});if(this.nativeHelper){const control:NativeOwnerInputControl={op:'input',input:Buffer.from(body).toString('base64')};this.nativeInputs.push({id,control:JSON.stringify(control)+'\n'});this.pumpNativeInput();}else this.child!.stdin.write(body);});
165
+ return new Promise((resolveReply,reject)=>{const onAbort=()=>{this.closeInBackground();reject(abort());};signal?.addEventListener('abort',onAbort,{once:true});const finish=(callback:(value:any)=>void,value:any)=>{signal?.removeEventListener('abort',onAbort);callback(value);};const timer=setTimeout(()=>{this.lose('Local runtime deadline');finish(reject,new WorkerFailure('Local runtime deadline'));},this.loaded?this.deadlines.callMs+(op==='embed'?this.deadlines.embedTextMs*((input as {texts?:unknown[]}).texts?.length??0):0):this.deadlines.startMs);this.pending.set(id,{resolve:value=>{if(op!=='ready')this.loaded=true;finish(resolveReply,value);},reject:error=>finish(reject,error),timer});if(this.nativeHelper){const control:NativeOwnerInputControl={op:'input',input:Buffer.from(body).toString('base64')};this.nativeInputs.push({id,control:JSON.stringify(control)+'\n'});this.pumpNativeInput();}else this.child!.stdin.write(body);});
156
166
  }
157
167
  private pumpNativeInput(){if(this.stopped||this.nativeInputId!==undefined)return;const next=this.nativeInputs.shift();if(next){this.nativeInputId=next.id;this.child!.stdin.write(next.control);}}
158
168
  private rejectAll(error:Error){for(const p of this.pending.values()){clearTimeout(p.timer);p.reject(error);}this.pending.clear();this.nativeInputs=[];this.nativeInputId=undefined;}
@@ -235,7 +245,7 @@ export class SemanticRecallController {
235
245
  // Whether this session ran a search since the controller was created or last closed; it survives idle stops, so a cut after a long quiet spell still indexes.
236
246
  private searched=false;
237
247
  private readonly listeners=new Set<(state:SemanticState,reason:string|null)=>void>();private announced='disabled\0';private announcing=false;
238
- constructor(config?:SemanticRecallConfig,private readonly timing:{healDelaysMs?:readonly number[];healSteadyMs?:number;idleStopMs?:number}={}){if(config)this.configure(config);}
248
+ constructor(config?:SemanticRecallConfig,private readonly timing:{healDelaysMs?:readonly number[];healSteadyMs?:number;idleStopMs?:number;startDeadlineMs?:number;callDeadlineMs?:number;embedTextMs?:number}={}){if(config)this.configure(config);}
239
249
  /** Be told when `status().state` or `lastError` changes: once per change, after the synchronous work that caused it. Returns the unsubscribe. A listener that throws is ignored. */
240
250
  onStateChange(listener:(state:SemanticState,reason:string|null)=>void):()=>void{this.listeners.add(listener);return()=>{this.listeners.delete(listener);};}
241
251
  private touch(){if(this.announcing)return;this.announcing=true;queueMicrotask(()=>{this.announcing=false;const {state,lastError}=this.status(),key=state+'\0'+(lastError??'');if(key===this.announced)return;this.announced=key;for(const listener of [...this.listeners])try{listener(state,lastError);}catch{}});}
@@ -254,7 +264,7 @@ export class SemanticRecallController {
254
264
  * It is the conversation that left the view only (kind first, no full pass), a heal backoff in progress is not bypassed, and the idle limit applies afterwards. */
255
265
  indexAtCut(provider:()=>Snapshot){if(this.enabled&&!this.healTimer&&this.searched&&!this.active){this.active=true;this.armIdle();}this.schedule(provider);}
256
266
  private async runtime(project:string){if(this.idling)await this.idling;if(!this.enabled)throw abort();outsideProject(project,this.config!.runtimeRoot,'runtime');outsideProject(project,scratchParent(this.config!),'scratch');outsideRuntime(this.config!.runtimeRoot,scratchParent(this.config!),'scratch');if(this.worker&&this.recycle){const old=this.worker;await old.close();if(this.worker===old){this.worker=undefined;this.identity='';this.recycle=false;}if(!this.enabled)throw abort();}
257
- if(!this.worker){const worker=new LocalWorker(this.config!,project,error=>{if(this.worker!==worker)return;this.cleanupError=error instanceof Error?error.message:'Semantic cleanup failed';this.error=this.cleanupError;this.enabled=false;this.generation++;},reason=>{if(this.worker===worker)this.heal(new WorkerFailure(reason));});this.worker=worker;}if(!this.identity){const ready=await this.worker.call('ready');if(ready.dimensions!==768||ready.maxTokens!==8192||!ready.offline)throw new Error('Incompatible semantic model; EG2 768/8192 offline required');this.identity=hash({worker:ready.identity,chunk:'eg2-passage256-focused-disjoint-direct-v3'});}return this.worker;}
267
+ if(!this.worker){const worker=new LocalWorker(this.config!,project,error=>{if(this.worker!==worker)return;this.cleanupError=error instanceof Error?error.message:'Semantic cleanup failed';this.error=this.cleanupError;this.enabled=false;this.generation++;},reason=>{if(this.worker===worker)this.heal(new WorkerFailure(reason));},{startMs:this.timing.startDeadlineMs??START_DEADLINE_MS,callMs:this.timing.callDeadlineMs??CALL_DEADLINE_MS,embedTextMs:this.timing.embedTextMs??EMBED_TEXT_MS});this.worker=worker;}if(!this.identity){const ready=await this.worker.call('ready');if(ready.dimensions!==768||ready.maxTokens!==8192||!ready.offline)throw new Error('Incompatible semantic model; EG2 768/8192 offline required');this.identity=hash({worker:ready.identity,chunk:'eg2-passage256-focused-disjoint-direct-v3'});}return this.worker;}
258
268
  private path(snapshot:Snapshot){const directory=this.config!.indexDirectory;outsideProject(snapshot.project,directory,'index');return join(directory,hash({owner:this.ownerId,session:snapshot.session,project:resolve(snapshot.project)})+'.sqlite');}
259
269
  private async index(action:string,snapshot:Snapshot,extra:object={}){const path=this.path(snapshot),generation=this.generation,worker=await this.runtime(snapshot.project);if(!this.enabled||generation!==this.generation)throw abort();if(this.provider)this.recheck(this.provider,snapshot);this.indexFiles.add(path);const result=await worker.call('index',{path,identity:this.identity,action,projectDirectory:snapshot.project,...extra});if(!this.enabled||generation!==this.generation)throw abort();if(this.provider)this.recheck(this.provider,snapshot);return result;}
260
270
  private async prepare(provider:()=>Snapshot,full=true){