pi-tuck 1.0.8 → 1.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +15 -1
- package/README.md +5 -4
- package/dist/index.d.ts +5 -0
- package/dist/index.js +28 -1
- package/dist/src/build-id.d.ts +1 -1
- package/dist/src/build-id.js +1 -1
- package/dist/src/compaction.js +3 -3
- package/dist/src/diagnostics.d.ts +1 -1
- package/dist/src/semantic-recall.js +2 -2
- package/index.ts +22 -1
- package/package.json +9 -5
- package/scripts/semantic-worker.py +7 -0
- package/src/build-id.ts +1 -1
- package/src/compaction.ts +3 -3
- package/src/semantic-recall.ts +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -3,7 +3,21 @@
|
|
|
3
3
|
All notable changes to pi-tuck. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
|
|
4
4
|
[Semantic Versioning](https://semver.org/).
|
|
5
5
|
|
|
6
|
-
## [1.0.
|
|
6
|
+
## [1.0.9] - 2026-11-01
|
|
7
|
+
|
|
8
|
+
### Changed
|
|
9
|
+
|
|
10
|
+
- pi-tuck needs Pi 1.2.0 or later, the oldest Pi that has every hook it uses and that its tests pass on. On an older Pi, pi-tuck stays off,
|
|
11
|
+
Pi's own compaction runs, and pi-tuck says so once.
|
|
12
|
+
|
|
13
|
+
### Fixed
|
|
14
|
+
|
|
15
|
+
- Semantic recall is Metal only, and now checks it. pi-tuck's own embedding server serves on the CPU, with only a log line, when it finds no
|
|
16
|
+
Metal device; before starting it, pi-tuck asks it for its devices and turns semantic recall off with "Semantic recall needs Metal; word
|
|
17
|
+
recall still works" when none is Metal. An embedding server a host configures is its owner's choice and is not checked.
|
|
18
|
+
- `tuck_search` takes a result back as returned: each entry of `priorResults` is a source's `sourceSha` and `read`, unchanged. It asked for
|
|
19
|
+
`start` and `end`, which no result carried.
|
|
20
|
+
|
|
7
21
|
|
|
8
22
|
### Fixed
|
|
9
23
|
|
package/README.md
CHANGED
|
@@ -55,10 +55,10 @@ pi install npm:pi-tuck
|
|
|
55
55
|
|
|
56
56
|
Then start a new Pi session. That is all. pi-tuck takes over Pi's compaction, both automatic and `/compact`. You change no setting.
|
|
57
57
|
|
|
58
|
-
- Needs Pi 1.
|
|
58
|
+
- Needs Pi 1.2.0 or later (tested on 1.2.0 and 1.4.0), on macOS or Linux. On an older Pi, pi-tuck stays off, Pi's own compaction runs, and pi-tuck says so once. Full features on macOS with Apple silicon; elsewhere, compaction and word recall.
|
|
59
59
|
- On macOS with Apple silicon, the first session also downloads a local search model in the background (about 390 MB, once). Until it is ready, search works by keyword. See [Semantic search](#semantic-search).
|
|
60
|
-
- Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.
|
|
61
|
-
- To pin a version, add it: `pi install npm:pi-tuck@1.0.
|
|
60
|
+
- Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.9`.
|
|
61
|
+
- To pin a version, add it: `pi install npm:pi-tuck@1.0.9`.
|
|
62
62
|
|
|
63
63
|
## How it works
|
|
64
64
|
|
|
@@ -104,7 +104,7 @@ The plan: [`benchmark/PREREG.md`](benchmark/PREREG.md), [`benchmark/PREREG-32K.m
|
|
|
104
104
|
Known limits today:
|
|
105
105
|
|
|
106
106
|
- pi-tuck's summary does not explain why things happened. It holds the words that were said, not a model's reading of them. The agent must look things up. Each look-up costs a tool call.
|
|
107
|
-
- Small windows: an upstream that spends far more context on tool schemas than others (some OpenRouter upstreams do) can leave a small input room too little space, and pi-tuck then refuses to cut rather than overflow. This is what made 1.0.1 refuse at a 32k window with Pi's default 16,384 reserve. Since 1.0.4 the notice says when the provider is counting far above pi-tuck's estimate. In 1.0.4, after an oversize tool result or a "Nothing to compact" error from Pi, a session could keep refusing later prompts. 1.0.5 fixes both. Pin a different upstream, lower the reserve, or use a larger window; see [Small windows](#small-windows).
|
|
107
|
+
- Small windows: an upstream that spends far more context on tool schemas than others (some OpenRouter upstreams do) can leave a small input room too little space, and pi-tuck then refuses to cut rather than overflow. This is what made 1.0.1 refuse at a 32k window with Pi's default 16,384 reserve. Since 1.0.4 the notice says when the provider is counting far above pi-tuck's estimate. In 1.0.4, after an oversize tool result or a "Nothing to compact" error from Pi, a session could keep refusing later prompts. 1.0.5 fixes both. Since 1.0.8, a fresh tool result too large for the room is clipped (its head and tail, with a `tuck_recall` locator to the rest) instead of refusing the run. Pin a different upstream, lower the reserve, or use a larger window; see [Small windows](#small-windows).
|
|
108
108
|
- Semantic search (`tuck_search`) needs a Mac with Metal (Apple silicon). Elsewhere, compaction and `tuck_recall`'s word search work, there is no `tuck_search`, and nothing is downloaded.
|
|
109
109
|
|
|
110
110
|
## Benchmarks
|
|
@@ -114,6 +114,7 @@ Each row says what was measured, under which conditions, and where the data is.
|
|
|
114
114
|
| What | Result | Conditions | Source |
|
|
115
115
|
|---|---|---|---|
|
|
116
116
|
| Tasks resolved, 32k window | Pi's built-in compaction 94 of 150; pi-tuck 120 of 150 (+17.3 points, 95% lower bound +10.7); observation masking 117 of 150 (pi-tuck +2.0, lower bound −2.7: no difference shown). Turns per task: 9.4 (built-in), 24.0 (pi-tuck), 11.9 (masking). Tokens per session: 6.10 M, 6.39 M, 7.02 M. Spend per arm for the five sessions: $0.87, $0.55, $0.94 | pi-tuck 1.0.5 (commit `aa30824`, the 1.0.5 code; 1.0.6 and 1.0.7 are not measured). DeepSeek V4.1 Flash pinned to one upstream, window 32,768, reserve 16,384, 5 chains of 30 SWE-bench Verified tasks, paired by task, one-sided bootstrap over tasks. **Limits:** only pi-tuck is held to the 16,384-token room (the built-in sent a request over it in 97 of 150 tasks, masking in 73, so the window is nominal for them). The difference by chain is +10.0, −3.3, +50.0, +16.7 and +13.3 points; leaving out chain t03 it is +9.2. The bootstrap resamples tasks, not chains. The pi-tuck arm ran at a different time from the others. One model, one window. pi-tuck refused to cut mid-run in 5 tasks, where one fresh tool result was larger than the room; they count as unresolved. In the same setup pi-tuck 1.0.3 resolved 72 of 150, with 52 tasks lost because an earlier refusal carried forward; 1.0.5 fixes that | [`benchmark/RESULTS-32K-ACCEPT.md`](benchmark/RESULTS-32K-ACCEPT.md) |
|
|
117
|
+
| The 1.0.8 candidate against 1.0.7, 32k window | 0 dead tasks in both arms. Resolved 46 of 60 in both (+0.0 points, 95% lower bound −6.7). No mid-run refusal in either arm. Turns per task 25.37 (1.0.7) and 26.10 (1.0.8) | Pre-registered gate, passed as registered. pi-tuck `a2e7d1a` (the tool-result clip and the cut at the end of a run, on 1.0.6) against 1.0.7. Released 1.0.8 adds 1.0.7's Metal-only recall to that build. DeepSeek V4.1 Flash pinned to one upstream, window 32,768, reserve 16,384, chains t01 and t02, 60 paired tasks, both arms run together. One model, one window, one run per arm; 60 tasks cannot show a difference smaller than the bound. The run shows that the change introduced no refusal; no refusal occurred in the 1.0.7 arm for the clip to prevent | [`benchmark/RESULTS-V108.md`](benchmark/RESULTS-V108.md) |
|
|
117
118
|
| Cut time, live | pi-tuck median 0.087 s (90th percentile 0.205 s); Pi's built-in median 268.8 s (90th percentile 421.6 s). Context carried after a cut, median: 27,751 against 29,241 tokens | **Interim.** pi-tuck 1.0.1, Pi 1.4.0, SWE-bench Verified chains of 30 tasks, 131,072-token window, reply reserve 32,768, local model `mac-studio/main` writing Pi's summaries. 27 cuts per arm, 5 chains. The built-in time depends on that model and on server load. pi-tuck's time is an upper bound read from the session file | [`benchmark/RESULTS-INTERIM.md`](benchmark/RESULTS-INTERIM.md) section 1 |
|
|
118
119
|
| Cut time, paired replay | pi-tuck median 0.062 s (range 0.005 to 0.48 s); local-model summarizer 193.4 s (77.3 to 413.3 s); frontier summarizer 96.1 s (39.1 to 267.8 s; measured on the retired model `openai/gpt-6-astra`; feeds no test). Context carried after a cut, median: 28,006 (pi-tuck), 26,424 (local), 25,555 (frontier) tokens | **Interim.** The same recorded sessions cut at the same positions. pi-tuck: 10 recordings, 52 cuts. Summarizers: 6 recordings, 28 and 29 cuts. pi-tuck's time leaves out the background search index | [`benchmark/RESULTS-INTERIM.md`](benchmark/RESULTS-INTERIM.md) section 2 |
|
|
119
120
|
| Billed input, cutting at 60% of the room instead of 100% | −17.9% at 128k, −14.0% at 32k with automatic prompt caching; −17.0% and −9.6% with explicit cache breakpoints. Cuts go from 6 to 12 (128k) and from 84 to 157 (32k); none frees under a tenth of the room | Simulated billing, not provider bills. Prompt caching priced. Build `1bda64b` (the default shipped in 1.0.3). 5 recorded live sessions. 128k = 131,072 window, reserve 32,768. 32k = 32,768 window, reserve 16,384 | [`rnd/replay32/RESULT-lean.md`](rnd/replay32/RESULT-lean.md) |
|
package/dist/index.d.ts
CHANGED
|
@@ -8,4 +8,9 @@ export { ensureSemanticRuntime } from "./src/setup.js";
|
|
|
8
8
|
/** pi-tuck as a Pi extension: compaction and history recall always; semantic recall too, unless `pi-tuck.json` in Pi's agent folder
|
|
9
9
|
* says `{"semanticRecall": false}`. Its runtime is set up in the background the first time a session starts. The same file's `"cutAt"`
|
|
10
10
|
* (a number above 0 and at most 1) is the fraction of the input room a cut is triggered at, 1 by default. */
|
|
11
|
+
/** The oldest Pi with every hook and API pi-tuck uses, and the oldest its tests pass on (Pi 1.2 added the custom message that starts a
|
|
12
|
+
* run to before_agent_start, which the prompt's cut counts). */
|
|
13
|
+
export declare const PI_FLOOR = "1.2.0";
|
|
14
|
+
/** Whether a Pi version is at or above the floor. A version that does not parse is taken as supported. */
|
|
15
|
+
export declare function piSupported(version: string): boolean;
|
|
11
16
|
export default function (pi: ExtensionAPI): Promise<void>;
|
package/dist/index.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { getAgentDir } from "@earendil-works/pi-coding-agent";
|
|
1
|
+
import { VERSION, getAgentDir } from "@earendil-works/pi-coding-agent";
|
|
2
2
|
import { readFileSync, realpathSync } from "node:fs";
|
|
3
3
|
import { join, sep } from "node:path";
|
|
4
4
|
import { installCompaction } from "./src/compaction.js";
|
|
@@ -13,7 +13,34 @@ export { ensureSemanticRuntime } from "./src/setup.js";
|
|
|
13
13
|
/** pi-tuck as a Pi extension: compaction and history recall always; semantic recall too, unless `pi-tuck.json` in Pi's agent folder
|
|
14
14
|
* says `{"semanticRecall": false}`. Its runtime is set up in the background the first time a session starts. The same file's `"cutAt"`
|
|
15
15
|
* (a number above 0 and at most 1) is the fraction of the input room a cut is triggered at, 1 by default. */
|
|
16
|
+
/** The oldest Pi with every hook and API pi-tuck uses, and the oldest its tests pass on (Pi 1.2 added the custom message that starts a
|
|
17
|
+
* run to before_agent_start, which the prompt's cut counts). */
|
|
18
|
+
export const PI_FLOOR = "1.2.0";
|
|
19
|
+
/** Whether a Pi version is at or above the floor. A version that does not parse is taken as supported. */
|
|
20
|
+
export function piSupported(version) {
|
|
21
|
+
const parts = /^(\d+)\.(\d+)\.(\d+)/.exec(version)?.slice(1).map(Number), floor = PI_FLOOR.split(".").map(Number);
|
|
22
|
+
if (!parts)
|
|
23
|
+
return true;
|
|
24
|
+
for (let i = 0; i < 3; i++)
|
|
25
|
+
if (parts[i] !== floor[i])
|
|
26
|
+
return parts[i] > floor[i];
|
|
27
|
+
return true;
|
|
28
|
+
}
|
|
16
29
|
export default async function (pi) {
|
|
30
|
+
// On an older Pi pi-tuck stays off, so Pi's own compaction runs, and says so once.
|
|
31
|
+
if (!piSupported(VERSION)) {
|
|
32
|
+
let told = false;
|
|
33
|
+
pi.on("session_start", (_event, ctx) => {
|
|
34
|
+
if (told)
|
|
35
|
+
return;
|
|
36
|
+
told = true;
|
|
37
|
+
try {
|
|
38
|
+
ctx.ui.notify(`pi-tuck needs Pi ${PI_FLOOR} or later (this is Pi ${VERSION}); it is off, and Pi's own compaction runs.`, "warning");
|
|
39
|
+
}
|
|
40
|
+
catch { /* ended */ }
|
|
41
|
+
});
|
|
42
|
+
return;
|
|
43
|
+
}
|
|
17
44
|
const semanticRecall = createSemanticRecall();
|
|
18
45
|
// `/tuck-semantic off` holds for the rest of the session, including over a setup that finishes later.
|
|
19
46
|
let off = false;
|
package/dist/src/build-id.d.ts
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export declare const BUILD_ID = "
|
|
1
|
+
export declare const BUILD_ID = "c4e6e6034a54ff7a893ee62484e54cf471964be3435aa587af8a288dcbf41314";
|
package/dist/src/build-id.js
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export const BUILD_ID = "
|
|
1
|
+
export const BUILD_ID = "c4e6e6034a54ff7a893ee62484e54cf471964be3435aa587af8a288dcbf41314";
|
package/dist/src/compaction.js
CHANGED
|
@@ -35,7 +35,7 @@ const RecallInput = Type.Union([
|
|
|
35
35
|
Type.Object({ index: Type.Integer({ minimum: 0, maximum: Number.MAX_SAFE_INTEGER }), query: Type.Optional(query), ...paging }, { additionalProperties: false }),
|
|
36
36
|
Type.Object(paging, { additionalProperties: false }),
|
|
37
37
|
]);
|
|
38
|
-
const SearchInput = Type.Object({ query, priorResults: Type.Optional(Type.Array(Type.Object({
|
|
38
|
+
const SearchInput = Type.Object({ query, priorResults: Type.Optional(Type.Array(Type.Object({ sourceSha: Type.String({ minLength: 64, maxLength: 64 }), read: Type.Object({ entryId, offset: Type.Integer({ minimum: 0 }), limit: Type.Integer({ minimum: 0 }) }, { additionalProperties: false }) }, { additionalProperties: false }), { maxItems: 20 })) }, { additionalProperties: false });
|
|
39
39
|
const reply = (value) => ({ content: [{ type: "text", text: JSON.stringify(value) }], details: {} });
|
|
40
40
|
/** The compactor's verdict that this attempt cannot be made: the cut it needs cannot be made, or the request cannot fit. Like every failure but
|
|
41
41
|
* a `CommitFault`, it ends with its attempt (`forgetFailure`), and the next attempt plans afresh from the session as it is. */
|
|
@@ -397,7 +397,7 @@ export async function installCompaction(pi, options = {}) {
|
|
|
397
397
|
});
|
|
398
398
|
if (search)
|
|
399
399
|
pi.registerTool({ name: "tuck_search", label: "History search", parameters: SearchInput,
|
|
400
|
-
description: "Find historical information relevant to a question, including different wording and related qualifications. Combines keyword and local embedding search. Knowing a name or identifier does not imply keyword-only lookup is sufficient. Returns original evidence up to 20000 characters; not exhaustive. Semantic search may be off or incomplete, which is not the same as no matches. Read exact originals and their context with tuck_recall at the returned locators. priorResults lists source
|
|
400
|
+
description: "Find historical information relevant to a question, including different wording and related qualifications. Combines keyword and local embedding search. Knowing a name or identifier does not imply keyword-only lookup is sufficient. Returns original evidence up to 20000 characters; not exhaustive. Semantic search may be off or incomplete, which is not the same as no matches. Read exact originals and their context with tuck_recall at the returned locators. priorResults lists sources already read: pass each returned source's sourceSha and read unchanged.",
|
|
401
401
|
async execute(_id, raw, signal, _update, ctx) {
|
|
402
402
|
access();
|
|
403
403
|
if (!Check(SearchInput, raw))
|
|
@@ -411,7 +411,7 @@ export async function installCompaction(pi, options = {}) {
|
|
|
411
411
|
// The key is taken in the same synchronous step as the first lexical pass. After the await the second pass runs only if it moved (see eligibilityKey); a moved key alone invalidates nothing.
|
|
412
412
|
// The snapshot id check covers every in-contract change (any branch change moves the id); the key-gated second pass is defence in depth for out-of-contract callers.
|
|
413
413
|
const provider = semanticProvider(ctx), initialSnapshot = provider(), initialKey = eligibilityKey(ctx), words = wordOffer(), wordText = JSON.stringify(words);
|
|
414
|
-
const related = await semantic.search(input.query, provider, signal, input.priorResults ?? [], words);
|
|
414
|
+
const related = await semantic.search(input.query, provider, signal, (input.priorResults ?? []).map(({ sourceSha, read }) => ({ entryId: read.entryId, sourceSha, start: read.offset, end: read.offset + read.limit })), words);
|
|
415
415
|
access();
|
|
416
416
|
if (signal?.aborted)
|
|
417
417
|
throw Object.assign(new Error('Semantic recall cancelled'), { name: 'AbortError' });
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { type RecallPolicy } from './recall.js';
|
|
2
2
|
import type { SessionEntry } from '@earendil-works/pi-coding-agent';
|
|
3
|
-
export declare const DIAGNOSTIC_BUILD = "
|
|
3
|
+
export declare const DIAGNOSTIC_BUILD = "c4e6e6034a54ff7a893ee62484e54cf471964be3435aa587af8a288dcbf41314";
|
|
4
4
|
export interface Receipt {
|
|
5
5
|
id: string;
|
|
6
6
|
build: string;
|
|
@@ -4,7 +4,7 @@ import { readFile, rm, mkdtemp, mkdir } from 'node:fs/promises';
|
|
|
4
4
|
import { lstatSync, realpathSync } from 'node:fs';
|
|
5
5
|
import { resolve, relative, join, dirname, basename, isAbsolute, sep } from 'node:path';
|
|
6
6
|
import { fileURLToPath } from 'node:url';
|
|
7
|
-
import { defaultRoot, runtimeConfigPath } from './assets.js';
|
|
7
|
+
import { LLAMA, defaultRoot, runtimeConfigPath } from './assets.js';
|
|
8
8
|
import { createRecall } from './recall.js';
|
|
9
9
|
import { buildGlobalIndexById } from './tuck/core/global-indices.js';
|
|
10
10
|
// Session-owned mutable scratch is outside verified software and caller projects.
|
|
@@ -256,7 +256,7 @@ class LocalWorker {
|
|
|
256
256
|
const worker = this.config.workerPath ?? fileURLToPath(new URL(import.meta.url.endsWith('.ts') ? '../scripts/semantic-worker.py' : '../../scripts/semantic-worker.py', import.meta.url));
|
|
257
257
|
const osNames = this.nativeHelper ? ['SYSTEMROOT', 'WINDIR', 'LANG', 'LC_ALL'] : ['SystemRoot', 'SYSTEMROOT', 'WINDIR', 'LANG', 'LC_ALL'];
|
|
258
258
|
const osEnvironment = Object.fromEntries(osNames.flatMap(name => typeof process.env[name] === 'string' ? [[name, process.env[name]]] : []));
|
|
259
|
-
const args = ['-I', '-B', '-u', worker], env = { ...osEnvironment, TMPDIR: this.scratch, TMP: this.scratch, TEMP: this.scratch, PI_SEMANTIC_RUNTIME_ROOT: this.config.runtimeRoot };
|
|
259
|
+
const args = ['-I', '-B', '-u', worker], env = { ...osEnvironment, TMPDIR: this.scratch, TMP: this.scratch, TEMP: this.scratch, PI_SEMANTIC_RUNTIME_ROOT: this.config.runtimeRoot, PI_SEMANTIC_OWN_SERVER_SHA256: LLAMA.files['llama-server'] };
|
|
260
260
|
this.jobId = randomUUID();
|
|
261
261
|
const launch = { executable: this.config.pythonPath, args, cwd: this.scratch, env, jobId: this.jobId, stdinMode: 'stream' };
|
|
262
262
|
const launchLine = JSON.stringify(launch) + '\n';
|
package/index.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { getAgentDir, type ExtensionAPI, type ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
1
|
+
import { VERSION, getAgentDir, type ExtensionAPI, type ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
2
2
|
import { readFileSync, realpathSync } from "node:fs";
|
|
3
3
|
import { join, sep } from "node:path";
|
|
4
4
|
import { installCompaction } from "./src/compaction";
|
|
@@ -17,7 +17,28 @@ export { ensureSemanticRuntime } from "./src/setup";
|
|
|
17
17
|
/** pi-tuck as a Pi extension: compaction and history recall always; semantic recall too, unless `pi-tuck.json` in Pi's agent folder
|
|
18
18
|
* says `{"semanticRecall": false}`. Its runtime is set up in the background the first time a session starts. The same file's `"cutAt"`
|
|
19
19
|
* (a number above 0 and at most 1) is the fraction of the input room a cut is triggered at, 1 by default. */
|
|
20
|
+
/** The oldest Pi with every hook and API pi-tuck uses, and the oldest its tests pass on (Pi 1.2 added the custom message that starts a
|
|
21
|
+
* run to before_agent_start, which the prompt's cut counts). */
|
|
22
|
+
export const PI_FLOOR = "1.2.0";
|
|
23
|
+
/** Whether a Pi version is at or above the floor. A version that does not parse is taken as supported. */
|
|
24
|
+
export function piSupported(version: string): boolean {
|
|
25
|
+
const parts = /^(\d+)\.(\d+)\.(\d+)/.exec(version)?.slice(1).map(Number), floor = PI_FLOOR.split(".").map(Number);
|
|
26
|
+
if (!parts) return true;
|
|
27
|
+
for (let i = 0; i < 3; i++) if (parts[i] !== floor[i]) return parts[i]! > floor[i]!;
|
|
28
|
+
return true;
|
|
29
|
+
}
|
|
30
|
+
|
|
20
31
|
export default async function (pi: ExtensionAPI) {
|
|
32
|
+
// On an older Pi pi-tuck stays off, so Pi's own compaction runs, and says so once.
|
|
33
|
+
if (!piSupported(VERSION)) {
|
|
34
|
+
let told = false;
|
|
35
|
+
pi.on("session_start", (_event, ctx) => {
|
|
36
|
+
if (told) return;
|
|
37
|
+
told = true;
|
|
38
|
+
try { ctx.ui.notify(`pi-tuck needs Pi ${PI_FLOOR} or later (this is Pi ${VERSION}); it is off, and Pi's own compaction runs.`, "warning"); } catch { /* ended */ }
|
|
39
|
+
});
|
|
40
|
+
return;
|
|
41
|
+
}
|
|
21
42
|
const semanticRecall = createSemanticRecall();
|
|
22
43
|
// `/tuck-semantic off` holds for the rest of the session, including over a setup that finishes later.
|
|
23
44
|
let off = false;
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-tuck",
|
|
3
|
-
"version": "1.0.
|
|
4
|
-
"description": "Deterministic compaction for Pi
|
|
3
|
+
"version": "1.0.9",
|
|
4
|
+
"description": "Deterministic compaction for Pi with no model call, plus exact recall of earlier history. Search by meaning on Apple silicon.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi-package",
|
|
7
7
|
"pi",
|
|
@@ -9,7 +9,11 @@
|
|
|
9
9
|
"compaction",
|
|
10
10
|
"context",
|
|
11
11
|
"recall",
|
|
12
|
-
"semantic-search"
|
|
12
|
+
"semantic-search",
|
|
13
|
+
"pi-coding-agent",
|
|
14
|
+
"coding-agent",
|
|
15
|
+
"context-window",
|
|
16
|
+
"deterministic"
|
|
13
17
|
],
|
|
14
18
|
"license": "MIT",
|
|
15
19
|
"author": "Rustem Galyavin",
|
|
@@ -54,8 +58,8 @@
|
|
|
54
58
|
"prepack": "npm run build"
|
|
55
59
|
},
|
|
56
60
|
"peerDependencies": {
|
|
57
|
-
"@earendil-works/pi-ai": "
|
|
58
|
-
"@earendil-works/pi-coding-agent": "
|
|
61
|
+
"@earendil-works/pi-ai": ">=1.2.0",
|
|
62
|
+
"@earendil-works/pi-coding-agent": ">=1.2.0",
|
|
59
63
|
"typebox": "*"
|
|
60
64
|
},
|
|
61
65
|
"devDependencies": {
|
|
@@ -110,6 +110,13 @@ def load():
|
|
|
110
110
|
# CPU did (0.7 s against 39 s for 8 passages at load 44-76). Where Metal cannot start (a VM, no GPU) the server exits or never becomes
|
|
111
111
|
# healthy, and semantic recall goes off with that reason; word recall keeps working. A busy machine can take minutes to start it; the
|
|
112
112
|
# limit stays under the host's start deadline (START_DEADLINE_MS, 600 s).
|
|
113
|
+
# pi-tuck's own pinned server serves on the CPU, with only a log line, when it finds no Metal device: it must list one. A server a host
|
|
114
|
+
# configures is its owner's choice and is not checked.
|
|
115
|
+
own=os.environ.get('PI_SEMANTIC_OWN_SERVER_SHA256')
|
|
116
|
+
if own and RUNTIME['read_json'](ROOT/'assets.sha256.json').get(executable.relative_to(ROOT).as_posix())==own:
|
|
117
|
+
try:listed=subprocess.run([str(executable),'--list-devices'],stdin=subprocess.DEVNULL,capture_output=True,text=True,env=env,timeout=120)
|
|
118
|
+
except (OSError,subprocess.TimeoutExpired):raise Refusal('metal_unavailable')
|
|
119
|
+
if not any(line.strip().startswith('MTL') for line in (listed.stdout+'\n'+listed.stderr).splitlines()):raise Refusal('metal_unavailable')
|
|
113
120
|
server=launch(540)
|
|
114
121
|
if server is None:raise Refusal('metal_unavailable')
|
|
115
122
|
return server
|
package/src/build-id.ts
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
export const BUILD_ID = "
|
|
1
|
+
export const BUILD_ID = "c4e6e6034a54ff7a893ee62484e54cf471964be3435aa587af8a288dcbf41314";
|
package/src/compaction.ts
CHANGED
|
@@ -39,7 +39,7 @@ const RecallInput = Type.Union([
|
|
|
39
39
|
Type.Object({ index: Type.Integer({ minimum: 0, maximum: Number.MAX_SAFE_INTEGER }), query: Type.Optional(query), ...paging }, { additionalProperties: false }),
|
|
40
40
|
Type.Object(paging, { additionalProperties: false }),
|
|
41
41
|
]);
|
|
42
|
-
const SearchInput = Type.Object({query,priorResults:Type.Optional(Type.Array(Type.Object({
|
|
42
|
+
const SearchInput = Type.Object({query,priorResults:Type.Optional(Type.Array(Type.Object({sourceSha:Type.String({minLength:64,maxLength:64}),read:Type.Object({entryId,offset:Type.Integer({minimum:0}),limit:Type.Integer({minimum:0})},{additionalProperties:false})},{additionalProperties:false}),{maxItems:20}))},{additionalProperties:false});
|
|
43
43
|
const reply = (value: unknown) => ({ content: [{ type: "text" as const, text: JSON.stringify(value) }], details: {} });
|
|
44
44
|
/** The compactor's verdict that this attempt cannot be made: the cut it needs cannot be made, or the request cannot fit. Like every failure but
|
|
45
45
|
* a `CommitFault`, it ends with its attempt (`forgetFailure`), and the next attempt plans afresh from the session as it is. */
|
|
@@ -348,7 +348,7 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
|
|
|
348
348
|
},
|
|
349
349
|
});
|
|
350
350
|
if (search) pi.registerTool({name:"tuck_search",label:"History search",parameters:SearchInput,
|
|
351
|
-
description:"Find historical information relevant to a question, including different wording and related qualifications. Combines keyword and local embedding search. Knowing a name or identifier does not imply keyword-only lookup is sufficient. Returns original evidence up to 20000 characters; not exhaustive. Semantic search may be off or incomplete, which is not the same as no matches. Read exact originals and their context with tuck_recall at the returned locators. priorResults lists source
|
|
351
|
+
description:"Find historical information relevant to a question, including different wording and related qualifications. Combines keyword and local embedding search. Knowing a name or identifier does not imply keyword-only lookup is sufficient. Returns original evidence up to 20000 characters; not exhaustive. Semantic search may be off or incomplete, which is not the same as no matches. Read exact originals and their context with tuck_recall at the returned locators. priorResults lists sources already read: pass each returned source's sourceSha and read unchanged.",
|
|
352
352
|
async execute(_id,raw,signal,_update,ctx){
|
|
353
353
|
access();if(!Check(SearchInput,raw))throw new Error("Invalid broad search input");const input=structuredClone(raw);
|
|
354
354
|
recorder.capture("recall-action","authorized-native-tool-invocation-no-result-copy",input);
|
|
@@ -359,7 +359,7 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
|
|
|
359
359
|
// The key is taken in the same synchronous step as the first lexical pass. After the await the second pass runs only if it moved (see eligibilityKey); a moved key alone invalidates nothing.
|
|
360
360
|
// The snapshot id check covers every in-contract change (any branch change moves the id); the key-gated second pass is defence in depth for out-of-contract callers.
|
|
361
361
|
const provider=semanticProvider(ctx),initialSnapshot=provider(),initialKey=eligibilityKey(ctx),words=wordOffer(),wordText=JSON.stringify(words);
|
|
362
|
-
const related=await semantic.search(input.query,provider,signal,input.priorResults??[],words);
|
|
362
|
+
const related=await semantic.search(input.query,provider,signal,(input.priorResults??[]).map(({sourceSha,read})=>({entryId:read.entryId,sourceSha,start:read.offset,end:read.offset+read.limit})),words);
|
|
363
363
|
access();if(signal?.aborted)throw Object.assign(new Error('Semantic recall cancelled'),{name:'AbortError'});const now=provider();
|
|
364
364
|
if(initialSnapshot.id!==now.id||(eligibilityKeyMoved(ctx,initialKey)&&wordText!==JSON.stringify(wordOffer())))return{...reply({status:"snapshot_invalidated",notice:"Search source eligibility or version changed; no earlier evidence delivered. Request a fresh query."}),isError:true};
|
|
365
365
|
const page=budgetSemantic(related,20000-wordText.length-1);
|
package/src/semantic-recall.ts
CHANGED
|
@@ -5,7 +5,7 @@ import {lstatSync,realpathSync} from 'node:fs';
|
|
|
5
5
|
import {resolve,relative,join,dirname,basename,isAbsolute,sep} from 'node:path';
|
|
6
6
|
import {fileURLToPath} from 'node:url';
|
|
7
7
|
import type {ExtensionContext,SessionEntry} from '@earendil-works/pi-coding-agent';
|
|
8
|
-
import {defaultRoot,runtimeConfigPath} from './assets';
|
|
8
|
+
import {LLAMA,defaultRoot,runtimeConfigPath} from './assets';
|
|
9
9
|
import {createRecall,type RecallPolicy,type ReadableEntry} from './recall';
|
|
10
10
|
import {buildGlobalIndexById} from './tuck/core/global-indices';
|
|
11
11
|
// Session-owned mutable scratch is outside verified software and caller projects.
|
|
@@ -146,7 +146,7 @@ class LocalWorker {
|
|
|
146
146
|
const worker=this.config.workerPath??fileURLToPath(new URL(import.meta.url.endsWith('.ts')?'../scripts/semantic-worker.py':'../../scripts/semantic-worker.py',import.meta.url));
|
|
147
147
|
const osNames=this.nativeHelper?['SYSTEMROOT','WINDIR','LANG','LC_ALL']:['SystemRoot','SYSTEMROOT','WINDIR','LANG','LC_ALL'];
|
|
148
148
|
const osEnvironment=Object.fromEntries(osNames.flatMap(name=>typeof process.env[name]==='string'?[[name,process.env[name]!]]:[]));
|
|
149
|
-
const args=['-I','-B','-u',worker],env={...osEnvironment,TMPDIR:this.scratch,TMP:this.scratch,TEMP:this.scratch,PI_SEMANTIC_RUNTIME_ROOT:this.config.runtimeRoot};
|
|
149
|
+
const args=['-I','-B','-u',worker],env={...osEnvironment,TMPDIR:this.scratch,TMP:this.scratch,TEMP:this.scratch,PI_SEMANTIC_RUNTIME_ROOT:this.config.runtimeRoot,PI_SEMANTIC_OWN_SERVER_SHA256:LLAMA.files['llama-server']!};
|
|
150
150
|
this.jobId=randomUUID();const launch:NativeOwnerLaunch={executable:this.config.pythonPath,args,cwd:this.scratch,env,jobId:this.jobId,stdinMode:'stream'};
|
|
151
151
|
const launchLine=JSON.stringify(launch)+'\n';if(this.nativeHelper&&Buffer.byteLength(launchLine)>2*1048576)throw new Error('Native owner launch bound');
|
|
152
152
|
const child=spawn(this.nativeHelper?this.config.nativeOwnerHelperPath!:this.config.pythonPath,this.nativeHelper?['--run']:args,{cwd:this.scratch,env:this.nativeHelper?osEnvironment:env,stdio:'pipe',detached:!this.nativeHelper});
|