pi-repl-py 0.6.12 → 0.6.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,8 +1,9 @@
1
1
  # Architecture
2
2
 
3
- pi-repl runs in **two processes**. pi hosts the TypeScript extension, and the extension manages a
4
- separate Python `ipykernel` process where user code runs. The host talks to that kernel using the
5
- standard Jupyter protocol. There is no Python middleman and no private framing layer between them.
3
+ pi-repl runs in **two processes**: pi hosts the TypeScript extension, which manages a separate
4
+ Python `ipykernel` process where user code runs, speaking the standard Jupyter protocol directly
5
+ (no Python middleman, no private framing). A cell can raise or wedge the kernel without taking pi
6
+ down; the host stays answerable.
6
7
 
7
8
  ```
8
9
  pi
@@ -16,150 +17,109 @@ pi
16
17
  └─ python -m ipykernel -f <connection-file> the evaluator
17
18
  ```
18
19
 
19
- The host is TypeScript, and the evaluator is Python in a separate process. This means a cell can
20
- raise an exception or make the kernel unusable without taking pi down. The host can still report
21
- what happened.
22
-
23
20
  ## Why the host speaks ZMTP itself
24
21
 
25
- The obvious way for a TypeScript host to drive an `ipykernel` is to load a ZMQ client
26
- library. That does not work here: libzmq's native Node bindings crash `bun`. So an earlier
27
- design put a Python middleman (`guest.py`) between the host and the kernel, translating a
28
- private JSON protocol over a file descriptor into the real Jupyter protocol.
29
-
30
- The current design removes the middleman. Instead of working around the missing library, the
31
- host implements the small slice of ZMTP 3.0 that a Jupyter client needs. ZMTP is the socket
32
- protocol used by Jupyter's channels: the host uses a DEALER socket for shell and control, and a
33
- SUB socket for iopub (`src/engine/zmtp.ts`).
34
- The payoff:
22
+ A TypeScript host cannot load libzmq's native Node bindings (they crash `bun`), and the earlier
23
+ Python middleman (`guest.py`) that translated a private JSON protocol is gone. The host instead
24
+ implements the small slice of ZMTP 3.0 a Jupyter client needs DEALER for shell/control, SUB for
25
+ iopub (`src/engine/zmtp.ts`). The payoff:
35
26
 
36
27
  - **one process boundary** instead of two;
37
28
  - **one standard protocol** (Jupyter) instead of a private one on top of it;
38
29
  - **no invented framing** to maintain;
39
- - **Messages are authenticated with HMAC.** The host signs and verifies Jupyter messages with
40
- the kernel's HMAC key. The earlier design used a nonce to prevent false completion messages;
41
- the standard message signature now provides that check.
30
+ - **messages are authenticated with HMAC** the host signs and verifies every message with the
31
+ kernel's HMAC key, replacing the old nonce that guarded against false completion messages.
42
32
 
43
33
  ## The Python environment (the venv)
44
34
 
45
- The evaluator is a real `ipykernel` kernel, so it needs a Python environment with
46
- `ipykernel` installed. This is a hard runtime dependency. A script cannot replace it.
47
- (`jupyter_client` is *not* needed: the host is the client.)
35
+ The evaluator is a real ipykernel, so it needs Python with `ipykernel` installed — a hard runtime
36
+ dependency (`jupyter_client` is *not* needed: the host is the client). A package install runs
37
+ `postinstall` (`scripts/setup-venv.mjs`), which builds a stable per-user venv at
38
+ `~/.pi/agent/pi-repl/venv/bin/python3` — stable because it sits outside the package dir that npm
39
+ replaces on each update. If `python3` or the network is missing at install time, it prints a
40
+ notice and the host falls back at runtime.
48
41
 
49
- When installed as a pi package, `npm install` runs `postinstall`
50
- (`scripts/setup-venv.mjs`), which builds a stable per-user venv:
42
+ At spawn, `resolvePythonPath` uses exactly one interpreter: the install venv, else `$PYTHON` /
43
+ `python3`. It never auto-picks a repo or cwd `.venv` such a venv may lack ipykernel, which
44
+ killed the kernel whenever cwd happened to contain one. The kernel starts in the session's cwd
45
+ and falls back to the host cwd if that directory is gone, so a stale cwd never prevents boot.
51
46
 
52
- ```
53
- ~/.pi/agent/pi-repl/venv/bin/python3
54
- ```
47
+ ## The kernel client
55
48
 
56
- That path is stable across updates because it sits outside the package's own directory,
57
- which npm replaces on each update. If `python3` or the network is missing at install time,
58
- `postinstall` prints a clear notice and the host falls back at runtime.
49
+ `KernelClient.start` spawns `python -m ipykernel -f <connection-file>` (a per-run connection file
50
+ in the temp dir), connects the three channels over ZMTP, and waits for `kernel_info_reply` before
51
+ declaring the kernel ready. Cells run as standard `execute_request`s, routed by `msg_id`:
59
52
 
60
- At spawn, `resolvePythonPath` uses exactly one interpreter, the install venv:
53
+ - **iopub** output: `stream`, `execute_result`, `display_data`, `error`, plus private-MIME
54
+ payloads for snapshot/restore/namespace data;
55
+ - **shell** — the authoritative `execute_reply` (status, ename, evalue);
56
+ - **control** — interrupts (`interrupt_request`) and shutdown.
61
57
 
62
- 1. `~/.pi/agent/pi-repl/venv` (the package install)
63
- 2. `$PYTHON`, then `python3` (only if the install venv is missing)
58
+ Four protocol details have contract tests.
64
59
 
65
- It deliberately does NOT auto-pick the repo's or current directory's `.venv`: a per-project
66
- venv is not guaranteed to have `ipykernel`, so preferring it (as earlier versions did) made
67
- the kernel die from a `ModuleNotFoundError` whenever cwd happened to contain such a venv. The
68
- kernel therefore always runs in the stable install environment. The kernel starts in the
69
- session's cwd, falling back to the host's own cwd if that directory no longer exists (a
70
- deleted project dir), so a stale cwd can never prevent the kernel from coming up.
60
+ **A cell settles only on two messages.** The shell reply and the iopub stream travel on different
61
+ connections, so a tiny reply can beat a large output. A cell completes only when **both** the
62
+ `execute_reply` and the matching iopub `status idle` (published after every byte) arrive;
63
+ settling on the reply alone would drop output still in flight.
71
64
 
72
- ## The kernel client
65
+ **Output is capped per channel and per line**, both announced with markers: channels accumulate
66
+ against `maxOutputChars` (checked within each message), and each line is capped at 4096 chars, so
67
+ one oversized line cannot own the budget while long JSON/reprs/errors pass whole.
73
68
 
74
- `KernelClient.start` spawns `python -m ipykernel -f <connection-file>` with a per-run
75
- connection file in the temp directory, connects the three channels as ZMTP sockets, and
76
- waits for a `kernel_info_reply` before declaring the kernel ready. Cells run as standard
77
- Jupyter `execute_request`s, routed by `msg_id`:
78
-
79
- - **iopub** carries output messages such as `stream`, `execute_result`, `display_data`, and
80
- `error`. It also carries private-MIME payloads for snapshot, restore, and namespace data.
81
- - **shell** carries the authoritative `execute_reply` (status, ename, evalue).
82
- - **control** carries interrupts (`interrupt_request`) and shutdown.
83
-
84
- Two details of this protocol are important enough to have dedicated contract tests.
85
-
86
- **A cell is not complete until two messages arrive.** The shell reply and the iopub output stream
87
- travel on different connections, so a tiny reply can arrive before a large output has
88
- finished draining on iopub. A cell settles only when **both** the `execute_reply` and the
89
- matching iopub `status idle` (published after every byte of output) have arrived. Settling
90
- on the reply alone would drop output that was still in flight.
91
-
92
- **Output is capped per channel and per line.** Each channel accumulates output against a character
93
- budget (`maxOutputChars`), checked within each message, so overflow trips the moment a message
94
- exceeds the budget rather than when it churns on. Each individual line is also capped at a generous
95
- length (4096 chars), so a single genuinely oversized line cannot own the whole budget — while
96
- legitimately long REPL output (JSON, reprs, errors) still passes through whole. Both truncations
97
- are announced with explicit markers so the model knows output was cut.
98
-
99
- **Cancellation is real.** An abort sends an `interrupt_request` on the control channel,
100
- which raises a genuine `KeyboardInterrupt` in the running cell; the namespace survives. As a
101
- backstop for cells wedged in C code (which ignore interrupts), the engine gives an aborted
102
- cell up to 20 seconds to settle and keeps the kernel if it does; only a cell that is still
103
- running after that grace is killed, and the next call rebuilds from the last snapshot.
104
-
105
- **History is off.** Every execute goes out with `store_history: false`. IPython's `In`/`Out`
106
- retention keeps every last-expression result object alive in the kernel, and that retention
107
- cannot be reclaimed from a user cell — deleting `Out` and `_`/`__`/`___` from `user_ns`
108
- followed by `gc.collect()` leaves the objects alive (measured: 62 MB idle grows past 400 MB
109
- after two bare big results and never comes back). Disabling history bounds the kernel to at
110
- most the latest result. The transcript is the record instead, and results still publish over
111
- iopub: single-mode execution calls `sys.displayhook` regardless of `store_history`.
69
+ **Cancellation is real.** An abort sends `interrupt_request`, raising a genuine
70
+ `KeyboardInterrupt`; the namespace survives. Cells wedged in C code (which ignore interrupts)
71
+ get a 20-second grace, then the kernel is killed and the next call rebuilds from the last
72
+ snapshot.
112
73
 
113
- ## Helpers loading
74
+ **History is off.** IPython's `In`/`Out` retention pins every last-expression result and cannot be
75
+ reclaimed from a user cell (measured: 62 MB → 400+ MB after two bare big results, unrecoverable),
76
+ so every execute goes out with `store_history: false`, bounding the kernel to the latest result.
77
+ Results still publish over iopub: single-mode execution calls `sys.displayhook` regardless.
114
78
 
115
- At boot, the kernel and the host both read the same merged helper list (project
116
- `.pi/helpers/` directories plus the global `~/.pi/agent/pi-repl/helpers/`), so what the
117
- prompt advertises is what the kernel holds. Both directories are optional; nothing ships
118
- with the package. The exact merge order is under "The fixed layout" below.
79
+ ## Helpers loading
119
80
 
120
- - **The kernel** executes each eligible `*.py` file in its namespace, so the file's definitions
121
- and imports become available.
122
- - **The host** reads the same files to build the helper list shown in the `execute` tool's
123
- prompt, so the model sees each `helper_description` verbatim.
81
+ At boot, the kernel and the host both read the same merged helper list (project `.pi/helpers/` up
82
+ to the git root, then the global `~/.pi/agent/pi-repl/helpers/`), so what the prompt advertises is
83
+ what the kernel holds: the kernel execs each eligible `*.py` into its namespace; the host reads
84
+ the same files' `helper_description` verbatim into the tool prompt. `_`-prefixed files are
85
+ skipped by both. Merge order and shadowing are under "The fixed layout" below.
124
86
 
125
- Both sides read the same list, so the names described to the model come from files the
126
- kernel also loads. A file renamed with a `_` prefix is skipped by both sides. The
127
- `promptGuidelines` are built once, when the `execute` tool is registered, so a helpers
128
- change needs a **session restart or `/reload`** to reach the prompt. The kernel also loads
129
- helpers only at boot.
87
+ The `promptGuidelines` are built once, when `execute` is registered, and the kernel loads helpers
88
+ only at boot, so a helpers change needs a **session restart or `/reload`**.
130
89
 
131
- There are no custom discovery intrinsics (`ls()` / `help()`) injected into a bare kernel.
132
- The model discovers what is loaded by listing the namespace with ordinary Python:
90
+ No discovery intrinsics (`ls()` / `help()`) are injected; list the namespace with ordinary Python:
133
91
 
134
92
  ```python
135
93
  [k for k in globals() if not k.startswith('_')]
136
94
  ```
137
95
 
138
- For the full helper contract, including descriptions, docstrings, and disabling, see
139
- [helpers.md](helpers.md).
96
+ Full contract: [helpers.md](helpers.md).
140
97
 
141
98
  ## Snapshots & honest resets
142
99
 
143
- After each successful cell, the host schedules a debounced snapshot. A private kernel cell
144
- pickles the kernel's `globals` (entry by entry, so one value that cannot be pickled costs
145
- only itself) and publishes the result back over a private MIME payload. The host stores it as
146
- `namespace.snapshot`, keyed to the session file under
147
- `~/.pi/agent/pi-repl/state/<session>/`.
148
-
149
- When a fresh engine is built, it restores that snapshot. Values are pickled entry by entry, and
150
- functions and classes defined in cells are captured by source and re-executed on restore (plain
151
- pickle cannot revive them, since they live in `__main__`). Bindings that still fail live
152
- handles, open resources, source-less functions are reported by name, never dropped silently.
153
- Entries are capped per-binding and in total (128 MiB default), and the snapshot file is written
154
- via temp-file-and-rename so a crash cannot corrupt the last good copy; old session snapshot
155
- directories are pruned to the newest 25, and snapshot dirs whose owning conversation file no
156
- longer exists in any project session root are swept entirely (deleting a conversation deletes
157
- its snapshots with it). "ephemeral" and the live session are always exempt. If the evaluator was rebuilt mid-session, the next
158
- cell's result is prefixed with a `<repl_engine_reset>` block that names what was revived and
159
- what was lost, so the model re-verifies before reusing state that may be gone. The human gets
160
- only a terse `ui.notify` toast ("repl kernel rebuilt, 3 names revived") instead of the marker;
161
- the two are derived from the same restore result, so they never disagree. A resumed
162
- conversation announces the same pair on its first cell, but only when the conversation has a
100
+ After each successful cell, a debounced snapshot pickles the kernel's `globals` entry by entry
101
+ (one un-picklable value costs only itself) and publishes it back over a private MIME payload; the
102
+ host stores it as `namespace.snapshot` under `~/.pi/agent/pi-repl/state/<session>/`.
103
+
104
+ A fresh engine restores that snapshot **in the background**: recovery is a quiet-gap job that
105
+ never runs ahead of a user cell, so a large revive does not delay the first cell; only a
106
+ mid-session rebuild (kernel death) forces the restore before the cell that found the kernel dead.
107
+ Functions and classes defined in cells are captured by source and re-executed on restore (plain
108
+ pickle cannot revive them in `__main__`); bindings that still fail are reported by name, never
109
+ dropped silently. Entries are capped per-binding and in total (128 MiB default); the file is
110
+ written via temp-file-and-rename so a crash cannot corrupt the last good copy; session dirs are
111
+ pruned to the newest 25, and dirs whose conversation file no longer exists are swept entirely
112
+ deleting a conversation deletes its snapshots. "ephemeral" and the live session are exempt.
113
+
114
+ A revive that never completes (a poisoned pickle) is bounded by an engine restore-cell watchdog
115
+ (`PI_REPL_BOOT_TIMEOUT_MS`, default 90s): the kernel is killed and the restore marked skipped
116
+ "wedged while reviving; skipped" instead of hanging the queue.
117
+
118
+ The first cell's result after a revive carries a `<repl_engine_reset>` block naming what was
119
+ revived and lost, so the model re-verifies before trusting state that may be gone; because
120
+ recovery is async, that is the first cell **after the restore completes** (usually the first cell
121
+ of a resumed conversation). The human gets only a terse `ui.notify` toast, derived from the same
122
+ restore result so the two never disagree. A resumed conversation announces only when it has a
163
123
  saved past; a first-ever session stays quiet.
164
124
 
165
125
  ## Failure modes
@@ -168,7 +128,7 @@ saved past; a first-ever session stays quiet.
168
128
  | --- | --- |
169
129
  | Cell throws | `error` status with traceback; kernel namespace intact |
170
130
  | Cell silent or wedged | the watchdog sends an `interrupt_request`; a caller abort kills the kernel only if it is still running after a 20-second grace |
171
- | Kernel dies | the running cell settles with an error; the next call builds a fresh kernel and restores the last snapshot |
131
+ | Kernel dies | the running cell settles with an error; the next call builds a fresh kernel and restores the last snapshot **before** the triggering cell |
172
132
  | Host exits | `process.on("exit")` SIGKILLs live kernels (a child does not die with its parent) |
173
133
  | Output flood | capped per channel, truncation announced |
174
134
 
@@ -181,14 +141,14 @@ saved past; a first-ever session stays quiet.
181
141
  snapshot/restore round-trips, output caps, silence timeout, abort, and rebuilding from a
182
142
  snapshot after the kernel dies.
183
143
 
184
- The gate is `just check`. It runs Biome formatting and linting, dead-code checks, and host tests.
185
- `just integration` adds the real-kernel suite.
144
+ The gate is `just check` (Biome formatting/lint, dead-code checks, host tests); `just integration`
145
+ adds the real-kernel suite.
186
146
 
187
147
  ## The fixed layout
188
148
 
189
- There is no configuration file. Most state lives under one directory in the user's home. A
190
- small number of environment variables can still change runtime behavior, such as the silence
191
- watchdog timeout.
149
+ There is no configuration file. Most state lives under one directory in the user's home. A small
150
+ number of environment variables can still change runtime behavior, such as the silence watchdog
151
+ timeout.
192
152
 
193
153
  ```
194
154
  ~/.pi/agent/pi-repl/
@@ -197,13 +157,24 @@ watchdog timeout.
197
157
  state/ per-session namespace snapshots
198
158
  ```
199
159
 
200
- Helpers merge project and global dirs: `resolveHelperDirs` walks from the working
201
- directory up to the git root collecting `.pi/helpers/`, then appends
202
- `~/.pi/agent/pi-repl/helpers`. Both the prompt loader and the kernel's `readHelperSources`
203
- walk the same ordered list with first-seen-wins, so a project helper shadows the same-named
204
- global one and both sides are guaranteed to agree. The venv is built automatically, and the
205
- interpreter follows the order above. No setting is needed. The per-cell silence watchdog is
206
- off by default (`PI_REPL_TIMEOUT_MS=0`: a silent but working cell may run on).
160
+ State dirs are keyed `<project-slug>__<conversation>` so two conversations that share a filename
161
+ can never share a snapshot (a pre-slug dir migrates on the owning conversation's next start), and
162
+ `EngineLifecycle.acquire` binds each engine to its conversation, tearing a foreign engine down
163
+ before building a new one sessions cannot bleed into each other.
164
+
165
+ Helpers merge project and global dirs: `resolveHelperDirs` walks up to the git root collecting
166
+ `.pi/helpers/`, then appends the global dir; the prompt loader and the kernel's `readHelperSources`
167
+ walk the same ordered list first-seen-wins, so a project helper shadows a same-named global one
168
+ and both sides agree. No setting is needed. The per-cell silence watchdog is off by default
169
+ (`PI_REPL_TIMEOUT_MS=0`).
170
+
171
+ Boot is bounded regardless: kernel start and helpers preload are kernel cells with no deadline of
172
+ their own, and `acquire()` dedupes, so one wedged boot (an npm update swapping the venv under a
173
+ live kernel, a hanging helper import) would hang the first cell and every cell after. The
174
+ lifecycle races each boot attempt against `PI_REPL_BOOT_TIMEOUT_MS` (default 90s): a wedged
175
+ attempt is killed and retried once with the snapshot deliberately skipped — "wedged while
176
+ reviving; skipped" — and a second wedge fails loudly. The same deadline bounds a restore whose
177
+ unpickling never returns.
207
178
 
208
179
  ## Reference documentation
209
180
 
package/index.ts CHANGED
@@ -8,6 +8,7 @@ import { withSkillsBlock } from "./src/extension/skill-hook.js";
8
8
  import { EngineManager, pruneOrphanedSnapshotDirs, pruneSnapshotDirs } from "./src/engine/index.js";
9
9
  import { ExecuteCellComponent, type ExecuteDetails, type ExecuteRenderState } from "./src/extension/render.js";
10
10
  import { EngineLifecycle, formatResetToast } from "./src/extension/session-engine.js";
11
+ import { conversationName, resolveStateDir } from "./src/extension/state-layout.js";
11
12
  import { EXECUTE_DESCRIPTION, buildExecutePromptGuidelines, EXECUTE_PROMPT_SNIPPET } from "./src/extension/tool-meta.js";
12
13
 
13
14
  const executeSchema = Type.Object({
@@ -58,28 +59,44 @@ export default function (pi: ExtensionAPI) {
58
59
  const pendingErrorResults = new Map<string, { details: ExecuteDetails }>();
59
60
 
60
61
  const lifecycle = new EngineLifecycle<EngineManager>({
61
- create() {
62
+ // --- boot deadline: bounds kernel start + helpers preload (recovery is a background
63
+ // --- quiet-gap job, bounded by the engine's own restore-cell watchdog). An npm update
64
+ // --- swaps the venv and helpers under a live kernel, and the first boot after it can
65
+ // --- wedge; without this the first cell hangs forever, because acquire() dedupes onto
66
+ // --- the same hung boot. ---
67
+ bootTimeoutMs: Number(process.env.PI_REPL_BOOT_TIMEOUT_MS ?? 90_000) || 90_000,
68
+ create(skipRestore = false) {
62
69
  const { cwd, sessionFile } = location;
63
- const sessionKey = sessionFile ? basename(sessionFile).replace(/\.jsonl$/, "") : undefined;
64
- // --- kernel namespace state lives under ~/.pi/agent/pi-repl, keyed by session, so it never clutters the project ---
65
- const stateDir = join(homedir(), ".pi", "agent", "pi-repl", "state", sessionKey ?? "ephemeral");
66
- // --- keep the state root from growing one dir per session forever; the live dir is exempt ---
67
- if (sessionKey) {
70
+ // --- kernel namespace state lives under ~/.pi/agent/pi-repl/state, keyed by
71
+ // --- <project-slug>__<conversation>, so it never clutters the project and two
72
+ // --- conversations can never share a snapshot dir (resolveStateDir migrates any
73
+ // --- pre-slug legacy dir on start). Ephemeral sessions get no snapshot at all. ---
74
+ const stateRoot = join(homedir(), ".pi", "agent", "pi-repl", "state");
75
+ let snapshot: { path: string } | undefined;
76
+ let currentDir: string | undefined;
77
+ if (sessionFile) {
78
+ const { dir, snapshotPath } = resolveStateDir(stateRoot, sessionFile);
79
+ currentDir = basename(dir);
80
+ snapshot = { path: snapshotPath };
81
+ // --- keep the state root from growing one dir per session forever; the live dir is exempt ---
68
82
  try {
69
- pruneSnapshotDirs(join(stateDir, ".."), 25, sessionKey);
83
+ pruneSnapshotDirs(stateRoot, 25, currentDir);
70
84
  } catch {}
71
85
  // --- cascade deletions: if a conversation is deleted, its snapshots die with it.
72
86
  // --- sessionFile is sessions/<project-root>/<name>.jsonl, so the sessions root is
73
87
  // --- two parent hops up; dirs whose conversation file exists in no project root
74
- // --- (and that aren't this session or the ephemeral fallback) are swept. ---
88
+ // --- (and that aren't this session or the ephemeral fallback) are swept, in both
89
+ // --- the legacy bare-name and slug-keyed formats. ---
75
90
  try {
76
- pruneOrphanedSnapshotDirs(join(stateDir, ".."), sessionFile ? dirname(dirname(sessionFile)) : undefined, sessionKey);
91
+ pruneOrphanedSnapshotDirs(stateRoot, sessionFile ? dirname(dirname(sessionFile)) : undefined, currentDir);
77
92
  } catch {}
78
93
  }
79
94
  return new EngineManager({
80
95
  cwd,
81
- // --- snapshots are keyed to a session file; ephemeral sessions get none ---
82
- snapshot: sessionKey ? { path: join(stateDir, "namespace.snapshot") } : undefined,
96
+ // --- snapshots are keyed to the conversation; ephemeral sessions get none.
97
+ // --- skipRestore is true on the lifecycle's retry after a wedged boot. ---
98
+ snapshot,
99
+ skipRestore,
83
100
  });
84
101
  },
85
102
  async dispose(engine) {
@@ -104,7 +121,8 @@ export default function (pi: ExtensionAPI) {
104
121
  // --- warm the engine (and its revive) in the background; no popup. ---
105
122
  // --- acquire() dedupes, so the first execute awaits this same in-flight boot ---
106
123
  location = { cwd: ctx.cwd, sessionFile: ctx.sessionManager.getSessionFile() ?? undefined };
107
- void lifecycle.acquire("startup").catch(() => {
124
+ const sessionKey = location.sessionFile ? conversationName(location.sessionFile) : undefined;
125
+ void lifecycle.acquire("startup", sessionKey).catch(() => {
108
126
  // --- boot/revive handled on the execute path; swallow so a background warm can never
109
127
  // --- surface an unhandled rejection. A resume's notice lands on the first cell. ---
110
128
  });
@@ -170,7 +188,8 @@ export default function (pi: ExtensionAPI) {
170
188
  // --- without this the host only renders the result once the first partial or the final result lands ---
171
189
  onUpdate?.({ content: [], details: {} });
172
190
  // --- previous engine died mid-session; acquire revives it ---
173
- const { engine: m } = await lifecycle.acquire("cell");
191
+ const sessionKey = location.sessionFile ? conversationName(location.sessionFile) : undefined;
192
+ const { engine: m } = await lifecycle.acquire("cell", sessionKey);
174
193
  try {
175
194
  // --- accumulate partial updates so the row height doesn't oscillate ---
176
195
  let streamed = "";
@@ -184,7 +203,7 @@ export default function (pi: ExtensionAPI) {
184
203
  // --- reset notice leads so the model reads that its namespace was rebuilt; the
185
204
  // --- human gets a terse notification instead of the marker, fire and forget ---
186
205
  const reset = lifecycle.takeResetNotice();
187
- if (reset?.notice) ctx?.ui?.notify?.(formatResetToast(reset.origin, reset.restore), "info");
206
+ if (reset?.notice) ctx?.ui?.notify?.(formatResetToast(reset.origin, reset.restore, reset.wedged), "info");
188
207
  const sections = [reset?.notice, r.stdout, r.stderr, r.result];
189
208
  const errorLines = r.error ? composeErrorLines(r.error) : undefined;
190
209
  if (r.status === "error" && errorLines) sections.push(errorLines.join("\n"));
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-repl-py",
3
- "version": "0.6.12",
3
+ "version": "0.6.14",
4
4
  "type": "module",
5
5
  "description": "A pi extension with a single tool: execute, running a TypeScript host with a persistent Python (ipykernel) evaluator and a user-configurable toolbox of functions.",
6
6
  "keywords": [
@@ -30,6 +30,12 @@ const DEFAULT_SNAPSHOT_DEBOUNCE_MS = 1500;
30
30
  /** Total snapshot size cap (base64 payload). Per-entry entries are capped at the same
31
31
  * bound; larger bindings are reported as skipped names. Mirrors the pi-codex scheme. */
32
32
  const DEFAULT_SNAPSHOT_MAX_BYTES = 128 * 1024 * 1024;
33
+ /** Quiet-gap window before the background restore fires after boot; never ahead of a user cell. */
34
+ const RESTORE_QUIET_MS = 250;
35
+ /** Deadline for one restore cell (a poisoned pickle can wedge the kernel's single queue for
36
+ * ever). The reaper kills the kernel and marks the restore skipped so the next call rebuilds
37
+ * honestly. Mirrors the boot deadline in the lifecycle; also settable per engine via env. */
38
+ const DEFAULT_RESTORE_DEADLINE_MS = 90_000;
33
39
 
34
40
  interface EngineExecuteError {
35
41
  /** Error class name, e.g. "TypeError". */
@@ -82,6 +88,8 @@ export interface EngineOptions {
82
88
  /** Total base64 payload cap; also the per-entry cap. Oversized entries are skipped with a reason. Default 128 MiB. */
83
89
  maxBytes?: number;
84
90
  };
91
+ /** Do not revive the snapshot on this engine (used after a wedged restore was detected once). */
92
+ skipRestore?: boolean;
85
93
  }
86
94
 
87
95
  // --- process-wide cleanup: a child does not die with its parent, so SIGKILL live kernels on exit ---
@@ -158,7 +166,12 @@ export function pruneOrphanedSnapshotDirs(
158
166
  for (const proj of readdirSync(sessionsRoot, { withFileTypes: true })) {
159
167
  if (!proj.isDirectory()) continue;
160
168
  for (const f of readdirSync(join(sessionsRoot, proj.name))) {
161
- if (f.endsWith(".jsonl")) liveNames.add(f.slice(0, -".jsonl".length));
169
+ if (!f.endsWith(".jsonl")) continue;
170
+ const name = f.slice(0, -".jsonl".length);
171
+ // --- both state-dir formats are live while their conversation lives: the legacy
172
+ // --- bare-name dir (pre-slug upgrade) and the slug-keyed dir (see state-layout) ---
173
+ liveNames.add(name);
174
+ liveNames.add(`${proj.name}__${name}`);
162
175
  }
163
176
  }
164
177
  } catch {
@@ -191,9 +204,20 @@ export class EngineManager {
191
204
  /** Last-seen top-level namespace names; snapshots are gated on this set changing. */
192
205
  private lastNamespaceNames?: string[];
193
206
  private pythonPath?: string;
207
+ /** The kernel whose namespace has (or is being) revived from the last snapshot. */
208
+ private restoredKernel?: KernelClient;
209
+ /** A wedged revive marks the engine: later kernels on this engine boot without restoring. */
210
+ private restoreSkipped: boolean;
211
+ private restoreTimer?: ReturnType<typeof setTimeout>;
212
+ private restoreResolve?: (result: RestoreResult | null) => void;
213
+ private restorePromise?: Promise<RestoreResult | null>;
214
+ private restoreSettledResult?: RestoreResult | null;
194
215
 
195
216
  constructor(options: EngineOptions = {}) {
196
217
  this.options = options;
218
+ this.restoreSkipped = options.skipRestore ?? false;
219
+ // no snapshot capability: recovery is trivially "nothing to revive"
220
+ if (!options.snapshot) this.settleRestore(null);
197
221
  }
198
222
 
199
223
  get isRunning(): boolean {
@@ -255,6 +279,9 @@ export class EngineManager {
255
279
  throw new Error("Engine has been shut down");
256
280
  }
257
281
  this.state = "running";
282
+ // recovery is not on the first call's critical path: revive this kernel in the
283
+ // first quiet gap (never ahead of a user cell) and settle restoreResult().
284
+ this.maybeScheduleRestore();
258
285
  }
259
286
 
260
287
  /** Abrupt teardown: SIGKILL the kernel; safe from process.on("exit"). */
@@ -302,6 +329,19 @@ export class EngineManager {
302
329
  this.kernel = undefined;
303
330
  this.startPromise = undefined;
304
331
  await this.start();
332
+ // --- a mid-session rebuild must revive the last snapshot BEFORE the cell that
333
+ // --- triggered it (unlike a session-start boot, where recovery runs in the
334
+ // --- background quiet gap and the first cell is served immediately). A wedged
335
+ // --- revive kills the new kernel too; boot a fresh one — the restore is now
336
+ // --- marked skipped, so the cell proceeds on live state instead of wedging. ---
337
+ await this.restoreWithReap().catch(() => null);
338
+ // read health through the getter: TS narrows this.kernel to undefined after the
339
+ // assignment above, but start() may have replaced it with a live kernel
340
+ if (!this.isRunning) {
341
+ this.kernel = undefined;
342
+ this.startPromise = undefined;
343
+ await this.start();
344
+ }
305
345
  }
306
346
  if (this.isShutdown()) {
307
347
  throw new Error("Engine has been shut down");
@@ -387,11 +427,123 @@ export class EngineManager {
387
427
  }
388
428
  }
389
429
 
390
- async restoreState(): Promise<RestoreResult | null> {
430
+ /** Resolves with this engine's recovery outcome: the revived names, or null when there was
431
+ * nothing to restore, restore was skipped, or restore failed. Never rejects. The restore runs
432
+ * as a background quiet-gap job, so this promise is the ONLY hook the lifecycle needs to
433
+ * announce a resume or rebuild — acquire() never awaits it. */
434
+ restoreResult(): Promise<RestoreResult | null> {
435
+ if (this.restoreSettledResult !== undefined) return Promise.resolve(this.restoreSettledResult);
436
+ if (!this.restorePromise) {
437
+ this.restorePromise = new Promise<RestoreResult | null>((resolve) => {
438
+ this.restoreResolve = resolve;
439
+ });
440
+ }
441
+ return this.restorePromise;
442
+ }
443
+
444
+ /** True when restoring was deliberately skipped (a prior revive wedged or the engine was
445
+ * built with skipRestore). Lets the lifecycle say exactly why a revival did not happen. */
446
+ restoreWasSkipped(): boolean {
447
+ return this.restoreSkipped;
448
+ }
449
+
450
+ private settleRestore(result: RestoreResult | null): void {
451
+ if (this.restoreSettledResult !== undefined) return;
452
+ this.restoreSettledResult = result;
453
+ this.restoreResolve?.(result);
454
+ this.restoreResolve = undefined;
455
+ }
456
+
457
+ /** Arm the background revive for the freshly booted kernel. It fires in the first quiet gap
458
+ * (the same rule as the debounced snapshot: never ahead of a user cell) and settles
459
+ * restoreResult(). A fresh engine therefore serves its first cell without waiting for the
460
+ * restore, while a mid-session rebuild (execute's zombie path) forces it synchronously. */
461
+ private maybeScheduleRestore(): void {
462
+ const config = this.options.snapshot;
463
+ if (!config) return;
464
+ if (this.restoreSkipped) {
465
+ this.settleRestore(null);
466
+ return;
467
+ }
468
+ if (this.kernel && this.kernel === this.restoredKernel) return; // this kernel already revived
469
+ if (!existsSync(config.path)) {
470
+ this.settleRestore(null);
471
+ return;
472
+ }
473
+ if (this.restoreTimer) return; // already armed
474
+ const arm = () => {
475
+ this.restoreTimer = setTimeout(() => {
476
+ this.restoreTimer = undefined;
477
+ // --- quiet-gap rule, mirroring scheduleSnapshot: a pickling restore must not
478
+ // --- queue ahead of the user's next cell on the kernel's single queue ---
479
+ if (this.inFlightCells > 0 || !this.kernel?.isRunning) {
480
+ arm();
481
+ return;
482
+ }
483
+ void this.runRestore();
484
+ }, RESTORE_QUIET_MS);
485
+ this.restoreTimer.unref?.();
486
+ };
487
+ arm();
488
+ }
489
+
490
+ /** Run the restore cell with a watchdog. A snapshot value whose unpickling never returns
491
+ * wedges the kernel's single queue forever; the reaper SIGKILLs the kernel and marks the
492
+ * restore skipped, so the next call rebuilds honestly ("wedged while reviving; skipped")
493
+ * instead of hanging every later cell behind the restore. */
494
+ private async restoreWithReap(): Promise<RestoreResult | null> {
391
495
  const config = this.options.snapshot;
392
- if (!config) return null;
393
- if (!existsSync(config.path)) return null;
496
+ if (!config || !this.kernel || this.restoreSkipped) {
497
+ this.settleRestore(null);
498
+ return null;
499
+ }
500
+ if (this.kernel === this.restoredKernel) return this.restoreResult();
501
+ const deadlineMs =
502
+ Number(process.env.PI_REPL_BOOT_TIMEOUT_MS ?? this.options.env?.PI_REPL_BOOT_TIMEOUT_MS ?? 0) ||
503
+ DEFAULT_RESTORE_DEADLINE_MS;
504
+ const reaper = setTimeout(() => {
505
+ this.restoreSkipped = true;
506
+ this.settleRestore(null);
507
+ this.kernel?.kill();
508
+ }, deadlineMs);
509
+ reaper.unref?.();
510
+ try {
511
+ return await this.restoreState(false).catch(() => null);
512
+ } finally {
513
+ clearTimeout(reaper);
514
+ }
515
+ }
516
+
517
+ private async runRestore(): Promise<void> {
518
+ await this.restoreWithReap();
519
+ }
520
+
521
+ /** Revive this engine's kernel from the snapshot file. Idempotent per kernel: a second call
522
+ * shares the outcome of an in-flight restore instead of double-running the restore cell. */
523
+ async restoreState(skip = false): Promise<RestoreResult | null> {
524
+ // --- start unconditionally: direct callers may not have started the engine, and a
525
+ // --- wedged boot is detectable only while a boot attempt is actually under way ---
394
526
  await this.start();
527
+ if (skip) {
528
+ this.settleRestore(null);
529
+ return null;
530
+ }
531
+ const config = this.options.snapshot;
532
+ const kernel = this.kernel;
533
+ if (!config || !kernel || this.restoreSkipped) {
534
+ this.settleRestore(null);
535
+ return null;
536
+ }
537
+ if (kernel === this.restoredKernel) {
538
+ // already revived or reviving on this kernel: share the outcome, never double-run
539
+ return this.restoreResult();
540
+ }
541
+ // claim the kernel now so the quiet-gap scheduler cannot start a second restore cell
542
+ this.restoredKernel = kernel;
543
+ if (!existsSync(config.path)) {
544
+ this.settleRestore(null);
545
+ return null;
546
+ }
395
547
  try {
396
548
  const payload = JSON.parse(readFileSync(config.path, "utf8")) as {
397
549
  version?: number;
@@ -403,9 +555,12 @@ export class EngineManager {
403
555
  payload.version === 2
404
556
  ? (payload.entries ?? [])
405
557
  : Object.entries(payload.vars ?? {}).map(([name, b64]) => ({ name, kind: "value", payload: b64 }));
406
- const reply = await this.kernel!.restore(entries);
407
- return { path: config.path, restored: reply.restored, failed: reply.failed };
558
+ const reply = await kernel.restore(entries);
559
+ const result: RestoreResult = { path: config.path, restored: reply.restored, failed: reply.failed };
560
+ this.settleRestore(result);
561
+ return result;
408
562
  } catch {
563
+ this.settleRestore(null);
409
564
  return null;
410
565
  }
411
566
  }
@@ -14,7 +14,7 @@ export function buildPromptGuidelines(preloaded: string[]): string[] {
14
14
  return [
15
15
  "Write modern idiomatic Python.",
16
16
  "Find, filter, fetch, sample: narrow the output in Python, then print only the window that decides the next step (a head, a shape, a slice), not the whole.",
17
- "Make surgical, precise changes over rewrites or whole-file dumps: a small unique anchor, replace, verify, read the file back before trusting it.",
17
+ "Make surgical, precise changes over rewrites or whole-file dumps: replace, verify, read the file back before trusting it.",
18
18
  "Prefer to reuse existing variables, functions, imports, classes, and data from prior cells/namespaces over recomputing.",
19
19
  "If output begins with <repl_engine_reset>, the runtime rebuilt and the notebook was restored from the last snapshot; reverify surviving names before building on them.",
20
20
  ...(preloaded.length
@@ -517,6 +517,9 @@ function renderOutput(
517
517
  }
518
518
 
519
519
  if (details?.errorStack && details.errorStack.length > 0) {
520
+ // --- a traceback IS output: without this flag a pure-traceback error cell (no
521
+ // --- stdout/stderr/result, the common error shape) also rendered "no output" below it ---
522
+ renderedText = true;
520
523
  output.push(` ${OUTPUT_INDENT}${deps.fg("dim", "traceback:")}`);
521
524
  for (const line of details.errorStack) {
522
525
  const safe = sanitizeTuiOutput(line || " ");
@@ -529,6 +532,13 @@ function renderOutput(
529
532
  addWrapped(output, OUTPUT_INDENT, deps.fg("muted", message), width, deps, { sanitize: false });
530
533
  }
531
534
 
535
+ // --- bottom cushion: streams end with a newline, so blobs already render a trailing blank
536
+ // --- row; traceback and the placeholder have no such newline and would sit flush against
537
+ // --- the panel's bottom edge. Normalize: the panel always ends with one blank painted row.
538
+ // --- (SGR stripped before the blank test; the colored rows themselves stay untouched.) ---
539
+ const lastRow = output[output.length - 1];
540
+ if (lastRow !== undefined && lastRow.replace(SGR_PATTERN, "").trim() !== "") output.push("");
541
+
532
542
  // --- expanded cells render the whole output: the data cap bounds it ---
533
543
  if (output.length > 0 && hasCode) lines.push("");
534
544
  lines.push(...output);
@@ -1,4 +1,6 @@
1
- // revival is part of create() so a session that gets teardown without a session_start reload still revives
1
+ // The lifecycle owns boot, session binding, and the reset announcements. Recovery is a
2
+ // background engine job (the quiet-gap restore): the first tool call waits only for the
3
+ // kernel to come up, and the reset notice lands on the first cell AFTER the restore lands.
2
4
 
3
5
  import type { RestoreResult } from "../engine/index.js";
4
6
 
@@ -10,44 +12,59 @@ function summarizeNames(names: readonly string[], limit: number): string {
10
12
 
11
13
  /** The part of EngineManager this lifecycle needs; narrowed so tests can fake it. */
12
14
  export interface RevivableEngine {
13
- restoreState(): Promise<RestoreResult | null>;
15
+ /** Boot the kernel (and preload helpers), independent of snapshot recovery. */
16
+ start(skipRestore?: boolean): Promise<void>;
17
+ /** Resolves once this engine's recovery has settled: revived names, or null when there was
18
+ * nothing to revive, recovery was skipped, or recovery failed. Never rejects. */
19
+ restoreResult(): Promise<RestoreResult | null>;
20
+ /** True when recovery was deliberately skipped (a prior revive wedged). */
21
+ restoreWasSkipped(): boolean;
14
22
  /** True when this conversation's state dir already exists, so the engine was resumed. */
15
23
  hasSnapshotHistory(): boolean;
16
24
  }
17
25
 
18
26
  export interface EngineLifecycleDeps<E extends RevivableEngine> {
19
- /** Builds a fresh engine. Called at most once per lifecycle generation. */
20
- create(): E;
27
+ /** Builds a fresh engine. Called at most once per lifecycle generation; `skipRestore` is
28
+ * true on the retry after a wedged boot, so the poisoned snapshot cannot wedge twice. */
29
+ create(skipRestore?: boolean): E;
21
30
  /** Tears the current engine down, flushing its final snapshot. */
22
31
  dispose(engine: E): Promise<void>;
23
32
  /** Kill-then-rebuild when a wedged engine cannot serve the snapshot flush. */
24
33
  discard?(engine: E): Promise<void>;
34
+ /** Boot deadline in ms; a boot (kernel start + helpers preload) that outlives it is killed
35
+ * and retried fresh. Default 90s. Recovery is NOT inside this deadline: it runs in the
36
+ * background and is bounded by the engine's own restore-cell watchdog. */
37
+ bootTimeoutMs?: number;
25
38
  }
26
39
 
27
- /** `startup` restores then announces on the first cell when the conversation has a saved past; `cell` means an engine was rebuilt mid-session and announces immediately. */
40
+ /** `startup` restores then announces when the conversation has a saved past; `cell` means an engine was rebuilt mid-session and announces immediately. */
28
41
  export type AcquireOrigin = "startup" | "cell";
29
42
 
43
+ const DEFAULT_BOOT_TIMEOUT_MS = 90_000;
44
+
45
+ /** Model-facing body for a boot whose snapshot revive wedged and was skipped. */
46
+ function revivedNoticeBody(origin: AcquireOrigin): string {
47
+ const resumed = origin === "startup";
48
+ return resumed
49
+ ? "This session's evaluator wedged while reviving the saved namespace, so the snapshot was skipped and the namespace is empty."
50
+ : "The evaluator wedged while reviving its saved namespace, so the snapshot was skipped and the namespace is empty.";
51
+ }
52
+
30
53
  // --- Terse TUI toast for the human, separate from the model-facing cell marker: the user
31
54
  // --- asked for the classic subtle notification instead of a showy in-cell message. Counts
32
55
  // --- come from the same restore the marker describes, so the two never disagree. ---
33
- export function formatResetToast(origin: AcquireOrigin, restore: RestoreResult | null): string {
56
+ export function formatResetToast(origin: AcquireOrigin, restore: RestoreResult | null, wedged = false): string {
34
57
  const resumed = origin === "startup";
58
+ const verb = resumed ? "repl session resumed" : "repl kernel rebuilt";
59
+ if (wedged) return `${verb}, snapshot revive skipped (wedged)`;
35
60
  const revived = restore?.restored.length ?? 0;
36
61
  const lost = restore?.failed.length ?? 0;
37
62
  if (restore && revived > 0) {
38
63
  const counts = lost > 0 ? `, ${lost} lost` : "";
39
64
  const noun = revived === 1 ? "name" : "names";
40
- return resumed
41
- ? `repl session resumed, ${revived} ${noun} revived${counts}`
42
- : `repl kernel rebuilt, ${revived} ${noun} revived${counts}`;
65
+ return `${verb}, ${revived} ${noun} revived${counts}`;
43
66
  }
44
- return resumed
45
- ? restore === null
46
- ? "repl session resumed, nothing saved to revive"
47
- : "repl session resumed, nothing could be revived"
48
- : restore === null
49
- ? "repl kernel rebuilt, nothing saved to revive"
50
- : "repl kernel rebuilt, nothing could be revived";
67
+ return restore === null ? `${verb}, nothing saved to revive` : `${verb}, nothing could be revived`;
51
68
  }
52
69
 
53
70
  function formatEngineResetNotice(restore: RestoreResult | null, origin: AcquireOrigin): string {
@@ -103,38 +120,99 @@ function formatEngineResetNotice(restore: RestoreResult | null, origin: AcquireO
103
120
 
104
121
  export class EngineLifecycle<E extends RevivableEngine> {
105
122
  private engine?: E;
106
- private revival?: Promise<RestoreResult | null>;
107
123
  private pendingNotice?: string;
108
- private pendingReset?: { origin: AcquireOrigin; restore: RestoreResult | null };
124
+ private pendingReset?: { origin: AcquireOrigin; restore: RestoreResult | null; wedged: boolean };
109
125
  private teardown?: Promise<void>;
110
126
  /** First-build in progress. */
111
127
  private acquiring?: Promise<{ engine: E; restore: RestoreResult | null; created: boolean }>;
128
+ /** The conversation this engine was built for; a different key on acquire tears it down. */
129
+ private boundKey?: string;
112
130
 
113
131
  constructor(private readonly deps: EngineLifecycleDeps<E>) {}
114
132
 
115
- /** Built and revived on demand; awaited so callers never see an un-revived namespace. */
116
- async acquire(origin: AcquireOrigin): Promise<{ engine: E; restore: RestoreResult | null; created: boolean }> {
133
+ /** Race one BOOT attempt (kernel start + helpers preload) against the deadline. Recovery is
134
+ * deliberately outside this race: it runs as a background quiet-gap job and is bounded by the
135
+ * engine's own restore-cell watchdog. A failed start is soft — the first cell observes it and
136
+ * the caller rebuilds. */
137
+ private bootOnce(engine: E, deadlineMs: number): Promise<boolean> {
138
+ const work = Promise.resolve()
139
+ .then(() => engine.start(false))
140
+ .catch(() => {});
141
+ let timer: ReturnType<typeof setTimeout> | undefined;
142
+ const guard = new Promise<boolean>((resolve) => {
143
+ timer = setTimeout(() => resolve(false), deadlineMs);
144
+ timer.unref?.();
145
+ });
146
+ return Promise.race([work.then(() => true), guard]).finally(() => clearTimeout(timer));
147
+ }
148
+
149
+ /**
150
+ * Built and booted on demand; the snapshot restore proceeds in the background, so acquire()
151
+ * NEVER waits on it. The engine's revive is announced (reset notice + toast) on the first
152
+ * cell after it completes.
153
+ *
154
+ * `sessionKey` guards against sessions bleeding into each other: pi tears the old session
155
+ * down before starting the next, but a missed or out-of-order shutdown must never serve one
156
+ * conversation's engine and namespace to another — acquire for a different key tears the
157
+ * bound engine down (flushing its snapshot) before building the new one.
158
+ */
159
+ async acquire(
160
+ origin: AcquireOrigin,
161
+ sessionKey?: string,
162
+ ): Promise<{ engine: E; restore: RestoreResult | null; created: boolean }> {
163
+ if (sessionKey !== undefined && this.boundKey !== undefined && sessionKey !== this.boundKey) {
164
+ await this.teardownWith((engine) => this.deps.dispose(engine));
165
+ }
117
166
  if (this.engine) {
118
- return { engine: this.engine, restore: await this.revival!, created: false };
167
+ return { engine: this.engine, restore: null, created: false };
119
168
  }
120
169
  if (this.acquiring) return this.acquiring;
121
170
  const build = (async () => {
122
171
  while (this.teardown) await this.teardown;
123
172
  if (this.engine) {
124
173
  const held: E = this.engine;
125
- return { engine: held, restore: await this.revival!, created: false };
174
+ return { engine: held, restore: null, created: false };
126
175
  }
127
- const engine = this.deps.create();
176
+ this.boundKey = sessionKey;
177
+ const deadline = this.deps.bootTimeoutMs ?? DEFAULT_BOOT_TIMEOUT_MS;
178
+ let engine = this.deps.create();
128
179
  this.engine = engine;
129
- this.revival = engine.restoreState().catch(() => null);
130
- const restore = await this.revival;
131
- // --- mid-session rebuilds always announce; startup announces only when the
132
- // --- conversation has a saved past, so a first-ever session stays quiet ---
133
- if (origin === "cell" || (origin === "startup" && engine.hasSnapshotHistory())) {
134
- this.pendingNotice = formatEngineResetNotice(restore, origin);
135
- this.pendingReset = { origin, restore };
180
+ let booted = await this.bootOnce(engine, deadline);
181
+ if (!booted) {
182
+ // --- the kernel is alive but stuck; only a kill frees it. Retry once WITHOUT the
183
+ // --- snapshot, so a poisoned snapshot cannot wedge the session twice in a row. ---
184
+ await (this.deps.discard ?? this.deps.dispose)(engine);
185
+ engine = this.deps.create(true);
186
+ this.engine = engine;
187
+ booted = await this.bootOnce(engine, deadline);
188
+ if (!booted) {
189
+ await (this.deps.discard ?? this.deps.dispose)(engine);
190
+ this.engine = undefined;
191
+ this.boundKey = undefined;
192
+ throw new Error("evaluator boot timed out twice (kernel/helpers wedged); no session was started");
193
+ }
136
194
  }
137
- return { engine, restore, created: true };
195
+ // --- recovery is async and off the first call's critical path: the engine revives in
196
+ // --- the first quiet gap and the notice lands on the first cell AFTER it completes
197
+ // --- (index.ts takes it with takeResetNotice after the next execute). announce when
198
+ // --- mid-session rebuilds happen, or on startup for a conversation with a saved past;
199
+ // --- a first-ever session stays quiet. ---
200
+ const announce = origin === "cell" || (origin === "startup" && engine.hasSnapshotHistory());
201
+ void engine.restoreResult().then((restore) => {
202
+ if (this.engine !== engine) return; // a replacement engine took over; no stale notice
203
+ if (!announce) return;
204
+ const wedged = engine.restoreWasSkipped();
205
+ this.pendingNotice = wedged
206
+ ? [
207
+ "<repl_engine_reset>",
208
+ revivedNoticeBody(origin),
209
+ "Re-verify a variable before reusing it, especially inside shell interpolation.",
210
+ "</repl_engine_reset>",
211
+ ].join("\n")
212
+ : formatEngineResetNotice(restore, origin);
213
+ this.pendingReset = { origin, restore, wedged };
214
+ });
215
+ return { engine, restore: null, created: true };
138
216
  })();
139
217
  this.acquiring = build;
140
218
  try {
@@ -144,14 +222,17 @@ export class EngineLifecycle<E extends RevivableEngine> {
144
222
  }
145
223
  }
146
224
 
147
- /** Returns the pending reset notice exactly once (alongside its origin and restore result), then clears it. */
148
- takeResetNotice(): { notice: string; origin: AcquireOrigin; restore: RestoreResult | null } | undefined {
225
+ /** Returns the pending reset notice exactly once (alongside its origin, restore result, and
226
+ * whether the restore was skipped), then clears it. */
227
+ takeResetNotice():
228
+ | { notice: string; origin: AcquireOrigin; restore: RestoreResult | null; wedged: boolean }
229
+ | undefined {
149
230
  const reset = this.pendingReset;
150
231
  const notice = this.pendingNotice;
151
232
  this.pendingNotice = undefined;
152
233
  this.pendingReset = undefined;
153
234
  if (!notice || !reset) return undefined;
154
- return { notice, origin: reset.origin, restore: reset.restore };
235
+ return { notice, origin: reset.origin, restore: reset.restore, wedged: reset.wedged };
155
236
  }
156
237
 
157
238
  async shutdown(): Promise<void> {
@@ -166,7 +247,7 @@ export class EngineLifecycle<E extends RevivableEngine> {
166
247
  private async teardownWith(run: (engine: E) => Promise<void>): Promise<void> {
167
248
  const engine = this.engine;
168
249
  this.engine = undefined;
169
- this.revival = undefined;
250
+ this.boundKey = undefined;
170
251
  this.pendingNotice = undefined;
171
252
  if (!engine) return;
172
253
  const teardown = run(engine).finally(() => {
@@ -0,0 +1,45 @@
1
+ // --- where a conversation's kernel state lives: ~/.pi/agent/pi-repl/state/<key>.
2
+ // --- Keys must never collide across conversations, so the project-root slug joins the
3
+ // --- conversation name. Pre-slug legacy dirs (bare name) migrate to the slug key on the
4
+ // --- owning conversation's next start; the orphan sweep still recognizes both formats,
5
+ // --- so nothing live is ever swept and a deleted conversation loses all its snapshots. ---
6
+ import { existsSync, renameSync } from "node:fs";
7
+ import { basename, dirname, join } from "node:path";
8
+
9
+ /** The conversation's own name: the session file's basename without .jsonl (unique per conversation). */
10
+ export function conversationName(sessionFile: string): string {
11
+ return basename(sessionFile).replace(/\.jsonl$/, "");
12
+ }
13
+
14
+ /** Slug-keyed state dir name: unique among all conversations under one sessions root, so two
15
+ * conversations whose files happen to share a basename (copied/renamed session files) can never
16
+ * share a snapshot. */
17
+ export function sessionStateDirName(sessionFile: string): string {
18
+ return `${basename(dirname(sessionFile))}__${conversationName(sessionFile)}`;
19
+ }
20
+
21
+ /** The pre-slug dir name; still honored when migrating or scanning live conversations. */
22
+ function legacyStateDirName(sessionFile: string): string {
23
+ return conversationName(sessionFile);
24
+ }
25
+
26
+ /**
27
+ * Resolve a conversation's state dir and snapshot file, migrating a legacy bare-name dir to the
28
+ * slug key on first start. Two conversations whose files share a basename therefore never share a
29
+ * snapshot file: whichever starts first migrates the legacy dir to its own key; the other starts
30
+ * empty rather than bleeding into the first's namespace.
31
+ */
32
+ export function resolveStateDir(stateRoot: string, sessionFile: string): { dir: string; snapshotPath: string } {
33
+ const dir = join(stateRoot, sessionStateDirName(sessionFile));
34
+ const legacy = join(stateRoot, legacyStateDirName(sessionFile));
35
+ const dirSnap = join(dir, "namespace.snapshot");
36
+ const legacySnap = join(legacy, "namespace.snapshot");
37
+ if (legacy !== dir && !existsSync(dirSnap) && existsSync(legacySnap)) {
38
+ try {
39
+ renameSync(legacy, dir); // migrate; a racing rename (another conversation) just falls through
40
+ } catch {
41
+ // the slug dir already exists or the rename raced: keep it; legacy stays until the sweep sees it live or orphaned
42
+ }
43
+ }
44
+ return { dir, snapshotPath: join(dir, "namespace.snapshot") };
45
+ }