pi-repl-py 0.1.1 → 0.2.0

package/docs/philosophy.md

@@ -2,87 +2,95 @@
 
 ## The bet
 
-Most coding agents are a pile of point tools. A read tool, a bash tool, an edit
-tool, a find tool, each with its own schema, its own failure modes, its own
-token cost to describe. The model spends context deciding *which* tool, then
-*how* the output should be threaded into the next one.
+Most coding agents carry a toolbox of point tools: a read tool, a bash tool, an edit tool,
+a search tool, each with its own schema, its own failure modes, and its own token cost to
+describe. The model spends context deciding *which* tool to call, then *how* to thread one
+tool's output into the next.
 
-pi-repl makes the opposite bet: **give the model one persistent Python
-workspace and let it write the composition itself.** Configuration, state, and
-file access all happen in code, in one living namespace. The model's interface
-to the world never grows — the *code* it writes adapts instead.
+pi-repl makes the opposite bet: **give the model one persistent Python workspace, and let it
+write the composition itself.** Reading, running, searching, and editing all happen in code,
+in a single living namespace. The model's interface to the world never grows — the *code* it
+writes adapts instead.
 
-That is a "REPL" the way an agent actually wants one. Not an interactive
-lozenge to type into, but a long-lived working memory the model owns.
+This is what an agent actually wants from a "REPL". Not an interactive prompt to type into,
+but long-lived working memory the model owns.
 
 ## What persistence buys
 
-A separate-tool loop re-parses text every step. `read` returns a string, the
-agent pastes it, `grep` returns lines, the agent re-reads them. Every
-transformation is round-tripped through the transcript and billed as tokens.
+A point-tool loop re-parses text at every step. The `read` tool returns a string, so the
+agent pastes it back into context. The `grep` tool returns lines, so the agent re-reads
+them. Every transformation is round-tripped through the transcript and billed as tokens.
 
-In a persistent kernel, that work happens once and stays put:
+In a persistent kernel that work happens once and stays put:
 
-- a variable assigned in one cell is there in the next, and the next turn;
+- a variable assigned in one cell is still there in the next cell, and the next turn;
 - a function defined once is reusable for the whole session;
-- `bash()` returns a real `subprocess.CompletedProcess`, not a transcript
-  snippet, so the agent branches on `.returncode` and slices `.stdout` with
-  normal code.
+- `subprocess.run(...)` returns a structured result (`.returncode`, `.stdout`, `.stderr`)
+  the agent branches on with normal code — no re-parsing a tool's text output.
 
-The savings compound harder for small models. Holding a whole file in context
-to avoid re-reading it is expensive precisely when context is scarce; the
-kernel lets the model aggregate, filter, and store in code, printing only what
-the current step needs.
+The savings compound for small models. Holding a whole file in context to avoid re-reading
+it is expensive precisely when context is scarce. The kernel lets the model load, filter,
+and store in code, printing only what the current step needs.
 
 ## Why a real kernel
 
-pi-repl does not hand-roll an `exec` loop. It drives a genuine `ipython`
-kernel in a subprocess via `jupyter_client`. That buys:
+pi-repl does not hand-roll an `exec` loop. It drives a genuine `ipython` kernel in a
+separate process. That buys four things a script string passed to `exec` cannot give:
 
-- rich, real tracebacks instead of a wrapped `except`;
-- the full standard library and real `import` semantics;
-- last-expression capture;
-- a namespace that genuinely survives errors, instead of a script string passed
-to `exec`.
+- **rich, real tracebacks** instead of a wrapped `except`;
+- **real interrupts** — a stuck cell can be interrupted mid-run without losing the session;
+- **last-expression capture** (a cell's final expression becomes its result);
+- **a namespace that survives errors** — a cell that throws leaves the kernel, and
+  everything defined before it, intact.
 
-And it is an honest isolation boundary: the kernel is a separate process from
-pi. A cell can raise, or consume memory, or spin, and pi keeps answering because
-pi is not the process that failed. The host restores from the last completed
-snapshot and tells the model exactly what came back in a `<rlm_engine_reset>`
-notice. More in `ARCHITECTURE.md`.
+It is also an honest isolation boundary. The kernel is its own process, not part of pi. A
+cell that raises leaves pi answering and the namespace intact, because pi is not the process
+that failed. A cell that wedges the *whole* kernel instead stops cells from running until
+the next call notices the dead kernel and rebuilds it from the last completed snapshot.
+Either way the result carries a `<repl_engine_reset>` notice that names what the rebuild
+revived and what it lost, so the model re-verifies before trusting state that may be gone.
+(How that machinery works is in `ARCHITECTURE.md`.)
 
 ## The venv as part of the design
 
-Because the evaluator is real Python, it needs a real Python environment with
-`ipykernel` + `jupyter_client`. You cannot conjure that from nothing.
+Because the evaluator is real Python, it needs a real Python environment with `ipykernel`.
+You cannot conjure that from a script; it is a hard runtime dependency.
 
-The package's `postinstall` creates it once, at a stable user path
-(`~/.pi/agent/pi-repl/venv`), so a `pi install` ends with a working evaluator
-and updates do not lose it (the venv is outside the ephemeral package dir
-where it would vanish). At runtime the host resolves the interpreter in a
-short deterministic order (repo venv, cwd venv, the install venv, then
-`$PYTHON`/`python3`). The system interpreter is the fallback, never the
-assumption, because the whole tool quietly breaks if it silently runs in the
-wrong environment. This is a fact the toolbox functions' test in `help()`
-exist to keep visible.
+The package's `postinstall` creates it once, at a stable per-user path
+(`~/.pi/agent/pi-repl/venv`), so a `pi install` normally ends with a working evaluator. If
+`python3` or the network is missing at install time, `postinstall` prints a clear notice and
+the host falls back to `$PYTHON` or `python3` at runtime. Updates never lose it, because the
+venv lives outside the ephemeral package directory where it would vanish on every update.
+
+At runtime the host resolves the interpreter in a short, fixed order:
+
+1. the repo's own `.venv` (development)
+2. a venv in the current directory (per-project)
+3. the install venv at `~/.pi/agent/pi-repl/venv`
+4. `$PYTHON`, then `python3` (the fallback)
+
+The first one that exists wins. The system interpreter is the fallback, never the
+assumption, because the whole tool quietly breaks if it silently runs in the wrong
+environment. The tool's prompt tells the model this, so it does not leak the wrong
+assumption into commands.
 
 ## Trust, not a sandbox
 
-This is deliberately **not** a sandbox. The kernel runs with your user's
-permissions, can read and write anywhere you can, and the toolbox is trusted
-as written. If you need to guard against an untrusted model, this is the wrong
-tool: reach for a real sandbox the way you would for any untrusted user code.
-The philosophy prefers a sharp, honest tool over a pretend-safe one.
+This is deliberately **not a sandbox.** The kernel runs with your user's permissions, can
+read and write anywhere you can, and helpers are trusted as written. If you need to guard
+against an untrusted model, this is the wrong tool — reach for a real sandbox the way you
+would for any untrusted code. The philosophy here prefers a sharp, honest tool over a
+pretend-safe one.
 
 ## What it isn't
 
-- A subagent framework. There is no `rlm.run`. To delegate, the model spawns
-  a process with `bash()`.
-- A drop-in pi-tool parcel. It exposes one `execute` tool; everything else is
-  inside that workspace.
-- A replacement for your own editing/browsing tools required. It is there
-  when the working style above is worth it, dormant otherwise.
+- **A subagent framework.** There is no `repl.run`. To delegate, the model spawns a process
+  with `subprocess.run` (or `!cmd` / `%%bash`).
+- **A pi tool-rack.** It exposes one `execute` tool; everything else lives inside that
+  workspace.
+- **A replacement for your own editing and browsing tools.** It is there when the working
+  style above is worth it, and dormant otherwise.
 
-The trade-off is real and accepted: the agent pays a little more per-call to
-hold a heavier environment, and it gets back far fewer re-reads, fewer
-transcript round-trips, and sharper small-model behavior.
+The trade-off is real and accepted: the agent pays a little more per call to hold a heavier
+environment, and gets back far fewer re-reads, fewer transcript round-trips, and sharper
+small-model behaviour.

package/index.ts

@@ -4,8 +4,7 @@ import { basename, join } from "node:path";
 import { homedir } from "node:os";
 import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";
 import { Type } from "typebox";
-import { EngineBusyError, EngineManager } from "./src/engine/index.js";
-import { loadConfig } from "./src/extension/config.js";
+import { EngineManager } from "./src/engine/index.js";
 import { ExecuteCellComponent, type ExecuteDetails, type ExecuteRenderState } from "./src/extension/render.js";
 import { EngineLifecycle, summarizeNames } from "./src/extension/session-engine.js";
 import { EXECUTE_DESCRIPTION, buildExecutePromptGuidelines, EXECUTE_PROMPT_SNIPPET } from "./src/extension/tool-meta.js";
@@ -45,8 +44,6 @@ function composeErrorLines(error: { name: string; message: string; stack: string
 	return stack[0]?.trim() === header ? stack : [header, ...stack];
 }
 
-const CFG = loadConfig();
-
 export default function (pi: ExtensionAPI) {
 	pi.registerFlag("repl", {
 		type: "boolean",
@@ -67,9 +64,6 @@ export default function (pi: ExtensionAPI) {
 			const stateDir = join(homedir(), ".pi", "agent", "pi-repl", "state", sessionKey ?? "ephemeral");
 			return new EngineManager({
 				cwd,
-				pythonPath: CFG.pythonPath,
-				timeoutMs: CFG.timeoutMs,
-				toolboxDir: CFG.toolboxDir,
 				// --- snapshots are keyed to a session file; ephemeral sessions get none ---
 				snapshot: sessionKey ? { path: join(stateDir, "namespace.snapshot") } : undefined,
 			});
@@ -130,7 +124,7 @@ export default function (pi: ExtensionAPI) {
 		label: "execute",
 		description: EXECUTE_DESCRIPTION,
 		promptSnippet: EXECUTE_PROMPT_SNIPPET,
-		promptGuidelines: buildExecutePromptGuidelines(CFG.toolboxDir),
+		promptGuidelines: buildExecutePromptGuidelines(),
 		parameters: executeSchema,
 		renderShell: "self",
 		renderCall(args, theme, context) {
@@ -189,22 +183,10 @@ export default function (pi: ExtensionAPI) {
 				pendingErrorResults.set(toolCallId, { details });
 				throw new Error(text || "(no output)");
 			}
-			if (r.status === "aborted") {
-				// --- a busy kernel can't be interrupted mid-cell; discard+rebuild so the next run doesn't queue ---
-				await lifecycle.discard();
-			}
+			// --- an aborted cell was interrupted, not wedged: the kernel keeps running, nothing to discard ---
 			return result;
 		} catch (error) {
-			if (error instanceof EngineBusyError) {
-				// --- discard the wedged engine; the next cell revives from the last snapshot ---
-				await lifecycle.discard();
-				throw new Error(
-					"The evaluator was wedged by a previously interrupted cell and has been killed. " +
-						"Run the next cell to get a fresh evaluator revived from the last snapshot; " +
-						"anything newer than that snapshot is gone, so re-verify variables before reusing them.",
-				);
-			}
-			// --- a guest that died leaves the engine shutdown; drop it so the next cell rebuilds fresh ---
+			// --- a kernel that died (or was killed as the abort backstop) is down; drop it so the next cell rebuilds ---
 			if (m.isRunning === false) {
 				await lifecycle.discard();
 			}

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "pi-repl-py",
-	"version": "0.1.1",
+	"version": "0.2.0",
 	"type": "module",
 	"description": "A pi extension with a single tool: execute, running a TypeScript host with a persistent Python (ipykernel) evaluator and a user-configurable toolbox of functions.",
 	"keywords": [
@@ -13,7 +13,7 @@
 	"license": "MIT",
 	"repository": {
 		"type": "git",
-		"url": "https://github.com/k3-2o/pi-repl-py.git"
+		"url": "git+https://github.com/k3-2o/pi-repl-py.git"
 	},
 	"homepage": "https://github.com/k3-2o/pi-repl-py",
 	"bugs": {
@@ -35,14 +35,13 @@
 	"engines": {
 		"node": ">=22"
 	},
-"scripts": {
+	"scripts": {
 		"typecheck": "tsc --noEmit",
 		"format": "biome format --write .",
 		"lint": "biome check .",
 		"knip": "knip",
-		"check": "tsc --noEmit && biome check . && npm run test:ts && npm run test:py",
+		"check": "biome check . && npm run test:ts",
 		"test:ts": "bun test test/units.test.ts test/preview-core.test.ts",
-		"test:py": ".venv/bin/python -m pytest test/guest_contract.py -q",
 		"test:integ": "bun test test/engine.integration.test.ts",
 		"postinstall": "node scripts/setup-venv.mjs"
 	},
@@ -61,4 +60,4 @@
 		"typebox": "^1.3.11",
 		"typescript": "^5.6.0"
 	}
-}
+}

package/scripts/setup-venv.mjs

@@ -1,18 +1,27 @@
 #!/usr/bin/env node
 /**
- * postinstall: build the stable per-user Python venv the guest evaluator needs.
+ * postinstall: build the stable per-user Python venv the evaluator needs.
  *
- * The guest (src/engine/guest.py) runs a real ipykernel, which requires
- * `ipykernel` + `jupyter_client` installed in a Python environment. When this
- * is installed as a pi package there is no repo-local `.venv` (it's gitignored
- * and excluded from the npm tarball), so we create one at a stable path that
- * the engine also knows about:
+ * The evaluator (src/engine/kernel.ts) drives a real ipykernel directly over
+ * the Jupyter protocol; `ipykernel` is the only hard runtime dependency. When
+ * this is installed as a pi package there is no repo-local `.venv` (gitignored
+ * and excluded from the npm tarball), so we create one at a stable path the
+ * engine also knows about:
  *
  *   ~/.pi/agent/pi-repl/venv/bin/python3
  *
- * Failures are non-fatal: if there's no system python3 or no network, we print
- * a clear notice and let the engine fall back to '$PYTHON' or 'python3' at
- * runtime (where the user is told how to install the deps).
+ * HELPERS live ONLY in the user-owned config dir:
+ *
+ *   ~/.pi/agent/pi-repl/helpers/
+ *
+ * There is no helper/config folder anywhere in this package (no
+ * src/engine/helpers, no templates/). The helpers dir is created if missing,
+ * but no default helpers are seeded — the REPL itself already provides shell
+ * (via `!cmd`, `%%bash`, `subprocess`) and file IO (via `open`, `pathlib`).
+ * Users add their own helper .py files freely; existing files are never clobbered.
+ *
+ * Failures are non-fatal: if there's no system python3 or no network we print a
+ * clear notice and let the engine fall back to '$PYTHON' or 'python3' at runtime.
  */
 
 import { execSync } from "node:child_process";
@@ -22,13 +31,14 @@ import { join } from "node:path";
 
 const VENV_DIR = join(homedir(), ".pi", "agent", "pi-repl", "venv");
 const PY = join(VENV_DIR, "bin", "python3");
-const DEPS = ["ipykernel", "jupyter_client"];
+const DEPS = ["ipykernel"];
+const HELPERS_DIR = join(homedir(), ".pi", "agent", "pi-repl", "helpers");
 
-function log(msg) {
-  process.stdout.write(`[pi-repl] ${msg}\n`);
+function log(m) {
+  process.stdout.write(`[pi-repl] ${m}\n`);
 }
-function warn(msg) {
-  process.stderr.write(`[pi-repl] warning: ${msg}\n`);
+function warn(m) {
+  process.stderr.write(`[pi-repl] warning: ${m}\n`);
 }
 
 function findSystemPython() {
@@ -41,8 +51,20 @@ function findSystemPython() {
   return null;
 }
 
+// ---------------------------------------------------------------- helpers dir
+// The helpers dir is user-owned. We create it empty on install. The REPL
+// provides shell and file IO natively; helpers are for things the user adds
+// themselves (e.g. web_search, custom skills). Existing files are never clobbered.
+function seedHelpersDir() {
+  try {
+    mkdirSync(HELPERS_DIR, { recursive: true });
+  } catch (e) {
+    warn(`could not create the helpers dir (${e?.message ?? e}); custom helpers won't preload.`);
+  }
+}
+
 function main() {
-  // Already built from a previous install/run.
+  seedHelpersDir();
   if (existsSync(PY)) {
     log(`venv already present at ${VENV_DIR}`);
     return;
@@ -51,7 +73,7 @@ function main() {
   if (!systemPython) {
     warn(
       `no python3 found on PATH; could not create the evaluator venv. ` +
-        `Install python3 and run '${PY.slice(-60)} -m venv' manually, or set PI_REPL pythonPath.`
+        `Install python3 and run '${PY.slice(-60)} -m venv' manually, or set $PYTHON to point at one.`
     );
     return;
   }
@@ -64,7 +86,7 @@ function main() {
     log("done. The pi-repl evaluator will use this venv.");
   } catch (error) {
     warn(`could not build the evaluator venv (${error && error.message ? error.message : error}). `);
-    warn("You may need to run npm rebuild pi-repl after fixing python/network, or install ipykernel manually.");
+    warn("You must install ipykernel in that venv before the evaluator runs.");
   }
 }