pi-repl-py 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -21,7 +21,7 @@ the kernel stayed alive.
21
21
 
22
22
  ```bash
23
23
  # from a clone, one-time setup
24
- just setup # npm install + a project-local .venv with the guest deps
24
+ just setup # npm install + a project-local .venv with ipykernel
25
25
 
26
26
  # run a session
27
27
  pi --repl
@@ -33,48 +33,59 @@ A plain `pi` session is untouched; the extension is dormant until `--repl` is pa
33
33
  ## Installing as a pi package
34
34
 
35
35
  `npm install` runs a `postinstall` that creates the Python venv the evaluator needs, at a stable
36
- per-user path (`~/.pi/agent/pi-repl-venv`). If `python3` or the network is missing, it prints a
37
- clear notice. How the interpreter is resolved is in [docs/philosophy.md](docs/philosophy.md).
36
+ per-user path (`~/.pi/agent/pi-repl/venv`). If `python3` or the network is missing, it prints a
37
+ clear notice. How the interpreter is resolved is in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
38
38
 
39
39
  ## What you get
40
40
 
41
41
  - **A persistent namespace.** Variables, functions, imports, and data survive across cells and
42
42
  turns; snapshots preserve them across a best-effort restart.
43
43
  - **A real `ipython` kernel**, not a hand-rolled `exec` loop.
44
- - **Shell as values.** `bash("git log --oneline")` returns a `CompletedProcess` you read
45
- `.stdout`/`.stderr`/`.returncode` on.
44
+ - **Shell and file IO as plain Python.** `!cmd` and `%%bash` run shell fire-and-forget,
45
+ `subprocess.run(...)` brings the result back into a variable, and `open()` / `pathlib`
46
+ read and write files — no wrapper API to learn, and nothing extra to describe to the model.
46
47
  - **Error survival.** A cell that throws reports the traceback and the kernel keeps going.
47
48
  - **An honest evaluator.** If it restarts, it names what state it could revive and what it lost, so you don't trust memory that's gone.
48
49
 
49
- ## The toolbox
50
+ ## Helpers
50
51
 
51
- A small set of Python functions is preloaded into every kernel and surfaced to the
52
- model through the `execute` tool's prompt guidance (their signatures + one-line
53
- summaries are listed there, and `ls()`/`help()` discover them at runtime), so the
54
- model can call `read`, `write`, `edit`, and `bash` without reimplementing them.
55
- Set `toolboxDir` to point at your own folder.
52
+ A **helper** is a Python function you preload into every kernel. Drop a file in the one
53
+ helpers directory, restart the session, and the function is callable from the workspace —
54
+ for example, `helpers/double.py` exposing `def double` becomes `double(...)`. It ships
55
+ **empty** (shell and file IO are already plain Python), so a fresh install preloads nothing
56
+ until you add one. Each helper's `helper_description` is shown to the model verbatim;
57
+ the full contract lives in [docs/how-to-functions.md](docs/how-to-functions.md).
56
58
 
57
- The function list shown to the model is built when the `execute` tool is
58
- registered, so changing the toolbox (adding/removing a file, renaming one with a
59
- `_` prefix) needs a **session restart / `/reload`** for the prompt to reflect it —
60
- the kernel also only loads the toolbox at boot.
59
+ Everything the extension keeps lives under one folder in your home directory:
61
60
 
62
- - Adding a function (the file contract, docstrings, disabling): [docs/how-to-functions.md](docs/how-to-functions.md)
61
+ ```
62
+ ~/.pi/agent/pi-repl/
63
+ venv/ the Python interpreter + ipykernel
64
+ helpers/ your helpers (created empty on install; every *.py loads)
65
+ state/ per-session namespace snapshots
66
+ ```
67
+
68
+ The helpers directory is fixed at `~/.pi/agent/pi-repl/helpers` — no config file.
69
+
70
+ Changing a helper (adding/removing a file, renaming one with a `_` prefix) needs a
71
+ **session restart / `/reload`**: the prompt list is built when `execute` is registered and
72
+ the kernel execs helpers only at boot.
63
73
 
64
74
  ## Configuration
65
75
 
66
- `~/.pi/agent/pi-repl.json` (or `$PI_REPL_CONFIG`) sets `toolboxDir`, `pythonPath`, and timeouts.
67
- Full keys and path rules: [ARCHITECTURE.md](ARCHITECTURE.md).
76
+ There is deliberately no configuration file. Everything is arranged under
77
+ `~/.pi/agent/pi-repl/`: the venv, the fixed helpers dir, and the session state.
78
+ The Python interpreter is auto-resolved (the venv, else `$PYTHON`/`python3`).
68
79
 
69
80
  ## More
70
81
 
71
82
  - Why this design: [docs/philosophy.md](docs/philosophy.md)
72
- - How it works, the venv, config reference: [ARCHITECTURE.md](ARCHITECTURE.md)
83
+ - How it works, the venv, config reference: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
73
84
 
74
85
  ## It is not
75
86
 
76
87
  - A sandbox. The kernel runs with your permissions; the toolbox trusts you.
77
- - A subagent framework. There are no `rlm.run` subagents; spawn a process with `bash()`.
88
+ - A subagent framework. There is no `repl.run` API; spawn a process with `subprocess.run`.
78
89
  - A pi tool-rack. It is one `execute` tool with functions inside.
79
90
 
80
91
  ## License
@@ -0,0 +1,182 @@
1
+ # Architecture
2
+
3
+ pi-repl runs in **two processes**: the host lives inside pi and talks the standard Jupyter
4
+ protocol directly to a real `ipykernel` subprocess. There is no middleman and no invented
5
+ framing between them.
6
+
7
+ ```
8
+ pi
9
+ └─ extension (index.ts) registers `execute`; dormant until --repl
10
+ └─ EngineManager (src/engine/index.ts) venv resolution, lazy spawn,
11
+ │ the call queue, snapshots,
12
+ │ abort grace, teardown
13
+ └─ KernelClient (src/engine/kernel.ts) one ipykernel subprocess
14
+ ├─ ZMTP 3.0 (src/engine/zmtp.ts) the wire protocol, by hand
15
+ ├─ Jupyter session (src/engine/session.ts) framing + HMAC + JSON
16
+ └─ python -m ipykernel -f <connection-file> the evaluator
17
+ ```
18
+
19
+ The host is TypeScript; the evaluator is Python in its own process. That split is what makes
20
+ a bad cell survivable: a cell can raise, leak memory, or wedge the kernel without taking pi
21
+ down — and the host, being not the thing that failed, always gets to report what happened.
22
+
23
+ ## Why the host speaks ZMTP itself
24
+
25
+ The obvious way for a TypeScript host to drive an `ipykernel` is to load a ZMQ client
26
+ library. That does not work here: libzmq's native Node bindings crash `bun`. So an earlier
27
+ design put a Python middleman (`guest.py`) between the host and the kernel, translating a
28
+ private JSON protocol over a file descriptor into the real Jupyter protocol.
29
+
30
+ The current design removes the middleman. Instead of working around the missing library, the
31
+ host implements the small slice of ZMTP 3.0 a Jupyter client actually needs — a DEALER
32
+ socket for the shell and control channels, a SUB socket for iopub (`src/engine/zmtp.ts`).
33
+ The payoff:
34
+
35
+ - **one process boundary** instead of two;
36
+ - **one standard protocol** (Jupyter) instead of a private one on top of it;
37
+ - **no invented framing** to maintain;
38
+ - **HMAC signed and verified by the host itself.** The old design carried a nonce scheme to
39
+ stop a cell from forging its own completion signal; with the host signing every message
40
+ with the kernel's HMAC key, that concern is moot.
41
+
42
+ ## The Python environment (the venv)
43
+
44
+ The evaluator is a real `ipykernel` kernel, so it needs a Python environment with
45
+ `ipykernel` installed. That is a hard runtime dependency — you cannot fake it with a script.
46
+ (`jupyter_client` is *not* needed: the host is the client.)
47
+
48
+ When installed as a pi package, `npm install` runs `postinstall`
49
+ (`scripts/setup-venv.mjs`), which builds a stable per-user venv:
50
+
51
+ ```
52
+ ~/.pi/agent/pi-repl/venv/bin/python3
53
+ ```
54
+
55
+ That path is stable across updates because it sits outside the package's own directory,
56
+ which npm replaces on each update. If `python3` or the network is missing at install time,
57
+ `postinstall` prints a clear notice and the host falls back at runtime.
58
+
59
+ At spawn, `resolvePythonPath` picks the interpreter in this order:
60
+
61
+ 1. the repo's own `.venv` (development)
62
+ 2. a venv in the current directory (per-project)
63
+ 3. `~/.pi/agent/pi-repl/venv` (package install)
64
+ 4. `$PYTHON`, then `python3` (the fallback)
65
+
66
+ The first one that exists wins. The tool's prompt tells the model it runs in a project-local
67
+ venv, not the system interpreter, so it does not leak the wrong assumption into commands.
68
+
69
+ ## The kernel client
70
+
71
+ `KernelClient.start` spawns `python -m ipykernel -f <connection-file>` with a per-run
72
+ connection file in the temp directory, connects the three channels as ZMTP sockets, and
73
+ waits for a `kernel_info_reply` before declaring the kernel ready. Cells run as standard
74
+ Jupyter `execute_request`s, routed by `msg_id`:
75
+
76
+ - **iopub** carries the output — `stream`, `execute_result`, `display_data`, and `error`
77
+ messages, plus private-MIME payloads that carry snapshot, restore, and namespace data.
78
+ - **shell** carries the authoritative `execute_reply` (status, ename, evalue).
79
+ - **control** carries interrupts (`interrupt_request`) and shutdown.
80
+
81
+ Two wire subtleties are load-bearing, and both are pinned by the contract tests.
82
+
83
+ **A cell is not done until two things arrive.** The shell reply and the iopub output stream
84
+ travel on different connections, so a tiny reply can arrive before a large output has
85
+ finished draining on iopub. A cell settles only when **both** the `execute_reply` and the
86
+ matching iopub `status idle` (published after every byte of output) have arrived. Settling
87
+ on the reply alone would drop output that was still in flight.
88
+
89
+ **Output is capped per channel.** Each channel accumulates output against a character budget
90
+ (`maxOutputChars`). Overflow is flagged per message — a single 10 MB print trips the cap
91
+ within that one message, not only once a later message exhausts the budget — and the host
92
+ appends an explicit truncation marker so the model knows output was cut.
93
+
94
+ **Cancellation is real.** An abort sends an `interrupt_request` on the control channel,
95
+ which raises a genuine `KeyboardInterrupt` in the running cell; the namespace survives. As a
96
+ backstop for cells wedged in C code (which ignore interrupts), the engine's abort grace then
97
+ kills the kernel after 500 ms, and the next call rebuilds it from the last snapshot.
98
+
99
+ ## Helpers loading
100
+
101
+ At boot, the kernel and the host both read **one** helpers directory — the fixed
102
+ `~/.pi/agent/pi-repl/helpers`, created empty on install with nothing seeded into it. There is
103
+ no shipped toolbox that merges in.
104
+
105
+ - **The kernel** execs each `*.py` into its namespace, so the file's functions become
106
+ callable.
107
+ - **The host** reads the same files to build the helper list shown in the `execute` tool's
108
+ prompt, so the model sees each `helper_description` verbatim.
109
+
110
+ Because both sides read the same directory, anything the model is told about is also
111
+ callable, and a file renamed with a `_` prefix is skipped by both. The
112
+ `promptGuidelines` are built once, when the `execute` tool is registered, so a helpers
113
+ change needs a **session restart or `/reload`** to reach the prompt — and the kernel loads
114
+ helpers only at boot anyway.
115
+
116
+ There are no custom discovery intrinsics (`ls()` / `help()`) injected into a bare kernel.
117
+ The model discovers what is loaded by listing the namespace with ordinary Python:
118
+
119
+ ```python
120
+ [k for k in globals() if not k.startswith('_')]
121
+ ```
122
+
123
+ For the full helper contract — the description, the docstring, disabling — see
124
+ [how-to-functions.md](how-to-functions.md).
125
+
126
+ ## Snapshots & honest resets
127
+
128
+ After each successful cell, the host schedules a debounced snapshot. A private kernel cell
129
+ pickles the kernel's `globals` (entry by entry, so one value that cannot be pickled costs
130
+ only itself) and publishes the result back over a private MIME payload. The host stores it as
131
+ `namespace.snapshot`, keyed to the session file under
132
+ `~/.pi/agent/pi-repl/state/<session>/`.
133
+
134
+ When a fresh engine is built, it restores that snapshot. Whatever could not be pickled —
135
+ live handles, some objects — is reported by name. If the evaluator was rebuilt mid-session,
136
+ the result is prefixed with a `<repl_engine_reset>` block that names what was revived and
137
+ what was lost, so the model re-verifies before reusing state that may be gone.
138
+
139
+ ## Failure modes
140
+
141
+ | Failure | Behaviour |
142
+ | --- | --- |
143
+ | Cell throws | `error` status with traceback; kernel namespace intact |
144
+ | Cell silent or wedged | the watchdog sends an `interrupt_request`; a caller abort then kills the kernel after a 500 ms grace |
145
+ | Kernel dies | the running cell settles with an error; the next call builds a fresh kernel and restores the last snapshot |
146
+ | Host exits | `process.on("exit")` SIGKILLs live kernels (a child does not die with its parent) |
147
+ | Output flood | capped per channel, truncation announced |
148
+
149
+ ## Testing
150
+
151
+ - **Host (fast):** `test/units.test.ts` covers engine orchestration, rendering, and config;
152
+ `test/preview-core.test.ts` covers the preview logic.
153
+ - **Contract (slow):** `test/engine.integration.test.ts` boots a real kernel per engine and
154
+ proves the guarantees the old Python contract pinned: persistence across cells,
155
+ error-survival, output attribution, helpers loading, snapshot/restore round-trips, output
156
+ caps, silence timeout, abort, and rebuild-from-snapshot after a dead kernel.
157
+
158
+ The gate is `just check` — biome (format + lint) plus the host tests. `just integration`
159
+ adds the real-kernel suite.
160
+
161
+ ## The fixed layout
162
+
163
+ There is no configuration file and no knobs. Everything lives under one directory in the
164
+ user's home, and the default is the only option:
165
+
166
+ ```
167
+ ~/.pi/agent/pi-repl/
168
+ venv/ the Python interpreter + ipykernel
169
+ helpers/ the helpers directory — every *.py loads into every kernel
170
+ state/ per-session namespace snapshots
171
+ ```
172
+
173
+ The helpers directory is fixed at `~/.pi/agent/pi-repl/helpers` (matching the kernel's
174
+ `readHelperSources` default), so both sides are guaranteed to read the same directory. The
175
+ venv is built automatically and the interpreter resolved by the order above — no setting
176
+ needed. The per-cell silence watchdog is off by default (`PI_REPL_TIMEOUT_MS=0`: a silent
177
+ but working cell may run on).
178
+
179
+ ## Reference documentation
180
+
181
+ - Philosophy and design rationale: [philosophy.md](philosophy.md)
182
+ - Adding a helper: [how-to-functions.md](how-to-functions.md)
@@ -1,107 +1,110 @@
1
- # How to add a toolbox function
1
+ # How to write a helper
2
2
 
3
- A toolbox function is one `.py` file that pi-repl loads into every kernel and
4
- surfaces to the model through the `execute` tool's prompt guidance (its
5
- signature + one-line summary appears in `promptGuidelines`). Add a file, and it
6
- shows up wherever the toolbox is read.
3
+ A **helper** is a Python function you write once that becomes available to the agent in
4
+ every `pi --repl` session. You drop a `.py` file into one folder, restart the session, and
5
+ the function is callable from the workspace like a bookmark for code the agent keeps
6
+ reaching for.
7
7
 
8
- > **When a change shows up.** The kernel loads the toolbox at boot, and the
9
- > `execute` tool builds its function list at registration (module load), so a
10
- > toolbox change (add/remove a file, rename one with a `_` prefix) is picked up
11
- > by a **session restart / `/reload`** — not mid-session.
8
+ This guide shows the smallest helper that works, then explains the three parts every
9
+ helper file has and the habits that make a helper useful.
12
10
 
13
- ## Where functions live
11
+ ## Prerequisites
14
12
 
15
- By default the extension ships four (`read`, `write`, `edit`, `bash`) in
16
- `src/engine/toolbox/`. To use your **own** set, set `toolboxDir` in your
17
- config:
13
+ - A working `pi --repl` session (the extension is installed see the README).
14
+ - The helpers folder, `~/.pi/agent/pi-repl/helpers/`. It is created empty on install.
18
15
 
19
- ```jsonc
20
- // ~/.pi/agent/pi-repl.json
21
- { "toolboxDir": "~/.pi/agent/pi-repl-functions" }
22
- ```
23
-
24
- Use an absolute path or a `~`-prefixed one (`~` expands to your home). A bare relative
25
- path resolves from the process working directory, which is not reliable, so prefer
26
- an absolute path for a stable per-user folder. Point `toolboxDir` at a directory and
27
- every `*.py` there is loaded. Note: it **replaces** the shipped defaults; you do not
28
- get built-ins plus yours, unless you copy the built-ins into your folder too.
16
+ ## The smallest helper
29
17
 
30
- ## The file contract
18
+ Create this file:
31
19
 
32
- Every toolbox file must:
20
+ ```python
21
+ # ~/.pi/agent/pi-repl/helpers/double.py
22
+ helper_description = """double(x) — multiply a value by two."""
33
23
 
34
- 1. have a `def` whose signature is the real call an agent would use, and
35
- 2. may declare `function_description` (a short one-line summary shown in the
36
- prompt).
24
+ def double(x):
25
+ """Return x * 2. Works on ints, floats, and lists."""
26
+ return x * 2
27
+ ```
37
28
 
38
- A minimal, valid file:
29
+ Restart the session (`/reload`, or relaunch `pi --repl`), then check it loaded:
39
30
 
40
31
  ```python
41
- # pi-repl-functions/summarize.py
42
- function_description = """Return a first-sentence summary of a text."""
43
-
44
- __all__ = ["summarize"]
32
+ print([k for k in globals() if not k.startswith('_')])
33
+ # ['double', ...]
45
34
 
46
- def summarize(text, limit=1):
47
- return ". ".join(text.split(". ")[:limit]) + "."
35
+ print(double(21))
36
+ # 42
48
37
  ```
49
38
 
50
- That is everything. `summarize` loads into the kernel and the `execute` tool's
51
- prompt guidance shows `summarize(text, limit=1)` after the next session restart.
39
+ If `double` appears in the namespace and runs, it's loaded. That is the whole loop: write
40
+ the file, restart, use it.
52
41
 
53
- ## The two pieces the loader reads
42
+ ## How it actually works
54
43
 
55
- **1. The signature comes from the `def`, not the description.**
56
- Arguments are read from the actual `def` line rather than hand-copied into a
57
- docstring, so the signature the model sees tracks the code for a normal
58
- single-line signature. Change `def summarize(text,
59
- limit=1):` to `limit=200`, and after the next session restart the prompt
60
- updates to match.
44
+ Two things happen with the file you wrote:
61
45
 
62
- **2. The description, from `function_description`, optional.**
63
- Used as the one-line summary in the `execute` tool's prompt guidance. If you
64
- omit it, the function is still advertised (by signature), just without a
65
- one-liner.
46
+ 1. **The kernel execs it at boot.** The file's source runs in the kernel before the first
47
+ cell, so `def double` defines a callable `double` in the workspace. Separately, the
48
+ filename sets the label the prompt uses to advertise it `double.py` is listed as
49
+ `double`. Keep the two identical (see *Common mistakes*).
66
50
 
67
- Each file should also give the function a real docstring (the text under
68
- `def`). That docstring is shown by `help(name)` in the kernel and carries the
69
- deeper usage and gotchas. It does not go into the execute tool's prompt
70
- guidance (only the one-line `function_description` does). Keep it for
71
- details, the venv note, and edge cases.
51
+ 2. **The model sees the description.** The `execute` tool's prompt lists each helper's
52
+ `helper_description`, rendered **verbatim**. Nothing is parsed: the text you put between
53
+ the triple-quotes is exactly what the model reads.
72
54
 
73
- ## How much to document
55
+ Shell and file IO are not helpers — they are already ordinary Python
56
+ (`subprocess.run`, `!cmd`, `%%bash`, `open`, `pathlib`). A helper is only worth writing for
57
+ the fragile or opaque part the model can't reliably reconstruct on its own: a `web_search`
58
+ with provider failover, a client wrapper, a conversion it keeps getting wrong.
74
59
 
75
- `function_description` is the summary; the `def` docstring is the detail. A
76
- good `function_description` is one line ("Run a shell command and return its
77
- result."). A good docstring explains arguments, return value, and any
78
- non-obvious behavior, including environment facts the model needs
79
- ("the evaluator runs in a project-local venv, not the system python").
60
+ ## The three parts of a helper file
80
61
 
81
- ## Disabling a file without deleting it
62
+ | Part | Required? | What it does | Who sees it |
63
+ |---|---|---|---|
64
+ | `def name(...)` | yes | the implementation | runs in the kernel |
65
+ | `helper_description` | no | one line the model reads first | the prompt, verbatim |
66
+ | a docstring | no | the full detail | `print(name.__doc__)` on demand |
82
67
 
83
- Rename the file to start with an underscore: `_test_helper.py`. The loader
84
- **(and the execute tool's prompt guidance)** skip underscore-prefixed files, so
85
- it never reaches the kernel or the model. Use this for scratch or internal
86
- helpers.
68
+ **The function** is the implementation. Its public name must match the filename: `double.py`
69
+ exposes `double`. Keep the `def` name identical so callers and the namespace agree.
87
70
 
88
- ## Good practice
71
+ **The description** is the model's first impression. It is the only part that reaches the
72
+ prompt, and it is billed into context every turn, so it wants to be short. Two habits help
73
+ (not requirements — the loader parses nothing):
89
74
 
90
- - One function per file, name matches the function.
91
- - Keep `function_description` one line. Everything else goes in the docstring.
92
- - Let the signature carry the truth; the description says what it's *for*.
93
- - A function that can hang (a shell call, network) should say so in its
94
- docstring so the model knows the trade-off.
75
+ - Start with the call shape: `double(x) multiply a value by two.`
76
+ - Add an "Instead of:" line naming the hand-rolled code it replaces, so the model reaches
77
+ for the helper rather than rewriting the raw call. *Good:* `Instead of: subprocess.run
78
+ with a hand-rolled kill-on-timeout.` *Weak:* `Instead of: doing it manually.`
95
79
 
96
- ## Confirming it worked
80
+ **The docstring** holds the depth — argument types and defaults, return value, error
81
+ behaviour, environment facts ("this runs in a project-local venv, not the system Python").
82
+ It never enters the prompt. The model reads it on demand with `print(name.__doc__)` or
83
+ `help(name)` when the description is not enough.
97
84
 
98
- At a `pi --repl` prompt, run a cell:
85
+ ## Common mistakes
99
86
 
100
- ```python
101
- print(ls()) # list what's loaded
102
- print(help('summarize')) # signature + full docstring details
103
- ```
87
+ - **Description buried in detail.** Move anything beyond the call shape and one consequence
88
+ into the docstring. A long description bills into every turn.
89
+ - **The helper decides too much.** A helper should own the murky part (the call, the
90
+ parsing), not the decision. If it picks the command, the routing, or the judgement, the
91
+ model stops reasoning.
92
+ - **No docstring.** Without one, the model sees only a signature and the gotchas vanish.
93
+ - **Top-level side effects.** The file runs at boot in every kernel. Keep module-level code
94
+ to definitions — no prints, no network, no slow imports at module scope.
95
+ - **Filename and `def` name differ.** `helpers/foo.py` exposing `def bar` confuses callers
96
+ and the prompt. Keep them identical.
97
+
98
+ ## Disable a file without deleting it
99
+
100
+ Rename it with a leading underscore (`_scratch.py`). The loader skips any file whose name
101
+ starts with `_`, so it never reaches the kernel or the prompt. Handy for scratch work.
102
+
103
+ ## Checklist
104
104
 
105
- If `summarize` shows up in `ls()` and `help`, it loaded. The `execute` tool's
106
- prompt guidance also lists it (same first-line summary) after the next session
107
- restart.
105
+ - [ ] Filename matches the public `def` name
106
+ - [ ] `helper_description` is short and starts with the call shape
107
+ - [ ] Depth lives in the docstring, not the description
108
+ - [ ] No top-level side effects
109
+ - [ ] Session restarted or `/reload`-ed
110
+ - [ ] Name appears in `globals()` and the function runs
@@ -2,87 +2,95 @@
2
2
 
3
3
  ## The bet
4
4
 
5
- Most coding agents are a pile of point tools. A read tool, a bash tool, an edit
6
- tool, a find tool, each with its own schema, its own failure modes, its own
7
- token cost to describe. The model spends context deciding *which* tool, then
8
- *how* the output should be threaded into the next one.
5
+ Most coding agents carry a toolbox of point tools: a read tool, a bash tool, an edit tool,
6
+ a search tool, each with its own schema, its own failure modes, and its own token cost to
7
+ describe. The model spends context deciding *which* tool to call, then *how* to thread one
8
+ tool's output into the next.
9
9
 
10
- pi-repl makes the opposite bet: **give the model one persistent Python
11
- workspace and let it write the composition itself.** Configuration, state, and
12
- file access all happen in code, in one living namespace. The model's interface
13
- to the world never grows — the *code* it writes adapts instead.
10
+ pi-repl makes the opposite bet: **give the model one persistent Python workspace, and let it
11
+ write the composition itself.** Reading, running, searching, and editing all happen in code,
12
+ in a single living namespace. The model's interface to the world never grows — the *code* it
13
+ writes adapts instead.
14
14
 
15
- That is a "REPL" the way an agent actually wants one. Not an interactive
16
- lozenge to type into, but a long-lived working memory the model owns.
15
+ This is what an agent actually wants from a "REPL". Not an interactive prompt to type into,
16
+ but long-lived working memory the model owns.
17
17
 
18
18
  ## What persistence buys
19
19
 
20
- A separate-tool loop re-parses text every step. `read` returns a string, the
21
- agent pastes it, `grep` returns lines, the agent re-reads them. Every
22
- transformation is round-tripped through the transcript and billed as tokens.
20
+ A point-tool loop re-parses text at every step. The `read` tool returns a string, so the
21
+ agent pastes it back into context. The `grep` tool returns lines, so the agent re-reads
22
+ them. Every transformation is round-tripped through the transcript and billed as tokens.
23
23
 
24
- In a persistent kernel, that work happens once and stays put:
24
+ In a persistent kernel that work happens once and stays put:
25
25
 
26
- - a variable assigned in one cell is there in the next, and the next turn;
26
+ - a variable assigned in one cell is still there in the next cell, and the next turn;
27
27
  - a function defined once is reusable for the whole session;
28
- - `bash()` returns a real `subprocess.CompletedProcess`, not a transcript
29
- snippet, so the agent branches on `.returncode` and slices `.stdout` with
30
- normal code.
28
+ - `subprocess.run(...)` returns a structured result (`.returncode`, `.stdout`, `.stderr`)
29
+ the agent branches on with normal code no re-parsing a tool's text output.
31
30
 
32
- The savings compound harder for small models. Holding a whole file in context
33
- to avoid re-reading it is expensive precisely when context is scarce; the
34
- kernel lets the model aggregate, filter, and store in code, printing only what
35
- the current step needs.
31
+ The savings compound for small models. Holding a whole file in context to avoid re-reading
32
+ it is expensive precisely when context is scarce. The kernel lets the model load, filter,
33
+ and store in code, printing only what the current step needs.
36
34
 
37
35
  ## Why a real kernel
38
36
 
39
- pi-repl does not hand-roll an `exec` loop. It drives a genuine `ipython`
40
- kernel in a subprocess via `jupyter_client`. That buys:
37
+ pi-repl does not hand-roll an `exec` loop. It drives a genuine `ipython` kernel in a
38
+ separate process. That buys four things a script string passed to `exec` cannot give:
41
39
 
42
- - rich, real tracebacks instead of a wrapped `except`;
43
- - the full standard library and real `import` semantics;
44
- - last-expression capture;
45
- - a namespace that genuinely survives errors, instead of a script string passed
46
- to `exec`.
40
+ - **rich, real tracebacks** instead of a wrapped `except`;
41
+ - **real interrupts** a stuck cell can be interrupted mid-run without losing the session;
42
+ - **last-expression capture** (a cell's final expression becomes its result);
43
+ - **a namespace that survives errors** a cell that throws leaves the kernel, and
44
+ everything defined before it, intact.
47
45
 
48
- And it is an honest isolation boundary: the kernel is a separate process from
49
- pi. A cell can raise, or consume memory, or spin, and pi keeps answering because
50
- pi is not the process that failed. The host restores from the last completed
51
- snapshot and tells the model exactly what came back in a `<rlm_engine_reset>`
52
- notice. More in `ARCHITECTURE.md`.
46
+ It is also an honest isolation boundary. The kernel is its own process, not part of pi. A
47
+ cell that raises leaves pi answering and the namespace intact, because pi is not the process
48
+ that failed. A cell that wedges the *whole* kernel instead stops cells from running until
49
+ the next call notices the dead kernel and rebuilds it from the last completed snapshot.
50
+ Either way the result carries a `<repl_engine_reset>` notice that names what the rebuild
51
+ revived and what it lost, so the model re-verifies before trusting state that may be gone.
52
+ (How that machinery works is in `ARCHITECTURE.md`.)
53
53
 
54
54
  ## The venv as part of the design
55
55
 
56
- Because the evaluator is real Python, it needs a real Python environment with
57
- `ipykernel` + `jupyter_client`. You cannot conjure that from nothing.
56
+ Because the evaluator is real Python, it needs a real Python environment with `ipykernel`.
57
+ You cannot conjure that from a script; it is a hard runtime dependency.
58
58
 
59
- The package's `postinstall` creates it once, at a stable user path
60
- (`~/.pi/agent/pi-repl-venv`), so a `pi install` ends with a working evaluator
61
- and updates do not lose it (the venv is outside the ephemeral package dir
62
- where it would vanish). At runtime the host resolves the interpreter in a
63
- short deterministic order (repo venv, cwd venv, the install venv, then
64
- `$PYTHON`/`python3`). The system interpreter is the fallback, never the
65
- assumption, because the whole tool quietly breaks if it silently runs in the
66
- wrong environment. This is a fact the toolbox functions' test in `help()`
67
- exist to keep visible.
59
+ The package's `postinstall` creates it once, at a stable per-user path
60
+ (`~/.pi/agent/pi-repl/venv`), so a `pi install` normally ends with a working evaluator. If
61
+ `python3` or the network is missing at install time, `postinstall` prints a clear notice and
62
+ the host falls back to `$PYTHON` or `python3` at runtime. Updates never lose it, because the
63
+ venv lives outside the ephemeral package directory where it would vanish on every update.
64
+
65
+ At runtime the host resolves the interpreter in a short, fixed order:
66
+
67
+ 1. the repo's own `.venv` (development)
68
+ 2. a venv in the current directory (per-project)
69
+ 3. the install venv at `~/.pi/agent/pi-repl/venv`
70
+ 4. `$PYTHON`, then `python3` (the fallback)
71
+
72
+ The first one that exists wins. The system interpreter is the fallback, never the
73
+ assumption, because the whole tool quietly breaks if it silently runs in the wrong
74
+ environment. The tool's prompt tells the model this, so it does not leak the wrong
75
+ assumption into commands.
68
76
 
69
77
  ## Trust, not a sandbox
70
78
 
71
- This is deliberately **not** a sandbox. The kernel runs with your user's
72
- permissions, can read and write anywhere you can, and the toolbox is trusted
73
- as written. If you need to guard against an untrusted model, this is the wrong
74
- tool: reach for a real sandbox the way you would for any untrusted user code.
75
- The philosophy prefers a sharp, honest tool over a pretend-safe one.
79
+ This is deliberately **not a sandbox.** The kernel runs with your user's permissions, can
80
+ read and write anywhere you can, and helpers are trusted as written. If you need to guard
81
+ against an untrusted model, this is the wrong tool — reach for a real sandbox the way you
82
+ would for any untrusted code. The philosophy here prefers a sharp, honest tool over a
83
+ pretend-safe one.
76
84
 
77
85
  ## What it isn't
78
86
 
79
- - A subagent framework. There is no `rlm.run`. To delegate, the model spawns
80
- a process with `bash()`.
81
- - A drop-in pi-tool parcel. It exposes one `execute` tool; everything else is
82
- inside that workspace.
83
- - A replacement for your own editing/browsing tools required. It is there
84
- when the working style above is worth it, dormant otherwise.
87
+ - **A subagent framework.** There is no `repl.run`. To delegate, the model spawns a process
88
+ with `subprocess.run` (or `!cmd` / `%%bash`).
89
+ - **A pi tool-rack.** It exposes one `execute` tool; everything else lives inside that
90
+ workspace.
91
+ - **A replacement for your own editing and browsing tools.** It is there when the working
92
+ style above is worth it, and dormant otherwise.
85
93
 
86
- The trade-off is real and accepted: the agent pays a little more per-call to
87
- hold a heavier environment, and it gets back far fewer re-reads, fewer
88
- transcript round-trips, and sharper small-model behavior.
94
+ The trade-off is real and accepted: the agent pays a little more per call to hold a heavier
95
+ environment, and gets back far fewer re-reads, fewer transcript round-trips, and sharper
96
+ small-model behaviour.