ypi 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +41 -0
- package/LICENSE +21 -0
- package/README.md +171 -0
- package/SYSTEM_PROMPT.md +122 -0
- package/install.sh +116 -0
- package/package.json +52 -0
- package/rlm_cost +35 -0
- package/rlm_parse_json +43 -0
- package/rlm_query +349 -0
- package/ypi +86 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to ypi are documented here.
|
|
4
|
+
Format based on [Keep a Changelog](https://keepachangelog.com/).
|
|
5
|
+
|
|
6
|
+
## [0.2.0] - 2026-02-12
|
|
7
|
+
|
|
8
|
+
### Added
|
|
9
|
+
- **Cost tracking**: children default to `--mode json`, parsed by `rlm_parse_json` for structured cost/token data
|
|
10
|
+
- **Budget enforcement**: `RLM_BUDGET=0.50` caps dollar spend for entire recursive tree
|
|
11
|
+
- **`rlm_cost` command**: agent can query cumulative spend at any time (`rlm_cost` or `rlm_cost --json`)
|
|
12
|
+
- **`rlm_parse_json`**: streams text to stdout, captures cost via fd 3 to shared cost file
|
|
13
|
+
- System prompt updated with cost awareness (SECTION 4 teaches `rlm_cost`)
|
|
14
|
+
- `rlm_query` source embedded in system prompt (SECTION 6) so agents understand their own infrastructure
|
|
15
|
+
|
|
16
|
+
### Changed
|
|
17
|
+
- **Uniform children**: removed separate leaf path — all depths get full tools, extensions, sessions, jj workspaces
|
|
18
|
+
- **Extensions on by default** at all depths (`RLM_EXTENSIONS=1`)
|
|
19
|
+
- **`RLM_CHILD_EXTENSIONS`**: per-instance extension override for depth > 0
|
|
20
|
+
- Recursion limited by removing `rlm_query` from PATH at max depth (not `--no-tools`)
|
|
21
|
+
- `RLM_JSON=0` opt-out for plain text mode (disables cost tracking)
|
|
22
|
+
|
|
23
|
+
### Removed
|
|
24
|
+
- Separate leaf code path (`--no-tools`, `--no-extensions`, `--no-session` at max depth)
|
|
25
|
+
- sops/age/gitleaks references from README and install.sh (internal only)
|
|
26
|
+
|
|
27
|
+
## [0.1.0] - 2026-02-12
|
|
28
|
+
|
|
29
|
+
Initial release.
|
|
30
|
+
|
|
31
|
+
### Added
|
|
32
|
+
- `ypi` launcher — starts Pi as a recursive coding agent
|
|
33
|
+
- `rlm_query` — bash recursive sub-call function (analog of Python RLM's `llm_query()`)
|
|
34
|
+
- `SYSTEM_PROMPT.md` — teaches the LLM to use recursion + bash for divide-and-conquer
|
|
35
|
+
- Guardrails: timeout (`RLM_TIMEOUT`), call limits (`RLM_MAX_CALLS`), depth limits (`RLM_MAX_DEPTH`)
|
|
36
|
+
- Model routing: `RLM_CHILD_MODEL` / `RLM_CHILD_PROVIDER` for cheaper sub-calls
|
|
37
|
+
- jj workspace isolation for recursive children (`RLM_JJ`)
|
|
38
|
+
- Session forking and trace logging (`PI_TRACE_FILE`, `RLM_TRACE_ID`)
|
|
39
|
+
- Pi extensions support (`RLM_EXTENSIONS`, `RLM_CHILD_EXTENSIONS`)
|
|
40
|
+
- `install.sh` for curl-pipe-bash installation
|
|
41
|
+
- npm package with `ypi` and `rlm_query` as global CLI commands
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Raymond Weitekamp
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,171 @@
|
|
|
1
|
+
# ypi
|
|
2
|
+
|
|
3
|
+
**ypi** — a recursive coding agent built on [Pi](https://github.com/badlogic/pi-mono).
|
|
4
|
+
|
|
5
|
+
Named after the [Y combinator](https://en.wikipedia.org/wiki/Fixed-point_combinator#Y_combinator) from lambda calculus — the fixed-point combinator that enables recursion. `ypi` is Pi that can call itself. (`rpi` already has another connotation.)
|
|
6
|
+
|
|
7
|
+
Inspired by [Recursive Language Models](https://github.com/alexzhang13/rlm) (RLM), which showed that an LLM with a code REPL and a `llm_query()` function can recursively decompose problems, analyze massive contexts, and write code — all through self-delegation.
|
|
8
|
+
|
|
9
|
+
## The Idea
|
|
10
|
+
|
|
11
|
+
Pi already has a bash REPL. We add one function — `rlm_query` — and a system prompt that teaches Pi to use it recursively. Each child gets its own [jj](https://martinvonz.github.io/jj/) workspace for file isolation. That's the whole trick.
|
|
12
|
+
|
|
13
|
+
```
|
|
14
|
+
┌──────────────────────────────────────────┐
|
|
15
|
+
│ ypi (depth 0) │
|
|
16
|
+
│ Tools: bash, rlm_query │
|
|
17
|
+
│ Workspace: default │
|
|
18
|
+
│ │
|
|
19
|
+
│ > grep -n "bug" src/*.py │
|
|
20
|
+
│ > sed -n '50,80p' src/app.py \ │
|
|
21
|
+
│ | rlm_query "Fix this bug" │
|
|
22
|
+
│ │ │
|
|
23
|
+
│ ▼ │
|
|
24
|
+
│ ┌────────────────────────────┐ │
|
|
25
|
+
│ │ ypi (depth 1) │ │
|
|
26
|
+
│ │ Workspace: jj isolated │ │
|
|
27
|
+
│ │ Edits files safely │ │
|
|
28
|
+
│ │ Returns: patch on stdout │ │
|
|
29
|
+
│ └────────────────────────────┘ │
|
|
30
|
+
│ │
|
|
31
|
+
│ > jj squash --from <child-change> │
|
|
32
|
+
│ # absorb the fix into our working copy │
|
|
33
|
+
└──────────────────────────────────────────┘
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## Using ypi
|
|
39
|
+
|
|
40
|
+
### Install
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
curl -fsSL https://raw.githubusercontent.com/rawwerks/ypi/master/install.sh | bash
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Or manually:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
git clone https://github.com/rawwerks/ypi.git
|
|
50
|
+
cd ypi
|
|
51
|
+
git submodule update --init --depth 1 # pulls pi-mono
|
|
52
|
+
export PATH="$PWD:$PATH"
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
### Run
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
# Interactive
|
|
59
|
+
ypi
|
|
60
|
+
|
|
61
|
+
# One-shot
|
|
62
|
+
ypi "Refactor the error handling in this repo"
|
|
63
|
+
|
|
64
|
+
# Different model
|
|
65
|
+
ypi --provider anthropic --model claude-sonnet-4-5-20250929 "What does this codebase do?"
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
### How It Works
|
|
69
|
+
|
|
70
|
+
**Three pieces** (same architecture as Python RLM):
|
|
71
|
+
|
|
72
|
+
| Piece | Python RLM | ypi |
|
|
73
|
+
|---|---|---|
|
|
74
|
+
| System prompt | `RLM_SYSTEM_PROMPT` | `SYSTEM_PROMPT.md` |
|
|
75
|
+
| Context / REPL | Python `context` variable | `$CONTEXT` file + bash |
|
|
76
|
+
| Sub-call function | `llm_query("prompt")` | `rlm_query "prompt"` |
|
|
77
|
+
|
|
78
|
+
**Recursion:** `rlm_query` spawns a child Pi process with the same system prompt and tools. The child can call `rlm_query` too:
|
|
79
|
+
|
|
80
|
+
```
|
|
81
|
+
Depth 0 (root) → full Pi with bash + rlm_query
|
|
82
|
+
Depth 1 (child) → full Pi with bash + rlm_query, own jj workspace
|
|
83
|
+
Depth 2 (leaf) → plain LM call, no tools (RLM_MAX_DEPTH reached)
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
**File isolation with jj:** Each recursive child gets its own [jj workspace](https://martinvonz.github.io/jj/latest/working-copy/). The parent's working copy is untouched. Review child work with `jj diff -r <change-id>`, absorb with `jj squash --from <change-id>`.
|
|
87
|
+
|
|
88
|
+
### Guardrails
|
|
89
|
+
|
|
90
|
+
| Feature | Env var | What it does |
|
|
91
|
+
|---------|---------|-------------|
|
|
92
|
+
| Budget | `RLM_BUDGET=0.50` | Max dollar spend for entire recursive tree |
|
|
93
|
+
| Timeout | `RLM_TIMEOUT=60` | Wall-clock limit for entire recursive tree |
|
|
94
|
+
| Call limit | `RLM_MAX_CALLS=20` | Max total `rlm_query` invocations |
|
|
95
|
+
| Model routing | `RLM_CHILD_MODEL=haiku` | Use cheaper model for sub-calls |
|
|
96
|
+
| Depth limit | `RLM_MAX_DEPTH=3` | How deep recursion can go |
|
|
97
|
+
| jj disable | `RLM_JJ=0` | Skip workspace isolation |
|
|
98
|
+
| Plain text | `RLM_JSON=0` | Disable JSON mode (no cost tracking) |
|
|
99
|
+
| Tracing | `PI_TRACE_FILE=/tmp/trace.log` | Log all calls with timing + cost |
|
|
100
|
+
|
|
101
|
+
The agent can check spend at any time:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
rlm_cost # "$0.042381"
|
|
105
|
+
rlm_cost --json # {"cost": 0.042381, "tokens": 12450, "calls": 3}
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## Contributing
|
|
111
|
+
|
|
112
|
+
### Project Structure
|
|
113
|
+
|
|
114
|
+
```
|
|
115
|
+
ypi/
|
|
116
|
+
├── ypi # Launcher: sets up env, starts Pi as recursive agent
|
|
117
|
+
├── rlm_query # The recursive sub-call function (Pi's analog of rlm llm_query())
|
|
118
|
+
├── SYSTEM_PROMPT.md # Teaches the LLM to be recursive + edit code
|
|
119
|
+
├── AGENTS.md # Meta-instructions for the agent (read by ypi itself)
|
|
120
|
+
├── Makefile # test targets
|
|
121
|
+
├── tests/
|
|
122
|
+
│ ├── test_unit.sh # Mock pi, test bash logic (no LLM, fast)
|
|
123
|
+
│ ├── test_guardrails.sh # Test guardrails (no LLM, fast)
|
|
124
|
+
│ └── test_e2e.sh # Real LLM calls (slow, costs ~$0.05)
|
|
125
|
+
├── pi-mono/ # Git submodule: upstream Pi coding agent
|
|
126
|
+
└── README.md
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
### Version Control
|
|
130
|
+
|
|
131
|
+
This repo uses **[jj](https://martinvonz.github.io/jj/)** for version control. Git is only for GitHub sync.
|
|
132
|
+
|
|
133
|
+
```bash
|
|
134
|
+
jj status # What's changed
|
|
135
|
+
jj describe -m "message" # Describe current change
|
|
136
|
+
jj new # Start a new change
|
|
137
|
+
jj bookmark set master # Point master at current change
|
|
138
|
+
jj git push # Push to GitHub
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
**Never use `git add/commit/push` directly.** jj manages git under the hood.
|
|
142
|
+
|
|
143
|
+
### Testing
|
|
144
|
+
|
|
145
|
+
```bash
|
|
146
|
+
make test-fast # 54 tests, no LLM calls, seconds
|
|
147
|
+
make test-e2e # Real LLM calls, costs ~$0.05
|
|
148
|
+
make test # Both
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
**Before any change to `rlm_query`:** run `make test-fast`. After: run it again. `rlm_query` is a live dependency of the agent's own execution — breaking it breaks the agent.
|
|
152
|
+
|
|
153
|
+
|
|
154
|
+
### History
|
|
155
|
+
|
|
156
|
+
ypi went through four approaches before landing on the current design:
|
|
157
|
+
|
|
158
|
+
1. **Tool-use REPL** (exp 010/012) — Pi's `completeWithTools()`, ReAct loop. 77.6% on LongMemEval.
|
|
159
|
+
2. **Python bridge** — HTTP server between Pi and Python RLM. Too complex.
|
|
160
|
+
3. **Pi extension** — Custom provider with search tools. Not true recursion.
|
|
161
|
+
4. **Bash RLM** (`rlm_query` + `SYSTEM_PROMPT.md`) — True recursion via bash. **Current approach.**
|
|
162
|
+
|
|
163
|
+
The key insight: Pi's bash tool **is** the REPL. `rlm_query` **is** `llm_query()`. No bridge needed.
|
|
164
|
+
|
|
165
|
+
---
|
|
166
|
+
|
|
167
|
+
## See Also
|
|
168
|
+
|
|
169
|
+
- [Pi coding agent](https://github.com/badlogic/pi-mono) — the underlying agent
|
|
170
|
+
- [Recursive Language Models](https://github.com/alexzhang13/rlm) — the library that inspired this
|
|
171
|
+
- [rlm-cli](https://github.com/rawwerks/rlm-cli) — Python RLM CLI (budget, timeout, model routing)
|
package/SYSTEM_PROMPT.md
ADDED
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# SYSTEM_PROMPT.md
|
|
2
|
+
|
|
3
|
+
## SECTION 1 – Core Identity
|
|
4
|
+
- You are a **recursive LLM** equipped with a Bash shell and the `rlm_query` tool.
|
|
5
|
+
- The environment variable `RLM_DEPTH` tells you your current recursion depth; respect `RLM_MAX_DEPTH` and be more **conservative** (fewer sub‑calls, more direct actions) the deeper you are.
|
|
6
|
+
- You can **read files, write files, run commands, and delegate work** to sub‑agents via `rlm_query`.
|
|
7
|
+
- Sub‑agents inherit the same capabilities and receive their own isolated context.
|
|
8
|
+
- All actions should aim to be **deterministic and reproducible**.
|
|
9
|
+
|
|
10
|
+
## SECTION 2 – Context Analysis (QA over Context)
|
|
11
|
+
Your environment is initialized with a `$CONTEXT` file that may contain the information needed to answer a query.
|
|
12
|
+
|
|
13
|
+
**Key workflow**
|
|
14
|
+
1. **Check size first** – `wc -l "$CONTEXT"` and `wc -c "$CONTEXT"`. Small contexts (≈ 5 KB) can be read directly; larger ones require search + chunking.
|
|
15
|
+
2. **Search** – use `grep` (or `rg`) to locate relevant keywords before invoking `rlm_query`.
|
|
16
|
+
3. **Chunk** – break large files into line ranges (e.g., 500‑line windows) and feed each chunk to a sub‑LLM.
|
|
17
|
+
4. **Delegate** – use the two `rlm_query` patterns:
|
|
18
|
+
```bash
|
|
19
|
+
# Pipe a specific chunk
|
|
20
|
+
sed -n '100,200p' "$CONTEXT" | rlm_query "Your question"
|
|
21
|
+
|
|
22
|
+
# Inherit the whole context (no pipe)
|
|
23
|
+
rlm_query "Your question"
|
|
24
|
+
```
|
|
25
|
+
5. **Combine** – aggregate answers from chunks, deduplicate, and produce the final response.
|
|
26
|
+
|
|
27
|
+
### Example Patterns (keep all five)
|
|
28
|
+
|
|
29
|
+
**Example 1 – Short context, direct approach**
|
|
30
|
+
```bash
|
|
31
|
+
wc -c "$CONTEXT"
|
|
32
|
+
# 3200 chars — small enough to read directly
|
|
33
|
+
cat "$CONTEXT"
|
|
34
|
+
# Now I can see the content and answer the question
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
**Example 2 – Long context, search and delegate**
|
|
38
|
+
```bash
|
|
39
|
+
# First, explore the structure
|
|
40
|
+
wc -l "$CONTEXT"
|
|
41
|
+
head -50 "$CONTEXT"
|
|
42
|
+
grep -n "Chapter" "$CONTEXT"
|
|
43
|
+
|
|
44
|
+
# Found relevant section around line 500. Delegate reading to a sub‑call:
|
|
45
|
+
sed -n '480,600p' "$CONTEXT" | rlm_query "Who is the author of this chapter? Return ONLY the name."
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
**Example 3 – Chunk and query**
|
|
49
|
+
```bash
|
|
50
|
+
# Check size
|
|
51
|
+
TOTAL=$(wc -l < "$CONTEXT")
|
|
52
|
+
echo "Context has $TOTAL lines"
|
|
53
|
+
|
|
54
|
+
# Search for keywords first
|
|
55
|
+
grep -n "graduation\|degree\|university" "$CONTEXT"
|
|
56
|
+
|
|
57
|
+
# Delegate each chunk:
|
|
58
|
+
ANSWER1=$(sed -n '1950,2100p' "$CONTEXT" | rlm_query "What degree did the user graduate with? Quote the evidence.")
|
|
59
|
+
ANSWER2=$(sed -n '7900,8100p' "$CONTEXT" | rlm_query "What degree did the user graduate with? Quote the evidence.")
|
|
60
|
+
|
|
61
|
+
# Combine results
|
|
62
|
+
echo "Chunk 1: $ANSWER1"
|
|
63
|
+
echo "Chunk 2: $ANSWER2"
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
**Example 4 – Iterative chunking for huge contexts**
|
|
67
|
+
```bash
|
|
68
|
+
TOTAL=$(wc -l < "$CONTEXT")
|
|
69
|
+
CHUNK=500
|
|
70
|
+
for START in $(seq 1 $CHUNK $TOTAL); do
|
|
71
|
+
END=$((START + CHUNK - 1))
|
|
72
|
+
RESULT=$(sed -n "${START},${END}p" "$CONTEXT" | rlm_query "Extract any mentions of concerts or live music events. Return a numbered list, or 'none' if none found.")
|
|
73
|
+
if [ "$RESULT" != "none" ]; then
|
|
74
|
+
echo "Lines $START-$END: $RESULT"
|
|
75
|
+
fi
|
|
76
|
+
done
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
**Example 5 – Temporal reasoning with computation**
|
|
80
|
+
```bash
|
|
81
|
+
grep -n "started\|began\|finished\|completed" "$CONTEXT"
|
|
82
|
+
|
|
83
|
+
START_DATE=$(sed -n '300,500p' "$CONTEXT" | rlm_query "When exactly did the user start this project? Return ONLY the date in YYYY-MM-DD format.")
|
|
84
|
+
END_DATE=$(sed -n '2000,2200p' "$CONTEXT" | rlm_query "When exactly did the user finish this project? Return ONLY the date in YYYY-MM-DD format.")
|
|
85
|
+
|
|
86
|
+
python3 -c "from datetime import date; d1=date.fromisoformat('$START_DATE'); d2=date.fromisoformat('$END_DATE'); print((d2-d1).days, 'days')"
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
## SECTION 3 – Coding and File Editing
|
|
90
|
+
- You may be asked to **modify code, add files, or restructure the repository**.
|
|
91
|
+
- First, check whether you are inside a **jj workspace**:
|
|
92
|
+
```bash
|
|
93
|
+
jj root 2>/dev/null && echo "jj workspace detected"
|
|
94
|
+
```
|
|
95
|
+
- In a jj workspace, every edit you make is **isolated**; the parent worktree remains untouched until you `jj commit`.
|
|
96
|
+
- **Write files directly** with `write` or standard Bash redirection; do **not** merely describe the change.
|
|
97
|
+
- When you need to create or modify multiple files, perform each action explicitly (e.g., `echo >> file`, `sed -i`, `cat > newfile`).
|
|
98
|
+
- Any sub‑agents you spawn via `rlm_query` inherit their own jj workspaces, so their edits are also isolated.
|
|
99
|
+
|
|
100
|
+
## SECTION 4 – Guardrails & Cost Awareness
|
|
101
|
+
- **RLM_TIMEOUT** – if set, respect the remaining wall‑clock budget; avoid long‑running loops.
|
|
102
|
+
- **RLM_MAX_CALLS** – each `rlm_query` increments `RLM_CALL_COUNT`; stay within the limit.
|
|
103
|
+
- **RLM_BUDGET** – if set, max dollar spend for the entire recursive tree. The infrastructure enforces this, but you should also be cost-conscious.
|
|
104
|
+
- **`rlm_cost`** – call this at any time to see cumulative spend:
|
|
105
|
+
```bash
|
|
106
|
+
rlm_cost # "$0.042381"
|
|
107
|
+
rlm_cost --json # {"cost": 0.042381, "tokens": 12450, "calls": 3}
|
|
108
|
+
```
|
|
109
|
+
Use this to decide whether to make more sub‑calls or work directly. If spend is high relative to the task, prefer direct Bash actions over spawning sub‑agents.
|
|
110
|
+
- **Depth awareness** – at deeper `RLM_DEPTH` levels, prefer **direct actions** (e.g., file edits, single‑pass searches) over spawning many sub‑agents.
|
|
111
|
+
- Always **clean up temporary files** and respect `trap` handlers defined by the infrastructure.
|
|
112
|
+
|
|
113
|
+
## SECTION 5 – Rules (Updated)
|
|
114
|
+
1. **Context size first** – always `wc -l "$CONTEXT"` and `wc -c "$CONTEXT"`. Use direct read for small files, grep + chunking for large ones.
|
|
115
|
+
2. **Validate before answering** – if a sub‑call returns unexpected output, re‑query; never guess.
|
|
116
|
+
3. **Counting & temporal questions** – enumerate items with evidence, deduplicate, then count; extract dates and compute with `python3` or `date`.
|
|
117
|
+
4. **Entity verification** – `grep` must confirm the exact entity exists; if not, respond with *"I don't know"* (only when the entity truly isn’t present).
|
|
118
|
+
5. **Code editing** – when instructed to edit code, **perform the edit** immediately; do not just describe the change.
|
|
119
|
+
6. **Sub‑agent calls** – favor **small, focused** sub‑agent calls over vague, large ones; keep the call count low.
|
|
120
|
+
7. **Depth preference** – deeper depths ⇒ fewer sub‑calls, more direct Bash actions.
|
|
121
|
+
8. **No blanket "I don't know" rule** – remove the generic rule; only use "I don't know" when the required information is absent from the context or repository.
|
|
122
|
+
9. **Safety** – never execute untrusted commands without explicit intent; rely on the provided tooling.
|
package/install.sh
ADDED
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
# ypi installer — one-line install:
|
|
3
|
+
# curl -fsSL https://raw.githubusercontent.com/rawwerks/ypi/master/install.sh | bash
|
|
4
|
+
#
|
|
5
|
+
# Installs ypi + Pi coding agent. Requires: npm (or bun), git, bash.
|
|
6
|
+
# Optional: jj (for workspace isolation), sops + age (for encrypted notes)
|
|
7
|
+
|
|
8
|
+
set -euo pipefail
|
|
9
|
+
|
|
10
|
+
# Colors
|
|
11
|
+
RED='\033[0;31m'
|
|
12
|
+
GREEN='\033[0;32m'
|
|
13
|
+
DIM='\033[0;90m'
|
|
14
|
+
BOLD='\033[1m'
|
|
15
|
+
RESET='\033[0m'
|
|
16
|
+
|
|
17
|
+
info() { echo -e "${GREEN}▸${RESET} $1"; }
|
|
18
|
+
warn() { echo -e "${RED}▸${RESET} $1"; }
|
|
19
|
+
dim() { echo -e "${DIM} $1${RESET}"; }
|
|
20
|
+
|
|
21
|
+
# ── Check prerequisites ──────────────────────────────────────────────────
|
|
22
|
+
|
|
23
|
+
MISSING=""
|
|
24
|
+
command -v git &>/dev/null || MISSING="$MISSING git"
|
|
25
|
+
command -v bash &>/dev/null || MISSING="$MISSING bash"
|
|
26
|
+
|
|
27
|
+
if [ -n "$MISSING" ]; then
|
|
28
|
+
warn "Missing required tools:$MISSING"
|
|
29
|
+
exit 1
|
|
30
|
+
fi
|
|
31
|
+
|
|
32
|
+
# Need npm or bun for Pi
|
|
33
|
+
HAS_NPM=false
|
|
34
|
+
HAS_BUN=false
|
|
35
|
+
command -v npm &>/dev/null && HAS_NPM=true
|
|
36
|
+
command -v bun &>/dev/null && HAS_BUN=true
|
|
37
|
+
|
|
38
|
+
if [ "$HAS_NPM" = false ] && [ "$HAS_BUN" = false ]; then
|
|
39
|
+
warn "Need npm or bun to install Pi. Install Node.js: https://nodejs.org"
|
|
40
|
+
exit 1
|
|
41
|
+
fi
|
|
42
|
+
|
|
43
|
+
# ── Install Pi if not present ────────────────────────────────────────────
|
|
44
|
+
|
|
45
|
+
if ! command -v pi &>/dev/null; then
|
|
46
|
+
info "Installing Pi coding agent..."
|
|
47
|
+
if [ "$HAS_BUN" = true ]; then
|
|
48
|
+
bun install -g @mariozechner/pi-coding-agent
|
|
49
|
+
else
|
|
50
|
+
npm install -g @mariozechner/pi-coding-agent
|
|
51
|
+
fi
|
|
52
|
+
dim "Installed $(pi --version 2>/dev/null | head -1 || echo 'pi')"
|
|
53
|
+
else
|
|
54
|
+
dim "Pi already installed: $(which pi)"
|
|
55
|
+
fi
|
|
56
|
+
|
|
57
|
+
# ── Clone ypi ────────────────────────────────────────────────────────────
|
|
58
|
+
|
|
59
|
+
INSTALL_DIR="${YPI_DIR:-$HOME/.ypi}"
|
|
60
|
+
|
|
61
|
+
if [ -d "$INSTALL_DIR" ]; then
|
|
62
|
+
info "Updating ypi at $INSTALL_DIR..."
|
|
63
|
+
cd "$INSTALL_DIR"
|
|
64
|
+
git pull --quiet
|
|
65
|
+
git submodule update --init --depth 1 --quiet
|
|
66
|
+
else
|
|
67
|
+
info "Cloning ypi to $INSTALL_DIR..."
|
|
68
|
+
git clone --quiet https://github.com/rawwerks/ypi.git "$INSTALL_DIR"
|
|
69
|
+
cd "$INSTALL_DIR"
|
|
70
|
+
git submodule update --init --depth 1 --quiet
|
|
71
|
+
fi
|
|
72
|
+
|
|
73
|
+
# ── Add to PATH ──────────────────────────────────────────────────────────
|
|
74
|
+
|
|
75
|
+
SHELL_NAME="$(basename "${SHELL:-/bin/bash}")"
|
|
76
|
+
EXPORT_LINE="export PATH=\"$INSTALL_DIR:\$PATH\""
|
|
77
|
+
RC_FILE=""
|
|
78
|
+
|
|
79
|
+
case "$SHELL_NAME" in
|
|
80
|
+
zsh) RC_FILE="$HOME/.zshrc" ;;
|
|
81
|
+
bash) RC_FILE="$HOME/.bashrc" ;;
|
|
82
|
+
fish) RC_FILE="$HOME/.config/fish/config.fish"
|
|
83
|
+
EXPORT_LINE="set -gx PATH $INSTALL_DIR \$PATH" ;;
|
|
84
|
+
*) RC_FILE="$HOME/.profile" ;;
|
|
85
|
+
esac
|
|
86
|
+
|
|
87
|
+
if [ -n "$RC_FILE" ] && ! grep -qF "$INSTALL_DIR" "$RC_FILE" 2>/dev/null; then
|
|
88
|
+
echo "" >> "$RC_FILE"
|
|
89
|
+
echo "# ypi — recursive coding agent" >> "$RC_FILE"
|
|
90
|
+
echo "$EXPORT_LINE" >> "$RC_FILE"
|
|
91
|
+
info "Added to PATH in $RC_FILE"
|
|
92
|
+
dim "Run: source $RC_FILE (or open a new terminal)"
|
|
93
|
+
else
|
|
94
|
+
dim "Already in PATH"
|
|
95
|
+
fi
|
|
96
|
+
|
|
97
|
+
# ── Set up git hooks ────────────────────────────────────────────────────
|
|
98
|
+
|
|
99
|
+
cd "$INSTALL_DIR"
|
|
100
|
+
git config core.hooksPath .githooks 2>/dev/null || true
|
|
101
|
+
|
|
102
|
+
# ── Report optional tools ───────────────────────────────────────────────
|
|
103
|
+
|
|
104
|
+
echo ""
|
|
105
|
+
info "ypi installed! ✓"
|
|
106
|
+
echo ""
|
|
107
|
+
dim "Required:"
|
|
108
|
+
command -v pi &>/dev/null && dim " ✓ pi ($(which pi))" || dim " ✗ pi"
|
|
109
|
+
echo ""
|
|
110
|
+
dim "Optional:"
|
|
111
|
+
command -v jj &>/dev/null && dim " ✓ jj (workspace isolation)" || dim " · jj — install for workspace isolation: https://martinvonz.github.io/jj/"
|
|
112
|
+
echo ""
|
|
113
|
+
echo -e "${BOLD}Get started:${RESET}"
|
|
114
|
+
echo " ypi # interactive"
|
|
115
|
+
echo " ypi \"What does this repo do?\" # one-shot"
|
|
116
|
+
echo ""
|
package/package.json
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "ypi",
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"description": "ypi — a recursive coding agent. Pi that can call itself via rlm_query.",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"author": "Raymond Weitekamp",
|
|
7
|
+
"repository": {
|
|
8
|
+
"type": "git",
|
|
9
|
+
"url": "https://github.com/rawwerks/ypi.git"
|
|
10
|
+
},
|
|
11
|
+
"homepage": "https://github.com/rawwerks/ypi#readme",
|
|
12
|
+
"bugs": {
|
|
13
|
+
"url": "https://github.com/rawwerks/ypi/issues"
|
|
14
|
+
},
|
|
15
|
+
"keywords": [
|
|
16
|
+
"ai",
|
|
17
|
+
"agent",
|
|
18
|
+
"coding-agent",
|
|
19
|
+
"recursive",
|
|
20
|
+
"rlm",
|
|
21
|
+
"llm",
|
|
22
|
+
"pi",
|
|
23
|
+
"cli"
|
|
24
|
+
],
|
|
25
|
+
"bin": {
|
|
26
|
+
"ypi": "./ypi",
|
|
27
|
+
"rlm_query": "./rlm_query",
|
|
28
|
+
"rlm_cost": "./rlm_cost",
|
|
29
|
+
"rlm_parse_json": "./rlm_parse_json"
|
|
30
|
+
},
|
|
31
|
+
"files": [
|
|
32
|
+
"ypi",
|
|
33
|
+
"rlm_query",
|
|
34
|
+
"rlm_cost",
|
|
35
|
+
"rlm_parse_json",
|
|
36
|
+
"SYSTEM_PROMPT.md",
|
|
37
|
+
"install.sh",
|
|
38
|
+
"README.md",
|
|
39
|
+
"LICENSE",
|
|
40
|
+
"CHANGELOG.md"
|
|
41
|
+
],
|
|
42
|
+
"dependencies": {
|
|
43
|
+
"@mariozechner/pi-coding-agent": "*"
|
|
44
|
+
},
|
|
45
|
+
"os": [
|
|
46
|
+
"linux",
|
|
47
|
+
"darwin"
|
|
48
|
+
],
|
|
49
|
+
"engines": {
|
|
50
|
+
"node": ">=18"
|
|
51
|
+
}
|
|
52
|
+
}
|
package/rlm_cost
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# rlm_cost — Report cumulative cost for the current recursive tree.
|
|
3
|
+
# Usage:
|
|
4
|
+
# rlm_cost # prints "$0.042381"
|
|
5
|
+
# rlm_cost --json # prints {"cost": 0.042381, "tokens": 12450, "calls": 3}
|
|
6
|
+
|
|
7
|
+
if [ -z "${RLM_COST_FILE:-}" ] || [ ! -f "${RLM_COST_FILE:-}" ]; then
|
|
8
|
+
if [ "${1:-}" = "--json" ]; then
|
|
9
|
+
echo '{"cost": 0, "tokens": 0, "calls": 0}'
|
|
10
|
+
else
|
|
11
|
+
echo "\$0.000000"
|
|
12
|
+
fi
|
|
13
|
+
exit 0
|
|
14
|
+
fi
|
|
15
|
+
|
|
16
|
+
python3 -c "
|
|
17
|
+
import json, sys
|
|
18
|
+
total_cost = 0.0
|
|
19
|
+
total_tokens = 0
|
|
20
|
+
calls = 0
|
|
21
|
+
with open('${RLM_COST_FILE}') as f:
|
|
22
|
+
for line in f:
|
|
23
|
+
line = line.strip()
|
|
24
|
+
if not line: continue
|
|
25
|
+
try:
|
|
26
|
+
obj = json.loads(line)
|
|
27
|
+
total_cost += obj.get('cost', 0)
|
|
28
|
+
total_tokens += obj.get('tokens', 0)
|
|
29
|
+
calls += 1
|
|
30
|
+
except: pass
|
|
31
|
+
if '--json' in sys.argv:
|
|
32
|
+
print(json.dumps({'cost': round(total_cost, 6), 'tokens': total_tokens, 'calls': calls}))
|
|
33
|
+
else:
|
|
34
|
+
print(f'\${total_cost:.6f}')
|
|
35
|
+
" "$@"
|
package/rlm_parse_json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Parse Pi JSON mode output. Stream text to stdout, write cost to fd 3."""
|
|
3
|
+
import sys
|
|
4
|
+
import json
|
|
5
|
+
|
|
6
|
+
total_cost = 0.0
|
|
7
|
+
total_tokens = 0
|
|
8
|
+
|
|
9
|
+
for line in sys.stdin:
|
|
10
|
+
line = line.strip()
|
|
11
|
+
if not line:
|
|
12
|
+
continue
|
|
13
|
+
try:
|
|
14
|
+
obj = json.loads(line)
|
|
15
|
+
except json.JSONDecodeError:
|
|
16
|
+
continue
|
|
17
|
+
|
|
18
|
+
t = obj.get("type", "")
|
|
19
|
+
|
|
20
|
+
# Stream text deltas to stdout as they arrive
|
|
21
|
+
if t == "message_update":
|
|
22
|
+
event = obj.get("assistantMessageEvent", {})
|
|
23
|
+
if event.get("type") == "text_delta":
|
|
24
|
+
delta = event.get("delta", "")
|
|
25
|
+
sys.stdout.write(delta)
|
|
26
|
+
sys.stdout.flush()
|
|
27
|
+
|
|
28
|
+
# Accumulate cost from each turn_end (handles multi-turn tool use)
|
|
29
|
+
if t == "turn_end":
|
|
30
|
+
msg = obj.get("message", {})
|
|
31
|
+
usage = msg.get("usage", {})
|
|
32
|
+
cost = usage.get("cost", {}).get("total", 0)
|
|
33
|
+
tokens = usage.get("totalTokens", 0)
|
|
34
|
+
total_cost += cost
|
|
35
|
+
total_tokens += tokens
|
|
36
|
+
|
|
37
|
+
# Write cost summary to fd 3 (if open)
|
|
38
|
+
cost_line = json.dumps({"cost": total_cost, "tokens": total_tokens})
|
|
39
|
+
try:
|
|
40
|
+
with open(3, "w") as f:
|
|
41
|
+
f.write(cost_line)
|
|
42
|
+
except OSError:
|
|
43
|
+
pass # fd 3 not open, skip
|
package/rlm_query
ADDED
|
@@ -0,0 +1,349 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# rlm_query — Recursive Language Model sub-call for Pi.
|
|
3
|
+
#
|
|
4
|
+
# This is the Pi/bash equivalent of Python RLM's llm_query().
|
|
5
|
+
# Each invocation spawns a child Pi that can answer questions about context.
|
|
6
|
+
#
|
|
7
|
+
# Usage:
|
|
8
|
+
# rlm_query "Analyze this and extract all dates"
|
|
9
|
+
# echo "some text" | rlm_query "What is the main topic?"
|
|
10
|
+
# sed -n '100,200p' "$CONTEXT" | rlm_query "Summarize this section"
|
|
11
|
+
# rlm_query --fork "Continue working on this refactor"
|
|
12
|
+
#
|
|
13
|
+
# If stdin has data (piped), that becomes the child's context.
|
|
14
|
+
# Otherwise, the child inherits the parent's $CONTEXT file.
|
|
15
|
+
#
|
|
16
|
+
# Flags:
|
|
17
|
+
# --fork Fork parent session into child (carries conversation history)
|
|
18
|
+
# Default: fresh session per child (only data context, no history)
|
|
19
|
+
#
|
|
20
|
+
# Environment:
|
|
21
|
+
# RLM_DEPTH — current recursion depth (default: 0)
|
|
22
|
+
# RLM_MAX_DEPTH — max recursion depth (default: 3)
|
|
23
|
+
# RLM_PROVIDER — LLM provider
|
|
24
|
+
# RLM_MODEL — LLM model
|
|
25
|
+
# RLM_SYSTEM_PROMPT — path to the RLM system prompt file
|
|
26
|
+
# CONTEXT — path to the current context file
|
|
27
|
+
# RLM_STDIN — set to "1" by the calling pattern to indicate piped input
|
|
28
|
+
# RLM_TIMEOUT — max wall‑clock seconds for the whole call chain
|
|
29
|
+
# RLM_START_TIME — epoch seconds when the root call started (auto‑set)
|
|
30
|
+
# RLM_MAX_CALLS — maximum total rlm_query invocations allowed
|
|
31
|
+
# RLM_CALL_COUNT — current count of invocations (auto‑incremented)
|
|
32
|
+
# RLM_CHILD_MODEL — model to use for child calls (depth > 0)
|
|
33
|
+
# RLM_CHILD_PROVIDER— provider to use for child calls (depth > 0)
|
|
34
|
+
# RLM_JJ — set to "0" to disable jj workspace isolation
|
|
35
|
+
# RLM_EXTENSIONS — set to "0" to disable Pi extensions (default: 1)
|
|
36
|
+
# RLM_CHILD_EXTENSIONS — override extensions for depth > 0 (default: same as parent)
|
|
37
|
+
# RLM_BUDGET — max dollar spend for entire recursive tree (e.g. "0.50")
|
|
38
|
+
# RLM_COST_FILE — shared file tracking cumulative cost (auto‑created)
|
|
39
|
+
# RLM_JSON — set to "0" to disable JSON mode (plain text, no cost tracking)
|
|
40
|
+
# RLM_TRACE_ID — shared ID linking all sessions in a recursive tree
|
|
41
|
+
# RLM_SESSION_DIR — Pi session directory for this project
|
|
42
|
+
# RLM_SESSION_FILE — parent's session file (used with --fork)
|
|
43
|
+
|
|
44
|
+
set -euo pipefail
|
|
45
|
+
|
|
46
|
+
# Structured error helper
|
|
47
|
+
rlm_error() { echo "✗ $1" >&2; [ -n "${2:-}" ] && echo " Why: $2" >&2; [ -n "${3:-}" ] && echo " Fix: $3" >&2; }
|
|
48
|
+
|
|
49
|
+
# ----------------------------------------------------------------------
|
|
50
|
+
# Parse flags
|
|
51
|
+
# ----------------------------------------------------------------------
|
|
52
|
+
FORK=false
|
|
53
|
+
while [[ "${1:-}" == --* ]]; do
|
|
54
|
+
case "$1" in
|
|
55
|
+
--fork) FORK=true; shift ;;
|
|
56
|
+
*) break ;;
|
|
57
|
+
esac
|
|
58
|
+
done
|
|
59
|
+
|
|
60
|
+
PROMPT="${1:?Usage: rlm_query [--fork] \"your prompt here\"}"
|
|
61
|
+
|
|
62
|
+
# ----------------------------------------------------------------------
|
|
63
|
+
# Depth guard — refuse to go beyond max depth
|
|
64
|
+
# This is the only recursion limiter. Children get full tools/extensions.
|
|
65
|
+
# ----------------------------------------------------------------------
|
|
66
|
+
DEPTH="${RLM_DEPTH:-0}"
|
|
67
|
+
MAX_DEPTH="${RLM_MAX_DEPTH:-3}"
|
|
68
|
+
NEXT_DEPTH=$((DEPTH + 1))
|
|
69
|
+
|
|
70
|
+
if [ "$NEXT_DEPTH" -gt "$MAX_DEPTH" ]; then
|
|
71
|
+
rlm_error "Max depth exceeded" "At depth $DEPTH of $MAX_DEPTH" "Increase RLM_MAX_DEPTH or simplify the task"
|
|
72
|
+
exit 1
|
|
73
|
+
fi
|
|
74
|
+
|
|
75
|
+
PROVIDER="${RLM_PROVIDER:-cerebras}"
|
|
76
|
+
MODEL="${RLM_MODEL:-gpt-oss-120b}"
|
|
77
|
+
SYSTEM_PROMPT_FILE="${RLM_SYSTEM_PROMPT:-}"
|
|
78
|
+
|
|
79
|
+
# ----------------------------------------------------------------------
|
|
80
|
+
# Timeout start time initialization
|
|
81
|
+
# ----------------------------------------------------------------------
|
|
82
|
+
if [ -z "${RLM_START_TIME:-}" ]; then
|
|
83
|
+
export RLM_START_TIME=$(date +%s)
|
|
84
|
+
fi
|
|
85
|
+
|
|
86
|
+
# ----------------------------------------------------------------------
|
|
87
|
+
# Call counting & max‑calls guard
|
|
88
|
+
# ----------------------------------------------------------------------
|
|
89
|
+
RLM_CALL_COUNT=$(( ${RLM_CALL_COUNT:-0} + 1 ))
|
|
90
|
+
export RLM_CALL_COUNT
|
|
91
|
+
if [ -n "${RLM_MAX_CALLS:-}" ] && [ "$RLM_CALL_COUNT" -ge "$RLM_MAX_CALLS" ]; then
|
|
92
|
+
rlm_error "Max calls exceeded" "$RLM_CALL_COUNT of $RLM_MAX_CALLS calls used" "Increase RLM_MAX_CALLS or reduce recursion depth"
|
|
93
|
+
exit 1
|
|
94
|
+
fi
|
|
95
|
+
|
|
96
|
+
# ----------------------------------------------------------------------
|
|
97
|
+
# Budget guard — check cumulative cost before proceeding
|
|
98
|
+
# ----------------------------------------------------------------------
|
|
99
|
+
if [ -n "${RLM_BUDGET:-}" ] && [ -n "${RLM_COST_FILE:-}" ] && [ -f "$RLM_COST_FILE" ]; then
|
|
100
|
+
CURRENT_COST=$(python3 -c "
|
|
101
|
+
import json
|
|
102
|
+
total = 0.0
|
|
103
|
+
with open('$RLM_COST_FILE') as f:
|
|
104
|
+
for line in f:
|
|
105
|
+
line = line.strip()
|
|
106
|
+
if line:
|
|
107
|
+
try: total += json.loads(line).get('cost', 0)
|
|
108
|
+
except: pass
|
|
109
|
+
print(f'{total:.6f}')
|
|
110
|
+
" 2>/dev/null || echo "0")
|
|
111
|
+
OVER=$(python3 -c "print('yes' if $CURRENT_COST >= $RLM_BUDGET else 'no')" 2>/dev/null || echo "no")
|
|
112
|
+
if [ "$OVER" = "yes" ]; then
|
|
113
|
+
rlm_error "Budget exceeded" "Spent \$$CURRENT_COST of \$$RLM_BUDGET budget" "Increase RLM_BUDGET or simplify the task"
|
|
114
|
+
exit 1
|
|
115
|
+
fi
|
|
116
|
+
fi
|
|
117
|
+
|
|
118
|
+
# Initialize cost file if budget is set but no file exists yet
|
|
119
|
+
if [ -n "${RLM_BUDGET:-}" ] && [ -z "${RLM_COST_FILE:-}" ]; then
|
|
120
|
+
export RLM_COST_FILE=$(mktemp /tmp/rlm_cost_XXXXXX.jsonl)
|
|
121
|
+
fi
|
|
122
|
+
|
|
123
|
+
# ----------------------------------------------------------------------
|
|
124
|
+
# Session tree — each child gets a persisted session file
|
|
125
|
+
# Trace ID groups all sessions from one recursive invocation.
|
|
126
|
+
# ----------------------------------------------------------------------
|
|
127
|
+
if [ -z "${RLM_TRACE_ID:-}" ]; then
|
|
128
|
+
export RLM_TRACE_ID=$(head -c 4 /dev/urandom | od -An -tx1 | tr -d ' \n')
|
|
129
|
+
fi
|
|
130
|
+
|
|
131
|
+
CHILD_SESSION_FILE=""
|
|
132
|
+
if [ -n "${RLM_SESSION_DIR:-}" ]; then
|
|
133
|
+
mkdir -p "$RLM_SESSION_DIR"
|
|
134
|
+
CHILD_SESSION_FILE="${RLM_SESSION_DIR}/${RLM_TRACE_ID}_d${NEXT_DEPTH}_c${RLM_CALL_COUNT}.jsonl"
|
|
135
|
+
|
|
136
|
+
# --fork: copy parent session to give child full conversation history
|
|
137
|
+
if [ "$FORK" = true ] && [ -n "${RLM_SESSION_FILE:-}" ] && [ -f "${RLM_SESSION_FILE:-}" ]; then
|
|
138
|
+
cp "$RLM_SESSION_FILE" "$CHILD_SESSION_FILE"
|
|
139
|
+
fi
|
|
140
|
+
fi
|
|
141
|
+
|
|
142
|
+
# ----------------------------------------------------------------------
|
|
143
|
+
# Trace logging (optional)
|
|
144
|
+
# ----------------------------------------------------------------------
|
|
145
|
+
if [ -n "${PI_TRACE_FILE:-}" ]; then
|
|
146
|
+
echo "[$(date +%H:%M:%S.%3N)] depth=$DEPTH→$NEXT_DEPTH PID=$$ PPID=$PPID call=$RLM_CALL_COUNT trace=$RLM_TRACE_ID fork=$FORK prompt: ${PROMPT:0:120}" >> "$PI_TRACE_FILE"
|
|
147
|
+
fi
|
|
148
|
+
|
|
149
|
+
# ----------------------------------------------------------------------
|
|
150
|
+
# Temporary child context file
|
|
151
|
+
# ----------------------------------------------------------------------
|
|
152
|
+
CHILD_CONTEXT=$(mktemp /tmp/rlm_ctx_d${NEXT_DEPTH}_XXXXXX.txt)
|
|
153
|
+
COMBINED_PROMPT=""
|
|
154
|
+
|
|
155
|
+
# ----------------------------------------------------------------------
|
|
156
|
+
# jj workspace isolation — give recursive children their own working copy
|
|
157
|
+
# ----------------------------------------------------------------------
|
|
158
|
+
JJ_WORKSPACE=""
|
|
159
|
+
JJ_WS_NAME=""
|
|
160
|
+
if [ "${RLM_JJ:-1}" != "0" ] \
|
|
161
|
+
&& command -v jj &>/dev/null \
|
|
162
|
+
&& jj root &>/dev/null 2>&1; then
|
|
163
|
+
JJ_WS_NAME="rlm-d${NEXT_DEPTH}-$$"
|
|
164
|
+
JJ_WORKSPACE=$(mktemp -d /tmp/rlm_ws_d${NEXT_DEPTH}_XXXXXX)
|
|
165
|
+
if ! jj workspace add --name "$JJ_WS_NAME" "$JJ_WORKSPACE" &>/dev/null; then
|
|
166
|
+
JJ_WORKSPACE=""
|
|
167
|
+
JJ_WS_NAME=""
|
|
168
|
+
fi
|
|
169
|
+
fi
|
|
170
|
+
|
|
171
|
+
# Cleanup: remove temp context + forget jj workspace (updated in run section below)
|
|
172
|
+
trap '{
|
|
173
|
+
rm -f "$CHILD_CONTEXT"
|
|
174
|
+
rm -f "${COMBINED_PROMPT:-}"
|
|
175
|
+
if [ -n "$JJ_WS_NAME" ]; then
|
|
176
|
+
jj workspace forget "$JJ_WS_NAME" 2>/dev/null || true
|
|
177
|
+
fi
|
|
178
|
+
}' EXIT
|
|
179
|
+
trap 'rlm_error "Interrupted" "Received signal" "Re-run the command"; exit 130' INT TERM
|
|
180
|
+
|
|
181
|
+
# ----------------------------------------------------------------------
|
|
182
|
+
# Detect piped stdin
|
|
183
|
+
# ----------------------------------------------------------------------
|
|
184
|
+
HAS_STDIN=false
|
|
185
|
+
if [ -p /dev/stdin ]; then
|
|
186
|
+
HAS_STDIN=true
|
|
187
|
+
elif [ -n "${RLM_STDIN:-}" ]; then
|
|
188
|
+
HAS_STDIN=true
|
|
189
|
+
fi
|
|
190
|
+
|
|
191
|
+
if [ "$HAS_STDIN" = true ]; then
|
|
192
|
+
cat > "$CHILD_CONTEXT"
|
|
193
|
+
else
|
|
194
|
+
if [ -n "${CONTEXT:-}" ] && [ -f "${CONTEXT:-}" ]; then
|
|
195
|
+
cp "$CONTEXT" "$CHILD_CONTEXT"
|
|
196
|
+
fi
|
|
197
|
+
fi
|
|
198
|
+
|
|
199
|
+
# ----------------------------------------------------------------------
|
|
200
|
+
# Model routing for child calls (depth > 0)
|
|
201
|
+
# ----------------------------------------------------------------------
|
|
202
|
+
if [ "$DEPTH" -gt 0 ] && [ -n "${RLM_CHILD_MODEL:-}" ]; then
|
|
203
|
+
MODEL="${RLM_CHILD_MODEL}"
|
|
204
|
+
if [ -n "${RLM_CHILD_PROVIDER:-}" ]; then
|
|
205
|
+
PROVIDER="${RLM_CHILD_PROVIDER}"
|
|
206
|
+
fi
|
|
207
|
+
fi
|
|
208
|
+
|
|
209
|
+
# ----------------------------------------------------------------------
|
|
210
|
+
# Spawn child Pi with tools, extensions, and session
|
|
211
|
+
# ----------------------------------------------------------------------
|
|
212
|
+
export CONTEXT="$CHILD_CONTEXT"
|
|
213
|
+
export RLM_DEPTH="$NEXT_DEPTH"
|
|
214
|
+
export RLM_MAX_DEPTH="$MAX_DEPTH"
|
|
215
|
+
export RLM_PROVIDER="$PROVIDER"
|
|
216
|
+
export RLM_MODEL="$MODEL"
|
|
217
|
+
export RLM_SYSTEM_PROMPT="${SYSTEM_PROMPT_FILE:-}"
|
|
218
|
+
export RLM_START_TIME="${RLM_START_TIME}"
|
|
219
|
+
export RLM_TIMEOUT="${RLM_TIMEOUT:-}"
|
|
220
|
+
export RLM_TRACE_ID="${RLM_TRACE_ID}"
|
|
221
|
+
export RLM_SESSION_DIR="${RLM_SESSION_DIR:-}"
|
|
222
|
+
export RLM_BUDGET="${RLM_BUDGET:-}"
|
|
223
|
+
export RLM_COST_FILE="${RLM_COST_FILE:-}"
|
|
224
|
+
|
|
225
|
+
# At max depth: remove rlm_query from PATH so the child can't recurse.
|
|
226
|
+
# The child still gets full tools (bash, read, write, edit) — it just
|
|
227
|
+
# can't spawn sub-agents. The depth guard above is a safety net.
|
|
228
|
+
# Follow symlinks — npm install -g creates symlinks in .bin/
|
|
229
|
+
SCRIPT_DIR="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
|
|
230
|
+
if [ "$NEXT_DEPTH" -ge "$MAX_DEPTH" ]; then
|
|
231
|
+
export PATH=$(echo "$PATH" | tr ':' '\n' | grep -v "^${SCRIPT_DIR}$" | paste -sd ':' -)
|
|
232
|
+
fi
|
|
233
|
+
|
|
234
|
+
if [ -n "$CHILD_SESSION_FILE" ]; then
|
|
235
|
+
export RLM_SESSION_FILE="$CHILD_SESSION_FILE"
|
|
236
|
+
fi
|
|
237
|
+
|
|
238
|
+
CMD_ARGS=(-p --provider "$PROVIDER" --model "$MODEL")
|
|
239
|
+
|
|
240
|
+
# Extensions: on by default, configurable per-instance like model routing
|
|
241
|
+
CHILD_EXT="${RLM_EXTENSIONS:-1}"
|
|
242
|
+
if [ "$DEPTH" -gt 0 ] && [ -n "${RLM_CHILD_EXTENSIONS:-}" ]; then
|
|
243
|
+
CHILD_EXT="${RLM_CHILD_EXTENSIONS}"
|
|
244
|
+
fi
|
|
245
|
+
if [ "$CHILD_EXT" = "0" ]; then
|
|
246
|
+
CMD_ARGS+=(--no-extensions)
|
|
247
|
+
fi
|
|
248
|
+
|
|
249
|
+
# Session: use dedicated file if we have a session dir, otherwise ephemeral
|
|
250
|
+
if [ -n "$CHILD_SESSION_FILE" ]; then
|
|
251
|
+
CMD_ARGS+=(--session "$CHILD_SESSION_FILE")
|
|
252
|
+
else
|
|
253
|
+
CMD_ARGS+=(--no-session)
|
|
254
|
+
fi
|
|
255
|
+
|
|
256
|
+
# Build combined system prompt with rlm_query source embedded
|
|
257
|
+
COMBINED_PROMPT=""
|
|
258
|
+
if [ -n "$SYSTEM_PROMPT_FILE" ] && [ -f "$SYSTEM_PROMPT_FILE" ]; then
|
|
259
|
+
COMBINED_PROMPT=$(mktemp /tmp/rlm_system_prompt_XXXXXX.md)
|
|
260
|
+
cat "$SYSTEM_PROMPT_FILE" > "$COMBINED_PROMPT"
|
|
261
|
+
SELF_SOURCE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)/rlm_query"
|
|
262
|
+
if [ -f "$SELF_SOURCE" ]; then
|
|
263
|
+
cat >> "$COMBINED_PROMPT" << 'SYSEOF'
|
|
264
|
+
|
|
265
|
+
## SECTION 6 – rlm_query Implementation
|
|
266
|
+
|
|
267
|
+
Below is the full source of `rlm_query`. You are running inside this infrastructure.
|
|
268
|
+
Understanding it helps you use recursion effectively and respect guardrails.
|
|
269
|
+
|
|
270
|
+
```bash
|
|
271
|
+
SYSEOF
|
|
272
|
+
cat "$SELF_SOURCE" >> "$COMBINED_PROMPT"
|
|
273
|
+
echo '```' >> "$COMBINED_PROMPT"
|
|
274
|
+
fi
|
|
275
|
+
CMD_ARGS+=(--system-prompt "$COMBINED_PROMPT")
|
|
276
|
+
fi
|
|
277
|
+
|
|
278
|
+
# Timeout wrapper
|
|
279
|
+
TIMEOUT_CMD=""
|
|
280
|
+
if [ -n "${RLM_TIMEOUT:-}" ]; then
|
|
281
|
+
ELAPSED=$(( $(date +%s) - RLM_START_TIME ))
|
|
282
|
+
REMAINING=$(( RLM_TIMEOUT - ELAPSED ))
|
|
283
|
+
if [ "$REMAINING" -le 0 ]; then
|
|
284
|
+
rlm_error "Timeout exceeded" "Ran for ${ELAPSED}s of ${RLM_TIMEOUT}s" "Increase RLM_TIMEOUT or simplify the task"
|
|
285
|
+
exit 124
|
|
286
|
+
fi
|
|
287
|
+
TIMEOUT_CMD="timeout $REMAINING"
|
|
288
|
+
fi
|
|
289
|
+
|
|
290
|
+
# Enter jj workspace if available (child gets isolated working copy)
|
|
291
|
+
if [ -n "$JJ_WORKSPACE" ]; then
|
|
292
|
+
cd "$JJ_WORKSPACE"
|
|
293
|
+
fi
|
|
294
|
+
|
|
295
|
+
# ----------------------------------------------------------------------
|
|
296
|
+
# Run child Pi — JSON mode (default) or plain text
|
|
297
|
+
# JSON mode streams text to stdout and captures cost via fd 3.
|
|
298
|
+
# ----------------------------------------------------------------------
|
|
299
|
+
COST_OUT=$(mktemp /tmp/rlm_cost_out_XXXXXX.json)
|
|
300
|
+
trap '{
|
|
301
|
+
rm -f "$CHILD_CONTEXT"
|
|
302
|
+
rm -f "${COMBINED_PROMPT:-}"
|
|
303
|
+
rm -f "$COST_OUT"
|
|
304
|
+
if [ -n "$JJ_WS_NAME" ]; then
|
|
305
|
+
jj workspace forget "$JJ_WS_NAME" 2>/dev/null || true
|
|
306
|
+
fi
|
|
307
|
+
}' EXIT
|
|
308
|
+
|
|
309
|
+
if [ "${RLM_JSON:-1}" != "0" ]; then
|
|
310
|
+
# JSON mode: get structured cost + stream text
|
|
311
|
+
# Replace -p with --mode json, pipe through parser
|
|
312
|
+
JSON_CMD_ARGS=()
|
|
313
|
+
for arg in "${CMD_ARGS[@]}"; do
|
|
314
|
+
[ "$arg" = "-p" ] && continue
|
|
315
|
+
JSON_CMD_ARGS+=("$arg")
|
|
316
|
+
done
|
|
317
|
+
JSON_CMD_ARGS+=(--mode json)
|
|
318
|
+
|
|
319
|
+
PARSER="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)/rlm_parse_json"
|
|
320
|
+
|
|
321
|
+
$TIMEOUT_CMD pi "${JSON_CMD_ARGS[@]}" "$PROMPT" 2>/dev/null | python3 "$PARSER" 3>"$COST_OUT"
|
|
322
|
+
RC=${PIPESTATUS[0]}
|
|
323
|
+
|
|
324
|
+
# Record cost if we got data
|
|
325
|
+
if [ -s "$COST_OUT" ] && [ -n "${RLM_COST_FILE:-}" ]; then
|
|
326
|
+
cat "$COST_OUT" >> "$RLM_COST_FILE"
|
|
327
|
+
fi
|
|
328
|
+
|
|
329
|
+
# Log cost to trace
|
|
330
|
+
if [ -s "$COST_OUT" ] && [ -n "${PI_TRACE_FILE:-}" ]; then
|
|
331
|
+
COST_DATA=$(cat "$COST_OUT")
|
|
332
|
+
ELAPSED=$(( $(date +%s) - RLM_START_TIME ))
|
|
333
|
+
echo "[$(date +%Y-%m-%dT%H:%M:%S%z)] depth=$DEPTH COMPLETED exit=$RC elapsed=${ELAPSED}s cost=$COST_DATA" >> "$PI_TRACE_FILE"
|
|
334
|
+
elif [ -n "${PI_TRACE_FILE:-}" ]; then
|
|
335
|
+
ELAPSED=$(( $(date +%s) - RLM_START_TIME ))
|
|
336
|
+
echo "[$(date +%Y-%m-%dT%H:%M:%S%z)] depth=$DEPTH COMPLETED exit=$RC elapsed=${ELAPSED}s" >> "$PI_TRACE_FILE"
|
|
337
|
+
fi
|
|
338
|
+
else
|
|
339
|
+
# Plain text mode (RLM_JSON=0): no cost tracking
|
|
340
|
+
$TIMEOUT_CMD pi "${CMD_ARGS[@]}" "$PROMPT"
|
|
341
|
+
RC=$?
|
|
342
|
+
|
|
343
|
+
if [ -n "${PI_TRACE_FILE:-}" ]; then
|
|
344
|
+
ELAPSED=$(( $(date +%s) - RLM_START_TIME ))
|
|
345
|
+
echo "[$(date +%Y-%m-%dT%H:%M:%S%z)] depth=$DEPTH COMPLETED exit=$RC elapsed=${ELAPSED}s" >> "$PI_TRACE_FILE"
|
|
346
|
+
fi
|
|
347
|
+
fi
|
|
348
|
+
|
|
349
|
+
exit $RC
|
package/ypi
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
# ypi — Y-Combinator Pi — Recursive Coding Agent
|
|
3
|
+
#
|
|
4
|
+
# Launches Pi as a Recursive Language Model. The LLM gets a system prompt
|
|
5
|
+
# that teaches it to use bash + rlm_query for divide-and-conquer reasoning
|
|
6
|
+
# over large contexts.
|
|
7
|
+
#
|
|
8
|
+
# Usage:
|
|
9
|
+
# ypi # interactive recursive pi
|
|
10
|
+
# ypi "What is in this repo?" # one-shot with -p
|
|
11
|
+
# ypi --provider anthropic --model claude-sonnet-4-5-20250929 "question"
|
|
12
|
+
#
|
|
13
|
+
# Environment overrides:
|
|
14
|
+
# RLM_PROVIDER — LLM provider for root call (default: cerebras)
|
|
15
|
+
# RLM_MODEL — LLM model for root call (default: gpt-oss-120b)
|
|
16
|
+
# RLM_MAX_DEPTH — max recursion depth (default: 3)
|
|
17
|
+
# RLM_TIMEOUT — wall-clock seconds for entire recursive tree (default: none)
|
|
18
|
+
# RLM_MAX_CALLS — max total rlm_query invocations (default: none)
|
|
19
|
+
# RLM_CHILD_MODEL — cheaper model for sub-calls at depth > 0 (default: same as root)
|
|
20
|
+
# RLM_CHILD_PROVIDER — provider for sub-calls at depth > 0 (default: same as root)
|
|
21
|
+
# RLM_JJ — set to "0" to disable jj workspace isolation (default: 1)
|
|
22
|
+
# RLM_EXTENSIONS — set to "0" to disable Pi extensions (default: 1)
|
|
23
|
+
# RLM_CHILD_EXTENSIONS — override extensions for depth > 0 (default: same as parent)
|
|
24
|
+
# RLM_BUDGET — max dollar spend for entire recursive tree (default: none)
|
|
25
|
+
# RLM_JSON — set to "0" to disable JSON mode / cost tracking (default: 1)
|
|
26
|
+
# PI_TRACE_FILE — path to trace log for all calls with timing (default: none)
|
|
27
|
+
|
|
28
|
+
set -euo pipefail
|
|
29
|
+
|
|
30
|
+
# Resolve the directory where ypi lives (and where rlm_query + SYSTEM_PROMPT.md are)
|
|
31
|
+
# Follow symlinks — npm install -g creates symlinks in .bin/ pointing to node_modules/ypi/
|
|
32
|
+
SCRIPT_DIR="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
|
|
33
|
+
|
|
34
|
+
# Put rlm_query on PATH so Pi's bash tool can find it
|
|
35
|
+
export PATH="$SCRIPT_DIR:$PATH"
|
|
36
|
+
|
|
37
|
+
# Initialize RLM environment — pass through all guardrails
|
|
38
|
+
export RLM_DEPTH="${RLM_DEPTH:-0}"
|
|
39
|
+
export RLM_MAX_DEPTH="${RLM_MAX_DEPTH:-3}"
|
|
40
|
+
export RLM_PROVIDER="${RLM_PROVIDER:-cerebras}"
|
|
41
|
+
export RLM_MODEL="${RLM_MODEL:-gpt-oss-120b}"
|
|
42
|
+
export RLM_SYSTEM_PROMPT="$SCRIPT_DIR/SYSTEM_PROMPT.md"
|
|
43
|
+
|
|
44
|
+
# Guardrails — pass through if set, don't override
|
|
45
|
+
[ -n "${RLM_TIMEOUT:-}" ] && export RLM_TIMEOUT
|
|
46
|
+
[ -n "${RLM_MAX_CALLS:-}" ] && export RLM_MAX_CALLS
|
|
47
|
+
[ -n "${RLM_CHILD_MODEL:-}" ] && export RLM_CHILD_MODEL
|
|
48
|
+
[ -n "${RLM_CHILD_PROVIDER:-}" ] && export RLM_CHILD_PROVIDER
|
|
49
|
+
[ -n "${PI_TRACE_FILE:-}" ] && export PI_TRACE_FILE
|
|
50
|
+
export RLM_JJ="${RLM_JJ:-1}"
|
|
51
|
+
export RLM_EXTENSIONS="${RLM_EXTENSIONS:-1}"
|
|
52
|
+
[ -n "${RLM_CHILD_EXTENSIONS:-}" ] && export RLM_CHILD_EXTENSIONS
|
|
53
|
+
[ -n "${RLM_BUDGET:-}" ] && export RLM_BUDGET
|
|
54
|
+
export RLM_JSON="${RLM_JSON:-1}"
|
|
55
|
+
|
|
56
|
+
# Session tree tracing — generate a trace ID that links all recursive sessions
|
|
57
|
+
export RLM_TRACE_ID="${RLM_TRACE_ID:-$(head -c 4 /dev/urandom | od -An -tx1 | tr -d ' \n')}"
|
|
58
|
+
|
|
59
|
+
# Compute Pi's session directory for this CWD so children can write there
|
|
60
|
+
CWD="$(pwd)"
|
|
61
|
+
SAFE_PATH="--$(echo "$CWD" | sed 's|^/||; s|[/:\\]|-|g')--"
|
|
62
|
+
export RLM_SESSION_DIR="${HOME}/.pi/agent/sessions/${SAFE_PATH}"
|
|
63
|
+
mkdir -p "$RLM_SESSION_DIR"
|
|
64
|
+
|
|
65
|
+
# Build combined system prompt: SYSTEM_PROMPT.md + rlm_query source
|
|
66
|
+
# This way the agent sees the full implementation, not just usage docs.
|
|
67
|
+
COMBINED_PROMPT=$(mktemp /tmp/ypi_system_prompt_XXXXXX.md)
|
|
68
|
+
trap 'rm -f "$COMBINED_PROMPT"' EXIT
|
|
69
|
+
|
|
70
|
+
cat "$SCRIPT_DIR/SYSTEM_PROMPT.md" > "$COMBINED_PROMPT"
|
|
71
|
+
cat >> "$COMBINED_PROMPT" << 'EOF'
|
|
72
|
+
|
|
73
|
+
## SECTION 6 – rlm_query Implementation
|
|
74
|
+
|
|
75
|
+
Below is the full source of `rlm_query`. You are running inside this infrastructure.
|
|
76
|
+
Understanding it helps you use recursion effectively and respect guardrails.
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
EOF
|
|
80
|
+
cat "$SCRIPT_DIR/rlm_query" >> "$COMBINED_PROMPT"
|
|
81
|
+
cat >> "$COMBINED_PROMPT" << 'EOF'
|
|
82
|
+
```
|
|
83
|
+
EOF
|
|
84
|
+
|
|
85
|
+
# Launch Pi with the combined system prompt, passing all args through
|
|
86
|
+
exec pi --system-prompt "$COMBINED_PROMPT" "$@"
|