jev-planner 0.0.1 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +481 -2
- package/config.schema.json +264 -0
- package/dist/bin.cjs +19 -0
- package/dist/bin.d.cts +1 -0
- package/dist/bin.d.mts +1 -0
- package/dist/bin.mjs +20 -0
- package/dist/index.cjs +12 -0
- package/dist/index.d.cts +566 -0
- package/dist/index.d.mts +566 -0
- package/dist/index.mjs +2 -0
- package/dist/jev-CIzG0Rhl.mjs +2685 -0
- package/dist/jev-aqxfEUdB.cjs +2756 -0
- package/llms.txt +206 -0
- package/package.json +77 -5
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Jonatan Kruszewski
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,5 +1,484 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="https://raw.githubusercontent.com/rxova/jev-planner/main/apps/docs/public/logo.svg" alt="jev-planner" width="320">
|
|
3
|
+
</p>
|
|
4
|
+
|
|
1
5
|
# jev-planner
|
|
2
6
|
|
|
3
|
-
|
|
7
|
+
`jev-planner` creates repository-aware implementation plans by combining two or more AIs with
|
|
8
|
+
[TypeSafe Jev](https://docs.typesafe.ai/concepts/system-one):
|
|
9
|
+
|
|
10
|
+
1. Each agent — Codex and Claude by default — drafts a plan independently, all in parallel.
|
|
11
|
+
2. Jev scores completeness, feasibility, and risk coverage; chooses a finalizer; and decides
|
|
12
|
+
whether a cross-review would materially improve the plan.
|
|
13
|
+
3. When it would, each agent sees every other agent's plan and returns a revised, standalone plan,
|
|
14
|
+
and Jev evaluates again — up to `--review-rounds` times.
|
|
15
|
+
4. The selected agent merges the plans into one final implementation plan, unless Jev judges one
|
|
16
|
+
cross-reviewed plan final as it stands.
|
|
17
|
+
|
|
18
|
+
Steps 3 and 4 are the ones a run can skip, and skipping them is most of the wall clock: the agents
|
|
19
|
+
in a step run in parallel, but each step waits for the one before, and a step waits for its slowest
|
|
20
|
+
agent. In the runs on [Modes compared](https://jev-planner.com/learn/modes-compared/) a step took
|
|
21
|
+
from under a minute to four. See [Modes](#modes).
|
|
22
|
+
|
|
23
|
+
The cross-review is what those steps buy. Drafts are written blind; reading each other's plans, the
|
|
24
|
+
agents can correct each other's facts about the repository, drop the ideas that do not survive a
|
|
25
|
+
second opinion, take the other plan's strengths, and name the questions they still disagree on. In
|
|
26
|
+
the debate run on Modes compared, 7 of the 10 objections were about the repository, and all 10 were
|
|
27
|
+
accepted. Compare `round1/` with `round2/` in a run folder to see it in yours.
|
|
28
|
+
[How the cross-review improves a plan →](https://jev-planner.com/learn/how-it-works/#what-the-cross-review-improves)
|
|
29
|
+
|
|
30
|
+
The planner itself is [`packages/core`](https://github.com/rxova/jev-planner/tree/main/packages/core#readme),
|
|
31
|
+
an internal package bundled into this one; jev-planner adds Jev, TypeSafe's typed judge, and the
|
|
32
|
+
`jev-planner` command.
|
|
33
|
+
|
|
34
|
+
**[Documentation →](https://jev-planner.com/)**
|
|
35
|
+
|
|
36
|
+
## Agents
|
|
37
|
+
|
|
38
|
+
Pick the agents with `--agents`, two or more, comma-separated. One provider can be two of them,
|
|
39
|
+
under different names; see [one provider, several agents](#one-provider-several-agents).
|
|
40
|
+
|
|
41
|
+
| Id | AI | Kind | Needs |
|
|
42
|
+
| ---------- | ----------------------------------------------------------------------------- | --------- | ------------------ |
|
|
43
|
+
| `codex` | [Codex CLI](https://learn.chatgpt.com/docs/non-interactive-mode) | agent CLI | the CLI, logged in |
|
|
44
|
+
| `claude` | [Claude Code](https://docs.anthropic.com/en/docs/claude-code/getting-started) | agent CLI | the CLI, logged in |
|
|
45
|
+
| `deepseek` | [DeepSeek](https://api-docs.deepseek.com) | chat API | `DEEPSEEK_API_KEY` |
|
|
46
|
+
| `kimi` | [Kimi](https://platform.moonshot.ai) (Moonshot) | chat API | `MOONSHOT_API_KEY` |
|
|
47
|
+
| `glm` | [GLM](https://docs.z.ai) (Z.ai) | chat API | `ZAI_API_KEY` |
|
|
48
|
+
|
|
49
|
+
`jev-planner --help` prints the same list, generated from the registry.
|
|
50
|
+
|
|
51
|
+
- **Agent CLIs** inspect the repository themselves, read-only: Codex runs in its read-only sandbox,
|
|
52
|
+
Claude Code in plan mode with only `Read`, `Glob` and `Grep` and none of your MCP servers. They use
|
|
53
|
+
the CLIs' existing logins, so their calls consume your Codex and Claude subscription allowances,
|
|
54
|
+
not API keys.
|
|
55
|
+
- **Chat APIs** cannot open files. Each of their calls is sent with a snapshot of the repository:
|
|
56
|
+
the list of files git tracks, and the contents of the tracked top-level docs and manifests
|
|
57
|
+
(`AGENTS.md`, `CLAUDE.md`, `README.md`, `CONTRIBUTING.md`, `package.json`, …), within fixed size
|
|
58
|
+
limits. Only tracked files are read, so an ignored `.env` is never sent. Outside a git repository
|
|
59
|
+
the snapshot is empty. The model is told to name the files it would need rather than guess them.
|
|
60
|
+
|
|
61
|
+
Each agent keeps one conversation through a run. An agent CLI's draft session is continued for its
|
|
62
|
+
cross-review and the final synthesis (`codex exec resume`, `claude --resume`), so those stages start
|
|
63
|
+
with what it already read instead of exploring the repository again; a resumed Codex keeps its
|
|
64
|
+
read-only sandbox. A chat API is sent its earlier messages, so the repository snapshot goes once.
|
|
65
|
+
If a session cannot be continued, the call starts afresh with the whole prompt.
|
|
66
|
+
|
|
67
|
+
## One provider, several agents
|
|
68
|
+
|
|
69
|
+
An agent is a provider under a name. `--agents codex` is short for `codex:codex`, and
|
|
70
|
+
`--agents codex:sol,codex:terra` runs Codex twice, as two agents named `sol` and `terra`. Each has
|
|
71
|
+
its own session, draft and round files (`round1/sol.md`), and the overrides take the name:
|
|
72
|
+
|
|
73
|
+
```sh
|
|
74
|
+
jev-planner --agents codex:sol,codex:terra \
|
|
75
|
+
--model sol=gpt-5.6-sol --model terra=gpt-5.6-terra "Add caching to the search endpoint"
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
- **Names** are a letter, then letters, digits or `-`, at most 24 characters, read lowercased. A
|
|
79
|
+
name cannot be `auto`, `none`, `tie`, a Windows device name (`con`, `nul`, …) or another
|
|
80
|
+
provider's id.
|
|
81
|
+
- **Labels** tell them apart: the output, the peer reviews and Jev see `Codex (sol)` and
|
|
82
|
+
`Codex (terra)`.
|
|
83
|
+
- **Vary them.** Two agents with the same provider, model and effort get a warning on stderr, since
|
|
84
|
+
their drafts may barely differ; the run still goes ahead.
|
|
85
|
+
- **Name the one you mean.** Once a provider's agents are named, `--model codex=…` is an error that
|
|
86
|
+
lists them.
|
|
87
|
+
- **One quota.** Both draw on the same subscription or key, at the same time. A rate limit (HTTP 429) fails the call, and a failed call fails the run.
|
|
88
|
+
- `doctor` checks each provider once.
|
|
89
|
+
|
|
90
|
+
The CLIs keep those sessions as they keep any other: in `~/.codex/sessions` and
|
|
91
|
+
`~/.claude/projects`, and Claude's appear in its `/resume` list. `--no-resume` starts every call
|
|
92
|
+
afresh and keeps none, as before.
|
|
93
|
+
|
|
94
|
+
Every provider's API key, and Jev's, is removed from the environment of every agent subprocess:
|
|
95
|
+
an agent never sees another provider's credentials.
|
|
96
|
+
|
|
97
|
+
## Requirements
|
|
98
|
+
|
|
99
|
+
- Node.js 20.19 or newer
|
|
100
|
+
- For each agent CLI you select: the CLI, already logged in
|
|
101
|
+
- For each chat API you select: its API key in the environment
|
|
102
|
+
- A TypeSafe API key from <https://console.typesafe.ai/keys>
|
|
103
|
+
|
|
104
|
+
The implementation uses the official [`@typesafe-ai/sdk`](https://docs.typesafe.ai/sdk/javascript)
|
|
105
|
+
and defaults to the SDK's `jev-latest` model alias.
|
|
106
|
+
|
|
107
|
+
## Install
|
|
108
|
+
|
|
109
|
+
```sh
|
|
110
|
+
npm install -g jev-planner
|
|
111
|
+
export TYPESAFE_API_KEY="your-key"
|
|
112
|
+
jev-planner doctor
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Or run it without installing: `npx jev-planner "<coding task>"`.
|
|
116
|
+
|
|
117
|
+
To run it from a clone of this repository instead:
|
|
118
|
+
|
|
119
|
+
```sh
|
|
120
|
+
corepack enable
|
|
121
|
+
pnpm install
|
|
122
|
+
pnpm build
|
|
123
|
+
node packages/jev-planner/dist/bin.mjs --help
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
`doctor` checks the selected agents — each CLI is installed and logged in, each API key is set —
|
|
127
|
+
and the Jev key. `jev-planner doctor --agents codex,deepseek` checks that pair. It does not make a
|
|
128
|
+
paid model call.
|
|
129
|
+
|
|
130
|
+
## Use
|
|
131
|
+
|
|
132
|
+
Run from the repository you want the agents to inspect:
|
|
133
|
+
|
|
134
|
+
```sh
|
|
135
|
+
jev-planner "Add per-user rate limiting to the public API"
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
Write the result to a file:
|
|
139
|
+
|
|
140
|
+
```sh
|
|
141
|
+
jev-planner -o PLAN.md "Migrate the persistence layer from SQLite to Postgres"
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Target a different repository or provide a longer brief:
|
|
145
|
+
|
|
146
|
+
```sh
|
|
147
|
+
jev-planner --cwd ../my-app --file ./brief.md --output PLAN.md
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Plan with three agents, and pin one's model:
|
|
151
|
+
|
|
152
|
+
```sh
|
|
153
|
+
jev-planner --agents claude,deepseek,glm --model glm=glm-4.6 "Add a CSV export to the reports page"
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Use JSON in another tool:
|
|
157
|
+
|
|
158
|
+
```sh
|
|
159
|
+
jev-planner --json "Make image uploads resumable" | jq '.verdict, .plan'
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
See every option with `jev-planner --help`. Useful controls include:
|
|
163
|
+
|
|
164
|
+
- `--agents <provider[:name],…>` to choose two or more agents (default: `codex,claude`).
|
|
165
|
+
- `--model <name>=<model>`, repeatable, to override one agent's model.
|
|
166
|
+
- `--effort <name>=<level>`, repeatable, to override an agent CLI's reasoning effort. Levels are the
|
|
167
|
+
CLI's own (`low` … `xhigh` and more, per model) and are passed through unchecked.
|
|
168
|
+
- `--review-effort <name>=<level>`, repeatable, to use another effort for that agent's cross-reviews
|
|
169
|
+
and synthesis only, while its draft keeps `--effort`. The later stages edit plans rather than
|
|
170
|
+
explore the repository, so a lower effort is meant to make them quicker; that is not measured.
|
|
171
|
+
|
|
172
|
+
Model and effort overrides win over the CLIs' local configuration, such as `model` and
|
|
173
|
+
`model_reasoning_effort` in `~/.codex/config.toml`, for that run only. Codex on GPT-5.6-Terra at low
|
|
174
|
+
effort, with Claude at its defaults:
|
|
175
|
+
|
|
176
|
+
```sh
|
|
177
|
+
jev-planner --model codex=gpt-5.6-terra --effort codex=low "Add caching to the search endpoint"
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
- `--judge-model` to pin a TypeSafe model rather than use `jev-latest`.
|
|
181
|
+
- `--finalizer <name>` to override Jev's routing decision with one of the selected agents.
|
|
182
|
+
- `--finalizer none` to keep the cross-reviewed plan Jev rates stronger as it is, rather than
|
|
183
|
+
merge. It saves the last agent call, at the cost of the merge; on a tie, or when no cross-review
|
|
184
|
+
ran, the finalizer still runs.
|
|
185
|
+
- `--mode ultra` to always run the first cross-review rather than let Jev skip it, or `--mode fast`
|
|
186
|
+
to answer with the first draft Jev accepts on its own (below).
|
|
187
|
+
- `--review-rounds 0` to skip the cross-review entirely, or `1` to allow only one.
|
|
188
|
+
- `--straggler-grace <seconds>` to change how long a `balanced` or `fast` round waits for a slow
|
|
189
|
+
agent.
|
|
190
|
+
- `--review-mode debate` to have the agents critique and answer each other, and Jev rule on what
|
|
191
|
+
they still disagree about; `--claim-checks` to check the disputed repository claims too (below).
|
|
192
|
+
- `--no-resume` to start every agent call afresh rather than continue its draft session
|
|
193
|
+
([Agents](#agents)).
|
|
194
|
+
- `--verbose` to watch the agents work, then print Jev's typed verdict to stderr (below).
|
|
195
|
+
- `--rounds-dir <path>` to keep every round's plans somewhere other than `.jev-planner/`, or
|
|
196
|
+
`--no-rounds` to keep none (below).
|
|
197
|
+
- `--allow-any-task` to plan text that looks like a placeholder.
|
|
198
|
+
|
|
199
|
+
A task that is empty or a near-certain placeholder — the text `TODO`, `TBD` or `<coding task>`,
|
|
200
|
+
an unfilled `<…>`, `{{…}}` or `[…]` slot, or text with no letters — is rejected before any paid
|
|
201
|
+
call. Only the whole text is compared, so a brief that quotes a placeholder, or a short real task
|
|
202
|
+
such as `Add caching`, is planned as usual. `Planner.plan` runs the same check and throws
|
|
203
|
+
`TaskValidationError`; set `allowAnyTask: true` in its options to skip it.
|
|
204
|
+
|
|
205
|
+
## Config file
|
|
206
|
+
|
|
207
|
+
A `jev-planner.json` in the repository, the `--cwd` folder or the current one, sets up every run
|
|
208
|
+
from there. `--config <path>` reads another file, `--no-config` none. It is for the CLI only;
|
|
209
|
+
`Planner` never reads it.
|
|
210
|
+
|
|
211
|
+
```json
|
|
212
|
+
{
|
|
213
|
+
"$schema": "https://jev-planner.com/config.schema.json",
|
|
214
|
+
"agents": { "codex": { "model": "gpt-5.6-sol", "effort": "high" }, "claude": {} },
|
|
215
|
+
"mode": "ultra",
|
|
216
|
+
"runsDir": "planner-runs",
|
|
217
|
+
"output": "PLAN.md"
|
|
218
|
+
}
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
Each key stands for the flag of the same name: `agents` with each agent's `model`, `effort` and
|
|
222
|
+
`reviewEffort`, keyed by provider id or by a name that sets `provider`
|
|
223
|
+
(`"sol": { "provider": "codex" }`); `mode`, `reviewMode`, `reviewRounds`, `claimChecks`, `finalizer`, `judgeModel`,
|
|
224
|
+
`stragglerGrace` and `timeout` (seconds); `resume`, `rounds`, `json`, `verbose` and
|
|
225
|
+
`allowAnyTask`; `output`; `task` or `taskFile`; and `cwd`, only in a file passed with `--config`.
|
|
226
|
+
`runsDir` is a folder in which each run gets its own timestamped folder. Paths are relative to the
|
|
227
|
+
file, and an unknown key or a wrong type is an error that names the key.
|
|
228
|
+
|
|
229
|
+
- **Flags win**, setting by setting. `--model codex=gpt-x` beats the config's model for Codex only,
|
|
230
|
+
`--agents` drops the config's settings for the agents it leaves out, and every boolean has both
|
|
231
|
+
forms: `--json` and `--no-json`, `--resume` and `--no-resume`, and so on.
|
|
232
|
+
- **The task**: arguments or `--file` first, then the config's `task` or `taskFile`, then stdin. A
|
|
233
|
+
task piped while the config has one is an error, never silently ignored.
|
|
234
|
+
- **No secrets.** The file is meant to be committed. A key such as `apiKey` or `token` is rejected
|
|
235
|
+
with the environment variable to set instead.
|
|
236
|
+
|
|
237
|
+
The schema ships as `node_modules/jev-planner/config.schema.json` too. The
|
|
238
|
+
[config file guide](https://jev-planner.com/guides/config-file/) has the full table.
|
|
239
|
+
|
|
240
|
+
## Modes
|
|
241
|
+
|
|
242
|
+
A run's wall clock is not the number of agent calls — the agents in a round run in parallel — but
|
|
243
|
+
the number of rounds, because each one waits for the round before it. `--mode` decides how many a
|
|
244
|
+
run is allowed to spend.
|
|
245
|
+
|
|
246
|
+
| `--mode` | Cross-review | Final merge | Agent calls, N agents | Rounds |
|
|
247
|
+
| -------------------- | -------------------------------- | ----------------------------------------- | --------------------- | ------ |
|
|
248
|
+
| `fast` | Never | Skipped when one draft stands alone | N … N + 1 | 1 or 2 |
|
|
249
|
+
| `balanced` (default) | Only when Jev asks for one | Skipped when a reviewed plan stands alone | N + 1 … 3N + 1 | 2 … 4 |
|
|
250
|
+
| `ultra` | Always, plus Jev's optional pass | Always, unless `--finalizer none` | 2N + 1, or 3N + 1 | 3 or 4 |
|
|
251
|
+
|
|
252
|
+
At best, with the default two agents, `balanced` spends three agent calls in two rounds where `ultra`
|
|
253
|
+
spends five in three. When Jev asks for both reviews, it spends what `ultra` does.
|
|
254
|
+
|
|
255
|
+
`balanced` puts Jev's typed judgment in front of each round instead of after it:
|
|
256
|
+
|
|
257
|
+
- **The cross-review is Jev's to order.** It judges the drafts first, and the agents only revise
|
|
258
|
+
against each other when Jev rates the chance that another pass would materially improve the plan
|
|
259
|
+
at 0.65 or more. Below that, it is a whole round of agent calls the run does not make.
|
|
260
|
+
- **The merge is Jev's to waive.** After a cross-review every plan already answers the others, so
|
|
261
|
+
when Jev judges the strongest one final as it stands, the run answers with it rather than paying
|
|
262
|
+
an agent to rewrite it. A plan that has _not_ been cross-reviewed is never adopted this way: the
|
|
263
|
+
merge is the only place the agents' material comes together, so it always runs.
|
|
264
|
+
- **A round stops waiting for a straggler.** Once half the agents (rounded up) have answered, the rest get
|
|
265
|
+
`--straggler-grace` seconds (90 by default) before the round goes on without them, and their calls
|
|
266
|
+
are aborted rather than left running. A round never drops below two plans, so with two agents a
|
|
267
|
+
draft is always waited for; an agent dropped from a cross-review keeps its previous plan.
|
|
268
|
+
|
|
269
|
+
`ultra` always runs the first cross-review: every agent drafts, and every agent reviews every other,
|
|
270
|
+
whatever the drafts turned out to be. Jev may ask for one more pass, and the finalizer merges, unless
|
|
271
|
+
`--finalizer none` keeps the reviewed plan Jev rates stronger. `--review-rounds 0` removes the review.
|
|
272
|
+
Use it when the plan matters more than the wait.
|
|
273
|
+
|
|
274
|
+
`fast` spends the fewest rounds, and pays for it in scrutiny:
|
|
275
|
+
|
|
276
|
+
- **Jev judges each draft alone, as it arrives.** One at a time, in the order the agents answer,
|
|
277
|
+
Jev is asked whether that plan could go to an implementer as it stands. The first one it rates
|
|
278
|
+
at 0.5 or more is the answer, and the agents still drafting are stopped.
|
|
279
|
+
- **Otherwise it is `balanced` without the cross-review.** When Jev accepts no draft, it judges
|
|
280
|
+
them together and the finalizer merges them; no agent reviews another's plan.
|
|
281
|
+
- **It favours the quickest agent.** Whichever agent answers first is judged first, so a quick
|
|
282
|
+
plan that clears the bar beats a slower, better one nobody waited for.
|
|
283
|
+
- **An accepted plan was read by no other agent.** Nothing in it has been challenged; use `fast`
|
|
284
|
+
for tasks where a single good plan is enough.
|
|
285
|
+
- **Stopped work is still paid for.** An agent aborted mid-draft has already spent what it used,
|
|
286
|
+
and it counts as an agent call.
|
|
287
|
+
|
|
288
|
+
Each draft Jev judges alone is one TypeSafe call, so `fast` makes 1 … N + 1 of them. It has no
|
|
289
|
+
review round, so `--review-mode debate` and `--claim-checks` are rejected with it, and
|
|
290
|
+
`--review-rounds` is ignored. `--finalizer` only picks who merges when no draft is accepted.
|
|
291
|
+
|
|
292
|
+
[Modes compared](https://jev-planner.com/learn/modes-compared/) plans one real task in every mode,
|
|
293
|
+
with the time, rounds, calls and Jev's verdicts of each.
|
|
294
|
+
|
|
295
|
+
Every run prints what it spent on stderr, and `--json` includes it as `cost`:
|
|
296
|
+
|
|
297
|
+
```text
|
|
298
|
+
[jev-planner] balanced mode, 3 agent calls, 1 Jev call, 0 cross-review rounds, merged
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
A `fast` run that answers with an accepted draft ends with `selected`, and names the agents it
|
|
302
|
+
stopped:
|
|
303
|
+
|
|
304
|
+
```text
|
|
305
|
+
[jev-planner] fast mode, 2 agent calls, 1 Jev call, 0 cross-review rounds, selected, not waited for: codex
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
### Debate review (experimental)
|
|
309
|
+
|
|
310
|
+
`--review-mode debate` replaces the first cross-review with an exchange Jev can rule on. `balanced`
|
|
311
|
+
and `ultra` run it where they would run a cross-review; `fast` has none, and rejects it:
|
|
312
|
+
|
|
313
|
+
1. **Critiques.** Each agent lists numbered objections to every other plan, at most five per plan,
|
|
314
|
+
and tags the ones that make a claim about the repository `[repo]`. It writes no plan.
|
|
315
|
+
2. **Replies.** Each author accepts or rejects every objection to its own plan, by id, and returns
|
|
316
|
+
its revised plan.
|
|
317
|
+
3. **Disputes.** The rejected objections become disputes, the same claim against the same plan
|
|
318
|
+
merged whoever raised it. Jev rules on up to eight of them, critic, author or unclear, in the
|
|
319
|
+
same call as its usual verdict.
|
|
320
|
+
4. **A targeted pass.** When Jev asks for another pass, the agents revise against the disputes it
|
|
321
|
+
left open, not the whole verdict. The merge sees every dispute and its ruling.
|
|
322
|
+
|
|
323
|
+
`--claim-checks` (which implies `--review-mode debate`) adds a step before Jev rules: each disputed
|
|
324
|
+
`[repo]` claim goes to an agent that reads the repository and did not raise it, which answers
|
|
325
|
+
CONFIRM, REFUTE or UNKNOWN with the file that shows it. It needs two agent CLIs among the agents; a
|
|
326
|
+
chat API sees only a snapshot, so with fewer the checks are skipped and the run says so.
|
|
327
|
+
|
|
328
|
+
A debate costs 2N agent calls where a cross-review costs N, plus one call per agent that checks a
|
|
329
|
+
claim. It never runs with `--review-rounds 0`. The rounds folder keeps each critique, reply and check
|
|
330
|
+
as `<agent>.critique.md`, `.reply.md` and `.check.md`, beside `objections.json`, `replies.json` and
|
|
331
|
+
`disputes.json`; from code, `PlanRound.debate` and `PlanResult.debate` carry the same.
|
|
332
|
+
|
|
333
|
+
## Following a run round by round
|
|
334
|
+
|
|
335
|
+
Every run writes each round's plans as soon as the round ends, to a new folder under
|
|
336
|
+
`.jev-planner/` in the repository, named by the run's UTC start time:
|
|
337
|
+
|
|
338
|
+
```text
|
|
339
|
+
.jev-planner/
|
|
340
|
+
.gitignore `*`, so the folder never shows up in git
|
|
341
|
+
20260921-230512/
|
|
342
|
+
round1/ the independent drafts
|
|
343
|
+
codex.md
|
|
344
|
+
claude.md
|
|
345
|
+
verdict.json in balanced and fast mode; ultra judges only reviewed plans
|
|
346
|
+
timings.json how long the round and each call in it took, in milliseconds
|
|
347
|
+
round2/ only when a cross-review ran: the revised plans, and Jev's verdict
|
|
348
|
+
codex.md
|
|
349
|
+
claude.md
|
|
350
|
+
verdict.json
|
|
351
|
+
round3/ only when Jev asked for a second review
|
|
352
|
+
final/
|
|
353
|
+
plan.md the merged plan, headed by the agent that merged it, or selected from
|
|
354
|
+
verdict.json the verdict the merge followed
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
In `fast` mode, `round1/verdict.json` is the verdict that decided the run: the accepted draft's,
|
|
358
|
+
or the one Jev gave the drafts together. The verdicts of drafts it turned down alone are not saved.
|
|
359
|
+
A debate names its rounds' files differently ([Debate review](#debate-review-experimental)).
|
|
360
|
+
|
|
361
|
+
`--rounds-dir <path>` writes them somewhere else instead, relative to `--cwd`; that folder must be
|
|
362
|
+
new or empty, so two runs never mix. `--no-rounds` writes nothing.
|
|
363
|
+
|
|
364
|
+
```sh
|
|
365
|
+
jev-planner --rounds-dir rounds -o PLAN.md "Add caching to the search endpoint"
|
|
366
|
+
```
|
|
367
|
+
|
|
368
|
+
From code, `onRound` in `Planner.plan`'s options receives the same rounds as `PlanRound` objects.
|
|
369
|
+
|
|
370
|
+
## Watching the agents work
|
|
371
|
+
|
|
372
|
+
A draft can take minutes. `--verbose` streams what each agent is doing to stderr as it happens, one
|
|
373
|
+
line per step, prefixed with the agent:
|
|
374
|
+
|
|
375
|
+
```text
|
|
376
|
+
[jev-planner] Drafting independent plans with Codex and Claude…
|
|
377
|
+
[claude] Grep deploy|pages
|
|
378
|
+
[codex] I'll inspect the docs app and the workflows first.
|
|
379
|
+
[codex] $ /bin/zsh -lc "ls apps/docs .github/workflows"
|
|
380
|
+
[claude] Read apps/docs/astro.config.mjs
|
|
381
|
+
```
|
|
382
|
+
|
|
383
|
+
Codex and Claude run with JSON event output (`codex exec --json`, `claude --output-format
|
|
384
|
+
stream-json`), so every message, command and file read is shown as the agent reaches it. A chat API
|
|
385
|
+
agent answers in one response, so it shows only which model it is waiting on. From code, pass
|
|
386
|
+
`onAgentProgress` in `Planner.plan`'s options.
|
|
387
|
+
|
|
388
|
+
After each round, `--verbose` prints how long it took and how long each call in it took, and a
|
|
389
|
+
total at the end. These are the rounds of the `ultra` run on Modes compared:
|
|
390
|
+
|
|
391
|
+
```text
|
|
392
|
+
[jev-planner] Drafts: 2m58s (Claude 1m18s, Codex 2m58s)
|
|
393
|
+
[jev-planner] Review: 1m20s (Claude 1m01s, Codex 1m18s, Jev 1.4s)
|
|
394
|
+
[jev-planner] Review: 1m25s (Claude 1m04s, Codex 1m24s, Jev 1.2s)
|
|
395
|
+
[jev-planner] Final plan: 54s (Claude 54s)
|
|
396
|
+
```
|
|
397
|
+
|
|
398
|
+
The same numbers, in milliseconds, are in each round's `timings.json`, in `--json`'s `timings`,
|
|
399
|
+
and in `PlanRound.timings` and `PlanResult.timings` from code.
|
|
400
|
+
|
|
401
|
+
## Cost and data flow
|
|
402
|
+
|
|
403
|
+
With N agents, `--mode ultra` makes 2N + 1 agent calls: N drafts, N cross-reviews, and one final
|
|
404
|
+
synthesis. If Jev requests another pass, it makes N more. With the default two agents that is five
|
|
405
|
+
calls, or seven. `--mode balanced`, the default, makes as few as N + 1 — the drafts and the merge, when
|
|
406
|
+
Jev asks for no cross-review — and never more than `ultra` would. `--mode fast` makes N, or N + 1
|
|
407
|
+
when it has to merge; an agent it stops mid-draft still counts, and still bills what it used. `--finalizer none` drops the
|
|
408
|
+
synthesis when Jev rates one cross-reviewed plan stronger. Agent CLIs use the accounts logged into
|
|
409
|
+
them; chat APIs bill the key they are given.
|
|
410
|
+
|
|
411
|
+
Each evaluation uses one TypeSafe API call, and `balanced` spends one extra to judge the drafts;
|
|
412
|
+
`fast` spends one per draft it judges alone, plus one to judge them together when it accepts none. Jev sees
|
|
413
|
+
the task and the agents' plan text, not a direct repository snapshot. Chat APIs see the snapshot
|
|
414
|
+
described under [Agents](#agents). Every agent that cross-reviews, checks a claim or merges sees the
|
|
415
|
+
other agents' plans, which may contain file names or code details; a draft `fast` accepts is read by
|
|
416
|
+
no other agent. Do not run this on material you are not allowed to send to every
|
|
417
|
+
provider you select.
|
|
418
|
+
|
|
419
|
+
## Adding a new AI
|
|
420
|
+
|
|
421
|
+
Every agent comes from one list, `PROVIDERS` in `packages/core/src/providers/providers.ts`. The CLI flags, `--help`,
|
|
422
|
+
`doctor`, the prompts and Jev's choices are all built from it, so wiring up a new AI is one entry
|
|
423
|
+
there. Add the agent to `config.schema.json` too, and to its copy in `apps/docs/public/`: a test
|
|
424
|
+
fails until both list it.
|
|
425
|
+
|
|
426
|
+
An OpenAI-compatible chat API is one `openAICompatibleProvider` call:
|
|
427
|
+
|
|
428
|
+
```ts
|
|
429
|
+
openAICompatibleProvider({
|
|
430
|
+
id: 'qwen',
|
|
431
|
+
label: 'Qwen',
|
|
432
|
+
baseUrl: 'https://dashscope-intl.aliyuncs.com/compatible-mode/v1',
|
|
433
|
+
apiKeyEnv: 'DASHSCOPE_API_KEY',
|
|
434
|
+
model: 'qwen-max',
|
|
435
|
+
}),
|
|
436
|
+
```
|
|
437
|
+
|
|
438
|
+
An agent CLI that can run non-interactively and read-only, taking the prompt on stdin and printing
|
|
439
|
+
the plan on stdout, is one `cliProvider` call:
|
|
440
|
+
|
|
441
|
+
```ts
|
|
442
|
+
cliProvider({
|
|
443
|
+
id: 'acme',
|
|
444
|
+
label: 'Acme',
|
|
445
|
+
command: 'acme',
|
|
446
|
+
// Whatever makes this CLI answer once, read-only, without prompting.
|
|
447
|
+
args: ({ model, effort }) => [
|
|
448
|
+
'ask',
|
|
449
|
+
'--read-only',
|
|
450
|
+
...(model ? ['--model', model] : []),
|
|
451
|
+
...(effort ? ['--effort', effort] : []),
|
|
452
|
+
],
|
|
453
|
+
effort: true,
|
|
454
|
+
auth: ['whoami'],
|
|
455
|
+
}),
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
`effort: true` says `args` passes an effort on, so `--effort` is accepted for it. `auth` is
|
|
459
|
+
optional: arguments that exit 0 when the CLI is logged in, or a check function. So is `sessions`,
|
|
460
|
+
for a CLI that can continue a conversation: `start(overrides, id)` and `resume(overrides, id)`
|
|
461
|
+
return the arguments that keep one and continue it, and without it every call starts afresh. Both builders are
|
|
462
|
+
exported by this package, so a program using the library can build its own agents from them and
|
|
463
|
+
pass them to `Planner`.
|
|
464
|
+
|
|
465
|
+
## Development
|
|
466
|
+
|
|
467
|
+
```sh
|
|
468
|
+
pnpm --filter jev-planner test
|
|
469
|
+
pnpm run verify
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
The orchestration tests use fake agents and make no model calls.
|
|
473
|
+
[CONTRIBUTING.md](https://github.com/rxova/jev-planner/blob/main/CONTRIBUTING.md) has the rest.
|
|
474
|
+
|
|
475
|
+
## Why Jev is the arbiter
|
|
476
|
+
|
|
477
|
+
Jev does not generate prose or code. It returns constrained `choice`, `score`, and `noul` decisions
|
|
478
|
+
with probabilities. That makes it a good fit for the branch points in this workflow—quality scoring,
|
|
479
|
+
review routing, and finalizer selection—while the agents handle repository exploration and plan
|
|
480
|
+
writing.
|
|
481
|
+
|
|
482
|
+
## License
|
|
4
483
|
|
|
5
|
-
|
|
484
|
+
MIT
|