agent-ultramode 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -4
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
# ultra
|
|
2
2
|
|
|
3
|
+
[](https://www.npmjs.com/package/agent-ultramode)
|
|
4
|
+
[](https://github.com/maverick-tr/agent-ultramode/releases)
|
|
5
|
+
[](./LICENSE)
|
|
6
|
+
|
|
3
7
|
Took the [LLM-as-a-Verifier](https://github.com/llm-as-a-verifier/llm-as-a-verifier) paper (Kwok et al., 2026) and turned it into a simple `/ultra` command for coding agents.
|
|
4
8
|
|
|
5
9
|
Instead of shipping the agent's first attempt, `ultra` runs your task **N times in isolated git worktrees**, in parallel, then uses the **same model** as a verifier to pick the best result. When it is confident (or when most attempts succeeded) it applies the winning diff to your working tree; when the attempts diverge and it is not sure, it hands you the top candidates. No cross-model dependency, no extra services, no first-attempt lottery.
|
|
@@ -8,13 +12,13 @@ And unlike most "add a verifier" posts, I benchmarked it before believing it. Th
|
|
|
8
12
|
|
|
9
13
|
## Demo
|
|
10
14
|
|
|
11
|
-
<video src="https://github.com/
|
|
15
|
+
<video src="https://github.com/user-attachments/assets/134487dc-1d60-434b-a2dc-97cfd5942959" controls muted playsinline width="100%"></video>
|
|
12
16
|
|
|
13
|
-
|
|
17
|
+
[▶ Watch the demo](https://github.com/user-attachments/assets/134487dc-1d60-434b-a2dc-97cfd5942959) (55s, opencode + DeepSeek V4 Flash)
|
|
14
18
|
|
|
15
19
|
## The receipts
|
|
16
20
|
|
|
17
|
-
Benchmarked on **Terminal-Bench** with **DeepSeek V4 Flash 0731** as both the agent and the verifier, running inside **opencode** (N=5 attempts, same-model verifier). Verified end to end with
|
|
21
|
+
Benchmarked on **Terminal-Bench** with **DeepSeek V4 Flash 0731** as both the agent and the verifier, running inside **opencode** (N=5 attempts, same-model verifier). Verified end to end with **OpenCode**, **Claude Code**, and **cline** as the agent (the numbers above are the OpenCode run; a Claude Code number will be added once benchmarked). Works with any agent, not just opencode.
|
|
18
22
|
|
|
19
23
|
| Slice | base@1 (single shot) | ultra (best-of-N + verify) | oracle@5 (ceiling) |
|
|
20
24
|
|---|:---:|:---:|:---:|
|
|
@@ -161,7 +165,7 @@ The 15 tasks were deliberately failure-skewed, so the +9 points is not what you
|
|
|
161
165
|
|
|
162
166
|
- [x] **Standalone CLI** (`npx agent-ultramode`) so the loop runs anywhere, with any agent, no opencode required.
|
|
163
167
|
- [x] **Multiple models in one pass** (repeatable `--agent`): spread attempts across different models, one neutral verifier picks the best.
|
|
164
|
-
- [x] **OpenCode
|
|
168
|
+
- [x] **OpenCode, Claude Code, and cline** verified end to end.
|
|
165
169
|
- [ ] **First-class agent integrations** with tuned defaults, and a benchmark number, for Claude Code, Grok, Pi, and Codex.
|
|
166
170
|
- [ ] Native slash-command or MCP packaging per agent. Contributions welcome.
|
|
167
171
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-ultramode",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.2",
|
|
4
4
|
"description": "Best-of-N for coding agents with a same-model verifier: run your task N times in isolated git worktrees and apply the verified winner. A /ultra command for opencode.",
|
|
5
5
|
"module": "ultra.ts",
|
|
6
6
|
"main": "ultra.ts",
|