ohmyjev 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +49 -17
- package/adapters/codex.ts +42 -17
- package/adapters/omp.ts +22 -10
- package/adapters/pi.ts +32 -16
- package/adapters/shared.ts +35 -1
- package/hooks/policy.ts +137 -1
- package/package.json +56 -1
package/README.md
CHANGED
|
@@ -13,8 +13,8 @@ d88P 88888888 "Y88P" 888 888 888 888 888 "Y88888 888 "Y8888 Y88P
|
|
|
13
13
|
```
|
|
14
14
|
|
|
15
15
|
<p align="center">
|
|
16
|
-
|
|
17
|
-
One
|
|
16
|
+
<a href="https://typesafe.ai">Jev</a>, batteries included, for Claude Code, Codex, pi and omp.<br>
|
|
17
|
+
One install that makes your coding agent faster and safer: no permission prompts for safe commands, long output cut to what matters, the right skill loaded, and nothing irreversible until you say yes.
|
|
18
18
|
</p>
|
|
19
19
|
|
|
20
20
|
<p align="center">
|
|
@@ -40,13 +40,14 @@ d88P 88888888 "Y88P" 888 888 888 888 888 "Y88888 888 "Y8888 Y88P
|
|
|
40
40
|
---
|
|
41
41
|
|
|
42
42
|
Jev is TypeSafe's decision model. You send it some state and a few typed questions, and it sends back probabilities
|
|
43
|
-
in about
|
|
44
|
-
|
|
45
|
-
|
|
43
|
+
in about 200 ms for a fraction of a cent. That's cheap enough to ask about every step your agent takes, so ohmyjev asks
|
|
44
|
+
it everywhere a coding agent wastes your time or puts your work at risk: which command needs your OK, which part of a
|
|
45
|
+
long log matters, which skill a prompt needs, whether the agent is going in circles, and whether a command can be
|
|
46
|
+
undone. It runs as in-process hooks, so there's no extra process and no server to keep up.
|
|
46
47
|
|
|
47
|
-
The bash gate
|
|
48
|
-
|
|
49
|
-
|
|
48
|
+
Most of it makes the agent faster. The bash gate keeps it safe: an agent running unattended in bypass mode can't wipe a
|
|
49
|
+
directory, force push over `main` or upload your keys. When it blocks something, the agent tells you why and asks.
|
|
50
|
+
Answer yes and that exact command runs.
|
|
50
51
|
|
|
51
52
|
## Quick start
|
|
52
53
|
|
|
@@ -65,6 +66,19 @@ If `/omj` ends with `key: env TYPESAFE_API_KEY · typesafe · jev-1.13.0`, you'r
|
|
|
65
66
|
|
|
66
67
|
## What you get
|
|
67
68
|
|
|
69
|
+
### Faster
|
|
70
|
+
|
|
71
|
+
| Feature | What it does |
|
|
72
|
+
|---|---|
|
|
73
|
+
| **Auto-approve** | Where Claude Code or Codex would stop to ask you, a command Jev is sure only reads or can be undone, or an edit inside the repo with no credential in it, runs without the prompt. Plain reads like `ls`, `git status` and `grep` are cleared by code, with no Jev call at all. Off whenever you've set `policies`. |
|
|
74
|
+
| **Trim long output** | A shell output over 12000 characters is split into parts, and one Jev call rates which parts your request needs: the error, the failing test, the value you asked for. The rest becomes one line saying how much was left out. A 20000-character test log came back as 4600 with the one failure in it. |
|
|
75
|
+
| **Skill picker** | Claude Code sends the model a line for every skill you have on every prompt. ohmyjev leaves that list out and asks Jev whether a prompt needs a skill and which one, then names it beside your prompt. A second check makes sure the pick does the job, so a prompt no skill fits gets none. |
|
|
76
|
+
| **Stuck check** | After the third failed tool call in a request, Jev reads the failures. If they repeat one approach and fail the same way, the agent is told to stop, read the error and try something else, or ask you. |
|
|
77
|
+
| **Router** | Asks Jev once per request for a tier, an effort level and a risk score. Then it raises or lowers effort and picks the model for general-purpose subagents. With no key it falls back to Claude Code's built-in classifier, which reports no confidence, so then it only ever routes up. |
|
|
78
|
+
| **Auto-compact** | When the task changes after a finished step and the context is at least 40% full, it compacts the conversation. The summary keeps the new request in full. |
|
|
79
|
+
|
|
80
|
+
### Safer
|
|
81
|
+
|
|
68
82
|
| Feature | What it does |
|
|
69
83
|
|---|---|
|
|
70
84
|
| **Bash gate** | Denies a command when Jev rates it irreversible at 0.6 or more, or when it sends local files or secrets to an outside host (0.7), such as `curl` uploading `~/.ssh`. A command Jev calls irreversible with less confidence is denied too when it also aims to wipe something (0.7). Reversible work always passes, so deleting lines, dropping a co-author trailer or uninstalling a plugin goes through. Plain reads like `ls`, `git status` or `grep` skip Jev entirely: plain code checks them, so they cost nothing and wait for nothing. |
|
|
@@ -74,8 +88,11 @@ If `/omj` ends with `key: env TYPESAFE_API_KEY · typesafe · jev-1.13.0`, you'r
|
|
|
74
88
|
| **Your request** | Every gate also asks Jev whether your latest message asked for exactly this call. If it did (0.8), a gate's Jev deny lets the call through. So "delete the build folder" or "force push it" works, while the same command picked by the agent on its own is still denied. After a block, a plain "yes" works too: the gate shows Jev the call it just blocked next to your reply. Policy and path denies always stand. |
|
|
75
89
|
| **Injection screen** | When output from Bash, WebFetch, an MCP tool, or a Read outside the repo has instructions aimed at the model, it adds a note telling the model to treat that output as data. |
|
|
76
90
|
| **Done-check** | If the agent says it's done but nothing shows it ran a check, it blocks the stop once and tells the agent to verify. A turn that ran no tools is never blocked, since there was nothing to verify. |
|
|
77
|
-
|
|
78
|
-
|
|
91
|
+
|
|
92
|
+
### Tools and commands
|
|
93
|
+
|
|
94
|
+
| Feature | What it does |
|
|
95
|
+
|---|---|
|
|
79
96
|
| **`ask_jev`** | A tool the model can use to ask Jev about repo files or text without loading them into its own context. |
|
|
80
97
|
| **`browse`** | A tool that does a web task in a headless browser, with Jev picking each click and field: a Google Flights search in about 6 seconds. Off until you turn on `browse`. |
|
|
81
98
|
| **`/omj`** | This session's Jev calls, errors, cost, median latency and denies, and where the key comes from. It never prints the key. `/omj config` lists and changes settings. `/omj stats` opens a dashboard of every session in your browser. `/omj off` and `/omj on` switch ohmyjev for the session. `/ohmyjev` is the same command. |
|
|
@@ -86,18 +103,21 @@ If `/omj` ends with `key: env TYPESAFE_API_KEY · typesafe · jev-1.13.0`, you'r
|
|
|
86
103
|
flowchart LR
|
|
87
104
|
A[Claude calls a tool] --> B{ohmyjev asks Jev}
|
|
88
105
|
B -- clearly risky --> C[Denied, with the reason]
|
|
106
|
+
B -- clearly safe --> E
|
|
89
107
|
B -- fine or unsure --> D[Claude Code's normal permission flow]
|
|
90
108
|
B -- Jev down, slow or no key --> D
|
|
91
109
|
D --> E[Tool runs]
|
|
92
|
-
E -->
|
|
110
|
+
E --> T[Long output trimmed<br>to what the request needs]
|
|
111
|
+
T --> F{Output has instructions<br>aimed at the model?}
|
|
93
112
|
F -- yes --> G[Model gets a<br>'treat as data' note]
|
|
94
113
|
F -- no --> H[Output as is]
|
|
95
114
|
```
|
|
96
115
|
|
|
97
|
-
Anything ohmyjev
|
|
98
|
-
being down or slower than `timeoutMs` (1.5 s), a missing key, and any error inside ohmyjev. It never
|
|
99
|
-
|
|
100
|
-
|
|
116
|
+
Anything ohmyjev neither denies nor clears goes on to Claude Code's normal permission flow. That covers middling
|
|
117
|
+
answers, Jev being down or slower than `timeoutMs` (1.5 s), a missing key, and any error inside ohmyjev. It never adds a
|
|
118
|
+
prompt of its own, so it works fine with agents running in bypass mode. It only removes prompts: auto-approve answers
|
|
119
|
+
"allow" only where the engine was about to ask and your settings left the call open. Jev can be wrong though, so keep
|
|
120
|
+
your deny rules in `settings.json`.
|
|
101
121
|
|
|
102
122
|
## Install
|
|
103
123
|
|
|
@@ -160,8 +180,9 @@ codex plugin add ohmyjev@ohmyjev
|
|
|
160
180
|
```
|
|
161
181
|
|
|
162
182
|
Then start Codex and run `/hooks` once: Codex skips a plugin's hooks until you trust them. ohmyjev needs Node on your
|
|
163
|
-
`PATH` and the key in the shell that starts Codex. It gates `Bash`, `apply_patch` edits and MCP calls,
|
|
164
|
-
|
|
183
|
+
`PATH` and the key in the shell that starts Codex. It gates `Bash`, `apply_patch` edits and MCP calls, answers
|
|
184
|
+
Codex's permission prompts for calls Jev cleared, trims long output, screens it, catches repeated failures, and runs the
|
|
185
|
+
done-check. Saying yes after a block works the same way. Codex hooks can't change the model or add
|
|
165
186
|
a tool, so there is no router, no `ask_jev` and no `/omj` command. Put settings in `~/.codex/ohmyjev.json`, using the
|
|
166
187
|
names in [Settings](#settings):
|
|
167
188
|
|
|
@@ -210,6 +231,10 @@ pi has no settings screen for extensions, so put yours under an `ohmyjev` key in
|
|
|
210
231
|
| Exfil gate | WebFetch, MCP tools | MCP tools | `read` of a URL, MCP tools | MCP tools (pi has no fetch tool; `curl` goes through the bash gate) |
|
|
211
232
|
| Injection screen | Bash, WebFetch, MCP, Reads outside the repo | Bash, MCP | `bash`, `github`, URL reads, MCP, reads outside the repo | `bash`, `powershell`, MCP, reads outside the repo |
|
|
212
233
|
| Done-check | ✓ | ✓ | ✓ | ✓ |
|
|
234
|
+
| Auto-approve | ✓ | ✓ (PermissionRequest) | none: omp doesn't ask | none: pi doesn't ask |
|
|
235
|
+
| Trim long output | Bash | Bash (the trimmed text replaces the result) | `bash` | `bash`, `powershell` |
|
|
236
|
+
| Stuck check | ✓ | ✓ | ✓ | ✓ |
|
|
237
|
+
| Skill picker | ✓ | none: hooks can't remove the skill list | none | none |
|
|
213
238
|
| Router effort | ✓ | none: hooks can't set it | ✓ (left alone while you're on `auto`) | ✓ |
|
|
214
239
|
| Router subagent model | general-purpose subagents | none | subagents on the default `task` role | none: pi has no subagents |
|
|
215
240
|
| Router without a key | Claude Code's built-in classifier, up only | none | off: omp has no separate classifier call for extensions | the cheapest model you've set up, up only |
|
|
@@ -380,6 +405,10 @@ one for the session. To keep a change, run `/plugin configure ohmyjev@ohmyjev` i
|
|
|
380
405
|
| `autoCompact` | on | Compacts when the task changes. `compactMinPercent` (40) sets how full the context has to be first. |
|
|
381
406
|
| `askJev` | on | Gives the model the `ask_jev` tool. |
|
|
382
407
|
| `browse` | off | Gives the model the `browse` tool. Needs uv and a Chromium-based browser. |
|
|
408
|
+
| `autoApprove` | on | Skips the permission prompt for a call Jev is sure is safe. `approveConfidence` (0.9) and `approveRiskMax` (0.2) set how sure. |
|
|
409
|
+
| `trimOutput` | on | Cuts long shell output to what the request needs. `trimOver` (12000 characters) and `trimKeep` (0.3) tune it. |
|
|
410
|
+
| `stuckCheck` | on | Tells the agent to change course when its failures repeat. `stuck` (0.7) sets how sure Jev must be. |
|
|
411
|
+
| `skillPick` | on | Claude Code: leaves the skill list out and names the one skill a prompt needs. `skillGate` (0.3) and `skillMin` (0.4) tune it. |
|
|
383
412
|
| `policies` | empty | Your own rules, separated by `;`. |
|
|
384
413
|
| `allowPaths` | `~/.claude;$TMPDIR;/tmp` | Places outside the repo where writes are allowed. ohmyjev ignores any entry with `..` in it. You don't need it for a sibling worktree or another git repo the agent `cd`s into: those count as the repo. |
|
|
385
414
|
| `fastModel`, `balancedModel`, `deepModel` | `claude-haiku-5-5`, `claude-sonnet-5-5`, `claude-opus-5-5` | The model id for each router tier. |
|
|
@@ -410,6 +439,9 @@ a fixed size:
|
|
|
410
439
|
| Injection screen | The tool's output (up to 6000 characters), from Bash, WebFetch, MCP tools and Reads outside the repo |
|
|
411
440
|
| Done-check | The current request (600), up to five earlier requests (200 each), up to 20 of this turn's tool calls with their input (200 each) and outcome, and Claude's last message (1500) |
|
|
412
441
|
| Router | The request (up to 1500 characters) |
|
|
442
|
+
| Trim long output | Your latest message (1500) and the long output, in parts of about 1500 characters, up to 24 parts |
|
|
443
|
+
| Stuck check | Your latest message (600) and the last six failed calls: each one's input and the end of its output (300 each) |
|
|
444
|
+
| Skill picker | Each prompt (1500) with every skill's name and one-line description (300 each); then the picked skill's description (600) |
|
|
413
445
|
| `ask_jev` | The model's question, any text it passes (up to 20000 characters) and the repo files it names (8000 each, 80000 in total). It never sends files outside the repo |
|
|
414
446
|
| `browse` | The goal and URL to the exfil gate; then, each step, the page's controls and visible text (6000) and the goal to Jev, and the page's text to the injection screen. The websites see the browser like any visitor |
|
|
415
447
|
|
package/adapters/codex.ts
CHANGED
|
@@ -7,15 +7,17 @@
|
|
|
7
7
|
*/
|
|
8
8
|
import { mkdir, readFile, writeFile } from 'node:fs/promises'
|
|
9
9
|
import {
|
|
10
|
-
BASH_Q, EXFIL_Q, SCREEN_Q, STOP_Q, WRITE_Q, applyPatchPaths, callSummary, clip, denyText, gateBash, gateExfil, gateWrite,
|
|
10
|
+
BASH_Q, EXFIL_Q, SCREEN_Q, STOP_Q, WRITE_Q, applyPatchPaths, approveBash, approveWrite, callSummary, clip, clipTail, denyText, gateBash, gateExfil, gateWrite,
|
|
11
11
|
judgeStop, pathDenyText, plainRead, readConfig, sanitizeSid, screen, withPolicies, withPolicyQ, type Config,
|
|
12
12
|
} from '../hooks/policy.ts'
|
|
13
|
-
import { Session, keyOf, pathAllowed, repoRoot, type Env, type Fetch } from './shared.ts'
|
|
13
|
+
import { Session, keyOf, newStuck, pathAllowed, repoRoot, stuckNote, trimOutput, type Env, type Fetch, type Stuck } from './shared.ts'
|
|
14
14
|
|
|
15
15
|
const CLIP = 16000
|
|
16
|
-
type Tool = { tool: string; input: string; outcome: 'ok' | 'failed' }
|
|
17
|
-
type State = { request: string; previous: string[]; count: number; blockedBefore: string; lastBlocked: string; tools: Tool[]; pushedBackAt: number }
|
|
18
|
-
const EMPTY: State = { request: '', previous: [], count: 0, blockedBefore: '', lastBlocked: '', tools: [], pushedBackAt: -1 }
|
|
16
|
+
type Tool = { tool: string; input: string; outcome: 'ok' | 'failed'; output: string }
|
|
17
|
+
type State = { request: string; previous: string[]; count: number; blockedBefore: string; lastBlocked: string; tools: Tool[]; pushedBackAt: number; approved: string[]; stuck: Stuck }
|
|
18
|
+
const EMPTY: State = { request: '', previous: [], count: 0, blockedBefore: '', lastBlocked: '', tools: [], pushedBackAt: -1, approved: [], stuck: { failures: [], askedAt: 0 } }
|
|
19
|
+
/** A call Jev cleared, as PermissionRequest will see it again. */
|
|
20
|
+
const approvalKey = (tool: string, input: Input): string => `${tool}\u0000${str(input, 'command')}`
|
|
19
21
|
type Input = Record<string, unknown>
|
|
20
22
|
const str = (o: Input, k: string): string => (typeof o[k] === 'string' ? (o[k] as string) : '')
|
|
21
23
|
const obj = (v: unknown): Input => (v && typeof v === 'object' && !Array.isArray(v) ? (v as Input) : {})
|
|
@@ -63,7 +65,7 @@ export async function handle(input: Input, env: Env, fetchImpl?: Fetch): Promise
|
|
|
63
65
|
const prompt = str(input, 'prompt')
|
|
64
66
|
if (!prompt.trim() || /^\/\S+\s*$/.test(prompt.trim())) return undefined
|
|
65
67
|
if (st.request) st.previous = [...st.previous, clip(st.request, 200)].slice(-5)
|
|
66
|
-
Object.assign(st, { request: prompt, count: st.count + 1, blockedBefore: st.lastBlocked, lastBlocked: '', tools: [] })
|
|
68
|
+
Object.assign(st, { request: prompt, count: st.count + 1, blockedBefore: st.lastBlocked, lastBlocked: '', tools: [], stuck: newStuck() })
|
|
67
69
|
await save(env, sid, st)
|
|
68
70
|
return undefined
|
|
69
71
|
}
|
|
@@ -73,7 +75,14 @@ export async function handle(input: Input, env: Env, fetchImpl?: Fetch): Promise
|
|
|
73
75
|
const toolInput = obj(input.tool_input)
|
|
74
76
|
|
|
75
77
|
if (event === 'PreToolUse') {
|
|
76
|
-
const
|
|
78
|
+
const approve = async () => {
|
|
79
|
+
if (!c.autoApprove) return undefined
|
|
80
|
+
st.approved = [...st.approved, approvalKey(tool, toolInput)].slice(-20)
|
|
81
|
+
await save(env, sid, st)
|
|
82
|
+
return undefined
|
|
83
|
+
}
|
|
84
|
+
const blocked = async (j: { verdict: unknown; reason: string }, safe = false) => {
|
|
85
|
+
if (j.verdict === null && safe) return approve()
|
|
77
86
|
if (j.verdict !== 'deny') return undefined
|
|
78
87
|
st.lastBlocked = callSummary(tool, toolInput)
|
|
79
88
|
await save(env, sid, st)
|
|
@@ -81,13 +90,13 @@ export async function handle(input: Input, env: Env, fetchImpl?: Fetch): Promise
|
|
|
81
90
|
}
|
|
82
91
|
if (tool === 'Bash' && c.bashGate) {
|
|
83
92
|
const command = str(toolInput, 'command')
|
|
84
|
-
if (plainRead(command, c)) return
|
|
93
|
+
if (plainRead(command, c)) return approve()
|
|
85
94
|
const state = withPolicies({ command: clip(command, CLIP), cwd, ...(command.length > CLIP ? { truncated: true } : {}) }, c, st.request, st.blockedBefore)
|
|
86
95
|
const d = await x.decide('tool.call', tool, state, withPolicyQ(BASH_Q, c, st.request))
|
|
87
96
|
if (!d) return undefined
|
|
88
97
|
const j = gateBash(d.answers, c)
|
|
89
98
|
x.record(d, j.verdict, j.reason)
|
|
90
|
-
return blocked(j)
|
|
99
|
+
return blocked(j, approveBash(d.answers, c))
|
|
91
100
|
}
|
|
92
101
|
if (tool === 'apply_patch' && c.writeGate) {
|
|
93
102
|
const patch = str(toolInput, 'command')
|
|
@@ -102,7 +111,7 @@ export async function handle(input: Input, env: Env, fetchImpl?: Fetch): Promise
|
|
|
102
111
|
if (!d) return undefined
|
|
103
112
|
const j = gateWrite(d.answers, c)
|
|
104
113
|
x.record(d, j.verdict, j.reason)
|
|
105
|
-
return blocked(j)
|
|
114
|
+
return blocked(j, approveWrite(d.answers, c))
|
|
106
115
|
}
|
|
107
116
|
if (tool.startsWith('mcp__') && c.exfilGate) {
|
|
108
117
|
const state = withPolicies({ tool, input: clip(JSON.stringify(toolInput), 4000) }, c, st.request, st.blockedBefore)
|
|
@@ -115,19 +124,35 @@ export async function handle(input: Input, env: Env, fetchImpl?: Fetch): Promise
|
|
|
115
124
|
return undefined
|
|
116
125
|
}
|
|
117
126
|
|
|
127
|
+
if (event === 'PermissionRequest') {
|
|
128
|
+
const key = approvalKey(tool, toolInput)
|
|
129
|
+
if (!c.autoApprove || !st.approved.includes(key)) return undefined
|
|
130
|
+
st.approved = st.approved.filter(k => k !== key)
|
|
131
|
+
await save(env, sid, st)
|
|
132
|
+
return { hookSpecificOutput: { hookEventName: 'PermissionRequest', decision: { behavior: 'allow' } } }
|
|
133
|
+
}
|
|
134
|
+
|
|
118
135
|
if (event === 'PostToolUse') {
|
|
119
136
|
const response = input.tool_response
|
|
120
137
|
const code = obj(response).exit_code ?? obj(response).exitCode
|
|
121
|
-
|
|
138
|
+
const r = obj(response)
|
|
139
|
+
const shown = typeof response === 'string' ? response : typeof r.output === 'string' ? r.output : typeof r.stdout === 'string' ? r.stdout : JSON.stringify(response ?? '')
|
|
140
|
+
st.tools = [...st.tools, { tool, input: clip(JSON.stringify(toolInput), 200), outcome: typeof code === 'number' && code !== 0 ? 'failed' : 'ok', output: clipTail(shown, 300) } as Tool].slice(-20)
|
|
141
|
+
const failed = typeof code === 'number' && code !== 0
|
|
142
|
+
const stuck = failed ? await stuckNote(x, st.stuck, st.request, tool, toolInput, shown) : null
|
|
122
143
|
await save(env, sid, st)
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
144
|
+
const notes = stuck ? [stuck] : []
|
|
145
|
+
// a cut output replaces the result: Codex hooks can't edit it, but a PostToolUse block's reason stands in for it
|
|
146
|
+
const cut = tool === 'Bash' ? await trimOutput(x, st.request, shown) : null
|
|
147
|
+
if (cut) return { decision: 'block', reason: [cut, ...notes].join('\n') }
|
|
148
|
+
const context = () => (notes.length ? { hookSpecificOutput: { hookEventName: 'PostToolUse', additionalContext: notes.join('\n') } } : undefined)
|
|
149
|
+
if (!c.injectionScreen || !(tool === 'Bash' || tool.startsWith('mcp__')) || !shown.trim()) return context()
|
|
150
|
+
const d = await x.decide('tool.result', tool, { tool, content: clip(shown, 6000) }, SCREEN_Q)
|
|
151
|
+
if (!d) return context()
|
|
128
152
|
const s = screen(d.answers, c)
|
|
129
153
|
x.record(d, s.flagged ? 'flag' : null, s.reason)
|
|
130
|
-
|
|
154
|
+
if (s.flagged) notes.push(s.note)
|
|
155
|
+
return context()
|
|
131
156
|
}
|
|
132
157
|
|
|
133
158
|
if (event === 'Stop') {
|
package/adapters/omp.ts
CHANGED
|
@@ -4,13 +4,13 @@
|
|
|
4
4
|
import { readFile } from 'node:fs/promises'
|
|
5
5
|
import {
|
|
6
6
|
BASH_Q, DEFAULTS, EXFIL_Q, ROUTE_Q, SCREEN_Q, STOP_Q, SWITCHED_Q, WRITE_Q,
|
|
7
|
-
clip, decideRoute, denyText, gateBash, gateExfil, gateWrite, jevRouteLine, judgeStop, pathDenyText, callSummary, plainRead, configCommand, applyPatchPaths,
|
|
7
|
+
clip, decideRoute, denyText, gateBash, gateExfil, gateWrite, jevRouteLine, judgeStop, pathDenyText, callSummary, clipTail, plainRead, configCommand, applyPatchPaths,
|
|
8
8
|
BROWSE_DESCRIPTION, OMJ_HELP, OMJ_OFF, OMJ_ON, toggleArg,
|
|
9
9
|
keepInstructions, modelOf, readConfig, routeStep, screen, stepLine, withPolicies, withPolicyQ,
|
|
10
10
|
type Config, type Route,
|
|
11
11
|
} from '../hooks/policy.ts'
|
|
12
12
|
import {
|
|
13
|
-
ASK_DESCRIPTION, Session, answerAsk, runBrowse, keyOf, pathAllowed, repoRoot,
|
|
13
|
+
ASK_DESCRIPTION, Session, answerAsk, newStuck, runBrowse, stuckNote, trimOutput, keyOf, pathAllowed, repoRoot,
|
|
14
14
|
type Env, type Fetch,
|
|
15
15
|
} from './shared.ts'
|
|
16
16
|
|
|
@@ -193,16 +193,18 @@ function sameModel(left: HostModel | undefined, right: HostModel | undefined): b
|
|
|
193
193
|
)
|
|
194
194
|
}
|
|
195
195
|
|
|
196
|
-
function toolState(messages: unknown[], after: number): Array<{ tool: string; input: string; outcome: 'ok' | 'failed' | 'pending' }> {
|
|
196
|
+
function toolState(messages: unknown[], after: number): Array<{ tool: string; input: string; outcome: 'ok' | 'failed' | 'pending'; output: string }> {
|
|
197
197
|
const tail = messages.slice(after)
|
|
198
198
|
const outcomes = new Map<string, 'ok' | 'failed'>()
|
|
199
|
+
const outputs = new Map<string, string>()
|
|
199
200
|
for (const message of tail) {
|
|
200
201
|
const m = object(message)
|
|
201
202
|
if (m.role !== 'toolResult') continue
|
|
202
203
|
const id = typeof m.toolCallId === 'string' ? m.toolCallId : ''
|
|
203
204
|
if (id) outcomes.set(id, m.isError === true ? 'failed' : 'ok')
|
|
205
|
+
if (id) outputs.set(id, clipTail(text(m.content), 300))
|
|
204
206
|
}
|
|
205
|
-
const calls: Array<{ tool: string; input: string; outcome: 'ok' | 'failed' | 'pending' }> = []
|
|
207
|
+
const calls: Array<{ tool: string; input: string; outcome: 'ok' | 'failed' | 'pending'; output: string }> = []
|
|
206
208
|
for (const message of tail) {
|
|
207
209
|
const m = object(message)
|
|
208
210
|
if (m.role !== 'assistant' || !Array.isArray(m.content)) continue
|
|
@@ -214,6 +216,7 @@ function toolState(messages: unknown[], after: number): Array<{ tool: string; in
|
|
|
214
216
|
tool: typeof p.name === 'string' ? p.name : '',
|
|
215
217
|
input: clip(safeJson(p.arguments), 200),
|
|
216
218
|
outcome: outcomes.get(id) ?? 'pending',
|
|
219
|
+
output: outputs.get(id) ?? '',
|
|
217
220
|
})
|
|
218
221
|
}
|
|
219
222
|
}
|
|
@@ -247,6 +250,7 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
247
250
|
let liveRequest = ''
|
|
248
251
|
let lastBlocked = '' // the call a gate denied this turn
|
|
249
252
|
let blockedBefore = '' // what was denied in the turn before this request: a "yes" confirms it
|
|
253
|
+
let stuck = newStuck()
|
|
250
254
|
let userRequest = '' // the user's latest message: the gates ask whether it asked for the call
|
|
251
255
|
let compactPending = false
|
|
252
256
|
let classifiedPrompt: string | undefined
|
|
@@ -573,23 +577,30 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
573
577
|
pi.on('tool_result', async (rawEvent, ctx) => {
|
|
574
578
|
try {
|
|
575
579
|
touch(ctx)
|
|
576
|
-
if (!session || !c.enabled
|
|
580
|
+
if (!session || !c.enabled) return
|
|
577
581
|
const event = rawEvent as ToolResult
|
|
578
582
|
const tool = event.toolName
|
|
579
583
|
const input = object(event.input)
|
|
584
|
+
let resultText = event.content.map(part => part.type === 'text' && typeof part.text === 'string' ? part.text : '').filter(Boolean).join('\n')
|
|
585
|
+
const cut = tool === 'bash' ? await trimOutput(session, userRequest, resultText) : null
|
|
586
|
+
if (cut) resultText = cut
|
|
587
|
+
const base = cut ? [...event.content.filter(part => part.type !== 'text'), { type: 'text', text: cut }] : event.content
|
|
588
|
+
const stuckText = (event as { isError?: boolean }).isError ? await stuckNote(session, stuck, userRequest, tool, input, resultText) : null
|
|
589
|
+
const content = stuckText ? [...base, { type: 'text', text: stuckText }] : base
|
|
590
|
+
const changed = stuckText || cut
|
|
591
|
+
if (!c.injectionScreen) return changed ? { content } : undefined
|
|
580
592
|
let shouldScreen = tool === 'bash' || tool === 'github' || exfilTool(tool, input)
|
|
581
593
|
if (tool === 'read' && !urlRead(tool, input) && c.screenReads) {
|
|
582
594
|
const path = string(input, 'path')
|
|
583
595
|
shouldScreen = !sessionLocalWrite(path) && !(await pathAllowed(path, ctx.cwd, root, c, env))
|
|
584
596
|
}
|
|
585
|
-
if (!shouldScreen) return
|
|
586
|
-
const resultText = event.content.map(part => part.type === 'text' && typeof part.text === 'string' ? part.text : '').filter(Boolean).join('\n')
|
|
587
|
-
if (!resultText.trim()) return
|
|
597
|
+
if (!shouldScreen || !resultText.trim()) return changed ? { content } : undefined
|
|
588
598
|
const d = await session.decide('tool_result', tool, { tool, content: clip(resultText, 6000) }, SCREEN_Q)
|
|
589
|
-
if (!d) return
|
|
599
|
+
if (!d) return changed ? { content } : undefined
|
|
590
600
|
const screened = screen(d.answers, c)
|
|
591
601
|
session.record(d, screened.flagged ? 'flag' : null, screened.reason)
|
|
592
|
-
if (screened.flagged) return { content: [...
|
|
602
|
+
if (screened.flagged) return { content: [...content, { type: 'text', text: screened.note }] }
|
|
603
|
+
return changed ? { content } : undefined
|
|
593
604
|
} catch {
|
|
594
605
|
return
|
|
595
606
|
}
|
|
@@ -650,6 +661,7 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
650
661
|
userEffort = userThinking(ctx)
|
|
651
662
|
const prompt = typeof rawEvent.prompt === 'string' ? rawEvent.prompt : ''
|
|
652
663
|
if (prompt.trim() && !/^\/\S+\s*$/.test(prompt.trim())) {
|
|
664
|
+
stuck = newStuck()
|
|
653
665
|
userRequest = prompt
|
|
654
666
|
blockedBefore = lastBlocked
|
|
655
667
|
lastBlocked = ''
|
package/adapters/pi.ts
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
import { readFile } from 'node:fs/promises'
|
|
2
2
|
import {
|
|
3
3
|
BASH_Q, EXFIL_Q, ROUTE_Q, SCREEN_Q, STOP_Q, SWITCHED_Q, WRITE_Q,
|
|
4
|
-
TIER_LABELS, builtinRoute, clip, decideRoute, denyText, gateBash, gateExfil, gateWrite, jevRouteLine, judgeStop, pathDenyText, callSummary, plainRead, configCommand,
|
|
4
|
+
TIER_LABELS, builtinRoute, clip, decideRoute, denyText, gateBash, gateExfil, gateWrite, jevRouteLine, judgeStop, pathDenyText, callSummary, clipTail, plainRead, configCommand,
|
|
5
5
|
BROWSE_DESCRIPTION, BROWSE_SCHEMA, OMJ_HELP, OMJ_OFF, OMJ_ON, toggleArg,
|
|
6
6
|
keepInstructions, readConfig, routeStep, screen, stepLine, withPolicies, withPolicyQ,
|
|
7
7
|
type Config, type Route,
|
|
8
8
|
} from '../hooks/policy.ts'
|
|
9
9
|
import {
|
|
10
|
-
ASK_DESCRIPTION, ASK_SCHEMA, Session, answerAsk, runBrowse, keyOf, pathAllowed, repoRoot,
|
|
10
|
+
ASK_DESCRIPTION, ASK_SCHEMA, Session, answerAsk, newStuck, runBrowse, stuckNote, trimOutput, keyOf, pathAllowed, repoRoot,
|
|
11
11
|
type Env, type Fetch,
|
|
12
12
|
} from './shared.ts'
|
|
13
13
|
|
|
@@ -173,20 +173,23 @@ function messagesForStop(messages: HostMessage[]) {
|
|
|
173
173
|
}
|
|
174
174
|
}
|
|
175
175
|
|
|
176
|
-
const tools: Array<{ id: string; tool: string; input: string; outcome: 'ok' | 'failed' | 'pending' }> = []
|
|
176
|
+
const tools: Array<{ id: string; tool: string; input: string; outcome: 'ok' | 'failed' | 'pending'; output: string }> = []
|
|
177
177
|
const byId = new Map<string, (typeof tools)[number]>()
|
|
178
178
|
for (const message of messages.slice(lastRequest + 1)) {
|
|
179
179
|
if (message.role === 'assistant' && Array.isArray(message.content)) {
|
|
180
180
|
for (const part of message.content) {
|
|
181
181
|
const block = entry(part)
|
|
182
182
|
if (block.type !== 'toolCall' || typeof block.id !== 'string' || typeof block.name !== 'string') continue
|
|
183
|
-
const call = { id: block.id, tool: block.name, input: clip(json(block.arguments), 200), outcome: 'pending' as
|
|
183
|
+
const call = { id: block.id, tool: block.name, input: clip(json(block.arguments), 200), outcome: 'pending' as 'ok' | 'failed' | 'pending', output: '' }
|
|
184
184
|
tools.push(call)
|
|
185
185
|
byId.set(call.id, call)
|
|
186
186
|
}
|
|
187
187
|
} else if (message.role === 'toolResult' && typeof message.toolCallId === 'string') {
|
|
188
188
|
const call = byId.get(message.toolCallId)
|
|
189
|
-
if (call)
|
|
189
|
+
if (call) {
|
|
190
|
+
call.outcome = message.isError ? 'failed' : 'ok'
|
|
191
|
+
call.output = clipTail(textOf(message.content), 300)
|
|
192
|
+
}
|
|
190
193
|
}
|
|
191
194
|
}
|
|
192
195
|
|
|
@@ -196,7 +199,7 @@ function messagesForStop(messages: HostMessage[]) {
|
|
|
196
199
|
state: {
|
|
197
200
|
current_request: clip(textOf(requests.at(-1)?.content), 600),
|
|
198
201
|
previous_requests: requests.slice(-6, -1).map(message => clip(textOf(message.content), 200)),
|
|
199
|
-
tools_this_turn: tools.slice(-20).map(({ tool, input, outcome }) => ({ tool, input, outcome })),
|
|
202
|
+
tools_this_turn: tools.slice(-20).map(({ tool, input, outcome, output }) => ({ tool, input, outcome, output })),
|
|
200
203
|
last_assistant_message: clip(textOf(lastAssistant?.content), 1500),
|
|
201
204
|
},
|
|
202
205
|
}
|
|
@@ -259,6 +262,7 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
259
262
|
let liveRequest = ''
|
|
260
263
|
let lastBlocked = '' // the call a gate denied this turn
|
|
261
264
|
let blockedBefore = '' // what was denied in the turn before this request: a "yes" confirms it
|
|
265
|
+
let stuck = newStuck()
|
|
262
266
|
let userRequest = '' // the user's latest message: the gates ask whether it asked for the call
|
|
263
267
|
let compactPending = false
|
|
264
268
|
let userThinking: ThinkingLevel = 'medium'
|
|
@@ -439,6 +443,7 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
439
443
|
pi.on('before_agent_start', guarded<BeforeAgentStartEvent, void>(async (event, ctx) => {
|
|
440
444
|
latestCtx = ctx
|
|
441
445
|
if (event.prompt.trim() && !/^\/\S+\s*$/.test(event.prompt.trim())) {
|
|
446
|
+
stuck = newStuck()
|
|
442
447
|
userRequest = event.prompt
|
|
443
448
|
blockedBefore = lastBlocked
|
|
444
449
|
lastBlocked = ''
|
|
@@ -519,27 +524,38 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
519
524
|
}
|
|
520
525
|
}))
|
|
521
526
|
|
|
522
|
-
pi.on('tool_result', guarded<ToolResultEvent, { content: Content[]; structuredContent?: unknown }>(async (
|
|
527
|
+
pi.on('tool_result', guarded<ToolResultEvent, { content: Content[]; structuredContent?: unknown }>(async (evt, ctx) => {
|
|
528
|
+
let event = evt
|
|
523
529
|
latestCtx = ctx
|
|
524
530
|
if (!c.enabled) return
|
|
525
531
|
const tool = event.toolName
|
|
526
532
|
const shell = tool === 'bash' || tool === 'powershell'
|
|
527
|
-
|
|
533
|
+
let text = event.content.filter((part): part is TextContent => part.type === 'text').map(part => part.text).join('\n')
|
|
534
|
+
const cut = shell ? await trimOutput(await ensureSession(ctx), userRequest, text) : null
|
|
535
|
+
if (cut) {
|
|
536
|
+
text = cut
|
|
537
|
+
event = { ...event, content: [...event.content.filter(part => part.type !== 'text'), { type: 'text', text: cut }] }
|
|
538
|
+
}
|
|
539
|
+
const stuckText = event.isError ? await stuckNote(await ensureSession(ctx), stuck, userRequest, tool, event.input, text) : null
|
|
540
|
+
const withStuck = stuckText
|
|
541
|
+
? { content: [...event.content, { type: 'text' as const, text: stuckText }], ...(Object.hasOwn(event, 'structuredContent') ? { structuredContent: event.structuredContent } : {}) }
|
|
542
|
+
: undefined
|
|
543
|
+
const changed = withStuck ?? (cut ? { content: event.content, ...(Object.hasOwn(event, 'structuredContent') ? { structuredContent: event.structuredContent } : {}) } : undefined)
|
|
544
|
+
if (!c.injectionScreen || (event.isError && !shell)) return changed
|
|
528
545
|
const candidate = shell || tool.startsWith('mcp__') || tool === 'read'
|
|
529
|
-
if (!candidate) return
|
|
546
|
+
if (!candidate) return changed
|
|
530
547
|
if (tool === 'read') {
|
|
531
|
-
if (!c.screenReads || await pathAllowed(arg(event.input, 'path'), ctx.cwd, root, c, env)) return
|
|
548
|
+
if (!c.screenReads || await pathAllowed(arg(event.input, 'path'), ctx.cwd, root, c, env)) return changed
|
|
532
549
|
}
|
|
533
|
-
|
|
534
|
-
if (!content.trim()) return
|
|
550
|
+
if (!text.trim()) return changed
|
|
535
551
|
const x = await ensureSession(ctx)
|
|
536
|
-
const d = await x.decide('tool.result', tool, { tool, content: clip(
|
|
537
|
-
if (!d) return
|
|
552
|
+
const d = await x.decide('tool.result', tool, { tool, content: clip(text, 6000) }, SCREEN_Q)
|
|
553
|
+
if (!d) return changed
|
|
538
554
|
const judged = screen(d.answers, c)
|
|
539
555
|
x.record(d, judged.flagged ? 'flag' : null, judged.reason)
|
|
540
|
-
if (!judged.flagged) return
|
|
556
|
+
if (!judged.flagged) return changed
|
|
541
557
|
return {
|
|
542
|
-
content: [...event.content, { type: 'text', text: judged.note }],
|
|
558
|
+
content: [...(changed?.content ?? event.content), { type: 'text', text: judged.note }],
|
|
543
559
|
...(Object.hasOwn(event, 'structuredContent') ? { structuredContent: event.structuredContent } : {}),
|
|
544
560
|
}
|
|
545
561
|
}))
|
package/adapters/shared.ts
CHANGED
|
@@ -9,7 +9,7 @@ import { dashboardHtml } from '../hooks/dashboard.ts'
|
|
|
9
9
|
import { ENDPOINTS, JevError, parseReply, pickKey, type Key } from '../hooks/jev.ts'
|
|
10
10
|
import {
|
|
11
11
|
BROWSE_TIMEOUT_MS, DASHBOARD, EMPTY_SESSION, EXFIL_Q, SCREEN_Q, browseCommand, browseReply, denyText, gateExfil, screen, withPolicies, withPolicyQ, absolute, buildQuestion, clip, expandRoot, hostPath, isUnder, normalize, parseLog, parseWorktrees, rawAbsolute, sanitizeSid,
|
|
12
|
-
splitList, statusText, summarize, type Answers, type Config, type LogEntry, type Questions, type SessionState, type Verdict,
|
|
12
|
+
STUCK_Q, trimChunks, trimRequest, trimmed, failureOf, judgeStuck, stuckDue, stuckState, splitList, statusText, summarize, type Failure, type Answers, type Config, type LogEntry, type Questions, type SessionState, type Verdict,
|
|
13
13
|
} from '../hooks/policy.ts'
|
|
14
14
|
|
|
15
15
|
export type Env = Readonly<Record<string, string | undefined>>
|
|
@@ -333,3 +333,37 @@ export async function runBrowse(
|
|
|
333
333
|
x.record(d, s.flagged ? 'flag' : null, s.reason)
|
|
334
334
|
return { text: s.flagged ? `${text}\n${s.note}` : text }
|
|
335
335
|
}
|
|
336
|
+
|
|
337
|
+
// --- stuck check: the same approach failing over and over ---
|
|
338
|
+
|
|
339
|
+
/** One request's failed calls; reset it when the user sends a new request. */
|
|
340
|
+
export type Stuck = { failures: Failure[]; askedAt: number }
|
|
341
|
+
export const newStuck = (): Stuck => ({ failures: [], askedAt: 0 })
|
|
342
|
+
|
|
343
|
+
/** Records a failed call and, when due, asks Jev whether the attempts repeat: the note for the model, or null. */
|
|
344
|
+
export async function stuckNote(x: Session, st: Stuck, request: string, tool: string, input: unknown, output: unknown): Promise<string | null> {
|
|
345
|
+
if (!x.c.enabled || !x.c.stuckCheck) return null
|
|
346
|
+
st.failures.push(failureOf(tool, input, output))
|
|
347
|
+
if (!stuckDue(st.failures.length, st.askedAt)) return null
|
|
348
|
+
st.askedAt = st.failures.length
|
|
349
|
+
const d = await x.decide('tool.result', 'stuck', stuckState(request, st.failures), STUCK_Q)
|
|
350
|
+
if (!d) return null
|
|
351
|
+
const note = judgeStuck(d.answers, x.c, st.failures.length)
|
|
352
|
+
x.record(d, note ? 'flag' : null, note ? 'stuck' : 'not stuck')
|
|
353
|
+
return note
|
|
354
|
+
}
|
|
355
|
+
|
|
356
|
+
// --- output trim ---
|
|
357
|
+
|
|
358
|
+
/** A long shell output cut to the parts the request needs; null to keep it whole. */
|
|
359
|
+
export async function trimOutput(x: Session, request: string, output: string): Promise<string | null> {
|
|
360
|
+
if (!x.c.enabled) return null
|
|
361
|
+
const chunks = trimChunks(output, x.c)
|
|
362
|
+
if (!chunks.length) return null
|
|
363
|
+
const { state, questions } = trimRequest(request, chunks)
|
|
364
|
+
const d = await x.decide('tool.result', 'trim', state, questions)
|
|
365
|
+
if (!d) return null
|
|
366
|
+
const out = trimmed(chunks, d.answers, x.c)
|
|
367
|
+
x.record(d, null, out ? `trimmed ${output.length} to ${out.length} characters` : 'kept whole')
|
|
368
|
+
return out
|
|
369
|
+
}
|
package/hooks/policy.ts
CHANGED
|
@@ -25,6 +25,17 @@ export const DEFAULTS = {
|
|
|
25
25
|
doneAsksUserMax: 0.5,
|
|
26
26
|
exfil: 0.7,
|
|
27
27
|
requested: 0.8,
|
|
28
|
+
autoApprove: true,
|
|
29
|
+
stuckCheck: true,
|
|
30
|
+
skillPick: true,
|
|
31
|
+
skillGate: 0.3,
|
|
32
|
+
skillMin: 0.4,
|
|
33
|
+
trimOutput: true,
|
|
34
|
+
trimOver: 12000,
|
|
35
|
+
trimKeep: 0.3,
|
|
36
|
+
stuck: 0.7,
|
|
37
|
+
approveConfidence: 0.9,
|
|
38
|
+
approveRiskMax: 0.2,
|
|
28
39
|
askJev: true,
|
|
29
40
|
browse: false,
|
|
30
41
|
autoCompact: true,
|
|
@@ -185,12 +196,33 @@ export const STOP_Q: Questions = {
|
|
|
185
196
|
claimed_done: noul('Does `last_assistant_message` say the task in `current_request` is complete?'),
|
|
186
197
|
verified: noul(
|
|
187
198
|
'Is there evidence the work was checked?',
|
|
188
|
-
'Tests, a build or the program ran and did not fail, output was quoted or inspected; `tools_this_turn` shows such a check with outcome ok',
|
|
199
|
+
'Tests, a build or the program ran and did not fail, output was quoted or inspected; `tools_this_turn` shows such a check with outcome ok and an `output` that confirms it',
|
|
189
200
|
'Only claims success, edited without running or inspecting anything, or every check is pending, backgrounded or failed',
|
|
190
201
|
),
|
|
191
202
|
asks_user: noul('Does `last_assistant_message` end by asking the user a question or reporting a blocker it cannot resolve?'),
|
|
192
203
|
at_boundary: noul('Did the last turn finish a unit of work rather than stop mid-step?'),
|
|
193
204
|
}
|
|
205
|
+
export const STUCK_Q: Questions = {
|
|
206
|
+
stuck: noul(
|
|
207
|
+
'Do the attempts in `failures` repeat one approach and fail the same way, so trying it again will not help?',
|
|
208
|
+
'The same command, edit or fix, re-run with small changes, and the same error or the same kind of failure each time',
|
|
209
|
+
'Different approaches, different errors, or each attempt gets further than the last',
|
|
210
|
+
),
|
|
211
|
+
}
|
|
212
|
+
/** A failed tool call, as the stuck check reads it. */
|
|
213
|
+
export type Failure = { tool: string; input: string; output: string }
|
|
214
|
+
export const failureOf = (tool: string, input: unknown, output: unknown): Failure =>
|
|
215
|
+
({ tool, input: clip(typeof input === 'string' ? input : JSON.stringify(input), 300), output: clipTail(output, 300) })
|
|
216
|
+
/** Ask after the third failure in a request, then after every third more: rarely, and only while it keeps failing. */
|
|
217
|
+
export const stuckDue = (failures: number, askedAt: number): boolean => failures >= 3 && failures - askedAt >= 3
|
|
218
|
+
export const stuckState = (request: string, failures: Failure[]) => ({ current_request: clip(request, 600), failures: failures.slice(-6) })
|
|
219
|
+
export function judgeStuck(a: Answers, c: Config, n: number): string | null {
|
|
220
|
+
const p = nv(a, 'stuck')
|
|
221
|
+
return p >= c.stuck
|
|
222
|
+
? `[ohmyjev] The last ${n} failed attempts repeat one approach and fail the same way (${f2(p)}). Stop repeating it: read the error, try a different approach, or ask the user.`
|
|
223
|
+
: null
|
|
224
|
+
}
|
|
225
|
+
|
|
194
226
|
export const SWITCHED_Q: Questions = {
|
|
195
227
|
switched_gears: noul('Is `current_request` a different task from `previous_requests`, so the earlier work is no longer needed?'),
|
|
196
228
|
}
|
|
@@ -204,6 +236,11 @@ export const clip = (text: unknown, n: number): string => {
|
|
|
204
236
|
const s = typeof text === 'string' ? text : ''
|
|
205
237
|
return s.length <= n ? s : s.slice(0, n - 1) + '…'
|
|
206
238
|
}
|
|
239
|
+
/** The end of a tool's output, where a test run's verdict and an error's last line are. */
|
|
240
|
+
export const clipTail = (text: unknown, n: number): string => {
|
|
241
|
+
const s = typeof text === 'string' ? text.trim() : ''
|
|
242
|
+
return s.length <= n ? s : '…' + s.slice(-(n - 1))
|
|
243
|
+
}
|
|
207
244
|
export const sanitizeSid = (sid: string): string => sid.replace(/[^\w-]/g, '') || 'unknown'
|
|
208
245
|
export const denyText = (reason: string): string => `ohmyjev blocked this: ${reason}. ${BLOCK_NOTICE}`
|
|
209
246
|
/** A path deny is a setting, not a hazard: the model must not route around it, but it should tell the user how to widen it. */
|
|
@@ -288,6 +325,21 @@ export function gateBash(a: Answers, c: Config): Judged {
|
|
|
288
325
|
return { verdict: null, reason: `${effect.choice} (${f2(effect.confidence)}), destructive ${f2(destructive)}, exfil ${f2(out)}` }
|
|
289
326
|
}
|
|
290
327
|
|
|
328
|
+
/**
|
|
329
|
+
* Whether the host may skip asking the person: Jev is sure the command only reads or makes a change git or a reinstall
|
|
330
|
+
* undoes, and sees no wipe and no data leaving. Policies make every call worth a look, so nothing is approved then.
|
|
331
|
+
*/
|
|
332
|
+
export function approveBash(a: Answers, c: Config): boolean {
|
|
333
|
+
const effect = ch(a, 'effect')
|
|
334
|
+
return c.autoApprove && !splitList(c.policies).length && effect.choice !== 'irreversible' && effect.confidence >= c.approveConfidence &&
|
|
335
|
+
nv(a, 'destructive_intent') < c.approveRiskMax && nv(a, 'exfiltrates') < c.approveRiskMax
|
|
336
|
+
}
|
|
337
|
+
/** The same for an edit inside the repo: no credential in it, not a secrets file. */
|
|
338
|
+
export function approveWrite(a: Answers, c: Config): boolean {
|
|
339
|
+
const kind = ch(a, 'kind')
|
|
340
|
+
return c.autoApprove && !splitList(c.policies).length && nv(a, 'contains_secret') < c.approveRiskMax && kind.choice !== 'secrets'
|
|
341
|
+
}
|
|
342
|
+
|
|
291
343
|
export function gateWrite(a: Answers, c: Config): Judged {
|
|
292
344
|
const kind = ch(a, 'kind')
|
|
293
345
|
const secret = nv(a, 'contains_secret')
|
|
@@ -723,3 +775,87 @@ export function browseReply(stdout: string, stderr = ''): string {
|
|
|
723
775
|
return JSON.stringify({ error: `browse could not run: ${why}. Chrome needs remote debugging allowed the first time.` })
|
|
724
776
|
}
|
|
725
777
|
}
|
|
778
|
+
|
|
779
|
+
// --- output trim: a long tool output cut to the parts the request needs ---
|
|
780
|
+
|
|
781
|
+
const TRIM_CHUNK = 1500
|
|
782
|
+
const TRIM_CHUNKS = 24
|
|
783
|
+
|
|
784
|
+
/** The output in line-aligned chunks of about TRIM_CHUNK characters, at most TRIM_CHUNKS of them; none when short. */
|
|
785
|
+
export function trimChunks(output: string, c: Config): string[] {
|
|
786
|
+
if (!c.trimOutput || output.length <= c.trimOver) return []
|
|
787
|
+
const chunks: string[] = []
|
|
788
|
+
let cur = ''
|
|
789
|
+
for (const line of output.split('\n')) {
|
|
790
|
+
if (cur && cur.length + line.length + 1 > TRIM_CHUNK) {
|
|
791
|
+
chunks.push(cur)
|
|
792
|
+
cur = ''
|
|
793
|
+
}
|
|
794
|
+
cur = cur ? `${cur}\n${line}` : line
|
|
795
|
+
}
|
|
796
|
+
if (cur) chunks.push(cur)
|
|
797
|
+
return chunks.length >= 4 && chunks.length <= TRIM_CHUNKS ? chunks : []
|
|
798
|
+
}
|
|
799
|
+
|
|
800
|
+
/** One question per chunk, all in one Jev call; the first and last chunks are always kept and not asked about. */
|
|
801
|
+
export function trimRequest(request: string, chunks: string[]) {
|
|
802
|
+
const middle = chunks.slice(1, -1)
|
|
803
|
+
const questions: Questions = Object.fromEntries(middle.map((_, i) => [`c${i + 1}`, noul(
|
|
804
|
+
`Does \`chunks.c${i + 1}\` hold something \`request\` needs: an error, a failing test, a result, or the data asked for?`,
|
|
805
|
+
'An error or warning, a failure, a value or line the request is about',
|
|
806
|
+
'Progress lines, passing items, boilerplate or data unrelated to the request',
|
|
807
|
+
)]))
|
|
808
|
+
return { state: { request: clip(request, 1500), chunks: Object.fromEntries(middle.map((t, i) => [`c${i + 1}`, t])) }, questions }
|
|
809
|
+
}
|
|
810
|
+
|
|
811
|
+
/** The output with the chunks Jev rated unrelated replaced by one line each run; null when too little would go. */
|
|
812
|
+
export function trimmed(chunks: string[], a: Answers, c: Config): string | null {
|
|
813
|
+
const keep = chunks.map((_, i) => i === 0 || i === chunks.length - 1 || nv(a, `c${i}`) >= c.trimKeep)
|
|
814
|
+
const total = chunks.reduce((n, t) => n + t.length, 0)
|
|
815
|
+
const kept = chunks.reduce((n, t, i) => n + (keep[i] ? t.length : 0), 0)
|
|
816
|
+
if (kept > total * 0.7) return null
|
|
817
|
+
const out: string[] = []
|
|
818
|
+
for (let i = 0; i < chunks.length; i++) {
|
|
819
|
+
if (keep[i]) out.push(chunks[i]!)
|
|
820
|
+
else if (keep[i - 1] !== false) {
|
|
821
|
+
let lines = 0
|
|
822
|
+
for (let j = i; j < chunks.length && !keep[j]; j++) lines += chunks[j]!.split('\n').length
|
|
823
|
+
out.push(`[ohmyjev: ${lines} lines left out as unrelated to the request. Rerun the command with grep or sed -n to see them.]`)
|
|
824
|
+
}
|
|
825
|
+
}
|
|
826
|
+
return out.join('\n')
|
|
827
|
+
}
|
|
828
|
+
|
|
829
|
+
// --- skill picker: the skill listing out of the prompt, Jev naming the one skill a request needs ---
|
|
830
|
+
|
|
831
|
+
export type Skill = { name: string; description: string }
|
|
832
|
+
|
|
833
|
+
/** One request: which skill fits, and three questions about whether the request needs one at all (TypeSafe's cookbook). */
|
|
834
|
+
export const skillQuestions = (skills: readonly Skill[]): Questions => ({
|
|
835
|
+
which: choice('Which skill in the list best fits `request`?', Object.fromEntries(skills.slice(0, 100).map(s => [s.name, clip(s.description, 300) || s.name]))),
|
|
836
|
+
acts_on_user_system: noul('Is the assistant asked to act on the user\'s files, accounts, devices or services, rather than only to explain or advise?'),
|
|
837
|
+
would_follow_documented_procedure: noul('Would a careful expert follow a specific documented procedure or set of commands, rather than answer from general understanding?'),
|
|
838
|
+
prose_suffices: noul('Could a knowledgeable generalist fully satisfy `request` in prose, with no tools and no access to the user\'s files?'),
|
|
839
|
+
})
|
|
840
|
+
|
|
841
|
+
/** The skill to name, when the request needs one (the gate's mean) and Jev's pick is clear enough; else null. */
|
|
842
|
+
export function pickSkill(a: Answers, c: Config): { name: string; gate: number; confidence: number } | null {
|
|
843
|
+
const gate = (nv(a, 'acts_on_user_system') + nv(a, 'would_follow_documented_procedure') + (1 - nv(a, 'prose_suffices'))) / 3
|
|
844
|
+
const w = ch(a, 'which')
|
|
845
|
+
if (!w || gate < c.skillGate || w.confidence < c.skillMin) return null
|
|
846
|
+
return { name: w.choice, gate, confidence: w.confidence }
|
|
847
|
+
}
|
|
848
|
+
|
|
849
|
+
/** The second request: does the picked skill do the specific thing asked? It can say no, so an only option isn't forced. */
|
|
850
|
+
export const fitsQuestion = (s: Skill): Questions => ({
|
|
851
|
+
fits: noul(
|
|
852
|
+
`Does the skill "${s.name}" (${clip(s.description, 600)}) do the specific thing \`request\` asks for?`,
|
|
853
|
+
'It is the documented procedure for exactly this task',
|
|
854
|
+
'It is about something else, only nearby, or the request needs no procedure',
|
|
855
|
+
),
|
|
856
|
+
})
|
|
857
|
+
export const skillFits = (a: Answers, c: Config): boolean => nv(a, 'fits') >= c.skillGate
|
|
858
|
+
|
|
859
|
+
export const skillNote = (name: string): string =>
|
|
860
|
+
`<skill_relevance>\nRelevant to the current request: the ${name} skill. Load it with the Skill tool (${name}) before you start, and ` +
|
|
861
|
+
'follow it. Ignore this if it does not fit what the user actually asked for.\n</skill_relevance>'
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ohmyjev",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.9.0",
|
|
4
4
|
"description": "Guardrails for coding agents, decided by TypeSafe's Jev: blocks destructive commands, catches prompt injection, pushes back on an unverified done and routes effort per turn. For Claude Code, pi and omp.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -136,6 +136,61 @@
|
|
|
136
136
|
"description": "Deny when Jev rates a call as sending local data out at least this probably (0-1)",
|
|
137
137
|
"default": 0.7
|
|
138
138
|
},
|
|
139
|
+
"autoApprove": {
|
|
140
|
+
"type": "boolean",
|
|
141
|
+
"description": "Where the host would ask you, run a command or in-repo edit Jev is sure is safe instead",
|
|
142
|
+
"default": true
|
|
143
|
+
},
|
|
144
|
+
"approveConfidence": {
|
|
145
|
+
"type": "number",
|
|
146
|
+
"description": "How sure Jev must be that a command only reads or is reversible (0-1)",
|
|
147
|
+
"default": 0.9
|
|
148
|
+
},
|
|
149
|
+
"approveRiskMax": {
|
|
150
|
+
"type": "number",
|
|
151
|
+
"description": "Destructive, data-leaving or credential probability under which a call is approved (0-1)",
|
|
152
|
+
"default": 0.2
|
|
153
|
+
},
|
|
154
|
+
"stuckCheck": {
|
|
155
|
+
"type": "boolean",
|
|
156
|
+
"description": "After repeated failed calls, ask Jev whether the agent is stuck on one approach and tell it to change course",
|
|
157
|
+
"default": true
|
|
158
|
+
},
|
|
159
|
+
"stuck": {
|
|
160
|
+
"type": "number",
|
|
161
|
+
"description": "How sure Jev must be that the failures repeat one approach before the note (0-1)",
|
|
162
|
+
"default": 0.7
|
|
163
|
+
},
|
|
164
|
+
"trimOutput": {
|
|
165
|
+
"type": "boolean",
|
|
166
|
+
"description": "Cut a long shell output to the parts your request needs, Jev rating each part",
|
|
167
|
+
"default": true
|
|
168
|
+
},
|
|
169
|
+
"trimOver": {
|
|
170
|
+
"type": "number",
|
|
171
|
+
"description": "Characters of shell output before trimming is considered",
|
|
172
|
+
"default": 12000
|
|
173
|
+
},
|
|
174
|
+
"trimKeep": {
|
|
175
|
+
"type": "number",
|
|
176
|
+
"description": "How likely Jev must rate a part as needed for it to stay (0-1)",
|
|
177
|
+
"default": 0.3
|
|
178
|
+
},
|
|
179
|
+
"skillPick": {
|
|
180
|
+
"type": "boolean",
|
|
181
|
+
"description": "Claude Code only: leave the skill listing out of every prompt; Jev names the one skill a request needs",
|
|
182
|
+
"default": true
|
|
183
|
+
},
|
|
184
|
+
"skillGate": {
|
|
185
|
+
"type": "number",
|
|
186
|
+
"description": "Claude Code only: how strongly Jev must rate a request as needing a skill (0-1)",
|
|
187
|
+
"default": 0.3
|
|
188
|
+
},
|
|
189
|
+
"skillMin": {
|
|
190
|
+
"type": "number",
|
|
191
|
+
"description": "Claude Code only: how sure Jev must be of which skill fits (0-1)",
|
|
192
|
+
"default": 0.4
|
|
193
|
+
},
|
|
139
194
|
"requested": {
|
|
140
195
|
"type": "number",
|
|
141
196
|
"description": "A gate's Jev deny passes when Jev rates your latest message as asking for exactly that call at least this probable (0-1). Policy and path denies always stand",
|