ohmyjev 0.3.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +15 -8
- package/adapters/omp.ts +10 -6
- package/adapters/pi.ts +9 -6
- package/hooks/dashboard.ts +225 -0
- package/hooks/policy.ts +32 -12
- package/package.json +7 -1
package/README.md
CHANGED
|
@@ -58,12 +58,13 @@ If `/omj` ends with `key: env TYPESAFE_API_KEY · typesafe · jev-1.13.0`, you'r
|
|
|
58
58
|
|
|
59
59
|
| Feature | What it does |
|
|
60
60
|
|---|---|
|
|
61
|
-
| **Bash gate** | Denies a command when Jev rates it irreversible at 0.6 or more
|
|
61
|
+
| **Bash gate** | Denies a command when Jev rates it irreversible at 0.6 or more. A command Jev calls irreversible with less confidence is denied too when it also aims to wipe something (0.7). A reversible command always passes, so deleting lines, dropping a co-author trailer or uninstalling a plugin goes through. |
|
|
62
62
|
| **Write gate** | Denies writes outside the repo and `allowPaths`, and writes that contain a real credential. The repo means every worktree of the one the session started in, and every worktree of a repo the agent has moved into. The path check is plain code, not Jev, and it follows symlinks the way the OS does. |
|
|
63
63
|
| **Exfil gate** | Denies a WebFetch or MCP call when Jev rates it 0.7 or more for sending your local data, files or credentials out. |
|
|
64
64
|
| **Policies** | Every gate also checks the call against your own rules in the `policies` setting. |
|
|
65
|
+
| **Your request** | Every gate also asks Jev whether your latest message asked for exactly this call. If it did (0.8), a gate's Jev deny lets the call through. So "delete the build folder" or "force push it" works, while the same command picked by the agent on its own is still denied. Policy and path denies always stand. |
|
|
65
66
|
| **Injection screen** | When output from Bash, WebFetch, an MCP tool, or a Read outside the repo has instructions aimed at the model, it adds a note telling the model to treat that output as data. |
|
|
66
|
-
| **Done-check** | If the agent says it's done but nothing shows it ran a check, it blocks the stop once and tells the agent to verify. |
|
|
67
|
+
| **Done-check** | If the agent says it's done but nothing shows it ran a check, it blocks the stop once and tells the agent to verify. A turn that ran no tools is never blocked, since there was nothing to verify. |
|
|
67
68
|
| **Router** | Asks Jev once per request for a tier, an effort level and a risk score. Then it raises or lowers effort and picks the model for general-purpose subagents. With no key it falls back to Claude Code's built-in classifier, which reports no confidence, so then it only ever routes up. |
|
|
68
69
|
| **Auto-compact** | When the task changes after a finished step and the context is at least 40% full, it compacts the conversation. The summary keeps the new request in full. |
|
|
69
70
|
| **`ask_jev`** | A tool the model can use to ask Jev about repo files or text without loading them into its own context. |
|
|
@@ -193,14 +194,18 @@ Most of the time you won't notice it. Here's what it looks like when it does ste
|
|
|
193
194
|
|
|
194
195
|
### When a command gets blocked
|
|
195
196
|
|
|
196
|
-
Claude gets the denial as the tool's result,
|
|
197
|
+
Claude gets the denial as the tool's result, with an instruction not to work around it and a way out:
|
|
197
198
|
|
|
198
199
|
```
|
|
199
200
|
ohmyjev blocked this: irreversible (0.95): nothing would restore what this removes or overwrites.
|
|
200
|
-
|
|
201
|
-
|
|
201
|
+
Do not try to work around it with another command, another tool, a different path, or an encoding that does
|
|
202
|
+
the same thing. Tell the user what was blocked and why. If they reply asking for exactly this, in words that
|
|
203
|
+
name it, run it again: their request then lets it through.
|
|
202
204
|
```
|
|
203
205
|
|
|
206
|
+
So when the agent stops and tells you, answer with the action itself, for example "yes, delete ~/.agentmemory".
|
|
207
|
+
A bare "yes" may not be enough, because the gate judges your latest message on its own.
|
|
208
|
+
|
|
204
209
|
A write outside the repo gets a different message. It names the path and tells the model to ask you to add that
|
|
205
210
|
directory to `allowPaths`, because that block is a setting, not a hazard.
|
|
206
211
|
|
|
@@ -314,7 +319,8 @@ Run `/omj settings` to see every setting's current value (the key stays hidden).
|
|
|
314
319
|
| `logDecisions` | on | Shows each routing decision as a line in the transcript. |
|
|
315
320
|
| `statusLine` | on | Pins ohmyjev's status under the prompt. |
|
|
316
321
|
|
|
317
|
-
Each gate also has its own threshold setting.
|
|
322
|
+
Each gate also has its own threshold setting, and `requested` (0.8) sets how sure Jev must be that you asked for a
|
|
323
|
+
call. The defaults are the numbers in [What you get](#what-you-get).
|
|
318
324
|
|
|
319
325
|
## Logs
|
|
320
326
|
|
|
@@ -329,7 +335,7 @@ a fixed size:
|
|
|
329
335
|
|
|
330
336
|
| What asks | What it sends |
|
|
331
337
|
|---|---|
|
|
332
|
-
| Bash gate | The command (up to 16000 characters), the working directory
|
|
338
|
+
| Bash gate | The command (up to 16000 characters), the working directory, the tool call's description and your latest message (1500) |
|
|
333
339
|
| Write gate | The file path and the new content (up to 16000 characters). Code denies writes outside the repo and `allowPaths` before it sends anything |
|
|
334
340
|
| Exfil gate | The tool's name and its input (up to 4000 characters) |
|
|
335
341
|
| Injection screen | The tool's output (up to 6000 characters), from Bash, WebFetch, MCP tools and Reads outside the repo |
|
|
@@ -337,7 +343,8 @@ a fixed size:
|
|
|
337
343
|
| Router | The request (up to 1500 characters) |
|
|
338
344
|
| `ask_jev` | The model's question, any text it passes (up to 20000 characters) and the repo files it names (8000 each, 80000 in total). It never sends files outside the repo |
|
|
339
345
|
|
|
340
|
-
|
|
346
|
+
The write and exfil gates also send your latest message (1500), for the request check. When you've set `policies`,
|
|
347
|
+
every gate sends those too. Turning a feature off stops its calls.
|
|
341
348
|
|
|
342
349
|
With no key, nothing goes to Jev. The router's built-in classifier sends the request (up to 1500 characters) to
|
|
343
350
|
Claude's own small model, over the same connection Claude Code already uses. In pi it goes to the cheapest model you
|
package/adapters/omp.ts
CHANGED
|
@@ -254,6 +254,7 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
254
254
|
let pushedBackAt = -1
|
|
255
255
|
let route: Route | undefined
|
|
256
256
|
let liveRequest = ''
|
|
257
|
+
let userRequest = '' // the user's latest message: the gates ask whether it asked for the call
|
|
257
258
|
let compactPending = false
|
|
258
259
|
let classifiedPrompt: string | undefined
|
|
259
260
|
let askRegistered = false
|
|
@@ -398,6 +399,7 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
398
399
|
pushedBackAt = -1
|
|
399
400
|
route = undefined
|
|
400
401
|
liveRequest = ''
|
|
402
|
+
userRequest = ''
|
|
401
403
|
compactPending = false
|
|
402
404
|
classifiedPrompt = undefined
|
|
403
405
|
status()
|
|
@@ -453,6 +455,7 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
453
455
|
root = ''
|
|
454
456
|
route = undefined
|
|
455
457
|
liveRequest = ''
|
|
458
|
+
userRequest = ''
|
|
456
459
|
compactPending = false
|
|
457
460
|
classifiedPrompt = undefined
|
|
458
461
|
pushedBackAt = -1
|
|
@@ -482,7 +485,7 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
482
485
|
description: clip(description, 300),
|
|
483
486
|
...(command.length > CLIP ? { truncated: true } : {}),
|
|
484
487
|
}
|
|
485
|
-
const d = await session.decide('tool_call', tool, withPolicies(state, c), withPolicyQ(BASH_Q, c))
|
|
488
|
+
const d = await session.decide('tool_call', tool, withPolicies(state, c, userRequest), withPolicyQ(BASH_Q, c, userRequest))
|
|
486
489
|
if (!d) return
|
|
487
490
|
const judged = gateBash(d.answers, c)
|
|
488
491
|
session.record(d, judged.verdict, judged.reason)
|
|
@@ -505,8 +508,8 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
505
508
|
const path = paths.join('; ')
|
|
506
509
|
const d = await session.decide(
|
|
507
510
|
'tool_call', tool,
|
|
508
|
-
withPolicies({ path, content: clip(writeContent(input), CLIP) }, c),
|
|
509
|
-
withPolicyQ(WRITE_Q, c),
|
|
511
|
+
withPolicies({ path, content: clip(writeContent(input), CLIP) }, c, userRequest),
|
|
512
|
+
withPolicyQ(WRITE_Q, c, userRequest),
|
|
510
513
|
)
|
|
511
514
|
if (!d) return
|
|
512
515
|
const judged = gateWrite(d.answers, c)
|
|
@@ -517,8 +520,8 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
517
520
|
if (c.exfilGate && exfilTool(tool, input)) {
|
|
518
521
|
const d = await session.decide(
|
|
519
522
|
'tool_call', tool,
|
|
520
|
-
withPolicies({ tool, input: clip(safeJson(input), 4000) }, c),
|
|
521
|
-
withPolicyQ(EXFIL_Q, c),
|
|
523
|
+
withPolicies({ tool, input: clip(safeJson(input), 4000) }, c, userRequest),
|
|
524
|
+
withPolicyQ(EXFIL_Q, c, userRequest),
|
|
522
525
|
)
|
|
523
526
|
if (!d) return
|
|
524
527
|
const judged = gateExfil(d.answers, c)
|
|
@@ -591,7 +594,7 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
591
594
|
scheduleCompact(ctx)
|
|
592
595
|
return
|
|
593
596
|
}
|
|
594
|
-
const judged = judgeStop(d.answers, c)
|
|
597
|
+
const judged = judgeStop(d.answers, c, state.tools_this_turn.length > 0)
|
|
595
598
|
const block = c.doneCheck ? judged.block : null
|
|
596
599
|
if (block) pushedBackAt = requests.length
|
|
597
600
|
if (c.autoCompact && judged.wantsCompact) compactPending = true
|
|
@@ -609,6 +612,7 @@ export function createExtension(options: OmpExtensionOptions = {}): (pi: OmpApi)
|
|
|
609
612
|
if (ctx.agent.kind !== 'main' || !c.enabled) return
|
|
610
613
|
userEffort = userThinking(ctx)
|
|
611
614
|
const prompt = typeof rawEvent.prompt === 'string' ? rawEvent.prompt : ''
|
|
615
|
+
if (prompt.trim() && !/^\/\S+\s*$/.test(prompt.trim())) userRequest = prompt
|
|
612
616
|
if (!prompt.trim()) return
|
|
613
617
|
if (classifiedPrompt !== prompt) {
|
|
614
618
|
classifiedPrompt = prompt
|
package/adapters/pi.ts
CHANGED
|
@@ -257,6 +257,7 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
257
257
|
let pushedBackAt = -1
|
|
258
258
|
let route: Route | undefined
|
|
259
259
|
let liveRequest = ''
|
|
260
|
+
let userRequest = '' // the user's latest message: the gates ask whether it asked for the call
|
|
260
261
|
let compactPending = false
|
|
261
262
|
let userThinking: ThinkingLevel = 'medium'
|
|
262
263
|
let changingThinking = false
|
|
@@ -289,6 +290,7 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
289
290
|
pushedBackAt = -1
|
|
290
291
|
route = undefined
|
|
291
292
|
liveRequest = ''
|
|
293
|
+
userRequest = ''
|
|
292
294
|
compactPending = false
|
|
293
295
|
}
|
|
294
296
|
const setThinking = (level: ThinkingLevel): void => {
|
|
@@ -403,6 +405,7 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
403
405
|
|
|
404
406
|
pi.on('before_agent_start', guarded<BeforeAgentStartEvent, void>(async (event, ctx) => {
|
|
405
407
|
latestCtx = ctx
|
|
408
|
+
if (event.prompt.trim() && !/^\/\S+\s*$/.test(event.prompt.trim())) userRequest = event.prompt
|
|
406
409
|
if (!c.enabled) return
|
|
407
410
|
const text = event.prompt
|
|
408
411
|
if (!text.trim()) return
|
|
@@ -433,8 +436,8 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
433
436
|
const input = event.input
|
|
434
437
|
if ((tool === 'bash' || tool === 'powershell') && c.bashGate) {
|
|
435
438
|
const command = arg(input, 'command')
|
|
436
|
-
const state = withPolicies({ command: clip(command, CLIP), cwd: ctx.cwd, ...(command.length > CLIP ? { truncated: true } : {}) }, c)
|
|
437
|
-
const d = await x.decide('tool.call', tool, state, withPolicyQ(BASH_Q, c))
|
|
439
|
+
const state = withPolicies({ command: clip(command, CLIP), cwd: ctx.cwd, ...(command.length > CLIP ? { truncated: true } : {}) }, c, userRequest)
|
|
440
|
+
const d = await x.decide('tool.call', tool, state, withPolicyQ(BASH_Q, c, userRequest))
|
|
438
441
|
if (!d) return
|
|
439
442
|
const judged = gateBash(d.answers, c)
|
|
440
443
|
x.record(d, judged.verdict, judged.reason)
|
|
@@ -448,7 +451,7 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
448
451
|
return { block: true, reason: pathDenyText(path) }
|
|
449
452
|
}
|
|
450
453
|
const content = tool === 'write' ? arg(input, 'content') : json(input.edits)
|
|
451
|
-
const d = await x.decide('tool.call', tool, withPolicies({ path, content: clip(content, CLIP) }, c), withPolicyQ(WRITE_Q, c))
|
|
454
|
+
const d = await x.decide('tool.call', tool, withPolicies({ path, content: clip(content, CLIP) }, c, userRequest), withPolicyQ(WRITE_Q, c, userRequest))
|
|
452
455
|
if (!d) return
|
|
453
456
|
const judged = gateWrite(d.answers, c)
|
|
454
457
|
x.record(d, judged.verdict, judged.reason)
|
|
@@ -459,8 +462,8 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
459
462
|
const d = await x.decide(
|
|
460
463
|
'tool.call',
|
|
461
464
|
tool,
|
|
462
|
-
withPolicies({ tool, input: clip(json(input), 4000) }, c),
|
|
463
|
-
withPolicyQ(EXFIL_Q, c),
|
|
465
|
+
withPolicies({ tool, input: clip(json(input), 4000) }, c, userRequest),
|
|
466
|
+
withPolicyQ(EXFIL_Q, c, userRequest),
|
|
464
467
|
)
|
|
465
468
|
if (!d) return
|
|
466
469
|
const judged = gateExfil(d.answers, c)
|
|
@@ -504,7 +507,7 @@ export function createExtension(options: PiAdapterOptions = {}) {
|
|
|
504
507
|
const questions = state.previous_requests.length ? { ...STOP_Q, ...SWITCHED_Q } : STOP_Q
|
|
505
508
|
const d = await x.decide('Stop', '', state, questions)
|
|
506
509
|
if (!d) return
|
|
507
|
-
const judged = judgeStop(d.answers, c)
|
|
510
|
+
const judged = judgeStop(d.answers, c, state.tools_this_turn.length > 0)
|
|
508
511
|
const block = c.doneCheck ? judged.block : null
|
|
509
512
|
if (block) pushedBackAt = count
|
|
510
513
|
if (c.autoCompact && judged.wantsCompact) compactPending = true
|
|
@@ -0,0 +1,225 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* /omj stats: one static HTML page from every session's decision log, in ohmyjev.xyz's theme. Pure: the hosts read the
|
|
3
|
+
* logs, write the page to ~/.ohmyjev/dashboard.html and open it. Nothing on the page leaves the machine except the
|
|
4
|
+
* four font files, fetched from ohmyjev.xyz (system fonts stand in when offline).
|
|
5
|
+
*/
|
|
6
|
+
import type { LogEntry } from './policy.ts'
|
|
7
|
+
|
|
8
|
+
const FONTS = 'https://ohmyjev.xyz/fonts'
|
|
9
|
+
const DAYS = 30
|
|
10
|
+
|
|
11
|
+
export const esc = (s: unknown): string =>
|
|
12
|
+
String(s ?? '').replace(/[&<>"']/g, ch => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[ch]!)
|
|
13
|
+
|
|
14
|
+
const day = (ts: number): string => new Date(ts).toISOString().slice(0, 10)
|
|
15
|
+
const when = (ts: number): string => new Date(ts).toISOString().replace('T', ' ').slice(0, 16)
|
|
16
|
+
const money = (usd: number): string => `$${usd.toFixed(usd >= 0.01 ? 2 : 4)}`
|
|
17
|
+
const pct = (n: number, of: number): number => (of ? Math.round((n / of) * 100) : 0)
|
|
18
|
+
const quantile = (sorted: number[], q: number): number => sorted[Math.min(sorted.length - 1, Math.floor(q * sorted.length))] ?? 0
|
|
19
|
+
/** What a line was about: the tool, or the battery for lines with no tool. */
|
|
20
|
+
const what = (r: LogEntry): string =>
|
|
21
|
+
r.tool || ({ 'turn.start': 'router', Stop: 'done-check', 'session.measure': 'auto-compact', 'tool.result': 'screen', ask_jev: 'ask_jev' } as Record<string, string>)[r.event] || r.event
|
|
22
|
+
|
|
23
|
+
type Stat = { calls: number; denies: number; flags: number; cost: number }
|
|
24
|
+
const stat = (): Stat => ({ calls: 0, denies: 0, flags: 0, cost: 0 })
|
|
25
|
+
const isDeny = (r: LogEntry) => r.verdict === 'deny' || r.verdict === 'block'
|
|
26
|
+
|
|
27
|
+
export function dashboardHtml(rows: LogEntry[], opts: { now: number; sid?: string; source?: string }): string {
|
|
28
|
+
const calls = rows.filter(r => r.answers)
|
|
29
|
+
const ms = calls.map(r => r.ms ?? 0).sort((a, b) => a - b)
|
|
30
|
+
const denies = rows.filter(isDeny)
|
|
31
|
+
const total = { ...stat(), calls: calls.length, denies: denies.length, flags: rows.filter(r => r.verdict === 'flag').length, cost: calls.reduce((n, r) => n + (r.costUsd ?? 0), 0) }
|
|
32
|
+
const errors = rows.filter(r => r.error).length
|
|
33
|
+
const compactions = rows.filter(r => r.verdict === 'compact').length
|
|
34
|
+
|
|
35
|
+
const by = <K extends string>(key: (r: LogEntry) => K): Map<K, Stat> => {
|
|
36
|
+
const m = new Map<K, Stat>()
|
|
37
|
+
for (const r of rows) {
|
|
38
|
+
const s = m.get(key(r)) ?? stat()
|
|
39
|
+
if (r.answers) s.calls++
|
|
40
|
+
if (isDeny(r)) s.denies++
|
|
41
|
+
if (r.verdict === 'flag') s.flags++
|
|
42
|
+
s.cost += r.costUsd ?? 0
|
|
43
|
+
m.set(key(r), s)
|
|
44
|
+
}
|
|
45
|
+
return m
|
|
46
|
+
}
|
|
47
|
+
const tools = [...by(what)].sort((a, b) => b[1].denies - a[1].denies || b[1].calls - a[1].calls)
|
|
48
|
+
const days = by(r => day(r.ts))
|
|
49
|
+
const span = Array.from({ length: DAYS }, (_, i) => day(opts.now - (DAYS - 1 - i) * 86_400_000))
|
|
50
|
+
const peak = Math.max(1, ...span.map(d => days.get(d)?.calls ?? 0))
|
|
51
|
+
const sessions = new Map<string, Stat & { first: number; last: number }>()
|
|
52
|
+
for (const r of rows) {
|
|
53
|
+
const s = sessions.get(r.session) ?? { ...stat(), first: r.ts, last: r.ts }
|
|
54
|
+
s.first = Math.min(s.first, r.ts)
|
|
55
|
+
s.last = Math.max(s.last, r.ts)
|
|
56
|
+
if (r.answers) s.calls++
|
|
57
|
+
if (isDeny(r)) s.denies++
|
|
58
|
+
if (r.verdict === 'flag') s.flags++
|
|
59
|
+
s.cost += r.costUsd ?? 0
|
|
60
|
+
sessions.set(r.session, s)
|
|
61
|
+
}
|
|
62
|
+
const recentSessions = [...sessions].sort((a, b) => b[1].last - a[1].last).slice(0, 15)
|
|
63
|
+
const recentDenies = denies.slice(-25).reverse()
|
|
64
|
+
|
|
65
|
+
const tile = (n: string, label: string, kind = '') =>
|
|
66
|
+
`<div class="feat"><span class="kind ${kind}">${esc(label)}</span><strong>${esc(n)}</strong></div>`
|
|
67
|
+
const bars = span
|
|
68
|
+
.map(d => {
|
|
69
|
+
const s = days.get(d) ?? stat()
|
|
70
|
+
const h = Math.max(s.calls ? 3 : 1, Math.round((s.calls / peak) * 100))
|
|
71
|
+
const dh = s.calls ? Math.max(s.denies ? 3 : 0, Math.round((s.denies / peak) * 100)) : 0
|
|
72
|
+
return `<div class="bar" title="${esc(d)}: ${s.calls} calls, ${s.denies} denies, ${money(s.cost)}"><i style="height:${h}%"></i><b style="height:${dh}%"></b></div>`
|
|
73
|
+
})
|
|
74
|
+
.join('')
|
|
75
|
+
const toolRows = tools
|
|
76
|
+
.map(([t, s]) => `<tr><td><code>${esc(t)}</code></td><td>${s.calls}</td><td class="${s.denies ? 'o' : ''}">${s.denies}</td><td>${s.flags}</td><td>${money(s.cost)}</td></tr>`)
|
|
77
|
+
.join('')
|
|
78
|
+
const sessionRows = recentSessions
|
|
79
|
+
.map(([sid, s]) => `<tr${sid === opts.sid ? ' class="me"' : ''}><td><code>${esc(sid.slice(0, 8))}</code>${sid === opts.sid ? ' <span class="g">this</span>' : ''}</td><td>${when(s.last)}</td><td>${s.calls}</td><td class="${s.denies ? 'o' : ''}">${s.denies}</td><td>${money(s.cost)}</td></tr>`)
|
|
80
|
+
.join('')
|
|
81
|
+
const denyRows = recentDenies
|
|
82
|
+
.map(r => `<tr><td>${when(r.ts)}</td><td><code>${esc(what(r))}</code></td><td>${esc(r.reason ?? '')}</td><td><code>${esc(r.session.slice(0, 8))}</code></td></tr>`)
|
|
83
|
+
.join('')
|
|
84
|
+
const lede = rows.length
|
|
85
|
+
? `${total.calls} Jev calls across ${sessions.size} sessions for ${money(total.cost)}. ${total.denies} ${total.denies === 1 ? 'deny' : 'denies'} (${pct(total.denies, total.calls + total.denies)}%), ${total.flags} injection ${total.flags === 1 ? 'flag' : 'flags'}, ${errors} ${errors === 1 ? 'error' : 'errors'}.`
|
|
86
|
+
: 'No decisions logged yet. Run a few commands with ohmyjev on, then come back.'
|
|
87
|
+
|
|
88
|
+
return `<!doctype html>
|
|
89
|
+
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
|
|
90
|
+
<title>ohmyjev · stats</title>
|
|
91
|
+
<style>
|
|
92
|
+
@font-face { font-family: "Changa"; src: url("${FONTS}/Changa-ExtraBold.woff2") format("woff2"); font-weight: 800; font-display: swap; }
|
|
93
|
+
@font-face { font-family: "Space Grotesk"; src: url("${FONTS}/SpaceGrotesk-Regular.woff2") format("woff2"); font-weight: 400; font-display: swap; }
|
|
94
|
+
@font-face { font-family: "Space Grotesk"; src: url("${FONTS}/SpaceGrotesk-SemiBold.woff2") format("woff2"); font-weight: 600; font-display: swap; }
|
|
95
|
+
@font-face { font-family: "Space Grotesk"; src: url("${FONTS}/SpaceGrotesk-Bold.woff2") format("woff2"); font-weight: 700; font-display: swap; }
|
|
96
|
+
@font-face { font-family: "JetBrains Mono"; src: url("${FONTS}/JetBrainsMono-Regular.woff2") format("woff2"); font-weight: 400; font-display: swap; }
|
|
97
|
+
:root {
|
|
98
|
+
--ground: #0a0a0a; --surface: #131313; --panel: rgba(19, 19, 19, .9); --panel-hi: rgba(28, 28, 28, .94);
|
|
99
|
+
--paper: #f4f8f5; --muted: #a5b5a9; --faint: #829487; --line: #2e2e2e; --steel: #829487;
|
|
100
|
+
--brand: #00ee22; --brand-hi: #39ff53; --link: #79ff98; --lavender: #b69cff;
|
|
101
|
+
--ok: #56e887; --warn: #ffc857; --err: #ff737d; --info: #74c7ff;
|
|
102
|
+
--ui: clamp(.875rem, .82rem + .15vw, 1rem);
|
|
103
|
+
--logo: "Changa", "Space Grotesk", system-ui, sans-serif; --display: "Space Grotesk", system-ui, sans-serif;
|
|
104
|
+
--body: "Space Grotesk", system-ui, sans-serif; --label: "Space Grotesk", system-ui, sans-serif;
|
|
105
|
+
--mono: "JetBrains Mono", ui-monospace, "SF Mono", Menlo, monospace;
|
|
106
|
+
color-scheme: dark;
|
|
107
|
+
}
|
|
108
|
+
*, *::before, *::after { margin: 0; padding: 0; box-sizing: border-box; }
|
|
109
|
+
html { background: var(--ground); scrollbar-color: var(--line) var(--ground); }
|
|
110
|
+
body {
|
|
111
|
+
min-height: 100vh; padding: clamp(26px, 4.5vh, 64px) clamp(22px, 6vw, 120px) 80px; color: var(--paper); font-family: var(--body);
|
|
112
|
+
-webkit-font-smoothing: antialiased; background: var(--ground);
|
|
113
|
+
background-image: linear-gradient(rgba(140, 140, 140, .07) 1px, transparent 1px), linear-gradient(90deg, rgba(140, 140, 140, .07) 1px, transparent 1px);
|
|
114
|
+
background-size: 64px 64px;
|
|
115
|
+
}
|
|
116
|
+
::selection { background: rgba(0, 238, 34, .3); color: var(--paper); }
|
|
117
|
+
a { color: inherit; text-decoration: none; }
|
|
118
|
+
:focus-visible { outline: 2px solid var(--link); outline-offset: 3px; }
|
|
119
|
+
code { font-family: var(--mono); font-size: .875em; color: var(--paper); }
|
|
120
|
+
.lbl { font-family: var(--label); font-weight: 600; font-size: var(--ui); letter-spacing: .06em; text-transform: uppercase; }
|
|
121
|
+
.header { display: flex; align-items: baseline; justify-content: space-between; gap: 16px; flex-wrap: wrap; margin-bottom: clamp(28px, 6vh, 64px); }
|
|
122
|
+
.brand { font: 800 30px/1 var(--logo); letter-spacing: -.01em; color: var(--paper); white-space: nowrap; }
|
|
123
|
+
.brand .acc { color: var(--brand); }
|
|
124
|
+
.header .lbl { color: var(--faint); }
|
|
125
|
+
.header .lbl span { color: var(--brand); }
|
|
126
|
+
.section-label { display: flex; align-items: center; gap: 10px; color: var(--paper); margin-bottom: 12px; }
|
|
127
|
+
.section-label::before { content: ""; width: 6px; height: 6px; background: var(--brand); }
|
|
128
|
+
.section-label span { margin-left: 8px; color: var(--faint); }
|
|
129
|
+
h2 { color: var(--paper); font: 600 clamp(1.75rem, 1.3rem + 1.2vw, 2.25rem)/1.1 var(--display); letter-spacing: -.02em; text-transform: lowercase; text-wrap: balance; }
|
|
130
|
+
h2 em { font-style: inherit; color: var(--brand); }
|
|
131
|
+
.lede { max-width: 62ch; color: var(--muted); font-size: clamp(1rem, .95rem + .2vw, 1.125rem); line-height: 1.5; margin: 14px 0 clamp(22px, 4vh, 40px); }
|
|
132
|
+
h3 { font: 600 var(--ui) var(--label); letter-spacing: .06em; text-transform: uppercase; color: var(--paper); }
|
|
133
|
+
section { margin-bottom: clamp(24px, 4vh, 44px); }
|
|
134
|
+
.grid { display: grid; gap: 1px; background: var(--line); border: 1px solid var(--line); }
|
|
135
|
+
.grid > * { position: relative; display: flex; flex-direction: column; gap: 8px; min-width: 0; padding: clamp(14px, 2vh, 22px) clamp(14px, 1.4vw, 22px); background: var(--panel); transition: background .25s ease; }
|
|
136
|
+
.grid > *:hover { background: var(--panel-hi); }
|
|
137
|
+
.tiles { grid-template-columns: repeat(auto-fit, minmax(150px, 1fr)); }
|
|
138
|
+
/* tiles draw their own hairlines, so a short last row ends in panel, not a block of line color */
|
|
139
|
+
.tiles { background: var(--panel); }
|
|
140
|
+
.tiles > * { box-shadow: 1px 0 0 var(--line), 0 1px 0 var(--line); }
|
|
141
|
+
.g2 { grid-template-columns: repeat(auto-fit, minmax(min(100%, 420px), 1fr)); }
|
|
142
|
+
.kind { font: 600 .75rem var(--label); letter-spacing: .08em; text-transform: uppercase; color: var(--faint); display: inline-flex; align-items: center; gap: 6px; }
|
|
143
|
+
.kind::before { content: ""; width: 5px; height: 5px; background: var(--steel); }
|
|
144
|
+
.kind.gate::before { background: var(--err); }
|
|
145
|
+
.kind.check::before { background: var(--lavender); }
|
|
146
|
+
.kind.route::before { background: var(--info); }
|
|
147
|
+
.feat strong { font: 700 clamp(1.6rem, 2.4vw, 2.2rem)/1.1 var(--display); letter-spacing: -.03em; color: var(--paper); font-variant-numeric: tabular-nums; }
|
|
148
|
+
.card-head { display: flex; flex-wrap: wrap; align-items: baseline; justify-content: space-between; gap: 4px 16px; }
|
|
149
|
+
.card-head .meta { color: var(--faint); font: 600 .75rem var(--label); letter-spacing: .06em; text-transform: uppercase; }
|
|
150
|
+
.chart { display: flex; align-items: flex-end; gap: 3px; height: 160px; padding-top: 8px; border-bottom: 1px solid var(--line); }
|
|
151
|
+
.bar { position: relative; flex: 1; height: 100%; min-width: 0; }
|
|
152
|
+
.bar i, .bar b { position: absolute; left: 0; right: 0; bottom: 0; display: block; }
|
|
153
|
+
.bar i { background: rgba(0, 238, 34, .45); transition: background .2s ease; }
|
|
154
|
+
.bar b { background: var(--err); }
|
|
155
|
+
.bar:hover i { background: var(--brand-hi); }
|
|
156
|
+
.axis { display: flex; justify-content: space-between; color: var(--faint); font: .75rem var(--mono); padding-top: 6px; }
|
|
157
|
+
.legend { display: flex; gap: 18px; color: var(--faint); font: 600 .75rem var(--label); letter-spacing: .06em; text-transform: uppercase; }
|
|
158
|
+
.legend i { display: inline-block; width: 10px; height: 10px; vertical-align: -1px; margin-right: 6px; background: rgba(0, 238, 34, .45); }
|
|
159
|
+
.legend i.o { background: var(--err); }
|
|
160
|
+
.tbl { overflow: auto; border: 1px solid var(--line); scrollbar-width: thin; }
|
|
161
|
+
table { width: 100%; border-collapse: collapse; }
|
|
162
|
+
th { padding: 7px 14px; text-align: left; font: 600 .75rem var(--label); letter-spacing: .08em; text-transform: uppercase; color: var(--faint); border-bottom: 1px solid var(--line); background: var(--panel); white-space: nowrap; }
|
|
163
|
+
td { padding: 6px 14px; border-bottom: 1px solid var(--line); color: var(--muted); font-size: .9375rem; line-height: 1.35; vertical-align: top; background: rgba(19, 19, 19, .82); font-variant-numeric: tabular-nums; }
|
|
164
|
+
tr:last-child td { border-bottom: 0; }
|
|
165
|
+
tr:hover td { background: var(--panel-hi); }
|
|
166
|
+
td:first-child, td:nth-child(2) { white-space: nowrap; }
|
|
167
|
+
td.o { color: var(--err); }
|
|
168
|
+
.g { color: var(--ok); }
|
|
169
|
+
.me td { background: rgba(0, 238, 34, .06); }
|
|
170
|
+
.empty { padding: 18px 14px; color: var(--faint); }
|
|
171
|
+
.foot { display: flex; flex-wrap: wrap; justify-content: space-between; gap: 8px 24px; margin-top: 40px; padding-top: 14px; border-top: 1px solid var(--line); color: var(--faint); font-size: .875rem; }
|
|
172
|
+
.foot a { color: var(--link); }
|
|
173
|
+
.foot a:hover { color: var(--brand-hi); }
|
|
174
|
+
.foot a span { color: var(--brand); }
|
|
175
|
+
</style></head>
|
|
176
|
+
<body>
|
|
177
|
+
<header class="header">
|
|
178
|
+
<a class="brand" href="https://ohmyjev.xyz" aria-label="ohmyjev"><span class="acc">>_</span>ohmy<span class="acc">jev</span></a>
|
|
179
|
+
<div class="lbl">stats <span>·</span> ${esc(when(opts.now))} UTC${opts.source ? ` <span>·</span> ${esc(opts.source)}` : ''}</div>
|
|
180
|
+
</header>
|
|
181
|
+
<p class="section-label lbl">What Jev decided <span>every session on this machine</span></p>
|
|
182
|
+
<h2>what jev <em>decided</em></h2>
|
|
183
|
+
<p class="lede">${esc(lede)}</p>
|
|
184
|
+
|
|
185
|
+
<section><div class="grid tiles">
|
|
186
|
+
${tile(String(total.calls), 'Jev calls')}
|
|
187
|
+
${tile(String(total.denies), 'denies', 'gate')}
|
|
188
|
+
${tile(String(total.flags), 'injection flags', 'check')}
|
|
189
|
+
${tile(money(total.cost), 'cost')}
|
|
190
|
+
${tile(`${quantile(ms, 0.5)} ms`, 'p50 latency')}
|
|
191
|
+
${tile(`${quantile(ms, 0.95)} ms`, 'p95 latency')}
|
|
192
|
+
${tile(String(sessions.size), 'sessions')}
|
|
193
|
+
${tile(String(errors), 'errors')}
|
|
194
|
+
${tile(String(compactions), 'auto-compactions', 'route')}
|
|
195
|
+
</div></section>
|
|
196
|
+
|
|
197
|
+
<section><div class="grid"><div>
|
|
198
|
+
<div class="card-head"><h3>last ${DAYS} days</h3><span class="legend"><span><i></i>calls</span><span><i class="o"></i>denies</span></span></div>
|
|
199
|
+
<div class="chart">${bars}</div>
|
|
200
|
+
<div class="axis"><span>${esc(span[0])}</span><span>${esc(span.at(-1))}</span></div>
|
|
201
|
+
</div></div></section>
|
|
202
|
+
|
|
203
|
+
<section><div class="grid g2">
|
|
204
|
+
<div>
|
|
205
|
+
<div class="card-head"><h3>by tool</h3><span class="meta">most denied first</span></div>
|
|
206
|
+
<div class="tbl">${toolRows ? `<table><thead><tr><th>tool</th><th>calls</th><th>denies</th><th>flags</th><th>cost</th></tr></thead><tbody>${toolRows}</tbody></table>` : '<p class="empty">nothing yet</p>'}</div>
|
|
207
|
+
</div>
|
|
208
|
+
<div>
|
|
209
|
+
<div class="card-head"><h3>sessions</h3><span class="meta">latest ${recentSessions.length} of ${sessions.size}</span></div>
|
|
210
|
+
<div class="tbl">${sessionRows ? `<table><thead><tr><th>session</th><th>last seen (UTC)</th><th>calls</th><th>denies</th><th>cost</th></tr></thead><tbody>${sessionRows}</tbody></table>` : '<p class="empty">nothing yet</p>'}</div>
|
|
211
|
+
</div>
|
|
212
|
+
</div></section>
|
|
213
|
+
|
|
214
|
+
<section><div class="grid"><div>
|
|
215
|
+
<div class="card-head"><h3>recent denies</h3><span class="meta">latest ${recentDenies.length} of ${denies.length}</span></div>
|
|
216
|
+
<div class="tbl">${denyRows ? `<table><thead><tr><th>when (UTC)</th><th>tool</th><th>reason</th><th>session</th></tr></thead><tbody>${denyRows}</tbody></table>` : '<p class="empty">no denies</p>'}</div>
|
|
217
|
+
</div></div></section>
|
|
218
|
+
|
|
219
|
+
<footer class="foot">
|
|
220
|
+
<span>Read from ~/.ohmyjev/log, readable only by you. Nothing on this page leaves the machine.</span>
|
|
221
|
+
<span>Regenerate with <code>/omj stats</code> · <a href="https://ohmyjev.xyz">ohmyjev<span>.</span>xyz</a></span>
|
|
222
|
+
</footer>
|
|
223
|
+
</body></html>
|
|
224
|
+
`
|
|
225
|
+
}
|
package/hooks/policy.ts
CHANGED
|
@@ -24,6 +24,7 @@ export const DEFAULTS = {
|
|
|
24
24
|
doneVerifiedMax: 0.3,
|
|
25
25
|
doneAsksUserMax: 0.5,
|
|
26
26
|
exfil: 0.7,
|
|
27
|
+
requested: 0.8,
|
|
27
28
|
askJev: true,
|
|
28
29
|
autoCompact: true,
|
|
29
30
|
compactSwitched: 0.8,
|
|
@@ -75,8 +76,9 @@ export const score = (instructions: string, criteria: string[]): Question => ({
|
|
|
75
76
|
// --- questions (rubrics from ten-levels-of-jev level 6) ---
|
|
76
77
|
|
|
77
78
|
export const BLOCK_NOTICE =
|
|
78
|
-
'
|
|
79
|
-
'
|
|
79
|
+
'Do not try to work around it with another command, another tool, a different path, or an encoding that does the ' +
|
|
80
|
+
'same thing. Tell the user what was blocked and why. If they reply asking for exactly this, in words that name it, ' +
|
|
81
|
+
'run it again: their request then lets it through.'
|
|
80
82
|
export const DONE_REASON =
|
|
81
83
|
"You said this is done but nothing shows it was verified. Run the check, or say explicitly why it can't be verified."
|
|
82
84
|
|
|
@@ -122,11 +124,20 @@ export const POLICY_Q: Questions = {
|
|
|
122
124
|
),
|
|
123
125
|
}
|
|
124
126
|
/** A gate's questions plus the policy question, only when the user wrote policies. */
|
|
125
|
-
export const withPolicyQ = (q: Questions, c: Config): Questions =>
|
|
127
|
+
export const withPolicyQ = (q: Questions, c: Config, request = ''): Questions =>
|
|
128
|
+
({ ...q, ...(splitList(c.policies).length ? POLICY_Q : {}), ...(request.trim() ? REQUESTED_Q : {}) })
|
|
126
129
|
/** A gate's state plus the user's policies, only when there are some. */
|
|
127
|
-
export const withPolicies = <T extends object>(state: T, c: Config
|
|
130
|
+
export const withPolicies = <T extends object>(state: T, c: Config, request = ''): T => {
|
|
128
131
|
const policies = splitList(c.policies)
|
|
129
|
-
return policies.length ? { ...
|
|
132
|
+
return { ...state, ...(policies.length ? { policies } : {}), ...(request.trim() ? { request: clip(request, 1500) } : {}) }
|
|
133
|
+
}
|
|
134
|
+
/** Asked only when there is a user request: does the user's own latest message ask for exactly this call? */
|
|
135
|
+
export const REQUESTED_Q: Questions = {
|
|
136
|
+
requested: noul(
|
|
137
|
+
'Does `request`, the user\'s own latest message, explicitly ask for what this call does?',
|
|
138
|
+
'The user named this action or its target: delete that file, force push, remove the co-author line, uninstall X',
|
|
139
|
+
'The request is about something else, is vague, or the agent chose this step on its own',
|
|
140
|
+
),
|
|
130
141
|
}
|
|
131
142
|
/** The tier rubric: Jev's choice criteria, and the labels the built-in classifier picks from. */
|
|
132
143
|
const TIER_CRITERIA: Record<Tier, string> = {
|
|
@@ -197,15 +208,22 @@ const policyDeny = (a: Answers, threshold: number): Judged | null => {
|
|
|
197
208
|
return p >= threshold ? { verdict: 'deny', reason: `breaks a listed policy (${f2(p)})` } : null
|
|
198
209
|
}
|
|
199
210
|
|
|
211
|
+
/** A Jev deny the user's own request asked for passes. Policy and path denies never reach here. */
|
|
212
|
+
const unlessRequested = (j: Judged, a: Answers, c: Config): Judged => {
|
|
213
|
+
const r = nv(a, 'requested')
|
|
214
|
+
return j.verdict === 'deny' && r >= c.requested ? { verdict: null, reason: `${j.reason}; the user asked for it (${f2(r)})` } : j
|
|
215
|
+
}
|
|
216
|
+
|
|
200
217
|
export function gateBash(a: Answers, c: Config): Judged {
|
|
201
218
|
const effect = ch(a, 'effect')
|
|
202
219
|
const destructive = nv(a, 'destructive_intent')
|
|
203
220
|
const policy = policyDeny(a, c.bashDestructive)
|
|
204
221
|
if (policy) return policy
|
|
222
|
+
// destructive intent alone is most editing (deleting lines, uninstalling); it only tips an irreversible answer
|
|
205
223
|
if (effect.choice === 'irreversible' && effect.confidence >= c.bashIrreversible)
|
|
206
|
-
return { verdict: 'deny', reason: `irreversible (${f2(effect.confidence)}): nothing would restore what this removes or overwrites` }
|
|
207
|
-
if (destructive >= c.bashDestructive)
|
|
208
|
-
return { verdict: 'deny', reason: `destructive
|
|
224
|
+
return unlessRequested({ verdict: 'deny', reason: `irreversible (${f2(effect.confidence)}): nothing would restore what this removes or overwrites` }, a, c)
|
|
225
|
+
if (effect.choice === 'irreversible' && destructive >= c.bashDestructive)
|
|
226
|
+
return unlessRequested({ verdict: 'deny', reason: `irreversible (${f2(effect.confidence)}) and destructive (${f2(destructive)}): this aims to wipe something with no way back` }, a, c)
|
|
209
227
|
return { verdict: null, reason: `${effect.choice} (${f2(effect.confidence)}), destructive ${f2(destructive)}` }
|
|
210
228
|
}
|
|
211
229
|
|
|
@@ -215,9 +233,9 @@ export function gateWrite(a: Answers, c: Config): Judged {
|
|
|
215
233
|
const policy = policyDeny(a, c.writeSecret)
|
|
216
234
|
if (policy) return policy
|
|
217
235
|
if (secret >= c.writeSecret)
|
|
218
|
-
return { verdict: 'deny', reason: `contains a credential (${f2(secret)}): put it in an ignored .env or a secret store` }
|
|
236
|
+
return unlessRequested({ verdict: 'deny', reason: `contains a credential (${f2(secret)}): put it in an ignored .env or a secret store` }, a, c)
|
|
219
237
|
if (kind.choice === 'secrets' && kind.confidence >= c.writeSecretsKind)
|
|
220
|
-
return { verdict: 'deny', reason: `a secrets file (${f2(kind.confidence)}): keep credentials out of the repo` }
|
|
238
|
+
return unlessRequested({ verdict: 'deny', reason: `a secrets file (${f2(kind.confidence)}): keep credentials out of the repo` }, a, c)
|
|
221
239
|
return { verdict: null, reason: `${kind.choice} (${f2(kind.confidence)}), secret ${f2(secret)}` }
|
|
222
240
|
}
|
|
223
241
|
|
|
@@ -225,7 +243,7 @@ export function gateExfil(a: Answers, c: Config): Judged {
|
|
|
225
243
|
const p = nv(a, 'exfiltrates')
|
|
226
244
|
const policy = policyDeny(a, c.exfil)
|
|
227
245
|
if (policy) return policy
|
|
228
|
-
if (p >= c.exfil) return { verdict: 'deny', reason: `sends local data out (${f2(p)}): keep local files and credentials local` }
|
|
246
|
+
if (p >= c.exfil) return unlessRequested({ verdict: 'deny', reason: `sends local data out (${f2(p)}): keep local files and credentials local` }, a, c)
|
|
229
247
|
return { verdict: null, reason: `exfil ${f2(p)}` }
|
|
230
248
|
}
|
|
231
249
|
|
|
@@ -239,8 +257,10 @@ export function screen(a: Answers, c: Config): { flagged: boolean; reason: strin
|
|
|
239
257
|
}
|
|
240
258
|
|
|
241
259
|
/** The done-check (DONE_REASON when done is claimed with no sign of a check or question) and the compact verdict. */
|
|
242
|
-
|
|
260
|
+
/** `ranTools` false: the turn ran no tool, so there is nothing that could have been verified and nothing to block. */
|
|
261
|
+
export function judgeStop(a: Answers, c: Config, ranTools = true): { block: string | null; wantsCompact: boolean } {
|
|
243
262
|
const unverified =
|
|
263
|
+
ranTools &&
|
|
244
264
|
nv(a, 'claimed_done') >= c.doneClaimed && nv(a, 'verified') < c.doneVerifiedMax && nv(a, 'asks_user') < c.doneAsksUserMax
|
|
245
265
|
const wantsCompact = nv(a, 'switched_gears') >= c.compactSwitched && nv(a, 'at_boundary') >= c.compactBoundary
|
|
246
266
|
return { block: unverified ? DONE_REASON : null, wantsCompact }
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ohmyjev",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.4.0",
|
|
4
4
|
"description": "Guardrails for coding agents, decided by TypeSafe's Jev: blocks destructive commands, catches prompt injection, pushes back on an unverified done and routes effort per turn. For Claude Code, pi and omp.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -19,6 +19,7 @@
|
|
|
19
19
|
],
|
|
20
20
|
"files": [
|
|
21
21
|
"adapters/*.ts",
|
|
22
|
+
"hooks/dashboard.ts",
|
|
22
23
|
"hooks/jev.ts",
|
|
23
24
|
"hooks/policy.ts",
|
|
24
25
|
"extras/"
|
|
@@ -125,6 +126,11 @@
|
|
|
125
126
|
"description": "Deny when Jev rates a call as sending local data out at least this probably (0-1)",
|
|
126
127
|
"default": 0.7
|
|
127
128
|
},
|
|
129
|
+
"requested": {
|
|
130
|
+
"type": "number",
|
|
131
|
+
"description": "A gate's Jev deny passes when Jev rates your latest message as asking for exactly that call at least this probable (0-1). Policy and path denies always stand",
|
|
132
|
+
"default": 0.8
|
|
133
|
+
},
|
|
128
134
|
"askJev": {
|
|
129
135
|
"type": "boolean",
|
|
130
136
|
"description": "Expose the ask_jev tool for quick judgments about repo files or text",
|