pi-jev-effort 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +58 -29
- package/decide.ts +61 -0
- package/index.ts +174 -79
- package/package.json +5 -2
package/README.md
CHANGED
|
@@ -5,11 +5,12 @@ Sets the thinking level of a [pi](https://pi.dev) session for every prompt, from
|
|
|
5
5
|
by how much of your quota is left.
|
|
6
6
|
|
|
7
7
|
```
|
|
8
|
-
|
|
9
|
-
[jev-effort] score=2.38 conf=0.54 budget=openrouter:58% burn=$0.00046/min ceiling=none high -> high
|
|
10
|
-
[jev-effort] score=2.36 conf=0.55 budget=local:0% burn=$0.00114/min ceiling=low high -> low
|
|
8
|
+
jev ▁▁█▁ medium · 57% · resets 2h11m
|
|
11
9
|
```
|
|
12
10
|
|
|
11
|
+
The four blocks are Jev's probability for each rung of the rubric — trivial, routine,
|
|
12
|
+
substantial, hard — so a split answer is visible rather than averaged away.
|
|
13
|
+
|
|
13
14
|
Jev is a System One model: it answers a typed question with a distribution instead of prose, so
|
|
14
15
|
one call returns a score on your rubric plus the confidence behind it. A judgement takes about
|
|
15
16
|
250ms and costs about $0.000015, and Jev bills input only.
|
|
@@ -50,28 +51,35 @@ There are no npm dependencies. Both transports take the same body and return the
|
|
|
50
51
|
|
|
51
52
|
## How a level gets chosen
|
|
52
53
|
|
|
53
|
-
**1. Jev
|
|
54
|
-
and returns
|
|
55
|
-
(resolved as `typesafe/jev-1.13-20260917`):
|
|
56
|
-
|
|
57
|
-
| Prompt | Score | Confidence | Latency | Cost |
|
|
58
|
-
|---|---|---|---|---|
|
|
59
|
-
| `list the files in this directory, nothing else` | 0.00 | 1.00 | — | — |
|
|
60
|
-
| `rename the variable foo to bar in utils.ts` | 0.43 | 0.57 | 281ms | $0.0000149 |
|
|
61
|
-
| `why does the run index drift from the runner status file after a crash` | 2.38 | 0.54 | — | — |
|
|
62
|
-
| `find why the scheduler deadlocks under concurrent compaction and fix it` | 2.87 | 0.87 | 235ms | $0.0000151 |
|
|
54
|
+
**1. Jev answers one scored question** about the prompt, against a four-level rubric — trivial,
|
|
55
|
+
routine, substantial, hard — and returns a probability for each rung. Measured against
|
|
56
|
+
`~typesafe/jev-latest` (resolved as `typesafe/jev-1.13-20260917`), about 250ms per call:
|
|
63
57
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
58
|
+
| Prompt | Distribution | Reads as |
|
|
59
|
+
|---|---|---|
|
|
60
|
+
| `안녕` / `show me the files in this directory` | `1.00 / 0 / 0 / 0` | certainly trivial |
|
|
61
|
+
| `fix the typo in the README` | `0.93 / 0.07 / 0 / 0` | trivial |
|
|
62
|
+
| `refactor this` | `0.07 / 0.07 / 0.80 / 0.06` | substantial |
|
|
63
|
+
| `fix this` | `0.47 / 0.09 / 0.43 / 0.01` | could be either, and says so |
|
|
64
|
+
|
|
65
|
+
**2. Cumulative mass decides whether to move**, not the average. The scale is ordinal, so the
|
|
66
|
+
question is how much of the answer sits at or beyond a level: move up to the highest level that
|
|
67
|
+
`P(score ≥ level) ≥ 0.3` reaches, or down to the lowest that `P(score ≤ level) ≥ 0.6` covers.
|
|
68
|
+
Upgrades clear a lower bar because thinking too much costs tokens while thinking too little costs
|
|
69
|
+
the answer.
|
|
70
|
+
|
|
71
|
+
The last row is why the average is the wrong summary. Its mean is 0.96 with a reported confidence
|
|
72
|
+
of 0.04 — round that and you land on a rung nothing voted for, at a confidence no threshold will
|
|
73
|
+
ever pass, and the level sticks wherever it happens to be. On cumulative mass the same answer moves
|
|
74
|
+
a session down from `high` to `medium` (`P(≤2) = 0.99`) and leaves it there, which is what a
|
|
75
|
+
genuinely ambiguous prompt deserves.
|
|
68
76
|
|
|
69
77
|
**3. The budget caps the result.** A ceiling from quota pressure is a hard cap, not an opinion — it
|
|
70
|
-
applies whether or not
|
|
78
|
+
applies whether or not the distribution could move anything.
|
|
71
79
|
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
80
|
+
Every prompt is judged, including one-word ones. A greeting is the easiest call Jev makes, and a
|
|
81
|
+
short follow-up like "continue" is sent with the previous reply and the tools that ran, so it is
|
|
82
|
+
judged against the work it continues rather than on its own two words.
|
|
75
83
|
|
|
76
84
|
## Budget, burn rate and reset
|
|
77
85
|
|
|
@@ -99,16 +107,30 @@ it resets counts as one threshold worse:
|
|
|
99
107
|
]
|
|
100
108
|
```
|
|
101
109
|
|
|
102
|
-
The footer carries it: `jev
|
|
110
|
+
The footer carries it: `jev ▁▂█▁ medium · 57% · resets 2h11m`. A `~` before the level —
|
|
111
|
+
`jev ▄▂▄▁ ~medium` — means the distribution leans somewhere else but did not clear its threshold,
|
|
112
|
+
so the level is being held rather than chosen.
|
|
103
113
|
|
|
104
114
|
## Command
|
|
105
115
|
|
|
106
116
|
```
|
|
107
|
-
/jev-effort #
|
|
117
|
+
/jev-effort # full readout: distribution, level, budget, burn, reset
|
|
118
|
+
/jev-effort last # the last ten judgements, one sparkline each
|
|
108
119
|
/jev-effort on # enable, and clear a manual pause
|
|
109
120
|
/jev-effort off # disable for this session
|
|
110
121
|
```
|
|
111
122
|
|
|
123
|
+
```
|
|
124
|
+
jev-effort on · via openrouter · level medium
|
|
125
|
+
last: "make it cleaner" → medium → medium
|
|
126
|
+
0 trivial minimal ███······· 0.34
|
|
127
|
+
1 routine low ██········ 0.20
|
|
128
|
+
2 substantial medium █████····· 0.46
|
|
129
|
+
3 hard high ·········· 0.00
|
|
130
|
+
score 1.13 · confidence 0.12 · P(≤2)=1.00
|
|
131
|
+
budget openrouter 57% · 1,203,164 tok / $0.031 in 5h · $0.00046/min
|
|
132
|
+
```
|
|
133
|
+
|
|
112
134
|
Change the level yourself with `/effort` or `Ctrl+Shift+E` and automatic routing pauses for the
|
|
113
135
|
session — your hands beat the model's judgement. A level change caused by switching models does not
|
|
114
136
|
count as manual, so clamping to a model's capabilities will not pause anything.
|
|
@@ -130,7 +152,7 @@ Optional, at `~/.pi/agent/jev-effort.json`. Defaults:
|
|
|
130
152
|
"minDowngradeConfidence": 0.6,
|
|
131
153
|
"floor": null,
|
|
132
154
|
"ceiling": null,
|
|
133
|
-
"minPromptChars":
|
|
155
|
+
"minPromptChars": 0,
|
|
134
156
|
"budget": {
|
|
135
157
|
"source": "auto",
|
|
136
158
|
"windowHours": 5,
|
|
@@ -146,9 +168,15 @@ Optional, at `~/.pi/agent/jev-effort.json`. Defaults:
|
|
|
146
168
|
}
|
|
147
169
|
```
|
|
148
170
|
|
|
149
|
-
`levels` maps rubric
|
|
150
|
-
them: `["low", "medium", "high", "max"]`. `
|
|
151
|
-
|
|
171
|
+
`levels` maps rubric rungs 0–3 onto pi thinking levels, so a model with `xhigh` and `max` can use
|
|
172
|
+
them: `["low", "medium", "high", "max"]`. `minUpgradeConfidence` and `minDowngradeConfidence` are
|
|
173
|
+
shares of the distribution, not Jev's reported confidence. `floor` and `ceiling` clamp every result,
|
|
174
|
+
and `minPromptChars` above 0 brings back a length guard if you want one. `JEV_EFFORT_DEBUG=1` prints
|
|
175
|
+
one line per judgement to stderr:
|
|
176
|
+
|
|
177
|
+
```
|
|
178
|
+
[jev-effort] ▁▁█▁ score=2.02 conf=0.85 P(<=3)=1.00 P(>=3)=0.09 budget=openrouter:57% ceiling=none medium -> medium
|
|
179
|
+
```
|
|
152
180
|
|
|
153
181
|
Pi clamps whatever it is given to what the model supports, so `minimal` can land as `low`. That is
|
|
154
182
|
pi, not this extension, and the status line shows what actually took effect.
|
|
@@ -165,8 +193,9 @@ judgement layer.
|
|
|
165
193
|
node --test
|
|
166
194
|
```
|
|
167
195
|
|
|
168
|
-
The tests cover the
|
|
169
|
-
mocked, transcript parsing and burn rate, rate-limit header shapes,
|
|
196
|
+
The tests cover the cumulative-mass rule against real distributions, the hard cap, the wire contract
|
|
197
|
+
of both transports with `fetch` mocked, transcript parsing and burn rate, rate-limit header shapes,
|
|
198
|
+
and the UTC window grid. For an
|
|
170
199
|
end-to-end check, point `baseUrl` at a local server that answers
|
|
171
200
|
|
|
172
201
|
```json
|
package/decide.ts
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Turning one Jev answer into a level.
|
|
3
|
+
*
|
|
4
|
+
* Jev returns a distribution over the rubric, and that distribution is the
|
|
5
|
+
* useful part. An expected score hides what a split answer is telling you: a
|
|
6
|
+
* vague prompt like "fix this" comes back 0.47 / 0.09 / 0.43 / 0.01, whose mean
|
|
7
|
+
* rounds to a middle level nothing voted for, carrying a confidence so low that
|
|
8
|
+
* no gate will ever pass it. So decisions are made on cumulative mass instead —
|
|
9
|
+
* the scale is ordinal, and "how much of the answer sits at or below this level"
|
|
10
|
+
* is the question that actually matters.
|
|
11
|
+
*/
|
|
12
|
+
|
|
13
|
+
export interface Judgement {
|
|
14
|
+
/** Expected score, kept for display. */
|
|
15
|
+
score: number;
|
|
16
|
+
/** Jev's own confidence in that score, kept for display. */
|
|
17
|
+
confidence: number;
|
|
18
|
+
/** Probability per rubric index, ascending. */
|
|
19
|
+
probabilities: number[];
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
export const atMost = (p: number[], index: number): number =>
|
|
23
|
+
p.slice(0, index + 1).reduce((a, b) => a + b, 0);
|
|
24
|
+
|
|
25
|
+
export const atLeast = (p: number[], index: number): number =>
|
|
26
|
+
p.slice(index).reduce((a, b) => a + b, 0);
|
|
27
|
+
|
|
28
|
+
export const mode = (p: number[]): number =>
|
|
29
|
+
p.reduce((best, v, i) => (v > p[best] ? i : best), 0);
|
|
30
|
+
|
|
31
|
+
export interface Gates {
|
|
32
|
+
minUpgradeConfidence: number;
|
|
33
|
+
minDowngradeConfidence: number;
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
/**
|
|
37
|
+
* The rubric index to move to, or null to stay. Upgrades look for the highest
|
|
38
|
+
* level that enough of the distribution reaches; downgrades look for the lowest
|
|
39
|
+
* level that enough of it falls under. Upgrades are checked first: when a split
|
|
40
|
+
* answer could justify both, the thinking is the cheaper mistake.
|
|
41
|
+
*/
|
|
42
|
+
export const chooseIndex = (current: number, p: number[], gates: Gates): number | null => {
|
|
43
|
+
for (let i = p.length - 1; i > current; i--) {
|
|
44
|
+
if (atLeast(p, i) >= gates.minUpgradeConfidence) return i;
|
|
45
|
+
}
|
|
46
|
+
for (let i = 0; i < current; i++) {
|
|
47
|
+
if (atMost(p, i) >= gates.minDowngradeConfidence) return i;
|
|
48
|
+
}
|
|
49
|
+
return null;
|
|
50
|
+
};
|
|
51
|
+
|
|
52
|
+
const BLOCKS = ["▁", "▂", "▃", "▄", "▅", "▆", "▇", "█"];
|
|
53
|
+
|
|
54
|
+
/** One glyph per rubric level, so a split answer is visible at a glance. */
|
|
55
|
+
export const sparkline = (p: number[]): string =>
|
|
56
|
+
p.map((v) => BLOCKS[Math.max(0, Math.min(7, Math.round(v * 7)))]).join("");
|
|
57
|
+
|
|
58
|
+
export const bar = (value: number, width = 10): string => {
|
|
59
|
+
const filled = Math.round(value * width);
|
|
60
|
+
return "█".repeat(filled) + "·".repeat(Math.max(0, width - filled));
|
|
61
|
+
};
|
package/index.ts
CHANGED
|
@@ -2,6 +2,15 @@ import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-a
|
|
|
2
2
|
import { homedir } from "node:os";
|
|
3
3
|
import { basename, join } from "node:path";
|
|
4
4
|
import { readFileSync } from "node:fs";
|
|
5
|
+
import {
|
|
6
|
+
atLeast,
|
|
7
|
+
atMost,
|
|
8
|
+
bar,
|
|
9
|
+
chooseIndex,
|
|
10
|
+
mode,
|
|
11
|
+
sparkline,
|
|
12
|
+
type Judgement,
|
|
13
|
+
} from "./decide.ts";
|
|
5
14
|
import {
|
|
6
15
|
humanizeReset,
|
|
7
16
|
openRouterCredits,
|
|
@@ -33,6 +42,7 @@ const ENDPOINTS: Record<Provider, { baseUrl: string; path: string; model: string
|
|
|
33
42
|
|
|
34
43
|
const CONFIG_PATH = join(homedir(), ".pi", "agent", "jev-effort.json");
|
|
35
44
|
const STATUS_KEY = "jev-effort";
|
|
45
|
+
const HISTORY = 10;
|
|
36
46
|
|
|
37
47
|
export interface Config {
|
|
38
48
|
enabled: boolean;
|
|
@@ -44,11 +54,13 @@ export interface Config {
|
|
|
44
54
|
timeoutMs: number;
|
|
45
55
|
/** Rubric index -> thinking level. */
|
|
46
56
|
levels: ThinkingLevel[];
|
|
57
|
+
/** Share of the distribution that must reach a level before moving up to it. */
|
|
47
58
|
minUpgradeConfidence: number;
|
|
59
|
+
/** Share of the distribution that must fall under a level before moving down to it. */
|
|
48
60
|
minDowngradeConfidence: number;
|
|
49
61
|
floor: ThinkingLevel | null;
|
|
50
62
|
ceiling: ThinkingLevel | null;
|
|
51
|
-
/**
|
|
63
|
+
/** Skip prompts shorter than this; 0 judges everything, which is the default. */
|
|
52
64
|
minPromptChars: number;
|
|
53
65
|
budget: BudgetConfig;
|
|
54
66
|
notify: boolean;
|
|
@@ -82,7 +94,7 @@ export const DEFAULTS: Config = {
|
|
|
82
94
|
minDowngradeConfidence: 0.6,
|
|
83
95
|
floor: null,
|
|
84
96
|
ceiling: null,
|
|
85
|
-
minPromptChars:
|
|
97
|
+
minPromptChars: 0,
|
|
86
98
|
budget: {
|
|
87
99
|
source: "auto",
|
|
88
100
|
windowHours: 5,
|
|
@@ -109,55 +121,61 @@ const RUBRIC = [
|
|
|
109
121
|
"conflicting constraints, work that needs a plan and verification before it can be trusted.",
|
|
110
122
|
] as const;
|
|
111
123
|
|
|
124
|
+
const RUBRIC_LABELS = ["trivial", "routine", "substantial", "hard"] as const;
|
|
125
|
+
|
|
112
126
|
const INSTRUCTIONS =
|
|
113
127
|
"A developer sent this prompt to a coding agent. How much step-by-step reasoning does " +
|
|
114
128
|
"answering it well require? Judge the work the prompt asks for, not how politely it is " +
|
|
115
|
-
"written and not how long it is."
|
|
129
|
+
"written and not how long it is. A short prompt continuing earlier work asks for as much " +
|
|
130
|
+
"as the work it continues.";
|
|
116
131
|
|
|
117
132
|
export const rankOf = (level: string): number => {
|
|
118
133
|
const i = LEVELS.indexOf(level as ThinkingLevel);
|
|
119
134
|
return i === -1 ? 0 : i;
|
|
120
135
|
};
|
|
121
136
|
|
|
122
|
-
const clampLevel = (
|
|
137
|
+
const clampLevel = (
|
|
138
|
+
level: ThinkingLevel,
|
|
139
|
+
floor: ThinkingLevel | null,
|
|
140
|
+
ceiling: ThinkingLevel | null,
|
|
141
|
+
): ThinkingLevel => {
|
|
123
142
|
let rank = rankOf(level);
|
|
124
|
-
if (
|
|
125
|
-
if (
|
|
143
|
+
if (floor) rank = Math.max(rank, rankOf(floor));
|
|
144
|
+
if (ceiling) rank = Math.min(rank, rankOf(ceiling));
|
|
126
145
|
return LEVELS[rank];
|
|
127
146
|
};
|
|
128
147
|
|
|
129
|
-
/**
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
const needed = delta > 0 ? cfg.minUpgradeConfidence : cfg.minDowngradeConfidence;
|
|
144
|
-
return confidence >= needed ? target : null;
|
|
148
|
+
/** Where the current level sits on the rubric: exact match, else nearest rank. */
|
|
149
|
+
export const indexOfLevel = (level: ThinkingLevel, levels: ThinkingLevel[]): number => {
|
|
150
|
+
const exact = levels.indexOf(level);
|
|
151
|
+
if (exact !== -1) return exact;
|
|
152
|
+
let best = 0;
|
|
153
|
+
let distance = Number.POSITIVE_INFINITY;
|
|
154
|
+
levels.forEach((candidate, i) => {
|
|
155
|
+
const d = Math.abs(rankOf(candidate) - rankOf(level));
|
|
156
|
+
if (d < distance) {
|
|
157
|
+
distance = d;
|
|
158
|
+
best = i;
|
|
159
|
+
}
|
|
160
|
+
});
|
|
161
|
+
return best;
|
|
145
162
|
};
|
|
146
163
|
|
|
147
164
|
/**
|
|
148
|
-
* The level to switch to, or null to stay put. The
|
|
149
|
-
* and has to clear its
|
|
150
|
-
* because a budget that has run out is not an opinion.
|
|
165
|
+
* The level to switch to, or null to stay put. The distribution decides the
|
|
166
|
+
* direction and has to clear its threshold; the ceiling is a hard cap that does
|
|
167
|
+
* not, because a budget that has run out is not an opinion.
|
|
151
168
|
*/
|
|
152
169
|
export const plan = (
|
|
153
170
|
current: ThinkingLevel,
|
|
154
|
-
|
|
155
|
-
confidence: number,
|
|
171
|
+
judgement: Judgement,
|
|
156
172
|
cfg: Config,
|
|
157
173
|
ceiling: ThinkingLevel | null,
|
|
158
174
|
): ThinkingLevel | null => {
|
|
159
|
-
const
|
|
160
|
-
const
|
|
175
|
+
const currentIndex = indexOfLevel(current, cfg.levels);
|
|
176
|
+
const target = chooseIndex(currentIndex, judgement.probabilities, cfg);
|
|
177
|
+
const proposed = target === null ? current : cfg.levels[target];
|
|
178
|
+
const capped = clampLevel(proposed, cfg.floor, ceiling);
|
|
161
179
|
return capped === current ? null : capped;
|
|
162
180
|
};
|
|
163
181
|
|
|
@@ -200,11 +218,6 @@ export const buildRoute = (provider: Provider, apiKey: string, cfg: Config): Rou
|
|
|
200
218
|
};
|
|
201
219
|
};
|
|
202
220
|
|
|
203
|
-
export interface Judgement {
|
|
204
|
-
score: number;
|
|
205
|
-
confidence: number;
|
|
206
|
-
}
|
|
207
|
-
|
|
208
221
|
export const classify = async (
|
|
209
222
|
state: Record<string, unknown>,
|
|
210
223
|
route: Route,
|
|
@@ -229,22 +242,39 @@ export const classify = async (
|
|
|
229
242
|
throw new Error(`${route.provider} ${res.status}: ${(await res.text()).slice(0, 300)}`);
|
|
230
243
|
}
|
|
231
244
|
const body = (await res.json()) as {
|
|
232
|
-
answers?: {
|
|
245
|
+
answers?: {
|
|
246
|
+
effort?: { score?: unknown; confidence?: unknown; probabilities?: Record<string, unknown> };
|
|
247
|
+
};
|
|
233
248
|
};
|
|
234
|
-
const
|
|
235
|
-
if (typeof score !== "number" || typeof confidence !== "number") return null;
|
|
236
|
-
|
|
249
|
+
const answer = body.answers?.effort;
|
|
250
|
+
if (typeof answer?.score !== "number" || typeof answer.confidence !== "number") return null;
|
|
251
|
+
const probabilities = RUBRIC.map((_, i) => {
|
|
252
|
+
const v = answer.probabilities?.[String(i)];
|
|
253
|
+
return typeof v === "number" ? v : 0;
|
|
254
|
+
});
|
|
255
|
+
if (probabilities.every((v) => v === 0)) return null;
|
|
256
|
+
return { score: answer.score, confidence: answer.confidence, probabilities };
|
|
237
257
|
};
|
|
238
258
|
|
|
259
|
+
interface Recorded extends Judgement {
|
|
260
|
+
at: number;
|
|
261
|
+
prompt: string;
|
|
262
|
+
from: ThinkingLevel;
|
|
263
|
+
to: ThinkingLevel;
|
|
264
|
+
}
|
|
265
|
+
|
|
239
266
|
export default function (pi: ExtensionAPI) {
|
|
240
267
|
const cfg = loadConfig();
|
|
241
268
|
let paused = false;
|
|
242
269
|
let lastApplied: ThinkingLevel | null = null;
|
|
243
|
-
let lastJudgement: Judgement | null = null;
|
|
244
270
|
let lastRoute: Route | null = null;
|
|
245
271
|
let lastBudget: Budget | null = null;
|
|
246
272
|
let observedLimit: RateLimit | null = null;
|
|
247
273
|
let balance: { at: number; remainingFraction: number } | null = null;
|
|
274
|
+
const history: Recorded[] = [];
|
|
275
|
+
// Context for a follow-up too short to judge on its own.
|
|
276
|
+
let previousReply: string | null = null;
|
|
277
|
+
let previousTools: string[] = [];
|
|
248
278
|
// Set while we call setThinkingLevel so the resulting event is not read as a
|
|
249
279
|
// manual override. Model changes clamp the level too, hence the timestamp.
|
|
250
280
|
let applying = false;
|
|
@@ -292,10 +322,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
292
322
|
if (b.source === "auto" || b.source === "headers") {
|
|
293
323
|
const l = observedLimit;
|
|
294
324
|
if (l?.limit && l.remaining !== undefined) {
|
|
295
|
-
const fraction = Math.max(0, Math.min(1, l.remaining / l.limit));
|
|
296
325
|
return {
|
|
297
326
|
source: "headers",
|
|
298
|
-
remainingFraction:
|
|
327
|
+
remainingFraction: Math.max(0, Math.min(1, l.remaining / l.limit)),
|
|
299
328
|
resetAt: l.resetAt ?? null,
|
|
300
329
|
spend,
|
|
301
330
|
exhaustsBeforeReset: false,
|
|
@@ -309,7 +338,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
309
338
|
if (!fresh) {
|
|
310
339
|
try {
|
|
311
340
|
const credits = await openRouterCredits(route.apiKey, cfg.timeoutMs);
|
|
312
|
-
balance = credits
|
|
341
|
+
balance = credits
|
|
342
|
+
? { at: Date.now(), remainingFraction: credits.remainingFraction }
|
|
343
|
+
: null;
|
|
313
344
|
} catch {
|
|
314
345
|
// A balance lookup is advisory; losing it must not change the turn.
|
|
315
346
|
}
|
|
@@ -353,22 +384,66 @@ export default function (pi: ExtensionAPI) {
|
|
|
353
384
|
const status = (ctx: ExtensionContext) => {
|
|
354
385
|
if (!ctx.hasUI) return;
|
|
355
386
|
if (!cfg.enabled || paused) {
|
|
356
|
-
ctx.ui.setStatus(STATUS_KEY, paused ? "jev
|
|
387
|
+
ctx.ui.setStatus(STATUS_KEY, paused ? "jev paused" : undefined);
|
|
357
388
|
return;
|
|
358
389
|
}
|
|
359
|
-
const
|
|
360
|
-
const
|
|
361
|
-
|
|
362
|
-
if (
|
|
363
|
-
|
|
390
|
+
const last = history[0];
|
|
391
|
+
const level = pi.getThinkingLevel() as ThinkingLevel;
|
|
392
|
+
let head = "jev auto";
|
|
393
|
+
if (last) {
|
|
394
|
+
// A tilde marks a level the judgement leans away from but could not move:
|
|
395
|
+
// the distribution pointed elsewhere without clearing its threshold.
|
|
396
|
+
const leaning = cfg.levels[mode(last.probabilities)];
|
|
397
|
+
const held = leaning !== level ? "~" : "";
|
|
398
|
+
head = `jev ${sparkline(last.probabilities)} ${held}${level}`;
|
|
364
399
|
}
|
|
400
|
+
const parts = [head];
|
|
401
|
+
const b = lastBudget;
|
|
402
|
+
if (b && b.remainingFraction !== null) parts.push(`${Math.round(b.remainingFraction * 100)}%`);
|
|
365
403
|
const reset = humanizeReset(b?.resetAt ?? null);
|
|
366
404
|
if (reset) parts.push(`resets ${reset}`);
|
|
367
405
|
ctx.ui.setStatus(STATUS_KEY, parts.join(" · "));
|
|
368
406
|
};
|
|
369
407
|
|
|
408
|
+
const readout = (): string => {
|
|
409
|
+
const last = history[0];
|
|
410
|
+
const level = pi.getThinkingLevel();
|
|
411
|
+
const state = !cfg.enabled ? "off" : paused ? "paused" : "on";
|
|
412
|
+
const lines = [
|
|
413
|
+
`jev-effort ${state} · via ${lastRoute?.provider ?? "unresolved"} · level ${level}`,
|
|
414
|
+
];
|
|
415
|
+
if (last) {
|
|
416
|
+
lines.push(`last: ${JSON.stringify(last.prompt.slice(0, 60))} → ${last.from} → ${last.to}`);
|
|
417
|
+
last.probabilities.forEach((v, i) => {
|
|
418
|
+
const label = (RUBRIC_LABELS[i] ?? String(i)).padEnd(11);
|
|
419
|
+
const level = (cfg.levels[i] ?? "?").padEnd(7);
|
|
420
|
+
lines.push(` ${i} ${label} ${level} ${bar(v)} ${v.toFixed(2)}`);
|
|
421
|
+
});
|
|
422
|
+
lines.push(
|
|
423
|
+
` score ${last.score.toFixed(2)} · confidence ${last.confidence.toFixed(2)} · ` +
|
|
424
|
+
`P(≤${mode(last.probabilities)})=${atMost(last.probabilities, mode(last.probabilities)).toFixed(2)}`,
|
|
425
|
+
);
|
|
426
|
+
} else {
|
|
427
|
+
lines.push("no judgement yet in this session");
|
|
428
|
+
}
|
|
429
|
+
const b = lastBudget;
|
|
430
|
+
if (b) {
|
|
431
|
+
const left = b.remainingFraction === null ? "?" : `${Math.round(b.remainingFraction * 100)}%`;
|
|
432
|
+
const reset = humanizeReset(b.resetAt);
|
|
433
|
+
lines.push(
|
|
434
|
+
`budget ${b.source} ${left}${reset ? ` (resets ${reset})` : ""} · ` +
|
|
435
|
+
`${b.spend.tokens.toLocaleString()} tok / $${b.spend.usd.toFixed(3)} in ` +
|
|
436
|
+
`${cfg.budget.windowHours}h · $${b.spend.burnUsdPerMin.toFixed(5)}/min`,
|
|
437
|
+
);
|
|
438
|
+
}
|
|
439
|
+
return lines.join("\n");
|
|
440
|
+
};
|
|
441
|
+
|
|
370
442
|
pi.on("session_start", async (_event, ctx) => {
|
|
371
443
|
levelChurnUntil = Date.now() + 2000;
|
|
444
|
+
history.length = 0;
|
|
445
|
+
previousReply = null;
|
|
446
|
+
previousTools = [];
|
|
372
447
|
if (cfg.enabled) {
|
|
373
448
|
const route = await resolveRoute(ctx);
|
|
374
449
|
if (!route) {
|
|
@@ -377,7 +452,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
377
452
|
"jev-effort: no key. Set TYPESAFE_API_KEY, or sign in to OpenRouter in pi.",
|
|
378
453
|
"warning",
|
|
379
454
|
);
|
|
380
|
-
ctx.ui.setStatus(STATUS_KEY, "jev
|
|
455
|
+
ctx.ui.setStatus(STATUS_KEY, "jev no key");
|
|
381
456
|
}
|
|
382
457
|
return;
|
|
383
458
|
}
|
|
@@ -392,6 +467,23 @@ export default function (pi: ExtensionAPI) {
|
|
|
392
467
|
if (seen) observedLimit = { ...observedLimit, ...seen };
|
|
393
468
|
});
|
|
394
469
|
|
|
470
|
+
pi.on("turn_end", async (event, _ctx) => {
|
|
471
|
+
// Remembered so the next prompt, however short, can be judged in context.
|
|
472
|
+
const content = (event.message as { content?: unknown } | undefined)?.content;
|
|
473
|
+
const text = typeof content === "string"
|
|
474
|
+
? content
|
|
475
|
+
: Array.isArray(content)
|
|
476
|
+
? content
|
|
477
|
+
.map((part) => (typeof part === "object" && part && "text" in part ? String((part as { text: unknown }).text) : ""))
|
|
478
|
+
.join(" ")
|
|
479
|
+
: "";
|
|
480
|
+
previousReply = text.trim().slice(0, 300) || previousReply;
|
|
481
|
+
const results = (event.toolResults ?? []) as { toolName?: string }[];
|
|
482
|
+
if (results.length) {
|
|
483
|
+
previousTools = [...new Set(results.map((r) => r.toolName ?? "").filter(Boolean))];
|
|
484
|
+
}
|
|
485
|
+
});
|
|
486
|
+
|
|
395
487
|
pi.on("model_select", async (_event, ctx) => {
|
|
396
488
|
// A model change can clamp the thinking level; that is not the user typing /effort.
|
|
397
489
|
levelChurnUntil = Date.now() + 1000;
|
|
@@ -414,9 +506,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
414
506
|
pi.on("before_agent_start", async (event, ctx) => {
|
|
415
507
|
if (!cfg.enabled || paused) return;
|
|
416
508
|
const prompt = (event.prompt ?? "").trim();
|
|
417
|
-
|
|
418
|
-
// them would drag the level back down in the middle of hard work.
|
|
419
|
-
if (prompt.length < cfg.minPromptChars) return;
|
|
509
|
+
if (prompt.length < Math.max(1, cfg.minPromptChars)) return;
|
|
420
510
|
|
|
421
511
|
const current = pi.getThinkingLevel() as ThinkingLevel;
|
|
422
512
|
let judgement: Judgement | null = null;
|
|
@@ -430,6 +520,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
430
520
|
project: basename(ctx.cwd),
|
|
431
521
|
model: ctx.model ? `${ctx.model.provider}/${ctx.model.id}` : null,
|
|
432
522
|
current_thinking_level: current,
|
|
523
|
+
continuing_work: previousReply !== null,
|
|
524
|
+
previous_reply: previousReply,
|
|
525
|
+
previous_tools: previousTools.length ? previousTools : null,
|
|
433
526
|
},
|
|
434
527
|
route,
|
|
435
528
|
cfg,
|
|
@@ -440,20 +533,19 @@ export default function (pi: ExtensionAPI) {
|
|
|
440
533
|
return;
|
|
441
534
|
}
|
|
442
535
|
if (!judgement) return;
|
|
443
|
-
lastJudgement = judgement;
|
|
444
536
|
|
|
445
537
|
const budget = await resolveBudget(lastRoute);
|
|
446
538
|
lastBudget = budget;
|
|
447
539
|
const ceiling = effectiveCeiling(budget);
|
|
448
|
-
const next = plan(current, judgement
|
|
540
|
+
const next = plan(current, judgement, cfg, ceiling);
|
|
541
|
+
const p = judgement.probabilities;
|
|
449
542
|
const left =
|
|
450
|
-
budget.remainingFraction === null
|
|
451
|
-
? "n/a"
|
|
452
|
-
: `${Math.round(budget.remainingFraction * 100)}%`;
|
|
543
|
+
budget.remainingFraction === null ? "n/a" : `${Math.round(budget.remainingFraction * 100)}%`;
|
|
453
544
|
log(
|
|
454
|
-
|
|
455
|
-
`
|
|
456
|
-
`
|
|
545
|
+
`${sparkline(p)} score=${judgement.score.toFixed(2)} conf=${judgement.confidence.toFixed(2)} ` +
|
|
546
|
+
`P(<=${indexOfLevel(current, cfg.levels)})=${atMost(p, indexOfLevel(current, cfg.levels)).toFixed(2)} ` +
|
|
547
|
+
`P(>=${indexOfLevel(current, cfg.levels)})=${atLeast(p, indexOfLevel(current, cfg.levels)).toFixed(2)} ` +
|
|
548
|
+
`budget=${budget.source}:${left} ceiling=${ceiling ?? "none"} ${current} -> ${next ?? current}`,
|
|
457
549
|
);
|
|
458
550
|
if (next) {
|
|
459
551
|
applying = true;
|
|
@@ -464,13 +556,21 @@ export default function (pi: ExtensionAPI) {
|
|
|
464
556
|
}
|
|
465
557
|
// Pi clamps to what the model supports, so record what actually took effect.
|
|
466
558
|
lastApplied = pi.getThinkingLevel() as ThinkingLevel;
|
|
467
|
-
if (cfg.notify && ctx.hasUI) ctx.ui.notify(`jev-effort: ${current}
|
|
559
|
+
if (cfg.notify && ctx.hasUI) ctx.ui.notify(`jev-effort: ${current} → ${next}`, "info");
|
|
468
560
|
}
|
|
561
|
+
history.unshift({
|
|
562
|
+
...judgement,
|
|
563
|
+
at: Date.now(),
|
|
564
|
+
prompt,
|
|
565
|
+
from: current,
|
|
566
|
+
to: (next ?? current) as ThinkingLevel,
|
|
567
|
+
});
|
|
568
|
+
history.length = Math.min(history.length, HISTORY);
|
|
469
569
|
status(ctx);
|
|
470
570
|
});
|
|
471
571
|
|
|
472
572
|
pi.registerCommand("jev-effort", {
|
|
473
|
-
description: "Jev-driven
|
|
573
|
+
description: "Jev-driven thinking level: status | on | off | last",
|
|
474
574
|
handler: async (args, ctx) => {
|
|
475
575
|
const arg = args.trim().toLowerCase();
|
|
476
576
|
if (arg === "on") {
|
|
@@ -478,26 +578,21 @@ export default function (pi: ExtensionAPI) {
|
|
|
478
578
|
paused = false;
|
|
479
579
|
} else if (arg === "off") {
|
|
480
580
|
cfg.enabled = false;
|
|
581
|
+
} else if (arg === "last") {
|
|
582
|
+
const lines = history.length
|
|
583
|
+
? history.map((h) => {
|
|
584
|
+
const moved = h.from === h.to ? `${h.to} held` : `${h.from} → ${h.to}`;
|
|
585
|
+
return `${sparkline(h.probabilities)} ${h.score.toFixed(2)} ${moved} ${h.prompt.slice(0, 40)}`;
|
|
586
|
+
})
|
|
587
|
+
: ["no judgements yet in this session"];
|
|
588
|
+
ctx.ui.notify(lines.join("\n"), "info");
|
|
589
|
+
return;
|
|
481
590
|
} else if (arg && arg !== "status") {
|
|
482
|
-
ctx.ui.notify("jev-effort: usage /jev-effort [on|off|
|
|
591
|
+
ctx.ui.notify("jev-effort: usage /jev-effort [status|on|off|last]", "warning");
|
|
483
592
|
return;
|
|
484
593
|
}
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
const b = lastBudget ?? (await resolveBudget(lastRoute));
|
|
488
|
-
lastBudget = b;
|
|
489
|
-
const left = b.remainingFraction === null ? "?" : `${Math.round(b.remainingFraction * 100)}%`;
|
|
490
|
-
const reset = humanizeReset(b.resetAt);
|
|
491
|
-
ctx.ui.notify(
|
|
492
|
-
`jev-effort ${state} · via ${lastRoute?.provider ?? "unresolved"} · ` +
|
|
493
|
-
`level ${pi.getThinkingLevel()}` +
|
|
494
|
-
(j ? ` · last score ${j.score.toFixed(2)} @ ${j.confidence.toFixed(2)}` : "") +
|
|
495
|
-
` · budget ${b.source} ${left}` +
|
|
496
|
-
(reset ? ` (resets ${reset})` : "") +
|
|
497
|
-
` · ${b.spend.tokens.toLocaleString()} tok / $${b.spend.usd.toFixed(3)} ` +
|
|
498
|
-
`in ${cfg.budget.windowHours}h`,
|
|
499
|
-
"info",
|
|
500
|
-
);
|
|
594
|
+
if (!lastBudget) lastBudget = await resolveBudget(lastRoute);
|
|
595
|
+
ctx.ui.notify(readout(), "info");
|
|
501
596
|
status(ctx);
|
|
502
597
|
},
|
|
503
598
|
});
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-jev-effort",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.2.0",
|
|
4
4
|
"description": "Pi extension that sets the session's thinking level per prompt from a TypeSafe Jev judgement, clamped by the quota you have left",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi-package",
|
|
@@ -30,6 +30,7 @@
|
|
|
30
30
|
},
|
|
31
31
|
"files": [
|
|
32
32
|
"index.ts",
|
|
33
|
+
"decide.ts",
|
|
33
34
|
"usage.ts",
|
|
34
35
|
"README.md",
|
|
35
36
|
"LICENSE"
|
|
@@ -38,6 +39,8 @@
|
|
|
38
39
|
"test": "node --test"
|
|
39
40
|
},
|
|
40
41
|
"pi": {
|
|
41
|
-
"extensions": [
|
|
42
|
+
"extensions": [
|
|
43
|
+
"./index.ts"
|
|
44
|
+
]
|
|
42
45
|
}
|
|
43
46
|
}
|