specpi 0.30.0 → 0.30.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/NPM_RELEASE.md +2 -2
- package/README.md +4 -5
- package/SECURITY_MODEL.md +10 -6
- package/extensions/jev-advisor/broker.mjs +87 -8
- package/extensions/jev-advisor/client.mjs +33 -10
- package/extensions/jev-advisor/consent.mjs +19 -7
- package/extensions/jev-advisor/gate.mjs +20 -5
- package/extensions/jev-advisor/index.ts +307 -189
- package/extensions/jev-advisor/ledger.mjs +16 -2
- package/extensions/jev-advisor/questions/gap.mjs +49 -86
- package/extensions/jev-advisor/questions/retention.mjs +5 -8
- package/extensions/jev-advisor/questions/sources.mjs +27 -56
- package/extensions/jev-advisor/sanitize.mjs +0 -0
- package/extensions/tool-wishlist/core.mjs +86 -0
- package/extensions/tool-wishlist/index.ts +81 -8
- package/extensions/workflow-controls/index.ts +23 -0
- package/package.json +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
## 0.30.1 - 2026-09-22
|
|
6
|
+
|
|
7
|
+
- Repair Jev's task context: read the active contract structurally, fall back to the bounded current request, and clear stale context on task, branch and session changes.
|
|
8
|
+
- Allocate the existing 1 KiB state budget by field. Preserve result samples and candidate/question correspondence; abstain locally when required evidence cannot fit. Record sample coverage and local refusals separately from classifier calls.
|
|
9
|
+
- Renew transmission consent with an accurate disclosure of sampled file contents, command output and request text. Redact question text as well as state, replace outside-workspace absolute paths, and use opaque source/cluster IDs. Consent now binds the transport scheme as well as the host.
|
|
10
|
+
- Rank delegation sources per `packet.jobs[]`, respecting the declared mode and preserving every selected source and ungated position. Store gap-triage opinions alongside the original observation, with a bounded existing-cluster shortlist. Opinions cannot change priority, canonical identity or human improvement authority.
|
|
11
|
+
- Retain object-shaped Score distributions, reject malformed probabilities and require a Choice distribution before applying its margin gate. Separate warnings from real elisions and count delivered effects rather than undelivered advice. Shell results are no longer eligible for shortening because their commands may not be safe to repeat.
|
|
12
|
+
- Close review findings at asynchronous boundaries: bind reports before consent and root lookup, recheck collection before sending and advisory authority inside locked writes, and abandon stale tool-result lookups. Reject transport redirects, redact quoted credentials and multiline keys before sampling, and retain committed-effect accounting when later notifications or report rendering fail.
|
|
13
|
+
- Keep all advisor systems off by default, production thresholds unchanged, compaction guidance withdrawn and the command guard separate. These repairs do not establish a task-quality or cost benefit; new controlled evaluations are still required.
|
|
14
|
+
- Isolate the base-package installer tests' home directory as well as their agent directory. The old fixtures could switch off the real command guard during validation; a regression now checks that a synthetic caller's enabled guard remains byte-for-byte unchanged.
|
|
15
|
+
- Trim the evaluations page to results, a short method and chart notes, without changing the published data. Add expandable diagrams for the six Jev systems, with theme-aware colors and contained scrolling on narrow screens.
|
|
16
|
+
|
|
3
17
|
## 0.30.0 - 2026-09-21
|
|
4
18
|
|
|
5
19
|
- Replace the evaluations page with Terminal-Bench 2.0. The suite that filled it was written in this repository, and it was retired for the reason it kept reporting: nearly every harness passed nearly every task, so the thing it was built to measure was the thing it could not see, and three rounds of harder tasks did not move it. A suite cannot mark its own homework. The page now carries somebody else's benchmark on tasks nobody here chose -- Pi with no harness, SpecPi + Jev, Oh My Pi and Claude Code, 334 scored attempts on one model through one endpoint.
|
package/NPM_RELEASE.md
CHANGED
|
@@ -6,7 +6,7 @@ Publishing, tags, deprecation, ownership changes and GitHub Releases require exp
|
|
|
6
6
|
|
|
7
7
|
## Prepare
|
|
8
8
|
|
|
9
|
-
1. This candidate is SpecPi **0.30.
|
|
9
|
+
1. This candidate is SpecPi **0.30.1**, paired with the Chat **0.14.0** VSIX. Browser QA **0.3.0** is already independently published; do not republish or edit its source as part of this release. Choose an unused version; update `package.json`, the dated `CHANGELOG.md` entry, README, and the website. Check Chat's version and VSIX download links. Preserve historical changelog entries.
|
|
10
10
|
2. Validate:
|
|
11
11
|
|
|
12
12
|
```sh
|
|
@@ -20,7 +20,7 @@ Publishing, tags, deprecation, ownership changes and GitHub Releases require exp
|
|
|
20
20
|
git diff --check
|
|
21
21
|
```
|
|
22
22
|
|
|
23
|
-
Installer/Pi lifecycle tests must use disposable state, never a live profile. `check:base` requires network access and verifies the real
|
|
23
|
+
Installer/Pi lifecycle tests must use disposable state, never a live profile. `check:base` requires network access and verifies the real eight-package base with isolated home/configuration paths. The base check now invokes the installed Browser QA Node bin for Chromium setup and offline doctor checks; scope `PLAYWRIGHT_BROWSERS_PATH` to a disposable test cache. Verify setup skip, core-only behavior, failure rollback, and BetterWright ownership migration in the installer regressions. Browser-cache/package bytes are outside managed rollback and uninstall. Review all default version changes, upstream lifecycle scripts, compatibility, and notices in `THIRD_PARTY.md`. Local dry runs disable provenance because they lack GitHub OIDC.
|
|
24
24
|
|
|
25
25
|
3. Review the diff, exact package manifest and artifact; obtain fresh read-only review for lifecycle, permissions and packaging changes.
|
|
26
26
|
4. After approval and passing PR checks, merge to `main`, then create the matching `v<version>` tag at the merge commit. Build Chat's VSIX with `npm --prefix vscode run package` and attach it to the GitHub Release. Publish the release to start the npm workflow and approve the protected `npm` environment if it requests review. Verify the website deployment and download links. If publication cannot complete, revert the release merge and use a new version for the next attempt.
|
package/README.md
CHANGED
|
@@ -28,7 +28,7 @@
|
|
|
28
28
|
|
|
29
29
|
---
|
|
30
30
|
|
|
31
|
-
SpecPi is a small starting point for the [Pi coding agent](https://pi.dev/). It is one opinionated setup for how the agent should work, not a marketplace of plugins.
|
|
31
|
+
SpecPi 0.30.1 is a small starting point for the [Pi coding agent](https://pi.dev/). It is one opinionated setup for how the agent should work, not a marketplace of plugins.
|
|
32
32
|
|
|
33
33
|
At the center are two built-in extensions. **Scope control** keeps each task to the files it said it would touch. The **improvement loop** turns repeated friction into small, tested changes to the setup, instead of letting prompts and workarounds pile up. Around those are eight hand-picked packages, each locked to an exact version and checked before anything installs, plus **SpecPi Chat**, a VS Code panel for working alongside the agent.
|
|
34
34
|
|
|
@@ -97,9 +97,8 @@ this run, so its cost is an upper bound.
|
|
|
97
97
|
|
|
98
98
|
<!-- /eval-summary -->
|
|
99
99
|
|
|
100
|
-
This run is still in progress.
|
|
101
|
-
[evaluations page](https://tannermidd.github.io/SpecPi/evaluations/)
|
|
102
|
-
what is missing and what is known to be wrong. The table above is regenerated
|
|
100
|
+
This run is still in progress. See the results and brief method on the
|
|
101
|
+
[evaluations page](https://tannermidd.github.io/SpecPi/evaluations/). The table above is regenerated
|
|
103
102
|
from the run data by `node scripts/tb2-site.mjs`, so it cannot drift from the
|
|
104
103
|
published figures.
|
|
105
104
|
|
|
@@ -122,7 +121,7 @@ Full setup options, package details, and requirements: [website](https://tannerm
|
|
|
122
121
|
|
|
123
122
|
| | |
|
|
124
123
|
| --- | --- |
|
|
125
|
-
| [Packages](https://tannermidd.github.io/SpecPi/#packages) | The
|
|
124
|
+
| [Packages](https://tannermidd.github.io/SpecPi/#packages) | The eight pinned packages and what each provides |
|
|
126
125
|
| [Scope control](https://tannermidd.github.io/SpecPi/wiki/#scope) | `/scope` commands and drift monitoring |
|
|
127
126
|
| [Improvement loop](https://tannermidd.github.io/SpecPi/#loop) | Local wishlist, `/harness-improvement`, and retirement with evidence |
|
|
128
127
|
| [SpecPi Chat](https://tannermidd.github.io/SpecPi/#vscode-chat) | VS Code frontend and VSIX install · [Chat guide](https://github.com/TannerMidd/SpecPi/blob/main/vscode/README.md) |
|
package/SECURITY_MODEL.md
CHANGED
|
@@ -18,19 +18,23 @@ Optional tool groups ship withdrawn. The `request_capability` tool lets the mode
|
|
|
18
18
|
|
|
19
19
|
The advisor is the only part of SpecPi that sends anything off this machine, and it ships off. With the master switch off there is no network call, no key read, no consent read and no prompt injection: the harness behaves exactly as it did before the extension existed.
|
|
20
20
|
|
|
21
|
-
What is sent: a summary object of at most 1 KB per call to `openrouter.ai`, which is where Jev is published and the default route, or to `api.typesafe.ai` when `JEV_BACKEND=typesafe` selects the direct API. Either way, over HTTPS. It carries tool names, byte counts, relative paths, entry counts, short descriptions and a bounded sample of lines from the material being judged. `sanitize.mjs`
|
|
21
|
+
What is sent: a summary object of at most 1 KB per call to `openrouter.ai`, which is where Jev is published and the default route, or to `api.typesafe.ai` when `JEV_BACKEND=typesafe` selects the direct API. Either way, over HTTPS. It carries tool names, byte counts, relative paths, entry counts, short descriptions and a bounded sample of lines from the material being judged. `sanitize.mjs` builds the wire state and sanitizes question text. Known credential, token, JWT, email and URL patterns are redacted, including quoted credential values and multiline private-key blocks before sampling; workspace paths become relative and other absolute paths are replaced. Source and cluster questions use opaque IDs, with their descriptions inside the state budget. Required fields have explicit allocations: samples may be shortened, but missing objectives, missing samples or candidate lists that cannot fit cause local abstention, not an evidence-free classifier call. Redaction reduces exposure; it does not guarantee that a model-authored string carries nothing sensitive.
|
|
22
22
|
|
|
23
|
-
Earlier versions of this document said file contents and command output were "refused outright". That was never accurate and is corrected here. Deciding whether a tool result is spent, or whether a fetched page is addressing the agent, cannot be done from byte counts alone, so `outline()` has always sent a sample of the result's own lines: up to six from the head and two from the tail, each collapsed to at most 80 characters and passed through the same redaction as everything else. As of 0.27.0 it also samples up to four lines evenly spaced through the middle, because an instruction planted in a fetched page is rarely in its first six lines and a digest that could never contain one would ask system 7 a question its own state made unanswerable. So the accurate statement is: **a bounded, redacted sample of at most twelve short lines per result, inside a 1 KB total budget** — not the whole content, and not none of it. What the sample cannot include is anything beyond that budget
|
|
23
|
+
Earlier versions of this document said file contents and command output were "refused outright". That was never accurate and is corrected here. Deciding whether a tool result is spent, or whether a fetched page is addressing the agent, cannot be done from byte counts alone, so `outline()` has always sent a sample of the result's own lines: up to six from the head and two from the tail, each collapsed to at most 80 characters and passed through the same redaction as everything else. As of 0.27.0 it also samples up to four lines evenly spaced through the middle, because an instruction planted in a fetched page is rarely in its first six lines and a digest that could never contain one would ask system 7 a question its own state made unanswerable. So the accurate statement is: **a bounded, redacted sample of at most twelve short lines per result, inside a 1 KB total budget** — not the whole content, and not none of it. What the sample cannot include is anything beyond that budget. The ledger records offered and retained sample counts and whether required evidence was present; that does not establish that the sample contains every relevant fact.
|
|
24
24
|
|
|
25
|
-
Consent is separate from the switch. The first time any system would transmit, an interactive dialog names the endpoint, the shape of the data and the byte budget. Without an
|
|
25
|
+
Consent is separate from the switch. The first time any system would transmit, an interactive dialog names the endpoint, the shape of the data and the byte budget. Without an existing valid grant, a noninteractive run cannot start transmitting. A previously granted consent may be reused without a UI. The grant lives in `<agent-dir>/specpi/jev/consent.json` and is bound to the destination origin, including its transport scheme. Transport rejects redirects rather than forwarding evidence to an unapproved destination. Schema 2 requires renewed consent because the earlier dialog incorrectly promised never to send file contents or command output. It now explicitly discloses sampled content, the current request/objective, best-effort redaction and any non-HTTPS endpoint override. `/jev forget` revokes it.
|
|
26
26
|
|
|
27
|
-
Every
|
|
27
|
+
Every advisor request attempt appends one line to `<agent-dir>/specpi/jev/transmissions.jsonl`: timestamp, system, question keys, state byte count, a SHA-256 of the sanitized state and questions, latency and the gated outcome. Local evidence refusals are marked `sent: false`, consume no call budget, and are counted separately from calls. Effect tags distinguish shortening, warnings, recorded assessments, notifications, capability proposals and queued steering; queueing does not prove the model consumed a message. The payload itself is never written. `/jev ledger` reads it back, so "only digests are sent" is checkable rather than asserted. The ledger is user data and survives uninstall.
|
|
28
28
|
|
|
29
29
|
The advisor also keeps `<agent-dir>/specpi/jev/usage.json`, the running call count for the current session, written atomically with owner-only permissions. It exists because the ledger has no session boundary in it, so nothing outside the advisor's own process could say what *this* session had spent, and the count is what SpecPi Chat shows beside each budget. It is the one file in this layer meant to be read by another process, and it is deliberately the least interesting one: counts per system, the budgets they are counted against, a session identifier and two timestamps. No state, no questions, no answers, not even the ledger's digests. It is written only while the master switch is on, so a layer nobody has enabled leaves no trace of having been installed, and the last session's counts are kept rather than deleted at shutdown, because "this has never run" and "the session that just ended spent its whole budget" are different facts and a reader should be able to tell them apart. One file serves the directory, as the settings file does, so where several Pi sessions share an agent directory it describes whichever wrote to it last; it carries a session identifier, two timestamps and an active flag so a reader can say which rather than having to assume.
|
|
30
30
|
|
|
31
|
-
The advisor holds no authority. It never grants a capability, never calls a tool and never allows one. Its only blocking action is to refuse a capability-gap report that looks like it carries a credential, which asks the model to rewrite its own text and discards nothing. Failure is silent, not closed: a timeout, HTTP error, missing key, missing consent, an exhausted per-system or session budget, or ungated confidence produces no advice, and the existing code path runs unchanged. Tool-result retention can shorten a large
|
|
31
|
+
The advisor holds no authority. It never grants a capability, never calls a tool and never allows one. Its only blocking action is to refuse a capability-gap report that looks like it carries a credential, which asks the model to rewrite its own text and discards nothing. Failure is silent, not closed: a timeout, HTTP error, missing key, missing consent, an exhausted per-system or session budget, or ungated confidence produces no advice, and the existing code path runs unchanged. Tool-result retention can shorten a large observational result before it is appended; it never shortens a write, edit, shell or error result. Shell success does not establish that a command is read-only or safe to repeat. Other replacements still suggest re-reading; preserving an immutable original result is not implemented.
|
|
32
32
|
|
|
33
|
-
|
|
33
|
+
Task context comes from the workflow owner's validated active contract, not a rendered heading or a search through stored conversations. A bounded current-request fallback is refreshed when the task changes and cleared on branch/session changes. Delegation ranking adapts `packet.jobs[]` independently, uses each declared mode, and preserves the exact source multiset and ungated positions.
|
|
34
|
+
|
|
35
|
+
Gap triage runs only after local collection is enabled. Reports bind their task and session before any awaited consent or root lookup; collection is rechecked before transmission. Locked writes recheck task authority and strip the advisory assessment if Jev was disabled while waiting, without preventing otherwise authorized local collection. A bounded shortlist comes from sanitized wishlist observations, never from Pi state. A persisted `assessment` is a Jev opinion based on the reporting model's account, **not independent evidence**. It cannot change the original impact or suggested fix, canonical identity, priority, qualification, selection or retirement. Suggested cluster matches require a human merge decision. Model-authored report arguments cannot inject these assessments.
|
|
36
|
+
|
|
37
|
+
Two of the six systems can add fixed advice to model context, and both are off by default like everything else here.
|
|
34
38
|
|
|
35
39
|
**System 5, progress detection**, is the only one that can change what the model does. On a confident verdict that a session has stopped making progress it adds one fixed, code-written line at a turn boundary. It ships set to `notify`, which tells the person and cannot change the session; `message`, which appends the line for the model to read, is a deliberate opt-in. It is written once per session and never retracted, it never aborts or blocks, and it is suppressed entirely when the same call reports that a person would have to answer something first. Nothing the model produced ever enters the transcript: the line is chosen from a fixed table by failure mode.
|
|
36
40
|
|
|
@@ -15,7 +15,7 @@
|
|
|
15
15
|
import { randomUUID } from "node:crypto";
|
|
16
16
|
import { SYSTEM_NAMES, loadSettings } from "./config.mjs";
|
|
17
17
|
import { ensureConsent } from "./consent.mjs";
|
|
18
|
-
import { buildState } from "./sanitize.mjs";
|
|
18
|
+
import { buildQuestions, buildState } from "./sanitize.mjs";
|
|
19
19
|
import { ask } from "./client.mjs";
|
|
20
20
|
import { payloadDigest, record } from "./ledger.mjs";
|
|
21
21
|
import { writeUsage } from "./usage.mjs";
|
|
@@ -164,8 +164,27 @@ export function createBroker(options = {}) {
|
|
|
164
164
|
* is the only point where the answers and the audit line exist together; its `decision` is
|
|
165
165
|
* handed back so the caller does not gate the same answers twice.
|
|
166
166
|
*/
|
|
167
|
-
const request = async ({
|
|
167
|
+
const request = async ({
|
|
168
|
+
system,
|
|
169
|
+
state,
|
|
170
|
+
questions,
|
|
171
|
+
ctx,
|
|
172
|
+
root,
|
|
173
|
+
profile,
|
|
174
|
+
maxBytes,
|
|
175
|
+
timeoutMs,
|
|
176
|
+
signal = ctx?.signal,
|
|
177
|
+
decide,
|
|
178
|
+
apply,
|
|
179
|
+
isCurrent = () => true,
|
|
180
|
+
}) => {
|
|
181
|
+
const startedGeneration = generation;
|
|
182
|
+
const currentRequest = () => startedGeneration === generation && !signal?.aborted && isCurrent();
|
|
168
183
|
const settings = readSettings();
|
|
184
|
+
if (!currentRequest()) {
|
|
185
|
+
return { ok: false, reason: "context-changed", answers: {} };
|
|
186
|
+
}
|
|
187
|
+
|
|
169
188
|
if (!settings.master) {
|
|
170
189
|
return { ok: false, reason: "master-off", answers: {} };
|
|
171
190
|
}
|
|
@@ -188,6 +207,25 @@ export function createBroker(options = {}) {
|
|
|
188
207
|
return { ok: false, reason: "budget-exhausted", answers: {} };
|
|
189
208
|
}
|
|
190
209
|
|
|
210
|
+
const built = buildState(state, { root, maxBytes, profile });
|
|
211
|
+
const wireQuestions = buildQuestions(questions, root);
|
|
212
|
+
if (!built.ok || !wireQuestions) {
|
|
213
|
+
const reason = !built.ok ? built.reason : "invalid-questions";
|
|
214
|
+
// A local abstention is not a transmission and consumes no call budget. Counts and
|
|
215
|
+
// coverage only: never record the material that could not safely be sent.
|
|
216
|
+
write({
|
|
217
|
+
system,
|
|
218
|
+
sent: false,
|
|
219
|
+
ok: false,
|
|
220
|
+
reason,
|
|
221
|
+
outcome: reason,
|
|
222
|
+
applied: false,
|
|
223
|
+
coverage: built.coverage,
|
|
224
|
+
});
|
|
225
|
+
|
|
226
|
+
return { ok: false, reason, answers: {} };
|
|
227
|
+
}
|
|
228
|
+
|
|
191
229
|
const consented = await resolveConsent(ctx, SYSTEM_LABELS[system] ?? system);
|
|
192
230
|
if (!consented) {
|
|
193
231
|
return { ok: false, reason: "no-consent", answers: {} };
|
|
@@ -195,22 +233,41 @@ export function createBroker(options = {}) {
|
|
|
195
233
|
|
|
196
234
|
// Settings can change while the dialog is open, and a session can end under it.
|
|
197
235
|
const current = readSettings();
|
|
236
|
+
if (!currentRequest()) {
|
|
237
|
+
return { ok: false, reason: "context-changed", answers: {} };
|
|
238
|
+
}
|
|
239
|
+
|
|
198
240
|
if (!current.master || current.systems[system] !== true) {
|
|
199
241
|
return { ok: false, reason: "master-off", answers: {} };
|
|
200
242
|
}
|
|
201
243
|
|
|
202
|
-
|
|
203
|
-
|
|
244
|
+
// Parallel result hooks can wait on the same consent prompt. Reserve only after rechecking.
|
|
245
|
+
if ((usedBySystem.get(system) ?? 0) >= current.budgets[system]) {
|
|
246
|
+
return { ok: false, reason: "system-budget-exhausted", answers: {} };
|
|
247
|
+
}
|
|
248
|
+
|
|
249
|
+
if (callsUsed >= current.budgets.total) {
|
|
250
|
+
return { ok: false, reason: "budget-exhausted", answers: {} };
|
|
251
|
+
}
|
|
252
|
+
|
|
253
|
+
const questionKeys = Object.keys(wireQuestions);
|
|
204
254
|
callsUsed += 1;
|
|
205
255
|
usedBySystem.set(system, (usedBySystem.get(system) ?? 0) + 1);
|
|
206
|
-
|
|
207
|
-
|
|
256
|
+
let result;
|
|
257
|
+
try {
|
|
258
|
+
result = await transport(built.state, wireQuestions, { timeoutMs, signal });
|
|
259
|
+
} catch {
|
|
260
|
+
result = { ok: false, reason: "transport-error", answers: {} };
|
|
261
|
+
}
|
|
262
|
+
|
|
208
263
|
// The session can end under a call that was never awaited, which is the normal shape of a
|
|
209
264
|
// turn-level system: the payload has already left the machine, and the answer now belongs
|
|
210
265
|
// to a session that no longer exists. It must not be acted on. It must still be recorded --
|
|
211
266
|
// the ledger's whole claim is that every transmission appears in it, and a run that sent 44
|
|
212
267
|
// and logged 43 is how this was found. So the line is written either way and says which.
|
|
213
|
-
const
|
|
268
|
+
const mayApply = () => currentRequest() && readSettings().master && readSettings().systems[system] === true;
|
|
269
|
+
let stale = !mayApply();
|
|
270
|
+
const committedEffects = new Set();
|
|
214
271
|
|
|
215
272
|
// A gate that throws must not turn into a failed call: the caller's own catch would have
|
|
216
273
|
// swallowed it anyway, and recording it as unapplied is the truthful line.
|
|
@@ -218,17 +275,38 @@ export function createBroker(options = {}) {
|
|
|
218
275
|
if (!stale && result.ok && typeof decide === "function") {
|
|
219
276
|
try {
|
|
220
277
|
outcome = decide(result.answers) ?? { applied: false };
|
|
278
|
+
if (typeof apply === "function") {
|
|
279
|
+
const delivered = await apply(outcome.decision, {
|
|
280
|
+
isCurrent: mayApply,
|
|
281
|
+
recordEffect: (effect) => committedEffects.add(effect),
|
|
282
|
+
});
|
|
283
|
+
outcome = { ...outcome, applied: false, ...delivered };
|
|
284
|
+
}
|
|
221
285
|
} catch {
|
|
222
286
|
outcome = { applied: false, gateThrew: true };
|
|
223
287
|
}
|
|
224
288
|
}
|
|
225
289
|
|
|
290
|
+
// A later notification or report-rendering failure cannot undo an already committed effect.
|
|
291
|
+
if (committedEffects.size > 0) {
|
|
292
|
+
outcome = {
|
|
293
|
+
...outcome,
|
|
294
|
+
applied: true,
|
|
295
|
+
effects: [...new Set([...(outcome.effects ?? []), ...committedEffects])],
|
|
296
|
+
};
|
|
297
|
+
}
|
|
298
|
+
|
|
299
|
+
// An application may await a dialog or a locked wishlist write. Do not attribute its
|
|
300
|
+
// delivered effects to a different session if that session changed while awaiting it.
|
|
301
|
+
stale ||= !currentRequest() || !readSettings().master || readSettings().systems[system] !== true;
|
|
226
302
|
write({
|
|
227
303
|
system,
|
|
304
|
+
sent: true,
|
|
228
305
|
questionKeys,
|
|
306
|
+
coverage: built.coverage,
|
|
229
307
|
stateBytes: built.bytes,
|
|
230
308
|
stateTruncated: built.truncated,
|
|
231
|
-
payloadSha256: payloadDigest({ state: built.state, questions }),
|
|
309
|
+
payloadSha256: payloadDigest({ state: built.state, questions: wireQuestions }),
|
|
232
310
|
ok: result.ok,
|
|
233
311
|
reason: result.ok ? undefined : result.reason,
|
|
234
312
|
// A sent payload whose answer arrived too late to use. Distinguished from a refusal,
|
|
@@ -237,6 +315,7 @@ export function createBroker(options = {}) {
|
|
|
237
315
|
// Whether the advice changed anything, and what it saved when the change was a
|
|
238
316
|
// shortening. Zero is a real answer here and means "asked, and kept the result whole".
|
|
239
317
|
applied: outcome.applied === true,
|
|
318
|
+
effects: Array.isArray(outcome.effects) ? outcome.effects : [],
|
|
240
319
|
// And why not, when nothing changed. Without this a system that asks and never acts is
|
|
241
320
|
// indistinguishable from one whose gate can never be satisfied, which is the exact
|
|
242
321
|
// failure the calibration pass had to go looking for by hand.
|
|
@@ -74,30 +74,51 @@ function unavailable(reason) {
|
|
|
74
74
|
* A Noul has no confidence, and inventing one would let a caller gate on a number the model never
|
|
75
75
|
* reported, so it stays undefined.
|
|
76
76
|
*/
|
|
77
|
-
function normalizeAnswer(raw) {
|
|
78
|
-
if (!raw || typeof raw !== "object") {
|
|
77
|
+
function normalizeAnswer(raw, question) {
|
|
78
|
+
if (!raw || typeof raw !== "object" || !question) {
|
|
79
79
|
return undefined;
|
|
80
80
|
}
|
|
81
81
|
|
|
82
|
-
|
|
82
|
+
const probability = (value) => Number.isFinite(value) && value >= 0 && value <= 1;
|
|
83
|
+
const confidence = probability(raw.confidence) ? raw.confidence : undefined;
|
|
84
|
+
const distribution = raw.probabilities;
|
|
85
|
+
const validDistribution =
|
|
86
|
+
distribution &&
|
|
87
|
+
typeof distribution === "object" &&
|
|
88
|
+
Object.keys(distribution).length > 0 &&
|
|
89
|
+
Object.values(distribution).every(probability);
|
|
90
|
+
|
|
91
|
+
if (question.type === "noul" && probability(raw.noul)) {
|
|
83
92
|
return { kind: "noul", value: raw.noul, probabilities: undefined, confidence: undefined };
|
|
84
93
|
}
|
|
85
94
|
|
|
86
|
-
if (typeof raw.choice === "string") {
|
|
95
|
+
if (question.type === "choice" && typeof raw.choice === "string" && Object.hasOwn(question.criteria, raw.choice)) {
|
|
87
96
|
return {
|
|
88
97
|
kind: "choice",
|
|
89
98
|
value: raw.choice,
|
|
90
|
-
probabilities:
|
|
91
|
-
|
|
99
|
+
probabilities:
|
|
100
|
+
validDistribution &&
|
|
101
|
+
!Array.isArray(distribution) &&
|
|
102
|
+
Object.keys(distribution).length === Object.keys(question.criteria).length &&
|
|
103
|
+
Object.keys(distribution).every((key) => Object.hasOwn(question.criteria, key))
|
|
104
|
+
? distribution
|
|
105
|
+
: undefined,
|
|
106
|
+
confidence,
|
|
92
107
|
};
|
|
93
108
|
}
|
|
94
109
|
|
|
95
|
-
if (
|
|
110
|
+
if (
|
|
111
|
+
question.type === "score" &&
|
|
112
|
+
Number.isFinite(raw.score) &&
|
|
113
|
+
raw.score >= 0 &&
|
|
114
|
+
raw.score <= question.criteria.length - 1
|
|
115
|
+
) {
|
|
96
116
|
return {
|
|
97
117
|
kind: "score",
|
|
98
118
|
value: raw.score,
|
|
99
|
-
|
|
100
|
-
|
|
119
|
+
// The documented Score distribution is an object; older routes also return arrays.
|
|
120
|
+
probabilities: validDistribution ? distribution : undefined,
|
|
121
|
+
confidence,
|
|
101
122
|
};
|
|
102
123
|
}
|
|
103
124
|
|
|
@@ -126,6 +147,8 @@ export async function ask(state, questions, options = {}) {
|
|
|
126
147
|
try {
|
|
127
148
|
const response = await fetch(endpoint(), {
|
|
128
149
|
method: "POST",
|
|
150
|
+
// Consent covers this origin, not a redirect target that could receive the same body.
|
|
151
|
+
redirect: "error",
|
|
129
152
|
headers: {
|
|
130
153
|
"content-type": "application/json",
|
|
131
154
|
authorization: `Bearer ${key}`,
|
|
@@ -143,7 +166,7 @@ export async function ask(state, questions, options = {}) {
|
|
|
143
166
|
const body = await response.json();
|
|
144
167
|
const answers = {};
|
|
145
168
|
for (const [name, raw] of Object.entries(body?.answers ?? {})) {
|
|
146
|
-
const normalized = normalizeAnswer(raw);
|
|
169
|
+
const normalized = normalizeAnswer(raw, Object.hasOwn(questions, name) ? questions[name] : undefined);
|
|
147
170
|
if (normalized) {
|
|
148
171
|
answers[name] = normalized;
|
|
149
172
|
}
|
|
@@ -20,6 +20,12 @@ import { MAX_STATE_BYTES } from "./sanitize.mjs";
|
|
|
20
20
|
* and a TYPESAFE_BASE_URL override) names itself, and so the rule below is true rather than
|
|
21
21
|
* aspirational: a grant is keyed on this string, so repointing the client really does ask again.
|
|
22
22
|
*/
|
|
23
|
+
export const CONSENT_SCHEMA = 2;
|
|
24
|
+
|
|
25
|
+
export function endpointOrigin() {
|
|
26
|
+
return new URL(baseUrl()).origin;
|
|
27
|
+
}
|
|
28
|
+
|
|
23
29
|
export function endpointLabel() {
|
|
24
30
|
const base = baseUrl();
|
|
25
31
|
try {
|
|
@@ -43,12 +49,12 @@ export function loadConsent() {
|
|
|
43
49
|
}
|
|
44
50
|
|
|
45
51
|
const stored = JSON.parse(fs.readFileSync(file, "utf8"));
|
|
46
|
-
if (stored?.schema !==
|
|
52
|
+
if (stored?.schema !== CONSENT_SCHEMA || stored.granted !== true || typeof stored.endpoint !== "string") {
|
|
47
53
|
return undefined;
|
|
48
54
|
}
|
|
49
55
|
|
|
50
56
|
// A grant is for the endpoint it was given for. Repointing the client asks again.
|
|
51
|
-
return stored.endpoint === endpointLabel() ? stored : undefined;
|
|
57
|
+
return stored.endpoint === endpointLabel() && stored.origin === endpointOrigin() ? stored : undefined;
|
|
52
58
|
} catch {
|
|
53
59
|
return undefined;
|
|
54
60
|
}
|
|
@@ -65,9 +71,11 @@ export function saveConsent() {
|
|
|
65
71
|
}
|
|
66
72
|
|
|
67
73
|
const stored = {
|
|
68
|
-
|
|
74
|
+
// Earlier consent incorrectly excluded file/command samples. It cannot authorize them.
|
|
75
|
+
schema: CONSENT_SCHEMA,
|
|
69
76
|
granted: true,
|
|
70
77
|
endpoint: endpointLabel(),
|
|
78
|
+
origin: endpointOrigin(),
|
|
71
79
|
maxStateBytes: MAX_STATE_BYTES,
|
|
72
80
|
grantedAt: new Date().toISOString(),
|
|
73
81
|
};
|
|
@@ -89,15 +97,19 @@ export function consentPath() {
|
|
|
89
97
|
return consentFile();
|
|
90
98
|
}
|
|
91
99
|
|
|
92
|
-
export const CONSENT_TITLE = "Allow SpecPi to send task summaries to Jev?";
|
|
100
|
+
export const CONSENT_TITLE = "Allow SpecPi to send task summaries and text samples to Jev?";
|
|
93
101
|
|
|
94
102
|
export function consentBody(systemLabel) {
|
|
95
103
|
return [
|
|
96
104
|
`${systemLabel} wants to ask TypeSafe's Jev classifier a question about this session.`,
|
|
97
105
|
"",
|
|
98
|
-
`What is sent: a
|
|
99
|
-
|
|
100
|
-
"It
|
|
106
|
+
`What is sent: a state object of at most ${MAX_STATE_BYTES} bytes to ${endpointLabel()}, with classifier questions.`,
|
|
107
|
+
`Transport: ${new URL(baseUrl()).protocol === "https:" ? "HTTPS" : "not HTTPS (configured endpoint override)"}.`,
|
|
108
|
+
"It may include the current request or objective, relative paths, capability-gap summaries,",
|
|
109
|
+
"and short sampled lines from file contents, command output and fetched content.",
|
|
110
|
+
"Known credential, email, URL and outside-workspace path patterns are redacted before sending.",
|
|
111
|
+
"Redaction is best effort, not a guarantee that every sensitive detail is removed. No session history is read.",
|
|
112
|
+
"Collection stays local unless you separately allow this advisor transmission.",
|
|
101
113
|
"",
|
|
102
114
|
"Every call is recorded locally in transmissions.jsonl with a hash of exactly what was sent,",
|
|
103
115
|
"which you can read with /jev ledger. Turn this off at any time with /jev off.",
|
|
@@ -167,14 +167,14 @@ export function thresholdsFor(system) {
|
|
|
167
167
|
export function nounTrue(answer, system) {
|
|
168
168
|
const limits = thresholdsFor(system);
|
|
169
169
|
|
|
170
|
-
return answer?.kind === "noul" && answer.value >= limits.high;
|
|
170
|
+
return answer?.kind === "noul" && Number.isFinite(answer.value) && answer.value >= limits.high && answer.value <= 1;
|
|
171
171
|
}
|
|
172
172
|
|
|
173
173
|
/** True when the Noul is confidently no. Not the negation of nounTrue: the middle band is silence. */
|
|
174
174
|
export function nounFalse(answer, system) {
|
|
175
175
|
const limits = thresholdsFor(system);
|
|
176
176
|
|
|
177
|
-
return answer?.kind === "noul" && answer.value <= limits.low;
|
|
177
|
+
return answer?.kind === "noul" && Number.isFinite(answer.value) && answer.value >= 0 && answer.value <= limits.low;
|
|
178
178
|
}
|
|
179
179
|
|
|
180
180
|
function topTwo(probabilities) {
|
|
@@ -193,7 +193,15 @@ function topTwo(probabilities) {
|
|
|
193
193
|
* the margin is a live test rather than a branch that silently never runs.
|
|
194
194
|
*/
|
|
195
195
|
export function choiceValue(answer, system) {
|
|
196
|
-
if (
|
|
196
|
+
if (
|
|
197
|
+
answer?.kind !== "choice" ||
|
|
198
|
+
!Number.isFinite(answer.confidence) ||
|
|
199
|
+
answer.confidence > 1 ||
|
|
200
|
+
!answer.probabilities ||
|
|
201
|
+
Array.isArray(answer.probabilities) ||
|
|
202
|
+
!Object.hasOwn(answer.probabilities, answer.value) ||
|
|
203
|
+
!Object.values(answer.probabilities).every((value) => Number.isFinite(value) && value >= 0 && value <= 1)
|
|
204
|
+
) {
|
|
197
205
|
return undefined;
|
|
198
206
|
}
|
|
199
207
|
|
|
@@ -203,7 +211,7 @@ export function choiceValue(answer, system) {
|
|
|
203
211
|
}
|
|
204
212
|
|
|
205
213
|
const { first, second } = topTwo(answer.probabilities);
|
|
206
|
-
if (answer.probabilities
|
|
214
|
+
if (answer.probabilities[answer.value] !== first || first - second < limits.margin) {
|
|
207
215
|
return undefined;
|
|
208
216
|
}
|
|
209
217
|
|
|
@@ -215,7 +223,14 @@ export function choiceValue(answer, system) {
|
|
|
215
223
|
* returned level is the rounded band; callers compare against their own rubric.
|
|
216
224
|
*/
|
|
217
225
|
export function scoreLevel(answer, system) {
|
|
218
|
-
if (
|
|
226
|
+
if (
|
|
227
|
+
answer?.kind !== "score" ||
|
|
228
|
+
!Number.isFinite(answer.confidence) ||
|
|
229
|
+
answer.confidence > 1 ||
|
|
230
|
+
!Number.isFinite(answer.value) ||
|
|
231
|
+
answer.value < 0 ||
|
|
232
|
+
answer.value > 9
|
|
233
|
+
) {
|
|
219
234
|
return undefined;
|
|
220
235
|
}
|
|
221
236
|
|