pi-openai-toolkit 0.15.1 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +60 -68
- package/README.zh.md +58 -66
- package/docs/configuration.md +9 -3
- package/docs/internals.md +31 -6
- package/package.json +11 -9
- package/src/auto-mode/classifier.ts +161 -165
- package/src/auto-mode/extension.ts +691 -661
- package/src/auto-mode/model-selection-guard.ts +52 -0
- package/src/auto-mode/reviewer.ts +325 -335
- package/src/auto-mode/transcript.ts +135 -133
- package/src/compaction-projection.ts +35 -31
- package/src/context-management/manual-compact.ts +433 -0
- package/src/context-management/tools.ts +11 -2
- package/src/context-management/window-manager.ts +35 -9
- package/src/extension-runtime.ts +64 -30
- package/src/payload-rewrite.ts +34 -64
- package/src/pi-context-hook.ts +117 -116
- package/src/remote-v2-client.ts +108 -33
package/docs/internals.md
CHANGED
|
@@ -9,6 +9,7 @@ Developer-facing reference for [pi-openai-toolkit](../README.md). The README kee
|
|
|
9
9
|
- [Configuration resolution](#configuration-resolution)
|
|
10
10
|
- [Remote Context protocol](#remote-context-protocol)
|
|
11
11
|
- [Rollover lifecycle](#rollover-lifecycle)
|
|
12
|
+
- [Managed manual compact](#managed-manual-compact)
|
|
12
13
|
- [Remote Compaction v2 wire contract](#remote-compaction-v2-wire-contract)
|
|
13
14
|
- [Auto Mode TUI review renderer](#auto-mode-tui-review-renderer)
|
|
14
15
|
- [Artifacts and debugging](#artifacts-and-debugging)
|
|
@@ -20,7 +21,7 @@ Developer-facing reference for [pi-openai-toolkit](../README.md). The README kee
|
|
|
20
21
|
|
|
21
22
|
The single global document is loaded through [src/config.ts](../src/config.ts). [src/config/legacy.ts](../src/config/legacy.ts) owns unversioned compatibility; [src/config/v2.ts](../src/config/v2.ts) validates v2 scopes, resolves exact policy, and records leaf origins. [src/config/policy.ts](../src/config/policy.ts) defines effective values independently of persisted session data. See the [configuration reference](configuration.md) for supported fields and migration limits.
|
|
22
23
|
|
|
23
|
-
Feature entrypoints resolve once per public operation and pass that immutable snapshot through awaited helpers. A separately named compaction producer is resolved from the same decoded document for its own identity. Subsequent callbacks reload;
|
|
24
|
+
Feature entrypoints resolve once per public operation and pass that immutable snapshot through awaited helpers. A separately named compaction producer is resolved from the same decoded document for its own identity. Subsequent independent callbacks reload; a managed manual handoff retains its initiating context-policy snapshot until selection is restored. There is no general cross-event provider transaction, project lookup, singleton cache, or config state writer. Internal legacy-shaped engine adapters preserve compaction/search algorithms without exposing v2 parsing to consumers. Cross-feature gateway policy belongs to the resolver, not the context enable switch.
|
|
24
25
|
|
|
25
26
|
Selected invalid policy blocks dependent operations; unrelated model errors remain reportable without replacing the selected model. An engaged auto gate stays blocking when its policy becomes invalid. Image choices are checked before auth, uploads, or paid requests. The human `/toolkit-config` command inspects effective values, validation, or a migration candidate without action APIs or network access. Migration is preview-only; source bytes and unknown/dormant content stay untouched.
|
|
26
27
|
|
|
@@ -75,13 +76,13 @@ State invariants the lifecycle code must keep honest:
|
|
|
75
76
|
- The gate uses the live session/branch identity from `ctx.sessionManager`; cached manager identity must not hide a valid current-window pair, and old or foreign-session evidence must remain rejected.
|
|
76
77
|
- Every rollover carries the most recent successful checkpoint path in its handoff message. The optional `thread_hint` is supplemental; a hint failure must not erase the local receipt.
|
|
77
78
|
- A successful rollover closes the pre-rollover checkpoint phase. The first turn in the new window reads the receipt once and resumes the active user task; it must not immediately create another checkpoint or call `new_context` as part of that handoff. A later checkpoint is only for a later rollover, and `canRolloverFromCurrentWindow()` enforces it: a window entered by a rollover has to produce a tool result from a tool that is not one of `new_context`, `get_context_remaining`, `history`, or `notes` before it can roll over again.
|
|
78
|
-
- Pi 0.
|
|
79
|
+
- Pi 0.87 separates conversation-only `context` handlers from full-transcript `context_with_system` handlers. Toolkit uses the latter to trim windows and re-anchor the merged prompt/tool state with `preserveSystemHead()`. The output owns its leading system message; Pi does not repair it after this phase.
|
|
79
80
|
- When even that rebuild would lose declared tools, the trim is refused and the full message list is sent: a window that still carries the previous turns is usable, a window with no tools silently degrades into a chat the model can only narrate. Each refusal is reported once per window and lost-tool set through `takeProjectionDiagnostics()`, which writes a `context.projection.tool_loadout_shrink` compaction artifact and warns in the UI.
|
|
80
81
|
- The manager also remembers the head it last anchored. When a boundary compaction retires the transcript entry that declared the tools, the branch itself no longer contains a system message, and the remembered head is re-anchored instead of sending a tool-less window. The cache is request-time only, refreshed by every projection, and cleared by `reset()`.
|
|
81
82
|
- `new_context` has no force escape hatch: a persisted successful notes checkpoint in the current window is always required before rollover. The model must wait for the notes result to be persisted before retrying.
|
|
82
83
|
- After a marker is accepted for sending, the manager keeps a session/window-anchored pending guard until that exact target marker is persisted or the session changes. A duplicate `new_context` during this persistence gap returns `started: false` and sends no second marker; projected/in-memory messages do not retire the guard.
|
|
83
84
|
- Budget checks are skipped until the current window has produced its own assistant usage. Acting on the previous window's usage anchor would burn the once-per-window reminder on a false alarm.
|
|
84
|
-
-
|
|
85
|
+
- Preparing a boundary never consumes its scheduled trim. The manager reconciles the latest persisted compaction and branch state, so only a durable matching boundary consumes it. Missing/late callbacks, aborted preparation and navigation across branches sharing a marker remain retryable. Automatic threshold/overflow and toolkit-owned manual maintenance only trim a matching scheduled window; a user manual `/compact` follows the managed handoff below.
|
|
85
86
|
- A window boundary is persisted as a `codex-context-window` custom message. `session_start` replays boundaries from the branch and rebuilds identity after forks.
|
|
86
87
|
- Retiring windows is a projection-level act: the session branch keeps every retired window until a compaction consumes the scheduled trim. `evaluateWindowBulk()` measures that gap and compares it with the model about to receive it, `decideBulkCliffAction()` maps the report onto the `leaveManagedMode` policy (`warn` default, `compact`), and `decideBulkCloseOut()` only allows the automatic compaction where a compaction can actually finish: never in print mode (the process exits with the turn and would kill the summary), and at turn end only while a rollover trim is queued, because the window policy cancels every other compaction. Each window and model is surfaced once, and every decision writes a `window-bulk.<action>` compaction artifact with its trigger.
|
|
87
88
|
- The close-out runs from `model_select` and `agent_settled`, never from `before_agent_start`: `ExtensionContext.compact()` aborts the running turn, and a hook inside turn processing is exactly the busy lane it must not fight.
|
|
@@ -89,17 +90,39 @@ State invariants the lifecycle code must keep honest:
|
|
|
89
90
|
|
|
90
91
|
---
|
|
91
92
|
|
|
93
|
+
### Managed manual compact
|
|
94
|
+
|
|
95
|
+
[src/context-management/manual-compact.ts](../src/context-management/manual-compact.ts) owns manual checkpoint duty. It uses public Pi lifecycle and selection APIs; it does not replace the built-in command or patch the scheduler. [src/context-management/window-manager.ts](../src/context-management/window-manager.ts) remains the sole owner of notes pairing, window identity, projection and trim.
|
|
96
|
+
|
|
97
|
+
1. An active managed `session_before_compact` with reason `manual` validates the source/target policies from one snapshot, actual permitted tools, remaining capacity, backend/account scope and approval eligibility. It records the original model/thinking and cancels the native compact attempt. Internal window-bulk maintenance has a separate in-process owner even though Pi also labels it `manual`.
|
|
98
|
+
2. The matching `session_compact_failed` callback launches duty only after Pi is idle. Public `setModel` changes the session selection, and a visible `sendUserMessage` rebuilds the target's normal prompt/tools. Delivery is observed through `message_start`; return from the fire-and-forget send API is not proof. A delivery timeout restores an abandoned idle selection.
|
|
99
|
+
3. The existing `new_context` gate additionally requires a successful paired notes result after this operation and any later delivered user message. It records the exact target window before queueing a context-only marker, and returns a terminating result. Ordinary tools are blocked during checkpoint duty. Normal `new_context` outside this operation is unchanged.
|
|
100
|
+
4. A terminating result alone does not stop mixed batches or queued messages. At `turn_end`, Toolkit requests abort without awaiting idle; context/request guards also refuse an unsafe continuation. The standard Responses and native Codex transports must not send an already-aborted continuation. Pi can deliver queued user content during that stop; it remains persisted and ordered for the original model.
|
|
101
|
+
5. At `agent_settled`, after Pi has flushed custom messages, the controller verifies the exact durable marker and fresh checkpoint, restores original model/thinking, and sends one visible receipt-reading continuation. Pi 0.87 defers that continuation until every settled handler has returned, so later observers still see an idle session. Restoration finishes before the next provider request; marker `message_end` alone is not proof of persistence.
|
|
102
|
+
|
|
103
|
+
Versioned `pi-openai-toolkit:manual-compact` custom entries record operation/session/branch/window identities, exact model keys, original thinking and phase. They contain no credentials, transcript copy or note contents. Reload/navigation reads the current branch and only recovers still-owned selection; it never retries notes, rollover or paid inference. A newer user model/thinking selection ends ownership. Failed restoration stays durable and blocks further provider requests until recovery or an explicit user selection.
|
|
104
|
+
|
|
105
|
+
An engaged Auto Mode gate grants a session/operation-scoped selection lease through [src/auto-mode/model-selection-guard.ts](../src/auto-mode/model-selection-guard.ts). An ineligible target is refused. While switching, a later eligibility change cannot silently disengage the gate: calls remain blocked. The lease neither enables Auto Mode nor widens its allowlist.
|
|
106
|
+
|
|
107
|
+
The underlying native compact promise still rejects its intentional cancellation. Empty/already-compacted sessions can fail before the extension hook. This is a separate handoff operation, not a fabricated successful native compaction. An observed user cancellation before the owned checkpoint stop restores selection without automatically resuming. Tests use the official Pi runtime, disk-backed sessions, isolated HOME/settings and synthetic Responses/native Codex traffic in [test/pi-managed-compact.test.ts](../test/pi-managed-compact.test.ts); they assert every request's model/thinking, marker/note order, retained tool head and queue preservation.
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
92
111
|
### Remote Compaction v2 wire contract
|
|
93
112
|
|
|
94
113
|
v2 is what an uncovered session gets. Any model that Remote Context declines, such as a gateway without an exact transport opt-in, can still compact through v2 when its API appears in `context.remoteCompaction.apis`. The two strategies never compete for one session.
|
|
95
114
|
|
|
96
115
|
A `compaction_trigger` item appended to the live streaming request yields one output item of `type: "compaction"` with non-empty `encrypted_content`, stored in `CompactionEntry.details.compactedWindow`. On later requests the opaque checkpoint is replayed ahead of live turns, with no text summary. Replay fails closed: if the summary anchor cannot be located, the request is aborted with a notification and a content-free failure artifact. The sentinel-only payload is never sent.
|
|
97
116
|
|
|
117
|
+
The incremental SSE consumer validates the first terminal event and reconciles its checkpoint with prior item evidence. A valid completed frame ends the operation even if the HTTP body stays open; reader cancellation is best-effort cleanup and is not awaited. Incomplete frames at EOF, conflicting checkpoint evidence and terminal errors cannot become successful checkpoints.
|
|
118
|
+
|
|
98
119
|
A v2 response with a missing or empty checkpoint is never stored, and the `nativeFallback` tier is skipped. Pi's own threshold drives the next attempt, which may use `remoteCompactModel` when configured. `remoteCompactModel` must resolve to the same effective base URL as the active model.
|
|
99
120
|
|
|
100
|
-
Pi 0.
|
|
121
|
+
Pi 0.87 makes `SessionManager` the source of model-visible context. An omitted `context.remoteCompaction.inputSource` keeps `"legacy"` behavior: first compaction reads `buildSessionProjection()` (falling back to Pi's preparation if unavailable), and recursion uses the opaque window plus the canonically projected branch tail. Both paths honor persisted context edits. Legacy mode still bypasses request-time extension transforms.
|
|
101
122
|
|
|
102
|
-
Setting `context.remoteCompaction.inputSource: "pi-context-hook"` opts into a narrow runtime bridge around Pi's public `ExtensionRunner.createContext()`. It adds a non-enumerable `ctx.projectContextForCompaction(messages)` method backed by `ExtensionRunner.emitContext()`.
|
|
123
|
+
Setting `context.remoteCompaction.inputSource: "pi-context-hook"` opts into a narrow runtime bridge around Pi's public `ExtensionRunner.createContext()`. It adds a non-enumerable `ctx.projectContextForCompaction(messages)` method backed by `ExtensionRunner.emitContext()`. This runs the ordered conversation-only and full-transcript phases on the canonical projection. If the bridge, session projection, or recursive summary anchor is unavailable, the extension cancels rather than sending unprojected history.
|
|
124
|
+
|
|
125
|
+
Retained-message matching uses the SDK projection and source entry IDs, including omissions and replacements made before compaction. Older retained compaction entries contribute no second summary. A compaction whose kept boundary is its own ID retains no earlier messages. Edits to post-checkpoint input are applied to the live tail; an edit targeting history already sealed in an opaque checkpoint blocks replay and recursive compaction with `checkpoint-context-edited`. The encrypted checkpoint cannot be selectively rewritten. Start a new session or navigate before that checkpoint to rebuild from editable history; the raw log remains unchanged.
|
|
103
126
|
|
|
104
127
|
New checkpoints record `inputProvenance: "pi-context-hook-v1"` or `"legacy-raw-context-v1"`, and replay/recursion reject missing or mode-mismatched markers without searching past the latest compaction. Retained `role: "custom"` messages are optional during replay because they may be changed or removed by context hooks; required user, assistant, and complete tool-call/result content remains ordered and fail-closed.
|
|
105
128
|
|
|
@@ -107,7 +130,9 @@ New checkpoints record `inputProvenance: "pi-context-hook-v1"` or `"legacy-raw-c
|
|
|
107
130
|
|
|
108
131
|
### Auto Mode TUI review renderer
|
|
109
132
|
|
|
110
|
-
|
|
133
|
+
Approval state is scoped to actual user-message delivery. Full structured user content is SHA-256 hashed independently of transcript truncation, and generation-owned classifier completions cannot overwrite newer samples. The bounded transcript selects recent user instructions first and reports omissions. The blocking reviewer and its read-only evidence loop share one deadline that also races uncooperative dependencies; timeout/cancellation is an unavailable review, never approval or a safety denial.
|
|
134
|
+
|
|
135
|
+
Auto Mode owns the review decision, but Pi owns the built-in tool-block component. Pi 0.87 exports `ToolExecutionComponent` without a public decorator interface, so `src/auto-mode/tool-review-tui.ts` installs a narrow, idempotent compatibility patch on its `render()` method. The patch only reads the component's existing tool-call ID and appends one bounded, single-line status after the normal block output; it never changes tool execution, event ordering, or provider payloads.
|
|
111
136
|
|
|
112
137
|
The extension updates an ephemeral per-call state through the same `tool_call` lifecycle that performs the review:
|
|
113
138
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-openai-toolkit",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.17.0",
|
|
4
4
|
"private": false,
|
|
5
5
|
"type": "module",
|
|
6
6
|
"engines": {
|
|
@@ -63,6 +63,7 @@
|
|
|
63
63
|
"src/context-management/window-request.ts",
|
|
64
64
|
"src/context-management/window-bulk.ts",
|
|
65
65
|
"src/context-management/window-manager.ts",
|
|
66
|
+
"src/context-management/manual-compact.ts",
|
|
66
67
|
"src/context-management/codex-provider.ts",
|
|
67
68
|
"src/context-management/history-notes.ts",
|
|
68
69
|
"src/context-management/tools.ts",
|
|
@@ -80,6 +81,7 @@
|
|
|
80
81
|
"src/auto-mode/extension.ts",
|
|
81
82
|
"src/auto-mode/tool-review-tui.ts",
|
|
82
83
|
"src/auto-mode/policy.ts",
|
|
84
|
+
"src/auto-mode/model-selection-guard.ts",
|
|
83
85
|
"src/auto-mode/prompt.ts",
|
|
84
86
|
"src/auto-mode/transcript.ts",
|
|
85
87
|
"src/auto-mode/reviewer.ts",
|
|
@@ -118,16 +120,16 @@
|
|
|
118
120
|
"typecheck": "tsc -p tsconfig.check.json"
|
|
119
121
|
},
|
|
120
122
|
"peerDependencies": {
|
|
121
|
-
"@earendil-works/pi-agent-core": ">=0.
|
|
122
|
-
"@earendil-works/pi-ai": ">=0.
|
|
123
|
-
"@earendil-works/pi-coding-agent": ">=0.
|
|
124
|
-
"@earendil-works/pi-tui": ">=0.
|
|
123
|
+
"@earendil-works/pi-agent-core": ">=0.87.0",
|
|
124
|
+
"@earendil-works/pi-ai": ">=0.87.0",
|
|
125
|
+
"@earendil-works/pi-coding-agent": ">=0.87.0",
|
|
126
|
+
"@earendil-works/pi-tui": ">=0.87.0"
|
|
125
127
|
},
|
|
126
128
|
"devDependencies": {
|
|
127
|
-
"@earendil-works/pi-agent-core": "0.
|
|
128
|
-
"@earendil-works/pi-ai": "0.
|
|
129
|
-
"@earendil-works/pi-coding-agent": "0.
|
|
130
|
-
"@earendil-works/pi-tui": "0.
|
|
129
|
+
"@earendil-works/pi-agent-core": "0.87.0",
|
|
130
|
+
"@earendil-works/pi-ai": "0.87.0",
|
|
131
|
+
"@earendil-works/pi-coding-agent": "0.87.0",
|
|
132
|
+
"@earendil-works/pi-tui": "0.87.0",
|
|
131
133
|
"@types/node": "24.12.4",
|
|
132
134
|
"bun": "1.2.23",
|
|
133
135
|
"typescript": "5.9.3"
|
|
@@ -1,165 +1,161 @@
|
|
|
1
|
-
import
|
|
2
|
-
import type {
|
|
3
|
-
import {
|
|
4
|
-
import {
|
|
5
|
-
import {
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
*
|
|
20
|
-
*
|
|
21
|
-
*
|
|
22
|
-
*
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
}
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
tracker.
|
|
39
|
-
tracker.
|
|
40
|
-
tracker.
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
*
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
}
|
|
56
|
-
|
|
57
|
-
export function
|
|
58
|
-
tracker.
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
const
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
if (
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
modelSpec
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
}
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
}
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
.
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
clearTimeout(timer);
|
|
163
|
-
params.signal?.removeEventListener("abort", abortFromCaller);
|
|
164
|
-
}
|
|
165
|
-
}
|
|
1
|
+
import { createHash } from "node:crypto";
|
|
2
|
+
import type { Api, Model } from "@earendil-works/pi-ai";
|
|
3
|
+
import type { ModelRegistry } from "@earendil-works/pi-coding-agent";
|
|
4
|
+
import { parseModelSpec } from "../runtime";
|
|
5
|
+
import { CLASSIFIER_SYSTEM_PROMPT } from "./prompt";
|
|
6
|
+
import { boundReviewText, type ClassifierRisk, type FastDecisionReason } from "./types";
|
|
7
|
+
|
|
8
|
+
/** One completed classification of the trajectory, tagged to a tool-call position. */
|
|
9
|
+
export type ScoreRecord = {
|
|
10
|
+
risk: ClassifierRisk;
|
|
11
|
+
/** Tool-call counter value when the sample was taken. */
|
|
12
|
+
scoredAtCall: number;
|
|
13
|
+
/** Fingerprint of the user turns the score was based on. */
|
|
14
|
+
authorizationVersion: string;
|
|
15
|
+
sampledAt: number;
|
|
16
|
+
};
|
|
17
|
+
|
|
18
|
+
/**
|
|
19
|
+
* Mutable per-session tracker for the non-blocking pre-scorer.
|
|
20
|
+
*
|
|
21
|
+
* The counters reproduce Codex's staleness rule: the agent keeps running while a
|
|
22
|
+
* classification is in flight, so a score is only trustworthy if it is recent
|
|
23
|
+
* enough and nothing the user said has changed underneath it.
|
|
24
|
+
*/
|
|
25
|
+
export type ScoreTracker = {
|
|
26
|
+
latestCallIndex: number;
|
|
27
|
+
latestScoredIndex: number;
|
|
28
|
+
latestFailedIndex: number;
|
|
29
|
+
score?: ScoreRecord;
|
|
30
|
+
};
|
|
31
|
+
|
|
32
|
+
export function createScoreTracker(): ScoreTracker {
|
|
33
|
+
return { latestCallIndex: 0, latestScoredIndex: 0, latestFailedIndex: 0 };
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
/** Reset everything: a new turn or a disengaged mode invalidates prior scoring. */
|
|
37
|
+
export function resetScoreTracker(tracker: ScoreTracker): void {
|
|
38
|
+
tracker.latestCallIndex = 0;
|
|
39
|
+
tracker.latestScoredIndex = 0;
|
|
40
|
+
tracker.latestFailedIndex = 0;
|
|
41
|
+
tracker.score = undefined;
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
/**
|
|
45
|
+
* Fingerprint of the user-authored conversation. Any new or edited user turn makes
|
|
46
|
+
* an existing score stale, because the score's authorization premise changed.
|
|
47
|
+
*/
|
|
48
|
+
export function authorizationVersion(userText: string): string {
|
|
49
|
+
return createHash("sha256").update(userText).digest("hex");
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
export function recordScoredCall(tracker: ScoreTracker, score: ScoreRecord): void {
|
|
53
|
+
tracker.score = score;
|
|
54
|
+
tracker.latestScoredIndex = Math.max(tracker.latestScoredIndex, score.scoredAtCall);
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
export function recordFailedCall(tracker: ScoreTracker, scoredAtCall: number): void {
|
|
58
|
+
tracker.latestFailedIndex = Math.max(tracker.latestFailedIndex, scoredAtCall);
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
export type FastDecisionInput = {
|
|
62
|
+
tracker: ScoreTracker;
|
|
63
|
+
/** Tool-call counter value for the call being gated. */
|
|
64
|
+
currentCallIndex: number;
|
|
65
|
+
authorizationVersion: string;
|
|
66
|
+
maxLag: number;
|
|
67
|
+
/** Retries after a denial and widened gates must not take the fast path. */
|
|
68
|
+
requiresSynchronousReview?: boolean;
|
|
69
|
+
};
|
|
70
|
+
|
|
71
|
+
/**
|
|
72
|
+
* Decide whether a cached classification can satisfy this gated call without
|
|
73
|
+
* running the blocking reviewer. Anything other than `low_risk` defers to the
|
|
74
|
+
* synchronous review; the classifier never denies an action on its own.
|
|
75
|
+
*/
|
|
76
|
+
export function fastApprovalEligible(input: FastDecisionInput): {
|
|
77
|
+
eligible: boolean;
|
|
78
|
+
reason: FastDecisionReason;
|
|
79
|
+
} {
|
|
80
|
+
const defer = (reason: FastDecisionReason) => ({ eligible: false, reason });
|
|
81
|
+
|
|
82
|
+
if (input.requiresSynchronousReview) return defer("requires_synchronous_review");
|
|
83
|
+
if (input.tracker.latestFailedIndex > input.tracker.latestScoredIndex) return defer("scoring_failure");
|
|
84
|
+
|
|
85
|
+
const score = input.tracker.score;
|
|
86
|
+
if (!score) return defer("missing_score");
|
|
87
|
+
|
|
88
|
+
const lag = input.currentCallIndex - score.scoredAtCall;
|
|
89
|
+
if (lag < 0 || lag > input.maxLag) return defer("stale_score");
|
|
90
|
+
if (score.authorizationVersion !== input.authorizationVersion) return defer("authorization_changed");
|
|
91
|
+
if (score.risk === "low") return { eligible: true, reason: "low_risk" };
|
|
92
|
+
return defer("elevated_risk");
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
export type ClassificationResult =
|
|
96
|
+
| { kind: ClassifierRisk; model: string }
|
|
97
|
+
| { kind: "failed"; reason: string };
|
|
98
|
+
|
|
99
|
+
/**
|
|
100
|
+
* Run one non-blocking classification. Output is a single token; anything else is a
|
|
101
|
+
* failed sample rather than a verdict, so a confused classifier costs a review and
|
|
102
|
+
* never an approval or a denial.
|
|
103
|
+
*/
|
|
104
|
+
export async function classifyTrajectory(params: {
|
|
105
|
+
registry: Pick<ModelRegistry, "find" | "complete">;
|
|
106
|
+
modelSpec: string;
|
|
107
|
+
prompt: string;
|
|
108
|
+
timeoutMs: number;
|
|
109
|
+
signal?: AbortSignal;
|
|
110
|
+
}): Promise<ClassificationResult> {
|
|
111
|
+
const parsed = parseModelSpec(params.modelSpec);
|
|
112
|
+
if (!parsed) {
|
|
113
|
+
return { kind: "failed", reason: `autoMode.classifier.model "${params.modelSpec}" is not provider/model-id.` };
|
|
114
|
+
}
|
|
115
|
+
const model = params.registry.find(parsed.provider, parsed.modelId) as Model<Api> | undefined;
|
|
116
|
+
if (!model) {
|
|
117
|
+
return { kind: "failed", reason: `Classifier model ${params.modelSpec} is not available.` };
|
|
118
|
+
}
|
|
119
|
+
const label = boundReviewText(`${model.provider}/${model.id}`, 256);
|
|
120
|
+
|
|
121
|
+
const controller = new AbortController();
|
|
122
|
+
const abortFromCaller = () => controller.abort();
|
|
123
|
+
if (params.signal?.aborted) controller.abort();
|
|
124
|
+
else params.signal?.addEventListener("abort", abortFromCaller);
|
|
125
|
+
const timer = setTimeout(() => controller.abort(), params.timeoutMs);
|
|
126
|
+
|
|
127
|
+
try {
|
|
128
|
+
const response = await params.registry.complete(
|
|
129
|
+
model,
|
|
130
|
+
{
|
|
131
|
+
systemPrompt: CLASSIFIER_SYSTEM_PROMPT,
|
|
132
|
+
messages: [
|
|
133
|
+
{
|
|
134
|
+
role: "user",
|
|
135
|
+
content: [{ type: "text", text: params.prompt }],
|
|
136
|
+
timestamp: Date.now(),
|
|
137
|
+
},
|
|
138
|
+
],
|
|
139
|
+
},
|
|
140
|
+
{ signal: controller.signal, cacheRetention: "none" },
|
|
141
|
+
);
|
|
142
|
+
if (controller.signal.aborted) return { kind: "failed", reason: "Classifier was cancelled or timed out." };
|
|
143
|
+
if (response.stopReason === "error" || response.stopReason === "aborted") {
|
|
144
|
+
return { kind: "failed", reason: response.errorMessage ?? `Classifier stopped: ${response.stopReason}` };
|
|
145
|
+
}
|
|
146
|
+
const text = response.content
|
|
147
|
+
.filter((block): block is { type: "text"; text: string } => block.type === "text")
|
|
148
|
+
.map((block) => block.text)
|
|
149
|
+
.join("")
|
|
150
|
+
.trim()
|
|
151
|
+
.toLowerCase();
|
|
152
|
+
if (/^high\b/.test(text)) return { kind: "high", model: label };
|
|
153
|
+
if (/^low\b/.test(text)) return { kind: "low", model: label };
|
|
154
|
+
return { kind: "failed", reason: `Classifier returned unreadable output: ${boundReviewText(text, 80)}` };
|
|
155
|
+
} catch (error) {
|
|
156
|
+
return { kind: "failed", reason: error instanceof Error ? error.message : String(error) };
|
|
157
|
+
} finally {
|
|
158
|
+
clearTimeout(timer);
|
|
159
|
+
params.signal?.removeEventListener("abort", abortFromCaller);
|
|
160
|
+
}
|
|
161
|
+
}
|