auto-model-router 0.2.9 → 0.2.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.omp-plugin/marketplace.json +2 -2
- package/README.md +16 -1
- package/docs/AGENTDOX-BRIDGE.md +92 -40
- package/omp-extension/embed-logic.ts +13 -3
- package/package.json +1 -1
- package/src/context/bridge.ts +57 -2
- package/src/context/types.ts +13 -0
- package/src/server/turn.ts +22 -6
- package/test/context-bridge.test.ts +94 -1
- package/test/embed-logic.test.ts +10 -3
- package/test/turn.test.ts +78 -0
- package/tools/agentdox-e2e.ts +47 -0
|
@@ -7,14 +7,14 @@
|
|
|
7
7
|
},
|
|
8
8
|
"metadata": {
|
|
9
9
|
"description": "auto-model-router: a local cost/complexity-aware model router for Oh My Pi, backed by OpenRouter",
|
|
10
|
-
"version": "0.2.
|
|
10
|
+
"version": "0.2.11",
|
|
11
11
|
"pluginRoot": "."
|
|
12
12
|
},
|
|
13
13
|
"plugins": [
|
|
14
14
|
{
|
|
15
15
|
"name": "auto-model-router",
|
|
16
16
|
"description": "Local cost/complexity-aware model router for Oh My Pi, backed by OpenRouter. Runs in-process, routes per turn by price and task complexity, with budget caps, mid-stream escalation, and cache-aware hysteresis.",
|
|
17
|
-
"version": "0.2.
|
|
17
|
+
"version": "0.2.11",
|
|
18
18
|
"author": {
|
|
19
19
|
"name": "drewappling",
|
|
20
20
|
"email": "drewappling@gmail.com"
|
package/README.md
CHANGED
|
@@ -580,11 +580,19 @@ and brief:
|
|
|
580
580
|
```bash
|
|
581
581
|
export AGENTDOX_URL=http://localhost:3003
|
|
582
582
|
export AGENTDOX_TOKEN=<pat with read+write on the scope>
|
|
583
|
-
export AGENTDOX_SCOPE=ashlands #
|
|
583
|
+
export AGENTDOX_SCOPE=ashlands # fallback only; see below
|
|
584
584
|
```
|
|
585
585
|
|
|
586
586
|
Setting a URL and a token is enough to turn it on.
|
|
587
587
|
|
|
588
|
+
The scope is **derived per workspace** from the directory basename
|
|
589
|
+
(`E:/projects/ashlands` → `ashlands`), and that derivation wins. `AGENTDOX_SCOPE` /
|
|
590
|
+
`context.defaultScope` is only a fallback for workspaces it cannot resolve, because one router
|
|
591
|
+
install serves every project on the machine — a slug pinned there would be sent for all of
|
|
592
|
+
them, injecting one project's context into another's work. A single configured token also
|
|
593
|
+
grants only the scopes it was minted for; for any other project the bridge degrades to inert
|
|
594
|
+
rather than writing somewhere wrong.
|
|
595
|
+
|
|
588
596
|
### It does not cost you a cache miss per turn
|
|
589
597
|
|
|
590
598
|
The context block sits at the front of the prompt, so re-fetching it every turn
|
|
@@ -620,6 +628,13 @@ shows which model produced which turn. Those messages feed back into the next
|
|
|
620
628
|
`context_assemble`, so the model you switch *to* inherits what the model you
|
|
621
629
|
switched *from* actually did.
|
|
622
630
|
|
|
631
|
+
A recorded turn is the whole **user-visible** turn, not one record per upstream
|
|
632
|
+
request. An agentic turn is a loop of dispatches — each tool round-trip finishes
|
|
633
|
+
with `tool_calls` and emits almost no text, and the last user message does not
|
|
634
|
+
move while the loop runs. So the router buffers the assistant's narration across
|
|
635
|
+
the loop and writes it once, together with the closing synthesis, when the
|
|
636
|
+
assistant actually yields back to the user.
|
|
637
|
+
|
|
623
638
|
Write-backs are queued, bounded, and never awaited: agentdox is an enrichment,
|
|
624
639
|
not a dependency. If it is unreachable the turn routes and dispatches normally,
|
|
625
640
|
and a pinned block keeps being served.
|
package/docs/AGENTDOX-BRIDGE.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# agentdox bridge — handoff
|
|
2
2
|
|
|
3
|
-
**Status:** implemented, typechecks clean,
|
|
4
|
-
through omp.
|
|
3
|
+
**Status:** implemented, typechecks clean, 439 tests pass, injection verified end-to-end
|
|
4
|
+
through omp. The write-back faults in §5 and §6 are **fixed**; `context.recordTurns` is on.
|
|
5
5
|
|
|
6
6
|
Design rationale (why it is built this way):
|
|
7
7
|
`E:/projects/agentdox/docs/architecture/router-context-bridge.md`.
|
|
@@ -86,44 +86,96 @@ Two more traps hit during this work:
|
|
|
86
86
|
- Long-running interactive omp sessions hold their own embedded routers from whenever they
|
|
87
87
|
started. Check `Get-Process omp` before trusting a result.
|
|
88
88
|
|
|
89
|
-
## 5.
|
|
90
|
-
|
|
91
|
-
**
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
89
|
+
## 5. FIXED — one record per dispatch, not per turn
|
|
90
|
+
|
|
91
|
+
**Symptom:** through omp the recorded assistant turn was near-empty —
|
|
92
|
+
`assistantChars=4` (literally `" high"`) while omp displayed several paragraphs. Session and
|
|
93
|
+
model attribution (`refs: ["model:…", "tier:…"]`) were always correct; only the assistant
|
|
94
|
+
*content* was wrong.
|
|
95
|
+
|
|
96
|
+
**Root cause — none of the three leads originally listed here.** The text was not
|
|
97
|
+
under-captured; the *wrong requests* were being recorded. A user-visible turn is not one
|
|
98
|
+
upstream request, it is a whole tool loop of them. Live ledger proof, one conversation key,
|
|
99
|
+
`wasted=0` and `attempt=0` on every row:
|
|
100
|
+
|
|
101
|
+
| dispatch | `finish_reason` | `toolLoopDepth` | completion tokens |
|
|
102
|
+
| --- | --- | --- | --- |
|
|
103
|
+
| 1 | `tool_calls` | 0 | 339 |
|
|
104
|
+
| 2–6 | `tool_calls` | 2, 4, 6, 8, 10 | 91, 68, 198, 78, 44 |
|
|
105
|
+
| 7 | **`stop`** | 12 | **596** |
|
|
106
|
+
| 8 | `tool_calls` | 0 *(next turn)* | 167 |
|
|
107
|
+
|
|
108
|
+
Each tool round-trip is its own dispatch, finishing with `tool_calls` and emitting almost no
|
|
109
|
+
`text` — the payload is tool calls. `" high"` was a stray word of preamble, a *complete*
|
|
110
|
+
record of a fragment rather than a truncated answer. Only the final `stop` dispatch carries
|
|
111
|
+
the synthesis. `recordTurn` fired on all ~13, and the last writer won.
|
|
112
|
+
|
|
113
|
+
The same root cause explains half the duplication in §7: `lastUserText` walks back to the last
|
|
114
|
+
`user` message, which does **not** move while a tool loop runs, so the identical user text was
|
|
115
|
+
appended once per round-trip too.
|
|
116
|
+
|
|
117
|
+
**Fix.** `TurnRecord` gained `turnEnded` (`finishReason !== "tool_calls"`, set in
|
|
118
|
+
`src/server/turn.ts`). The bridge buffers assistant fragments per conversation in a
|
|
119
|
+
process-local map and flushes **once**, when the assistant yields back to the user, writing
|
|
120
|
+
the loop's narration plus the closing synthesis as one message. Bounded by
|
|
121
|
+
`MAX_PENDING_CHARS` / `MAX_PENDING_CONVERSATIONS`, since a turn that dies without a terminal
|
|
122
|
+
dispatch never flushes. The terminal dispatch is appended past the char cap, so the model's
|
|
123
|
+
actual answer is never what gets dropped.
|
|
124
|
+
|
|
125
|
+
Covered by `test/context-bridge.test.ts` (loop records one turn; a running loop writes
|
|
126
|
+
nothing; interleaved conversations buffer independently) and `test/turn.test.ts` (the
|
|
127
|
+
`tool_calls` → `turnEnded=false` wiring). All four were verified to FAIL against the old
|
|
128
|
+
behavior. `tools/agentdox-e2e.ts` step 6 proves it against a live server: four dispatches →
|
|
129
|
+
exactly one user and one assistant message.
|
|
130
|
+
|
|
131
|
+
## 6. FIXED — harness utility calls, and the scope that leaked across projects
|
|
132
|
+
|
|
133
|
+
Two further faults surfaced the moment `recordTurns` was first switched on, both found by
|
|
134
|
+
reading what actually landed in agentdox.
|
|
135
|
+
|
|
136
|
+
**Utility calls were recorded as turns.** omp drives more than the agent through this
|
|
137
|
+
provider: it asks for a conversation title and a complexity rating, with `model: auto`, over
|
|
138
|
+
the same embedded router. Those answer *about* a conversation rather than participating in
|
|
139
|
+
one, and they finish with `stop`, so `turnEnded` alone does not exclude them. Three junk
|
|
140
|
+
sessions appeared immediately:
|
|
141
|
+
|
|
142
|
+
| recorded assistant text | what it really was |
|
|
143
|
+
| --- | --- |
|
|
144
|
+
| `high` | omp's complexity rating — **this is the original `" high"`** |
|
|
145
|
+
| `<title>Read memory and resume work</title>` | omp's title generation |
|
|
146
|
+
| `<title>Resume settlement 2D slice 3 streaming</title>` | omp's title generation |
|
|
147
|
+
|
|
148
|
+
The discriminator is the tool array: an agent always ships its tool schemas (`toolCount` 12,
|
|
149
|
+
prompts of 60k–90k), while utility calls ship none (`toolCount` 0, prompts of 222–841,
|
|
150
|
+
`task=chat`). `src/server/turn.ts` therefore records only when `req.tools.length > 0`. A
|
|
151
|
+
deliberately tool-less session is not transcribed — silence beats garbage, because every junk
|
|
152
|
+
record is re-injected into every later turn.
|
|
153
|
+
|
|
154
|
+
**`defaultScope` leaked one project's slug to all of them.** `omp-extension/embed-logic.ts`
|
|
155
|
+
resolved the header as `defaultScope !== "" ? defaultScope : derive(cwd)`, so a *scope-agnostic
|
|
156
|
+
global* overrode the *per-workspace* derivation. One router install serves every workspace, so
|
|
157
|
+
with `defaultScope: omp-router` set, an **ashlands** session shipped
|
|
158
|
+
`X-Agentdox-Scope: omp-router`: it injected omp-router's context into ashlands work and filed
|
|
159
|
+
ashlands turns under omp-router. The server always treated the field as a fallback ("the
|
|
160
|
+
configured default covers harnesses that send none"), so the two sides disagreed about the same
|
|
161
|
+
field. Now the workspace derivation wins and `defaultScope` is its fallback, matching the name
|
|
162
|
+
and the server. Same failure class as the `.mcp.json` lesson: a scope-specific value must never
|
|
163
|
+
live in a scope-agnostic file.
|
|
164
|
+
|
|
165
|
+
One consequence worth knowing: `context.token` is a single PAT, but a machine-wide router
|
|
166
|
+
serves N scopes. The omp-router PAT gets `403 no read access to scope "ashlands"`, so with the
|
|
167
|
+
scope now correct the bridge degrades to **inert** for other projects. Correct and safe, but it
|
|
168
|
+
means the bridge only helps projects the configured token actually grants. A multi-scope token
|
|
169
|
+
would fix that, at the cost of one credential reaching every project.
|
|
170
|
+
|
|
171
|
+
## 7. Also worth doing
|
|
172
|
+
|
|
173
|
+
- **Context pollution from test turns.** `context_assemble` includes recent session messages,
|
|
174
|
+
so router test turns feed back into the next block. The omp-router scope had accumulated 19
|
|
175
|
+
sessions of which 18 were noise (`hi`, `say hello`, `Reply with exactly the word: PONG`,
|
|
176
|
+
injection probes, `bridge e2e …`); they were deleted, and `tools/agentdox-e2e.ts` writes two
|
|
177
|
+
more every run. Consider a `sessionLimit` override for the bridge, or excluding
|
|
178
|
+
router-authored sessions.
|
|
127
179
|
- **`context.timeoutMs` is 3000ms** and failures degrade silently at `debug` level by design.
|
|
128
180
|
If agentdox is cold this can no-op invisibly. Consider logging the first failure at `warn`.
|
|
129
181
|
- **Four copies of this project exist** on this machine: this repo, the research checkout,
|
|
@@ -65,8 +65,9 @@ export interface EmbedConfig {
|
|
|
65
65
|
*
|
|
66
66
|
* The workspace basename is the one identifier that is already stable, already
|
|
67
67
|
* per-project, and requires no configuration — the same convention agentdox's
|
|
68
|
-
* own `project_ensure` slugs follow.
|
|
69
|
-
*
|
|
68
|
+
* own `project_ensure` slugs follow. It WINS over `context.defaultScope`,
|
|
69
|
+
* which is a fallback for workspaces it cannot resolve; see
|
|
70
|
+
* `buildProviderConfig`.
|
|
70
71
|
*/
|
|
71
72
|
export function deriveAgentdoxScope(cwd: string): string {
|
|
72
73
|
// Both separators: omp reports a Windows cwd with backslashes.
|
|
@@ -165,7 +166,16 @@ export function buildProviderConfig(
|
|
|
165
166
|
out.harnessId = cfg.server.harnessId;
|
|
166
167
|
}
|
|
167
168
|
if (cfg.context?.enabled === true) {
|
|
168
|
-
|
|
169
|
+
// The WORKSPACE wins. `defaultScope` is a scope-agnostic global — one
|
|
170
|
+
// router install serves every project on the machine — so letting it
|
|
171
|
+
// override the per-workspace derivation sends one project's slug for all
|
|
172
|
+
// of them: an ashlands session shipped `X-Agentdox-Scope: omp-router`,
|
|
173
|
+
// which both injected the wrong project's context and filed its turns
|
|
174
|
+
// under the wrong scope. The server treats this field as a fallback too
|
|
175
|
+
// ("the configured default covers harnesses that send none"), so the two
|
|
176
|
+
// sides now agree: most specific signal first.
|
|
177
|
+
const derived = deriveAgentdoxScope(cwd ?? "");
|
|
178
|
+
const scope = derived !== "" ? derived : cfg.context.defaultScope;
|
|
169
179
|
if (scope !== "") out.agentdoxScope = scope;
|
|
170
180
|
}
|
|
171
181
|
return out;
|
package/package.json
CHANGED
package/src/context/bridge.ts
CHANGED
|
@@ -49,6 +49,29 @@ function renderBlock(raw: string, maxChars: number): string {
|
|
|
49
49
|
].join("\n");
|
|
50
50
|
}
|
|
51
51
|
|
|
52
|
+
/**
|
|
53
|
+
* Cap on assistant text buffered for one in-flight turn, chars. A memory guard
|
|
54
|
+
* only, not a quality knob: a 200-round-trip loop must not buffer without
|
|
55
|
+
* limit. The dispatch that ENDS the turn is appended past this cap, so the
|
|
56
|
+
* model's actual answer is never the thing that gets dropped.
|
|
57
|
+
*/
|
|
58
|
+
const MAX_PENDING_CHARS = 64_000;
|
|
59
|
+
|
|
60
|
+
/**
|
|
61
|
+
* Cap on conversations buffering fragments at once. A turn that dies without a
|
|
62
|
+
* terminal dispatch (client disconnect, upstream error) leaves its buffer
|
|
63
|
+
* behind, so this map is bounded rather than trusted to drain.
|
|
64
|
+
*/
|
|
65
|
+
const MAX_PENDING_CONVERSATIONS = 64;
|
|
66
|
+
|
|
67
|
+
/** Appends a mid-loop fragment, bounded. Blank-line joined: separate thoughts. */
|
|
68
|
+
function appendFragment(prior: string, next: string): string {
|
|
69
|
+
if (next === "") return prior;
|
|
70
|
+
if (prior === "") return next.slice(0, MAX_PENDING_CHARS);
|
|
71
|
+
if (prior.length >= MAX_PENDING_CHARS) return prior;
|
|
72
|
+
return `${prior}\n\n${next}`.slice(0, MAX_PENDING_CHARS);
|
|
73
|
+
}
|
|
74
|
+
|
|
52
75
|
export function createContextBridge(opts: BridgeOptions): ContextBridge {
|
|
53
76
|
const { client, store, log, maxStalenessMs, maxBlockChars, recordTurns, maxQueue } = opts;
|
|
54
77
|
|
|
@@ -57,6 +80,10 @@ export function createContextBridge(opts: BridgeOptions): ContextBridge {
|
|
|
57
80
|
let queue: Promise<void> = Promise.resolve();
|
|
58
81
|
let queued = 0;
|
|
59
82
|
let closed = false;
|
|
83
|
+
// Assistant text buffered across an in-flight tool loop, keyed by
|
|
84
|
+
// conversation. Process-local by design: a turn never spans a restart, and
|
|
85
|
+
// losing a buffer whose turn already died costs nothing.
|
|
86
|
+
const pending = new Map<string, string>();
|
|
60
87
|
|
|
61
88
|
const shouldRefresh = (input: ContextResolveInput, pin: ContextPin | null): boolean => {
|
|
62
89
|
if (pin === null) return true;
|
|
@@ -115,7 +142,34 @@ export function createContextBridge(opts: BridgeOptions): ContextBridge {
|
|
|
115
142
|
|
|
116
143
|
recordTurn(rec: TurnRecord) {
|
|
117
144
|
if (!recordTurns || closed || rec.scope === "") return;
|
|
118
|
-
|
|
145
|
+
|
|
146
|
+
// Mid-loop dispatch: keep the fragment and wait for the turn to end.
|
|
147
|
+
// Writing here is what produced ~13 near-empty assistant messages per
|
|
148
|
+
// turn plus ~13 copies of an unchanged user message, which both lost
|
|
149
|
+
// the real answer and poisoned later context assembly.
|
|
150
|
+
if (!rec.turnEnded) {
|
|
151
|
+
if (rec.assistantText === "") return;
|
|
152
|
+
const prior = pending.get(rec.conversationKey);
|
|
153
|
+
if (prior === undefined && pending.size >= MAX_PENDING_CONVERSATIONS) {
|
|
154
|
+
log.debug("agentdox pending transcript budget full; dropping fragment", { conversations: pending.size });
|
|
155
|
+
return;
|
|
156
|
+
}
|
|
157
|
+
pending.set(rec.conversationKey, appendFragment(prior ?? "", rec.assistantText));
|
|
158
|
+
return;
|
|
159
|
+
}
|
|
160
|
+
|
|
161
|
+
// Turn over. Flush the whole loop's narration plus this dispatch's
|
|
162
|
+
// synthesis as ONE assistant message, attributed to the served model.
|
|
163
|
+
const buffered = pending.get(rec.conversationKey) ?? "";
|
|
164
|
+
pending.delete(rec.conversationKey);
|
|
165
|
+
const assistantText =
|
|
166
|
+
buffered === ""
|
|
167
|
+
? rec.assistantText
|
|
168
|
+
: rec.assistantText === ""
|
|
169
|
+
? buffered
|
|
170
|
+
: `${buffered}\n\n${rec.assistantText}`;
|
|
171
|
+
|
|
172
|
+
if (rec.userText === "" && assistantText === "") return;
|
|
119
173
|
if (queued >= maxQueue) {
|
|
120
174
|
log.debug("agentdox write-back queue full; dropping turn record", { queued });
|
|
121
175
|
return;
|
|
@@ -134,7 +188,7 @@ export function createContextBridge(opts: BridgeOptions): ContextBridge {
|
|
|
134
188
|
// every turn shows WHICH model produced it.
|
|
135
189
|
const refs = [`model:${rec.slug}`, `tier:${rec.tier}`];
|
|
136
190
|
if (rec.userText !== "") await client.append(sessionId, "user", rec.userText, []);
|
|
137
|
-
if (
|
|
191
|
+
if (assistantText !== "") await client.append(sessionId, "assistant", assistantText, refs);
|
|
138
192
|
})
|
|
139
193
|
.catch((err: unknown) => {
|
|
140
194
|
log.debug("agentdox write-back failed", { error: err instanceof Error ? err.message : String(err) });
|
|
@@ -150,6 +204,7 @@ export function createContextBridge(opts: BridgeOptions): ContextBridge {
|
|
|
150
204
|
|
|
151
205
|
close() {
|
|
152
206
|
closed = true;
|
|
207
|
+
pending.clear();
|
|
153
208
|
},
|
|
154
209
|
};
|
|
155
210
|
}
|
package/src/context/types.ts
CHANGED
|
@@ -49,10 +49,23 @@ export interface TurnRecord {
|
|
|
49
49
|
/** Title used if this is the first turn and a session must be created. */
|
|
50
50
|
title: string;
|
|
51
51
|
userText: string;
|
|
52
|
+
/** Text THIS dispatch produced. Fragments are joined across a tool loop. */
|
|
52
53
|
assistantText: string;
|
|
53
54
|
/** The slug that actually served the turn — the model attribution. */
|
|
54
55
|
slug: string;
|
|
55
56
|
tier: string;
|
|
57
|
+
/**
|
|
58
|
+
* Whether the assistant yielded control back to the user — i.e. the upstream
|
|
59
|
+
* finish reason was NOT `tool_calls`.
|
|
60
|
+
*
|
|
61
|
+
* A user-visible turn is many dispatches: every tool round-trip is its own
|
|
62
|
+
* request, and only the last carries the model's synthesis. The intermediate
|
|
63
|
+
* ones are almost pure tool calls with a few stray words of text, and the
|
|
64
|
+
* last *user* message does not move while the loop runs. False therefore
|
|
65
|
+
* means "buffer this fragment, the turn is still running" — recording it as
|
|
66
|
+
* a turn would write a near-empty answer and re-append the same user text.
|
|
67
|
+
*/
|
|
68
|
+
turnEnded: boolean;
|
|
56
69
|
}
|
|
57
70
|
|
|
58
71
|
export interface ContextBridge {
|
package/src/server/turn.ts
CHANGED
|
@@ -448,22 +448,38 @@ export async function runTurn(
|
|
|
448
448
|
|
|
449
449
|
// Record the settled turn into agentdox, attributed to the model that
|
|
450
450
|
// actually served it. Queued and never awaited: the transcript is an
|
|
451
|
-
// artifact of the turn, not a precondition for finishing it.
|
|
452
|
-
|
|
451
|
+
// artifact of the turn, not a precondition for finishing it. A
|
|
452
|
+
// `tool_calls` finish means the assistant is still working, so the bridge
|
|
453
|
+
// buffers the fragment rather than writing a near-empty turn.
|
|
454
|
+
//
|
|
455
|
+
// Only the agent's WORKING conversation is transcribed. A harness also
|
|
456
|
+
// drives utility calls through this same provider with `model: auto` —
|
|
457
|
+
// omp asks for a conversation title and a complexity rating — and those
|
|
458
|
+
// answer ABOUT a conversation instead of participating in one, which is
|
|
459
|
+
// where the junk records (`high`, `<title>…</title>`) came from. They are
|
|
460
|
+
// single-shot and carry NO tool schemas, while an agent always ships its
|
|
461
|
+
// tools, so the tool array is the discriminator. A deliberately
|
|
462
|
+
// tool-less session is therefore not transcribed: silence beats garbage,
|
|
463
|
+
// because every junk record is re-injected into every later turn.
|
|
464
|
+
if (doxActive && req.tools.length > 0) {
|
|
465
|
+
const userText = lastUserText(req);
|
|
466
|
+
const turnEnded = finishReason !== "tool_calls";
|
|
453
467
|
log.debug("agentdox record turn", {
|
|
454
|
-
|
|
468
|
+
conversationKey: req.conversationKey.slice(0, 8),
|
|
469
|
+
userChars: userText.length,
|
|
455
470
|
assistantChars: assistantText.length,
|
|
456
|
-
|
|
457
|
-
|
|
471
|
+
finishReason,
|
|
472
|
+
turnEnded,
|
|
458
473
|
});
|
|
459
474
|
bridge.recordTurn({
|
|
460
475
|
scope: doxScope,
|
|
461
476
|
conversationKey: req.conversationKey,
|
|
462
477
|
title: sessionTitle(req),
|
|
463
|
-
userText
|
|
478
|
+
userText,
|
|
464
479
|
assistantText,
|
|
465
480
|
slug: servedSlug ?? decision.slug,
|
|
466
481
|
tier: decision.tier,
|
|
482
|
+
turnEnded,
|
|
467
483
|
});
|
|
468
484
|
}
|
|
469
485
|
|
|
@@ -3,7 +3,7 @@ import { describe, expect, test } from "bun:test";
|
|
|
3
3
|
import type { AgentDoxClient } from "../src/context/agentdox.ts";
|
|
4
4
|
import { createContextBridge } from "../src/context/bridge.ts";
|
|
5
5
|
import { createContextStore } from "../src/context/store.ts";
|
|
6
|
-
import type { ContextResolveInput } from "../src/context/types.ts";
|
|
6
|
+
import type { ContextResolveInput, TurnRecord } from "../src/context/types.ts";
|
|
7
7
|
import { createLogger } from "../src/util/log.ts";
|
|
8
8
|
import { openDb } from "../src/util/sqlite.ts";
|
|
9
9
|
import { injectForTest } from "./helpers/inject.ts";
|
|
@@ -246,6 +246,7 @@ describe("context bridge write-back", () => {
|
|
|
246
246
|
assistantText: "done",
|
|
247
247
|
slug: "anthropic/claude-haiku-4.5",
|
|
248
248
|
tier: "simple",
|
|
249
|
+
turnEnded: true,
|
|
249
250
|
});
|
|
250
251
|
bridge.recordTurn({
|
|
251
252
|
scope: "ashlands",
|
|
@@ -255,6 +256,7 @@ describe("context bridge write-back", () => {
|
|
|
255
256
|
assistantText: "ok",
|
|
256
257
|
slug: "anthropic/claude-opus-4.5",
|
|
257
258
|
tier: "hard",
|
|
259
|
+
turnEnded: true,
|
|
258
260
|
});
|
|
259
261
|
await bridge.flush();
|
|
260
262
|
|
|
@@ -280,6 +282,7 @@ describe("context bridge write-back", () => {
|
|
|
280
282
|
assistantText: "a",
|
|
281
283
|
slug: "x",
|
|
282
284
|
tier: "simple",
|
|
285
|
+
turnEnded: true,
|
|
283
286
|
});
|
|
284
287
|
await bridge.flush();
|
|
285
288
|
expect(client.appended).toHaveLength(0);
|
|
@@ -287,6 +290,96 @@ describe("context bridge write-back", () => {
|
|
|
287
290
|
db.close();
|
|
288
291
|
}
|
|
289
292
|
});
|
|
293
|
+
|
|
294
|
+
/** One dispatch of a turn; `turnEnded` marks the one that yields to the user. */
|
|
295
|
+
function mkRecord(over: Partial<TurnRecord> & { turnEnded: boolean }): TurnRecord {
|
|
296
|
+
return {
|
|
297
|
+
scope: "ashlands",
|
|
298
|
+
conversationKey: "k1",
|
|
299
|
+
title: "movement fix",
|
|
300
|
+
userText: "fix movement",
|
|
301
|
+
assistantText: "",
|
|
302
|
+
slug: "z-ai/glm-5.3-flash",
|
|
303
|
+
tier: "simple",
|
|
304
|
+
...over,
|
|
305
|
+
};
|
|
306
|
+
}
|
|
307
|
+
|
|
308
|
+
test("a tool loop records one turn, not one record per dispatch", async () => {
|
|
309
|
+
const client = mkClient();
|
|
310
|
+
const { bridge, db } = mkBridge(client);
|
|
311
|
+
try {
|
|
312
|
+
// One user-visible turn: five tool round-trips, then the synthesis.
|
|
313
|
+
// Every dispatch carries the SAME unchanged user text — recording per
|
|
314
|
+
// dispatch appended it once per round-trip and buried the real answer
|
|
315
|
+
// under near-empty assistant messages.
|
|
316
|
+
for (const assistantText of ["let me look", "", "checking the ledger", "", "almost there"]) {
|
|
317
|
+
bridge.recordTurn(mkRecord({ assistantText, turnEnded: false }));
|
|
318
|
+
}
|
|
319
|
+
bridge.recordTurn(mkRecord({ assistantText: "fixed: the damping was inverted.", turnEnded: true }));
|
|
320
|
+
await bridge.flush();
|
|
321
|
+
|
|
322
|
+
expect(client.sessionsCreated).toBe(1);
|
|
323
|
+
const users = client.appended.filter((m) => m.role === "user");
|
|
324
|
+
const assistants = client.appended.filter((m) => m.role === "assistant");
|
|
325
|
+
expect(users).toHaveLength(1);
|
|
326
|
+
expect(assistants).toHaveLength(1);
|
|
327
|
+
// The loop's narration AND the closing synthesis survive, in order.
|
|
328
|
+
expect(assistants[0]?.content).toBe(
|
|
329
|
+
"let me look\n\nchecking the ledger\n\nalmost there\n\nfixed: the damping was inverted.",
|
|
330
|
+
);
|
|
331
|
+
expect(assistants[0]?.refs).toEqual(["model:z-ai/glm-5.3-flash", "tier:simple"]);
|
|
332
|
+
} finally {
|
|
333
|
+
db.close();
|
|
334
|
+
}
|
|
335
|
+
});
|
|
336
|
+
|
|
337
|
+
test("a tool loop still running writes nothing", async () => {
|
|
338
|
+
const client = mkClient();
|
|
339
|
+
const { bridge, db } = mkBridge(client);
|
|
340
|
+
try {
|
|
341
|
+
bridge.recordTurn(mkRecord({ assistantText: "let me look", turnEnded: false }));
|
|
342
|
+
await bridge.flush();
|
|
343
|
+
// The assistant has not answered yet. Writing here is what produced the
|
|
344
|
+
// 4-char transcripts, so mid-loop must stay silent.
|
|
345
|
+
expect(client.appended).toHaveLength(0);
|
|
346
|
+
expect(client.sessionsCreated).toBe(0);
|
|
347
|
+
} finally {
|
|
348
|
+
db.close();
|
|
349
|
+
}
|
|
350
|
+
});
|
|
351
|
+
|
|
352
|
+
test("interleaved conversations buffer independently", async () => {
|
|
353
|
+
const client = mkClient();
|
|
354
|
+
const { bridge, db } = mkBridge(client);
|
|
355
|
+
try {
|
|
356
|
+
bridge.recordTurn(mkRecord({ conversationKey: "k1", assistantText: "k1 narration", turnEnded: false }));
|
|
357
|
+
bridge.recordTurn(mkRecord({ conversationKey: "k2", assistantText: "k2 narration", turnEnded: false }));
|
|
358
|
+
bridge.recordTurn(mkRecord({ conversationKey: "k2", assistantText: "k2 answer", turnEnded: true }));
|
|
359
|
+
bridge.recordTurn(mkRecord({ conversationKey: "k1", assistantText: "k1 answer", turnEnded: true }));
|
|
360
|
+
await bridge.flush();
|
|
361
|
+
|
|
362
|
+
const assistants = client.appended.filter((m) => m.role === "assistant");
|
|
363
|
+
expect(assistants).toHaveLength(2);
|
|
364
|
+
expect(assistants[0]?.content).toBe("k2 narration\n\nk2 answer");
|
|
365
|
+
expect(assistants[1]?.content).toBe("k1 narration\n\nk1 answer");
|
|
366
|
+
} finally {
|
|
367
|
+
db.close();
|
|
368
|
+
}
|
|
369
|
+
});
|
|
370
|
+
|
|
371
|
+
test("a silent turn still records the user message", async () => {
|
|
372
|
+
const client = mkClient();
|
|
373
|
+
const { bridge, db } = mkBridge(client);
|
|
374
|
+
try {
|
|
375
|
+
bridge.recordTurn(mkRecord({ assistantText: "", turnEnded: true }));
|
|
376
|
+
await bridge.flush();
|
|
377
|
+
expect(client.appended.filter((m) => m.role === "user")).toHaveLength(1);
|
|
378
|
+
expect(client.appended.filter((m) => m.role === "assistant")).toHaveLength(0);
|
|
379
|
+
} finally {
|
|
380
|
+
db.close();
|
|
381
|
+
}
|
|
382
|
+
});
|
|
290
383
|
});
|
|
291
384
|
|
|
292
385
|
describe("context injection into the wire body", () => {
|
package/test/embed-logic.test.ts
CHANGED
|
@@ -115,7 +115,11 @@ describe("agentdox scope", () => {
|
|
|
115
115
|
expect(deriveAgentdoxScope("")).toBe("");
|
|
116
116
|
});
|
|
117
117
|
|
|
118
|
-
test("
|
|
118
|
+
test("the workspace derivation wins over the scope-agnostic defaultScope", () => {
|
|
119
|
+
// Regression: one router install serves every project on the machine, so a
|
|
120
|
+
// global `defaultScope` overriding the derivation made an ashlands session
|
|
121
|
+
// ship `X-Agentdox-Scope: omp-router` — wrong context injected, turns
|
|
122
|
+
// filed under the wrong project.
|
|
119
123
|
const base = {
|
|
120
124
|
server: { host: "127.0.0.1" },
|
|
121
125
|
profiles: [],
|
|
@@ -123,8 +127,11 @@ describe("agentdox scope", () => {
|
|
|
123
127
|
};
|
|
124
128
|
const derived = buildProviderConfig(1234, { ...base, context: { enabled: true, defaultScope: "" } }, "/x/ashlands");
|
|
125
129
|
expect(derived.agentdoxScope).toBe("ashlands");
|
|
126
|
-
const
|
|
127
|
-
expect(
|
|
130
|
+
const both = buildProviderConfig(1234, { ...base, context: { enabled: true, defaultScope: "omp-router" } }, "/x/ashlands");
|
|
131
|
+
expect(both.agentdoxScope).toBe("ashlands");
|
|
132
|
+
// The default only applies when the workspace yields nothing.
|
|
133
|
+
const fallback = buildProviderConfig(1234, { ...base, context: { enabled: true, defaultScope: "pinned" } }, "");
|
|
134
|
+
expect(fallback.agentdoxScope).toBe("pinned");
|
|
128
135
|
});
|
|
129
136
|
|
|
130
137
|
test("no scope header when the bridge is off", () => {
|
package/test/turn.test.ts
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import { describe, expect, test } from "bun:test";
|
|
2
2
|
import { createDisabledBridge } from "../src/context/bridge.ts";
|
|
3
|
+
import type { ContextBridge, TurnRecord } from "../src/context/types.ts";
|
|
3
4
|
import type { CatalogSource } from "../src/catalog/types.ts";
|
|
4
5
|
import type { EscalationConfig, RouterConfig } from "../src/config/types.ts";
|
|
5
6
|
import { EMPTY_USAGE, type Ledger, type LedgerEntry, type UsageCounts } from "../src/cost/types.ts";
|
|
@@ -553,3 +554,80 @@ describe("exploration reaches the ledger", () => {
|
|
|
553
554
|
expect(entries[0]?.exploredFrom).toBeNull();
|
|
554
555
|
});
|
|
555
556
|
});
|
|
557
|
+
|
|
558
|
+
describe("agentdox write-back sees the shape of the turn", () => {
|
|
559
|
+
/** An agent ships its tool schemas; a harness utility call does not. */
|
|
560
|
+
const AGENT_TOOL = { name: "read", description: "read a file", schemaBytes: 128 };
|
|
561
|
+
|
|
562
|
+
function mkRecordingBridge(): { bridge: ContextBridge; records: TurnRecord[] } {
|
|
563
|
+
const records: TurnRecord[] = [];
|
|
564
|
+
return {
|
|
565
|
+
records,
|
|
566
|
+
bridge: {
|
|
567
|
+
enabled: true,
|
|
568
|
+
resolve: () => Promise.resolve(null),
|
|
569
|
+
recordTurn: (rec) => {
|
|
570
|
+
records.push(rec);
|
|
571
|
+
},
|
|
572
|
+
flush: () => Promise.resolve(),
|
|
573
|
+
close: () => {},
|
|
574
|
+
},
|
|
575
|
+
};
|
|
576
|
+
}
|
|
577
|
+
|
|
578
|
+
test("a tool_calls finish is a fragment; only a yielding finish ends the turn", async () => {
|
|
579
|
+
// A user-visible turn is many dispatches. The orchestrator must tell the
|
|
580
|
+
// bridge which one actually handed control back, or the transcript records
|
|
581
|
+
// a near-empty answer per tool round-trip and re-appends the same user
|
|
582
|
+
// text every time.
|
|
583
|
+
const { router } = mkRouter([
|
|
584
|
+
mkDecision("simple", "cheap/model", { escalateTo: null }),
|
|
585
|
+
mkDecision("simple", "cheap/model", { escalateTo: null }),
|
|
586
|
+
]);
|
|
587
|
+
const { upstream } = mkUpstream([
|
|
588
|
+
{ kind: "chunks", chunks: [startChunk("cheap/model"), textChunk("let me look"), finishChunk("tool_calls"), usageChunk({}, 0.0001)] },
|
|
589
|
+
{ kind: "chunks", chunks: [startChunk("cheap/model"), textChunk("all done"), finishChunk("stop"), usageChunk({}, 0.0001)] },
|
|
590
|
+
]);
|
|
591
|
+
const { ledger } = mkLedger();
|
|
592
|
+
const { store } = mkConversations();
|
|
593
|
+
const { sink, errors } = mkSink();
|
|
594
|
+
const { bridge, records } = mkRecordingBridge();
|
|
595
|
+
// doxActive needs a scope; the request header supplies it. The tool
|
|
596
|
+
// schemas mark this as the agent's working conversation.
|
|
597
|
+
const req: NormRequest = { ...mkReq(), agentdoxScope: "proj", tools: [AGENT_TOOL] };
|
|
598
|
+
const deps = { config: mkConfig({ enabled: false }), router, upstream, ledger, conversations: store, catalog, context: bridge };
|
|
599
|
+
|
|
600
|
+
await runTurn(req, sink, deps, new AbortController().signal);
|
|
601
|
+
await runTurn(req, sink, deps, new AbortController().signal);
|
|
602
|
+
|
|
603
|
+
expect(errors).toHaveLength(0);
|
|
604
|
+
expect(records).toHaveLength(2);
|
|
605
|
+
expect(records[0]?.turnEnded).toBe(false);
|
|
606
|
+
expect(records[0]?.assistantText).toBe("let me look");
|
|
607
|
+
expect(records[1]?.turnEnded).toBe(true);
|
|
608
|
+
expect(records[1]?.assistantText).toBe("all done");
|
|
609
|
+
});
|
|
610
|
+
|
|
611
|
+
test("a harness utility call is never transcribed", async () => {
|
|
612
|
+
// omp drives title generation and complexity rating through this same
|
|
613
|
+
// provider with `model: auto`. They answer ABOUT the conversation
|
|
614
|
+
// ("high", "<title>…</title>") and carry NO tool schemas. Recording them
|
|
615
|
+
// created junk agentdox sessions that then fed back into every later
|
|
616
|
+
// context block.
|
|
617
|
+
const { router } = mkRouter([mkDecision("trivial", "cheap/model", { escalateTo: null })]);
|
|
618
|
+
const { upstream } = mkUpstream([
|
|
619
|
+
{ kind: "chunks", chunks: [startChunk("cheap/model"), textChunk("high"), finishChunk("stop"), usageChunk({}, 0.0001)] },
|
|
620
|
+
]);
|
|
621
|
+
const { ledger } = mkLedger();
|
|
622
|
+
const { store } = mkConversations();
|
|
623
|
+
const { sink, errors } = mkSink();
|
|
624
|
+
const { bridge, records } = mkRecordingBridge();
|
|
625
|
+
// Same scope, same provider — only the absent tool array differs.
|
|
626
|
+
const req: NormRequest = { ...mkReq(), agentdoxScope: "proj", tools: [] };
|
|
627
|
+
|
|
628
|
+
await runTurn(req, sink, { config: mkConfig({ enabled: false }), router, upstream, ledger, conversations: store, catalog, context: bridge }, new AbortController().signal);
|
|
629
|
+
|
|
630
|
+
expect(errors).toHaveLength(0);
|
|
631
|
+
expect(records).toHaveLength(0);
|
|
632
|
+
});
|
|
633
|
+
});
|
package/tools/agentdox-e2e.ts
CHANGED
|
@@ -93,6 +93,7 @@ bridge.recordTurn({
|
|
|
93
93
|
assistantText: "yes - refs carry model: and tier:.",
|
|
94
94
|
slug: "anthropic/claude-haiku-4.5",
|
|
95
95
|
tier: "simple",
|
|
96
|
+
turnEnded: true,
|
|
96
97
|
});
|
|
97
98
|
await bridge.flush();
|
|
98
99
|
|
|
@@ -118,6 +119,52 @@ if (mine !== undefined) {
|
|
|
118
119
|
);
|
|
119
120
|
}
|
|
120
121
|
|
|
122
|
+
// 6. A tool loop must record ONE turn, not one record per dispatch. This is the
|
|
123
|
+
// regression that made transcripts useless: every tool round-trip is its own
|
|
124
|
+
// dispatch, finishing with `tool_calls` and carrying an UNCHANGED last user
|
|
125
|
+
// message, so recording per dispatch wrote a near-empty assistant message
|
|
126
|
+
// and a duplicate user message per round-trip.
|
|
127
|
+
const loopKey = `${conversationKey}-loop`;
|
|
128
|
+
const loopTitle = `bridge e2e loop ${loopKey}`;
|
|
129
|
+
function loopDispatch(assistantText: string, turnEnded: boolean): void {
|
|
130
|
+
bridge.recordTurn({
|
|
131
|
+
scope,
|
|
132
|
+
conversationKey: loopKey,
|
|
133
|
+
title: loopTitle,
|
|
134
|
+
userText: "why did cache read fall?",
|
|
135
|
+
assistantText,
|
|
136
|
+
slug: "z-ai/glm-5.3-flash",
|
|
137
|
+
tier: "simple",
|
|
138
|
+
turnEnded,
|
|
139
|
+
});
|
|
140
|
+
}
|
|
141
|
+
for (const fragment of ["reading the ledger", "", "checking the cache column"]) loopDispatch(fragment, false);
|
|
142
|
+
loopDispatch("the breakpoint index drifted every turn.", true);
|
|
143
|
+
await bridge.flush();
|
|
144
|
+
|
|
145
|
+
const loopRes = await fetch(`${baseUrl}/sessions?scope=${encodeURIComponent(scope)}`, {
|
|
146
|
+
headers: { authorization: `Bearer ${token}` },
|
|
147
|
+
});
|
|
148
|
+
const loopSessions = (await loopRes.json()) as { id: string; title: string }[];
|
|
149
|
+
const loopSession = loopSessions.find((s) => s.title === loopTitle);
|
|
150
|
+
check("tool loop created a session", loopSession !== undefined, loopSession?.id ?? "not found");
|
|
151
|
+
|
|
152
|
+
if (loopSession !== undefined) {
|
|
153
|
+
const full = await fetch(`${baseUrl}/sessions/${loopSession.id}`, {
|
|
154
|
+
headers: { authorization: `Bearer ${token}` },
|
|
155
|
+
});
|
|
156
|
+
const session = (await full.json()) as { messages: { role: string; content: string }[] };
|
|
157
|
+
const users = session.messages.filter((m) => m.role === "user");
|
|
158
|
+
const assistants = session.messages.filter((m) => m.role === "assistant");
|
|
159
|
+
check("four dispatches wrote exactly one user message", users.length === 1, `${users.length} user messages`);
|
|
160
|
+
check("four dispatches wrote exactly one assistant message", assistants.length === 1, `${assistants.length} assistant messages`);
|
|
161
|
+
check(
|
|
162
|
+
"the loop's narration and the closing synthesis both survive",
|
|
163
|
+
assistants[0]?.content === "reading the ledger\n\nchecking the cache column\n\nthe breakpoint index drifted every turn.",
|
|
164
|
+
JSON.stringify(assistants[0]?.content ?? ""),
|
|
165
|
+
);
|
|
166
|
+
}
|
|
167
|
+
|
|
121
168
|
db.close();
|
|
122
169
|
console.log(failures === 0 ? "\nAll bridge e2e checks passed." : `\n${failures} check(s) failed.`);
|
|
123
170
|
process.exit(failures === 0 ? 0 : 1);
|