@adhd/agent-mcp 2.1.1 → 2.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. package/docs/marketing/.catalog/capabilities.md +1 -1
  2. package/docs/plan/accounting/SCOPE.md +316 -0
  3. package/docs/provider-call-audit.md +83 -1
  4. package/package.json +9 -12
  5. package/src/config.d.ts +59 -30
  6. package/src/config.d.ts.map +1 -1
  7. package/src/config.js +202 -205
  8. package/src/config.js.map +1 -1
  9. package/src/db/client.d.ts.map +1 -1
  10. package/src/db/client.js +6 -2
  11. package/src/db/client.js.map +1 -1
  12. package/src/db/migrate-runner.d.ts.map +1 -1
  13. package/src/db/migrate-runner.js.map +1 -1
  14. package/src/db/migrate.d.ts.map +1 -1
  15. package/src/db/migrate.js.map +1 -1
  16. package/src/db/schema.d.ts.map +1 -1
  17. package/src/db/schema.js.map +1 -1
  18. package/src/index.d.ts.map +1 -1
  19. package/src/index.js +20 -13
  20. package/src/index.js.map +1 -1
  21. package/src/logger.d.ts.map +1 -1
  22. package/src/logger.js +2 -2
  23. package/src/logger.js.map +1 -1
  24. package/src/scripts/agent-mcp-tail.d.ts.map +1 -1
  25. package/src/scripts/agent-mcp-tail.js +202 -122
  26. package/src/scripts/agent-mcp-tail.js.map +1 -1
  27. package/src/server.d.ts.map +1 -1
  28. package/src/server.js +7 -7
  29. package/src/server.js.map +1 -1
  30. package/src/store/agent-store.d.ts.map +1 -1
  31. package/src/store/agent-store.js.map +1 -1
  32. package/src/streaming/chat-gateway.d.ts.map +1 -1
  33. package/src/streaming/chat-gateway.js.map +1 -1
  34. package/src/streaming/event-bus.d.ts.map +1 -1
  35. package/src/streaming/event-bus.js.map +1 -1
  36. package/src/streaming/sse-server.d.ts.map +1 -1
  37. package/src/streaming/sse-server.js +2 -2
  38. package/src/streaming/sse-server.js.map +1 -1
  39. package/adhd-agent-mcp-2.1.1.tgz +0 -0
  40. package/src/__tests__/integration/harness.d.ts +0 -1027
  41. package/src/__tests__/integration/harness.d.ts.map +0 -1
  42. package/src/__tests__/integration/harness.js +0 -409
  43. package/src/__tests__/integration/harness.js.map +0 -1
  44. package/src/__tests__/integration/scripted-provider.d.ts +0 -43
  45. package/src/__tests__/integration/scripted-provider.d.ts.map +0 -1
  46. package/src/__tests__/integration/scripted-provider.js +0 -80
  47. package/src/__tests__/integration/scripted-provider.js.map +0 -1
  48. package/src/utils/load-env.d.ts +0 -2
  49. package/src/utils/load-env.d.ts.map +0 -1
  50. package/src/utils/load-env.js +0 -9
  51. package/src/utils/load-env.js.map +0 -1
@@ -83,7 +83,7 @@
83
83
  | Feature | Status | Tested | Substance | Notes |
84
84
  |---------|--------|--------|-----------|-------|
85
85
  | Persistent Storage (SQLite) | ✅ shipped | ✓ wiring.test | moderate | agents, sessions, tasks, task_events, task_usage; WAL mode; BUG-ORCH-012: FK cascade missing |
86
- | Environment Config (Hierarchical .env) | ✅ shipped | ✓ config.test | moderate | .adhd/.env, .env, system; ADHD_AGENT_* prefixed |
86
+ | Environment Config (Zero-Config `Environment<T>` cascade) | ✅ shipped | ✓ config.zero-config.test | moderate | Code-first `agentMcpEnvironmentSpec` (`@adhd/environment`) resolved live at construction — spec defaults → `~/.adhd/agent-mcp/production/config.yaml`/`config.local.yaml` layer files (optional) → `ADHD_AGENT_*` env vars; runs with zero files/env vars on disk. No dotenv `.env` files (the `.adhd/.env`/`.env` hierarchy was removed — see `packages/environment/ARCHITECTURE.md` §6) |
87
87
  | Structured Logging (Pino) | ✅ shipped | ✓ config.test | trivial | Configurable via ADHD_AGENT_LOG_LEVEL |
88
88
  | SSE Task Streaming | ✅ shipped | ❌ no test | substantial | Real-time task/completed notifications on separate port |
89
89
 
@@ -0,0 +1,316 @@
1
+ # SCOPE — agent-mcp usage accounting
2
+
3
+ > Plan type: **investigate + correct**. Goal: make agent-mcp's token/usage data model
4
+ > **accurate, unambiguous, and reconcilable** — so a caller can tell "what did this cost"
5
+ > (cumulative) apart from "will a call fit" (context size), and can attribute cost to tool
6
+ > calls. Written 2026-07-16 against `@adhd/agent-mcp` **v2.1.1**
7
+ > (`entrypoint/agent-mcp/package.json`).
8
+
9
+ ## Sourcing convention (read this first)
10
+
11
+ Every factual claim below is tagged with how it was established. **Nothing in this document
12
+ is asserted from model recall.**
13
+
14
+ | Tag | Meaning |
15
+ |---|---|
16
+ | `[DDL]` | Read directly from a Drizzle migration file under `entrypoint/agent-mcp/drizzle/*.sql` (the authoritative applied schema). File cited inline. |
17
+ | `[DB]` | Observed by a **read-only** (`sqlite3 -readonly` / `PRAGMA` / `SELECT`) inspection of a live database on 2026-07-16. DB path cited inline. |
18
+ | `[TOOL]` | Observed as literal output of the agent-mcp `usage_query` MCP tool on 2026-07-16. |
19
+ | `[ARITH]` | Derived by arithmetic from `[DB]`/`[TOOL]` numbers; the calculation is shown. |
20
+ | `[SRC-COMMENT]` | Quoted from a maintainer comment in repo source (migration comment). File + lines cited. |
21
+ | `[RESEARCH]` | From the 2026-07-16 research runs (memory episodes cited by UID) and/or vendor docs. **Snippet-sourced, MEDIUM confidence, NOT independently verified in this environment.** Treat as a lead to confirm, not fact. |
22
+
23
+ If a statement has no tag, it is a definition or a proposal, not a claim of fact.
24
+
25
+ ---
26
+
27
+ ## 1. Problem statement
28
+
29
+ Two tools reporting the "same" work produced numbers that looked contradictory, which
30
+ surfaced three real defects in how usage is modeled and exposed:
31
+
32
+ - agent-mcp reported a single task as **`inputTokens: 1,154,170`** while another tool showed a
33
+ whole session as **`input: 88 / cache read: 355,712 / total: 356,498`**. `[TOOL]` / (the
34
+ second is an opencode panel, see §5). These were never comparable — one is a **cumulative
35
+ running total that includes cache reads**, the other is a **per-turn snapshot that excludes
36
+ cache** — but nothing in the field names says so.
37
+ - The field name `input_tokens` does not signal that it is (a) **cumulative** across all model
38
+ calls and (b) **cache-inclusive** (`= uncached + cache_read`). `[ARITH]` (§3).
39
+ - Tool-call token consumption — a first-class cost driver — is **not stored at all**; only a
40
+ `tool_call_count` integer exists. `[DB]` (§4).
41
+
42
+ The maintainer already identified the cumulative-vs-peak half of this and partially fixed it
43
+ in migration `0008` (§3); this scope covers what remains.
44
+
45
+ ---
46
+
47
+ ## 2. Ground truth — the agent-mcp schema (what is stored, and at what grain)
48
+
49
+ DB inspected: **`/Users/nix/.adhd/agent-mcp/agents.db`** (Drizzle-managed SQLite). `[DB]`
50
+ Tables present: `agents, sessions, messages, tasks, task_events, task_usage,
51
+ composed_prompts, experiment_assignments, __drizzle_migrations`. `[DB]`
52
+
53
+ ### 2.1 `task_usage` — grain: **one row per task** (`task_id` PRIMARY KEY) `[DDL]` `[DB]`
54
+
55
+ Column provenance (each column traced to the migration that added it):
56
+
57
+ | Column | Added by | Meaning |
58
+ |---|---|---|
59
+ | `task_id` (PK), `root_task_id`, `agent_name`, `provider_type`, `model`, `created_at` | `0001_task_usage.sql` `[DDL]` | identity |
60
+ | `input_tokens`, `output_tokens`, `tool_call_count`, `model_calls`, `latency_ms`, `is_complete` | `0001_task_usage.sql` `[DDL]` | **cumulative** counters (see 0008 comment) |
61
+ | `stop_reason`, `max_tokens` | `0002_lumpy_silver_centurion.sql` `[DDL]` | |
62
+ | `cache_read_input_tokens`, `cache_creation_input_tokens` | `0003_nifty_shriek.sql` `[DDL]` | cumulative cache tokens |
63
+ | (table rebuilt to consolidate the above) | `0007_smart_callisto.sql` `[DDL]` | |
64
+ | `uncached_input_tokens`, `reasoning_tokens`, `peak_context_tokens`, `peak_context_at` | `0008_cache_and_peak_context_usage.sql` `[DDL]` | cache split + **peak (snapshot) high-water mark** |
65
+
66
+ `tool_call_count` and `model_calls` are **integer counts, not token sizes**. `[DDL]`
67
+
68
+ ### 2.2 `task_events` — grain: **one row per event** `[DB]`
69
+
70
+ Event-type counts across the DB at inspection time: `TOOL_CALL` 141, `TOOL_RESULT` 141,
71
+ `MODEL_REQUEST` 113, `MODEL_RESPONSE` 112, `TASK_COMPLETED` 17, `TASK_FAILED` 1. `[DB]`
72
+ Token data lives only in the JSON `payload`:
73
+
74
+ - **`MODEL_RESPONSE`** payload carries per-model-call tokens: `inputTokens, outputTokens,
75
+ cacheReadTokens, cacheCreationTokens`, plus a provider-native `rawUsage` object
76
+ (`prompt_tokens, completion_tokens, total_tokens, prompt_cache_hit_tokens,
77
+ prompt_cache_miss_tokens, prompt_tokens_details, completion_tokens_details`), plus
78
+ `stopReason, toolCallCount, hasContent`. `[DB]`
79
+ **Coverage is partial: only 48 of 112 `MODEL_RESPONSE` events actually carry `inputTokens`**;
80
+ the remaining 64 have only `stopReason/hasContent/toolCallCount`. `[DB]`
81
+ - `MODEL_REQUEST` payload: `messageCount, toolCount` — **no tokens**. `[DB]`
82
+ - `TOOL_CALL` payload: `tool, callId, arguments` — **no tokens**. `[DB]`
83
+ - `TOOL_RESULT` payload: `tool, callId, isError, result` — **no tokens**. `[DB]`
84
+
85
+ (`task_events` table created in `0000_nifty_sasquatch.sql`. `[DDL]`)
86
+
87
+ ### 2.3 `messages` — grain: one row per message — **no token columns at all** `[DB]`
88
+
89
+ Columns: `id, session_id, role, content, tool_calls, tool_results, created_at`. `[DB]`
90
+
91
+ ---
92
+
93
+ ## 3. The cumulative-vs-snapshot distinction (the core concept)
94
+
95
+ **Definitions.**
96
+ - **Cumulative** = a running total summed over every model call in a task/session.
97
+ Monotonically increasing. **Unbounded** — it can far exceed the model's context window and
98
+ says nothing about whether any single call fits.
99
+ - **Snapshot** = the size of a **single** model call's prompt. This — and only this — is what
100
+ the context window bounds; a call errors when its prompt exceeds the window.
101
+
102
+ **The maintainer already stated this problem verbatim** in
103
+ `drizzle/0008_cache_and_peak_context_usage.sql` (lines 8–12) `[SRC-COMMENT]`:
104
+
105
+ > "Peak context. Every column accumulated with `+=`; nothing tracked a MAX. So a run that
106
+ > billed 715K tokens across 24 calls was indistinguishable from one that held a 715K context,
107
+ > when the true high-water mark was 43K. Callers could not tell 'many small calls' (fix:
108
+ > preserve the cache) from 'one huge call' (fix: cap that input)."
109
+
110
+ `0008` added `peak_context_tokens` / `peak_context_at` to record the snapshot high-water mark
111
+ alongside the cumulative counters. `[DDL]` **This is the correct structural fix and is already
112
+ in place** — but the cumulative columns are still *named* as if they were sizes (see §7).
113
+
114
+ **Live falsification proof that `input_tokens` is cumulative, not context size** — task
115
+ `9aa2cb7b-4473-4789-a3fb-86ba1256f3de` `[TOOL]`:
116
+
117
+ ```
118
+ input_tokens = 1,154,170 (cumulative)
119
+ peak_context_tokens = 172,521 (snapshot — the real context size)
120
+ model_calls = 8
121
+ is_complete = 1 (ran to completion — never hit a context limit)
122
+ ```
123
+
124
+ If `1,154,170` were a single prompt it would exceed any current context window and the call
125
+ would have errored. It completed with a peak single-call size of `172,521`. Therefore
126
+ `input_tokens` is a **sum across the 8 calls**, not an occupancy. `[ARITH]`
127
+
128
+ **Which agent-mcp fields are which:**
129
+
130
+ | Field | Kind | Window-bounded? |
131
+ |---|---|---|
132
+ | `input_tokens`, `output_tokens`, `uncached_input_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`, `reasoning_tokens`, `tool_call_count`, `model_calls`, `latency_ms` | **cumulative** | No — unbounded |
133
+ | `peak_context_tokens`, `peak_context_at` | **snapshot** (max) | Yes |
134
+
135
+ ---
136
+
137
+ ## 4. Cache accounting
138
+
139
+ - **`input_tokens` is cache-INCLUSIVE**: `input_tokens = uncached_input_tokens +
140
+ cache_read_input_tokens`. Verified on task `9aa2cb7b`: `139,386 + 1,014,784 = 1,154,170`.
141
+ `[ARITH]` `[TOOL]`
142
+ - Cache tokens matter enormously to cost. Per the `0008` comment `[SRC-COMMENT]`: on
143
+ `deepseek-v4-flash`, **cache-hit vs cache-miss input differs 50× in price
144
+ ($0.0028/M vs $0.14/M)** — "the largest cost signal in the system." The comment records that
145
+ a context limiter once "silently collapse[d] cache hit rate from ~98% to ~5% (BUG-ORCH-008)
146
+ with no telemetry able to show it."
147
+ - The provider-native cache fields are captured in the `MODEL_RESPONSE` `rawUsage` payload
148
+ (`prompt_cache_hit_tokens`, `prompt_cache_miss_tokens`) `[DB]` — i.e. DeepSeek's OpenAI-
149
+ compatible convention where `prompt_tokens` **includes** cached tokens.
150
+ - `cache_creation_input_tokens` is **0** across the observed DeepSeek tasks `[TOOL]` — DeepSeek
151
+ has no separate cache-write billing class (cache hit vs miss only). Contrast Anthropic, which
152
+ bills cache *writes* at a premium — see §6, `[RESEARCH]`.
153
+
154
+ ---
155
+
156
+ ## 5. Reference: how opencode models the same thing (for contrast)
157
+
158
+ DB inspected read-only: **`/Users/nix/.local/share/opencode/opencode.db`** (SQLite, WAL). `[DB]`
159
+ opencode stores tokens at **both** grains, which is instructive:
160
+
161
+ - **Per assistant message** (`message.data` JSON): `tokens.{input, output, reasoning,
162
+ cache.write, cache.read, total}` + `cost`; finer still, per model step (`part` type
163
+ `step-finish`). `[DB]`
164
+ - **Per session** (dedicated columns on `session`): `tokens_input, tokens_output,
165
+ tokens_reasoning, tokens_cache_read, tokens_cache_write, cost` — these are a **cumulative
166
+ sum**, verified on a 1,928-message session where the session columns equalled `SUM()` over
167
+ every message's `tokens.*` to the digit (cumulative input there = 9,427,070). `[DB]` `[ARITH]`
168
+ - The usage **panel** number (`input 88 / cache read 355,712`) is a **per-turn snapshot of the
169
+ last assistant message**, not a session total: on a busy session the last message read
170
+ `input 46, cache.read 291,072` while the session cumulative input was 9.4M. `[DB]` `[ARITH]`
171
+ - opencode **compacts context**: a dedicated `part` type `compaction` (17 parts across 13
172
+ sessions), keyed `{auto, tail_start_id, type}`, prunes context before `tail_start_id`. `[DB]`
173
+ This is why its cumulative growth stays sublinear where agent-mcp's does not (agent-mcp has
174
+ no compaction mechanism in its schema `[DB]`).
175
+
176
+ **Takeaways for agent-mcp:** (a) opencode proves it is reasonable to keep BOTH per-call and
177
+ cumulative token records; (b) a headline "input" number can be a snapshot in one tool and a
178
+ cumulative in another — naming must disambiguate.
179
+
180
+ ---
181
+
182
+ ## 6. Reference: provider usage conventions `[RESEARCH — UNVERIFIED IN THIS ENVIRONMENT]`
183
+
184
+ The following provider-API field shapes are from the 2026-07-16 research (memory episodes,
185
+ §9) and vendor docs. They were **snippet-sourced, not independently exercised here**. Confirm
186
+ against the cited docs before implementing against them.
187
+
188
+ - **Anthropic (Messages API):** `input_tokens` **excludes** cache; cache is separate lines
189
+ `cache_read_input_tokens` (~0.1×) and `cache_creation_input_tokens` (~1.25×). Per-call.
190
+ `[RESEARCH]` — docs.anthropic.com.
191
+ - **OpenAI (Chat Completions):** `prompt_tokens` **includes** `prompt_tokens_details.cached_tokens`;
192
+ `completion_tokens_details.reasoning_tokens` is a subset of completion. `total_tokens =
193
+ prompt + completion`. Per-call. `[RESEARCH]` — platform.openai.com/docs.
194
+ - **OpenRouter:** per-generation; `native_tokens_prompt/completion` (provider tokenizer) vs
195
+ normalized `tokens_prompt/completion`; adds `cost` / `cache_discount`; cumulative is the
196
+ caller's own sum. `[RESEARCH]` — openrouter.ai/docs.
197
+
198
+ agent-mcp's `providerType: "openai"` path (used for DeepSeek) follows the **OpenAI/cache-
199
+ inclusive** convention, consistent with §4. `[DB]`
200
+
201
+ ---
202
+
203
+ ## 7. Corrections in scope
204
+
205
+ Ranked by impact on accuracy. Each states the defect (with evidence) and the fix.
206
+
207
+ 1. **Complete `MODEL_RESPONSE` usage capture.** Only 48/112 model responses carry
208
+ `inputTokens` `[DB]`; the other 64 are token-blind. Likely cause: streaming responses whose
209
+ usage arrives only in the terminal chunk, or tool-only turns. **Fix:** capture the final
210
+ usage chunk on every response; where a provider omits usage, tokenize locally as a fallback.
211
+ Without this, per-model-call analysis is lossy and `task_usage` cannot be reconciled against
212
+ the event log.
213
+ 2. **Store per-tool-call token size.** `TOOL_CALL`/`TOOL_RESULT` payloads carry no tokens `[DB]`.
214
+ **Fix:** tokenize the `TOOL_RESULT.result` payload when the event is written and store
215
+ `result_tokens` (optionally `arguments_tokens` on `TOOL_CALL`). See §8 for why this is the
216
+ correct approach rather than input-delta inference. This is the direct answer to the
217
+ open "tool call token approximation" question.
218
+ 3. **Disambiguate cumulative vs snapshot in names/exposure** (see Unresolved Q2, §10). The peak
219
+ columns already exist `[DDL]`; the cumulative columns still read like sizes. **Fix:** expose
220
+ token data under two clearly-named groups (a cumulative/"billed" group and a
221
+ snapshot/"context-fit" group) and document `input_tokens` as cumulative + cache-inclusive.
222
+ 4. **Single source of truth / reconciliation.** Given the coverage gap (#1), verify
223
+ `task_usage.input_tokens == Σ MODEL_RESPONSE.inputTokens`; if the aggregate is computed from
224
+ a complete provider counter while events are lossy, the two silently diverge. **Fix:** make
225
+ `task_usage` the sum of the (now-complete) events, reconcilable and auditable.
226
+ 5. **Per-call context-size series.** Only a single `peak_context_tokens` is kept `[DDL]`. A
227
+ per-model-call context series (derivable once #1 lands) is what reveals *growth over time*
228
+ and predicts a context-limit failure, not just the max.
229
+ 6. **(Larger, optional) Context compaction.** Not strictly accounting, but it is *why* agent-mcp
230
+ cumulative balloons where opencode's does not (§5) `[DB]`. Out of scope for the accounting
231
+ model itself; noted so the big cumulative numbers are read correctly (pair every cumulative
232
+ with `peak_context_tokens`).
233
+
234
+ ---
235
+
236
+ ## 8. Formulas
237
+
238
+ **Cumulative vs snapshot (from per-call usage):**
239
+ ```
240
+ cumulative_input = Σ_i input_tokens(call_i) # monotonic, unbounded
241
+ context_size(i) = input_tokens(call_i) # snapshot, window-bounded
242
+ peak_context = max_i input_tokens(call_i) # == task_usage.peak_context_tokens
243
+ ```
244
+
245
+ **Tool-call token consumption — approximation vs exact.**
246
+
247
+ The tempting approximation is the input delta on the model request that *follows* a tool call:
248
+ ```
249
+ Δ = input_tokens(call_{N+1}) − input_tokens(call_N)
250
+ = output_tokens(call_N) # the assistant's turn-N response, incl. the tool_call request itself
251
+ + Σ tool_result_tokens(turn N) # ALL tool results returned in turn N
252
+ + framework/system additions
253
+ ```
254
+ This is **imprecise and cannot attribute per-tool cost**, because:
255
+ - It also contains `output_tokens(call_N)`, which must be subtracted out.
256
+ - agent-mcp fires **parallel** tool calls (141 `TOOL_CALL` vs 112 `MODEL_RESPONSE` `[DB]`), so
257
+ the delta yields only the **turn total**, never per-tool.
258
+ - After first appearance a tool result is re-sent as **cache_read**, not fresh input — so the
259
+ "cost" of a tool result is its *first-appearance uncached* contribution, not the gross delta.
260
+
261
+ **Correct technique (proposed):** tokenize the `TOOL_RESULT.result` payload directly at write
262
+ time → `result_tokens`. Exact, per-tool, cache-independent. The input-delta formula above is
263
+ retained only as a per-*turn* cross-check:
264
+ ```
265
+ Σ tool_result_tokens(turn N) ≈ input_tokens(call_{N+1}) − input_tokens(call_N) − output_tokens(call_N)
266
+ ```
267
+
268
+ ---
269
+
270
+ ## 9. Unresolved questions
271
+
272
+ 1. **Standardize naming.** `input_tokens` (cumulative, cache-inclusive) reads identically to a
273
+ context size. What is the canonical field vocabulary that makes cumulative-vs-snapshot and
274
+ fresh-vs-cached unmistakable, and does it get applied at the DB column level, the
275
+ `usage_query` API level, or both? (Renaming DB columns is a migration + a breaking API
276
+ change — scope the blast radius.)
277
+ 2. **Structurally differentiate cumulative vs per-turn.** `peak_context_tokens` exists `[DDL]`,
278
+ but there is no per-call context series and the API returns cumulative and snapshot fields
279
+ flat, side by side. Should the `usage_query` response be restructured into explicit
280
+ `cumulative { … }` and `contextFit { … }` groups? Should per-model-call rows (from
281
+ `task_events`) be a first-class queryable, given only 48/112 currently carry tokens (§7 #1)?
282
+ 3. **Tool-call token approximation technique.** Confirm the §8 decision: store `result_tokens`
283
+ at write time (exact) vs derive from input deltas (per-turn only, cache-confounded). Open
284
+ sub-questions: which tokenizer to use when the provider's is unavailable; whether to also
285
+ store `arguments_tokens`; how to reconcile `result_tokens` against the subsequent call's
286
+ uncached delta as a validation gate.
287
+
288
+ ---
289
+
290
+ ## 10. References
291
+
292
+ **Repo (authoritative, read this session):**
293
+ - `entrypoint/agent-mcp/drizzle/0000_nifty_sasquatch.sql` — `task_events` table `[DDL]`
294
+ - `entrypoint/agent-mcp/drizzle/0001_task_usage.sql` — base `task_usage` `[DDL]`
295
+ - `entrypoint/agent-mcp/drizzle/0002_lumpy_silver_centurion.sql` — `stop_reason`, `max_tokens` `[DDL]`
296
+ - `entrypoint/agent-mcp/drizzle/0003_nifty_shriek.sql` — cache token columns `[DDL]`
297
+ - `entrypoint/agent-mcp/drizzle/0007_smart_callisto.sql` — table rebuild `[DDL]`
298
+ - `entrypoint/agent-mcp/drizzle/0008_cache_and_peak_context_usage.sql` — uncached/reasoning/peak
299
+ columns **and the maintainer comment quoted in §3–§4** (BUG-ORCH-008/009/010, FINDING-ORCH-007) `[SRC-COMMENT]` `[DDL]`
300
+ - `entrypoint/agent-mcp/package.json` — version 2.1.1
301
+
302
+ **Databases inspected read-only (2026-07-16):**
303
+ - `/Users/nix/.adhd/agent-mcp/agents.db` `[DB]`
304
+ - `/Users/nix/.local/share/opencode/opencode.db` `[DB]`
305
+
306
+ **Live tool output (2026-07-16):** agent-mcp `usage_query` — task `9aa2cb7b…` per-task row and
307
+ the 5-task aggregate (`totalInputTokens 1,927,244`). `[TOOL]`
308
+
309
+ **Research (memory episodes, MEDIUM confidence, snippet-sourced — `[RESEARCH]`):**
310
+ - Token accounting: `01KXP4MDPARC…` (opencode), `01KXP4MDPQR6…` (Claude Code), `01KXP4P2ZV7D…`
311
+ (OpenRouter), `01KXP4P30GZSHKNH2NC7YMQMYA` (context vs cumulative), `01KXP4P30NPWVXWZVXBH3E5CT2`
312
+ (588k-vs-356k worked example), `01KXP4P30SV99308NGEYM3G3GW` (cross-provider caching).
313
+ - Prior caching research: `01KX9Z06VE4MDV5KYB3H0A0ADT` (cross-provider), `01KX9Z06T924F6N9GH7XM940RX`
314
+ (Anthropic), `01KX9Z06V4B6M7S5R5RF2F4F6Z` (OpenAI), `01KX9Z06V72ZN13YX60QV99B1Z` (DeepSeek).
315
+ - Vendor docs referenced by the research (not fetched here): docs.anthropic.com,
316
+ platform.openai.com/docs, openrouter.ai/docs.
@@ -4,7 +4,7 @@ Audit of what agent-mcp sends to the provider API each turn and what it stores i
4
4
  the DB. Compared against the standard ReAct loop pattern used by OpenCode and Claude
5
5
  Code.
6
6
 
7
- **Date:** 2026-07-02
7
+ **Date:** 2026-07-02 · **Updated:** 2026-07-15 (added §6 — measured agent-mcp-vs-OpenCode benchmark)
8
8
  **Trigger:** investigating 262k input tokens for a 15-turn, 775-line implementation task.
9
9
 
10
10
  ---
@@ -156,6 +156,11 @@ The critical difference: **agent-mcp has no tool pruning.** Claude Code sends on
156
156
  the tools relevant to the current task. OpenCode gates tools by permission mode.
157
157
  agent-mcp sends every tool from every connected MCP server on every turn.
158
158
 
159
+ > **Measured update (2026-07-15):** §6 runs the identical task through both harnesses
160
+ > on the same DeepSeek model. Counter-intuitively OpenCode is the *heavier* one here
161
+ > — its `build` agent ships a **~32K** prefix per call vs agent-mcp's ~5K — so agent-mcp
162
+ > came out **~3× cheaper**, not more expensive. See [§6](#6-empirical-head-to-head-agent-mcp-vs-opencode-measured-2026-07-15).
163
+
159
164
  ---
160
165
 
161
166
  ## 4. Opportunities
@@ -252,3 +257,80 @@ This matches the observed average of ~17,500/turn × 15 turns = 262,590.
252
257
  conversation history is.** But the tool schemas ARE the dominant cost on
253
258
  early turns, and they compound the problem by bloating the prefix that
254
259
  rides along in every turn.
260
+
261
+ ---
262
+
263
+ ## 6. Empirical head-to-head: agent-mcp vs OpenCode (measured 2026-07-15)
264
+
265
+ Section 3 compares by *pattern*; this section is the *measured* run. It settles a
266
+ recurring claim that "the same task costs ~92K through agent-mcp but basically
267
+ nothing on OpenCode."
268
+
269
+ **Method.** Identical two prompts run on the **same model** (`deepseek-v4-flash`,
270
+ DeepSeek API) through both harnesses:
271
+ 1. *Tool-surface* — "describe your complete tool surface."
272
+ 2. *Packages+lint* — "list `packages/`, then run `nx lint <project>`, report the result."
273
+
274
+ Usage read from each tool's own store (not estimated): agent-mcp from `task_usage`
275
+ (`usage_query`); OpenCode from `~/.local/share/opencode/opencode.db` → `session`
276
+ (`tokens_input` = uncached/cache-miss, `tokens_cache_read` = cache-hit — verified by
277
+ reproducing OpenCode's own reported `cost` from the split). agent-mcp runs were
278
+ fresh ephemeral tasks on published **2.1.1** (post `BUG-ORCH-008` cache fix), after
279
+ the local `nx` graph was repaired (so no `nx` hangs inflate the call count).
280
+
281
+ **Result — the packages+lint task (the "92K" one):**
282
+
283
+ | metric | agent-mcp | OpenCode |
284
+ |---|---|---|
285
+ | system+tools **prefix per call** | **~5K** | **~32K** (`build` agent: full toolset every call) |
286
+ | **uncached input (billed full-price)** | **6,249** | **31,002** |
287
+ | cache-read input (cheap) | 47,616 | 129,408 |
288
+ | gross input (uncached + cache-read) | 53,865 | 160,410 |
289
+ | output | 1,590 | 509 |
290
+ | model calls | ~10 | ~4–5 |
291
+ | **cost** | **~$0.0015** (derived) | **$0.0049** (OpenCode-reported) |
292
+
293
+ DeepSeek rates back-solved from OpenCode's reported costs: uncached ≈ $0.14/M,
294
+ cache-read ≈ $0.003/M (~47× cheaper), output ≈ $0.28/M. agent-mcp does not emit a
295
+ `$` figure, so its cost is derived with those rates.
296
+
297
+ **Findings.**
298
+ 1. **The direction is opposite the claim.** On billed (uncached) tokens OpenCode
299
+ used **~5×** more; on cost, **~3×** more. agent-mcp is the *cheaper* harness on
300
+ these prompts, not 92K-worse.
301
+ 2. **Prefix size is the driver — and it confirms §3's thesis.** OpenCode ships a
302
+ ~32K system+tools prefix; agent-mcp ~5K (with `toolAdvertisement:"names"`, §4).
303
+ A fresh session's cold cache-miss therefore costs OpenCode ~31K full-price vs
304
+ agent-mcp ~6K. Caching makes both cheap in absolute terms (sub-cent), but the
305
+ **lean-prefix harness wins on the miss side.**
306
+ 3. **The "92K" was a units error.** The original agent-mcp figure was *gross input
307
+ with cache* (95% cache-hit → ~5K actually billed); comparing that to a billed
308
+ number overstates spend ~15×. It was further inflated to a bogus "1.13M" by the
309
+ per-event token fan-out in `agent-mcp-tail` — see `BUG-AGENTMCP-006`.
310
+ 4. **agent-mcp's real inefficiency is round-trips, not tokens.** It took ~10 calls
311
+ vs OpenCode's ~4–5, because its sandboxed shell forbids `&&`/pipes so the model
312
+ can't chain `ls && nx lint` in one call (§3: no tool pruning + fine-grained
313
+ tools). That costs **latency and gross tokens**, but caching absorbs the dollar
314
+ impact and the smaller prefix more than compensates on cost.
315
+
316
+ **Cross-references.**
317
+ - Cache correctness (why agent-mcp caches well on 2.1.1): `BUG-ORCH-008`
318
+ (cache-preserving compaction, shipped 2.1.1) — `BACKLOG.md`.
319
+ - Usage-telemetry gaps: `BUG-ORCH-009` + its 2026-07-15 follow-up (`usage_query`
320
+ `group_by`/`summary` still drop `uncachedInputTokens`) — `BACKLOG.md`.
321
+ - Measurement artifact that produced the bogus 1.13M: `BUG-AGENTMCP-006` (FIXED —
322
+ `agent-mcp-tail` now attaches usage only to the terminal event).
323
+ - Cost model this confirms: [`docs/ideas/context-and-cache-strategy.md`](../../../docs/ideas/context-and-cache-strategy.md)
324
+ §3 (cumulative billed input) and §6 (redesigned usage reporting).
325
+
326
+ **Caveat.** The measured OpenCode run is the permitted run. Earlier
327
+ non-interactive `opencode run` attempts appeared to "hang" — but that was a
328
+ harness-invocation error on the measurer's side, **not** a model, provider, or
329
+ OpenCode fault: the runs were launched **without granting permission to execute the
330
+ required tools** (e.g. `npx nx lint`), so OpenCode correctly blocked awaiting tool
331
+ approval, and the wait was then misread as a provider stall and killed. (A `pwd`
332
+ probe succeeding earlier was misleading — it did not require the same approval.)
333
+ Nothing here reflects on OpenCode's reliability. Cache warmth also differs between
334
+ harnesses (OpenCode's run was warm from prior calls, ~81% cache-read; agent-mcp
335
+ 88%), which affects the miss counts but not the qualitative conclusion — the
336
+ prefix-size gap dominates.
package/package.json CHANGED
@@ -1,28 +1,26 @@
1
1
  {
2
2
  "name": "@adhd/agent-mcp",
3
- "version": "2.1.1",
3
+ "version": "2.1.2",
4
4
  "type": "module",
5
5
  "bin": {
6
6
  "agent-mcp": "./src/index.js",
7
7
  "agent-mcp-tail": "./src/scripts/agent-mcp-tail.js"
8
8
  },
9
9
  "dependencies": {
10
- "@adhd/agent-store-runtime": "^2.1.0",
11
- "@adhd/agent-store-prompts": "^2.1.0",
12
- "@adhd/agent-engine-orchestrator": "^2.1.0",
10
+ "@adhd/agent-store-runtime": "^2.1.1",
11
+ "@adhd/agent-store-prompts": "^2.1.1",
12
+ "@adhd/agent-engine-orchestrator": "^2.1.1",
13
13
  "@modelcontextprotocol/sdk": "^1.29.0",
14
14
  "pino": "^10.3.1",
15
15
  "drizzle-orm": "^0.45.2",
16
16
  "better-sqlite3": "^12.10.0",
17
- "dotenv": "^17.4.2",
18
17
  "zod": "^4.4.3",
19
- "@adhd/agent-base-types": "^2.1.0",
20
- "@adhd/agent-engine-compiler": "^2.1.0",
18
+ "@adhd/environment": "^0.0.1",
19
+ "@adhd/environment-base-spec": "^0.0.1",
20
+ "@adhd/agent-base-types": "^2.1.1",
21
+ "@adhd/agent-engine-compiler": "^2.1.1",
21
22
  "tslib": "^2.3.0"
22
23
  },
23
- "devDependencies": {
24
- "@adhd/agent-base-types": "^2.1.0"
25
- },
26
24
  "main": "./src/index.js",
27
25
  "license": "MIT",
28
26
  "repository": {
@@ -36,6 +34,5 @@
36
34
  },
37
35
  "publishConfig": {
38
36
  "access": "public"
39
- },
40
- "module": "./src/index.js"
37
+ }
41
38
  }
package/src/config.d.ts CHANGED
@@ -1,27 +1,21 @@
1
+ import { Environment } from "@adhd/environment";
2
+ import type { EnvironmentSpec } from "@adhd/environment-base-spec";
1
3
  import type { EngineConfig } from "@adhd/agent-engine-orchestrator";
2
- export type GetProviderConfigOpts = {
3
- provider: "openai" | "anthropic" | "claudecli";
4
- secret?: string;
5
- url?: string;
6
- model?: string;
7
- inlineBaseURL?: string;
8
- inlineModel?: string;
9
- };
10
- export type ProviderConfigResolved = {
11
- secret?: string;
12
- baseURL?: string;
13
- model?: string;
14
- };
15
- export interface Config extends EngineConfig {
4
+ export interface AgentMcpConfig {
16
5
  readonly db: {
17
- readonly path: string;
6
+ readonly path: string | undefined;
18
7
  };
19
8
  readonly logging: {
20
9
  readonly level: string;
21
10
  };
22
- readonly server: EngineConfig["server"] & {
11
+ readonly queue: {
12
+ readonly concurrency: number;
13
+ };
14
+ readonly server: {
23
15
  readonly maxDepth: number;
24
16
  readonly maxToolLoops: number;
17
+ readonly defaultMaxTokens: number;
18
+ readonly contextLimit: number;
25
19
  readonly allowedAgents: readonly string[] | undefined;
26
20
  readonly registryDbPath: string;
27
21
  };
@@ -29,23 +23,58 @@ export interface Config extends EngineConfig {
29
23
  readonly kind: string;
30
24
  readonly port: number;
31
25
  };
32
- readonly sse: EngineConfig["sse"] & {
26
+ readonly sse: {
33
27
  readonly port: number;
34
28
  readonly host: string;
29
+ readonly baseUrl: string | undefined;
35
30
  };
36
- readonly security: {
37
- readonly envAllowlist: readonly string[];
38
- };
39
- readonly queue: EngineConfig["queue"];
40
- readonly plugins: EngineConfig["plugins"];
41
- getProviderConfig(opts: GetProviderConfigOpts): ProviderConfigResolved;
42
- resolveEnvRef(name: string): string | undefined;
43
- verifyEnvRefs(names: string[]): {
44
- missing: string[];
45
- disallowed: string[];
31
+ readonly plugins: {
32
+ readonly configPath: string | undefined;
33
+ readonly entries: readonly string[];
46
34
  };
47
- subprocessEnv(): Record<string, string>;
48
35
  }
49
- export declare function loadConfig(env?: NodeJS.ProcessEnv): Config;
50
- export declare const config: Config;
36
+ /**
37
+ * Exported so tests (and any other real consumer that needs an isolated
38
+ * instance — e.g. `__tests__/integration/harness.ts`) can construct their
39
+ * own `new Environment<AgentMcpConfig>('agent-mcp', agentMcpEnvironmentSpec, {...})`
40
+ * against a temp `adhdRoot` without touching the real machine's `~/.adhd`.
41
+ */
42
+ export declare const agentMcpEnvironmentSpec: EnvironmentSpec<AgentMcpConfig>;
43
+ export type GetProviderConfigOpts = {
44
+ provider: "openai" | "anthropic" | "claudecli";
45
+ secret?: string;
46
+ url?: string;
47
+ model?: string;
48
+ inlineBaseURL?: string;
49
+ inlineModel?: string;
50
+ };
51
+ export type ProviderConfigResolved = {
52
+ secret?: string;
53
+ baseURL?: string;
54
+ model?: string;
55
+ };
56
+ declare function getProviderConfig(opts: GetProviderConfigOpts): ProviderConfigResolved;
57
+ /**
58
+ * Snapshot of the current `process.env`, for subprocess spawning
59
+ * (`EngineConfig.subprocessEnv`, consumed by the `claudecli` provider and
60
+ * the stdio MCP-client transport to forward the parent's environment to a
61
+ * spawned child). Matches the pre-redesign `Config.subprocessEnv()`
62
+ * behavior exactly: the full current env snapshot, not prefix-filtered.
63
+ */
64
+ declare function subprocessEnv(): Record<string, string>;
65
+ export declare const env: Environment<AgentMcpConfig> & {
66
+ getProviderConfig: typeof getProviderConfig;
67
+ subprocessEnv: typeof subprocessEnv;
68
+ };
69
+ /**
70
+ * Builds the `EngineConfig`-shaped adapter the orchestrator layer (`server.ts`,
71
+ * `index.ts`) is injected with — the engine MUST NOT import `config.ts`
72
+ * directly (`packages/agent/agent-engine-orchestrator/src/interfaces.ts`).
73
+ * Derived live from `env.config` plus `env`'s own methods
74
+ * (`getProviderConfig`/`isEnvNameAllowed`/`subprocessEnv`) on every call, so
75
+ * it always reflects the current cascade (relevant for `at:'runtime'`/
76
+ * `secret` fields, though this spec declares none).
77
+ */
78
+ export declare function toEngineConfig(): EngineConfig;
79
+ export {};
51
80
  //# sourceMappingURL=config.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../../../../entrypoint/agent-mcp/src/config.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,iCAAiC,CAAC;AAIpE,MAAM,MAAM,qBAAqB,GAAG;IAChC,QAAQ,EAAE,QAAQ,GAAG,WAAW,GAAG,WAAW,CAAC;IAC/C,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,GAAG,CAAC,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,WAAW,CAAC,EAAE,MAAM,CAAC;CACxB,CAAC;AAEF,MAAM,MAAM,sBAAsB,GAAG;IACjC,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;CAClB,CAAC;AAEF,MAAM,WAAW,MAAO,SAAQ,YAAY;IACxC,QAAQ,CAAC,EAAE,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IACvC,QAAQ,CAAC,OAAO,EAAE;QAAE,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7C,QAAQ,CAAC,MAAM,EAAE,YAAY,CAAC,QAAQ,CAAC,GAAG;QACtC,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;QAC1B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;QAC9B,QAAQ,CAAC,aAAa,EAAE,SAAS,MAAM,EAAE,GAAG,SAAS,CAAC;QACtD,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAC;KACnC,CAAC;IACF,QAAQ,CAAC,SAAS,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IACrE,QAAQ,CAAC,GAAG,EAAE,YAAY,CAAC,KAAK,CAAC,GAAG;QAChC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;QACtB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;KACzB,CAAC;IACF,QAAQ,CAAC,QAAQ,EAAE;QAAE,QAAQ,CAAC,YAAY,EAAE,SAAS,MAAM,EAAE,CAAA;KAAE,CAAC;IAChE,QAAQ,CAAC,KAAK,EAAE,YAAY,CAAC,OAAO,CAAC,CAAC;IACtC,QAAQ,CAAC,OAAO,EAAE,YAAY,CAAC,SAAS,CAAC,CAAC;IAC1C,iBAAiB,CAAC,IAAI,EAAE,qBAAqB,GAAG,sBAAsB,CAAC;IACvE,aAAa,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAAC;IAChD,aAAa,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG;QAAE,OAAO,EAAE,MAAM,EAAE,CAAC;QAAC,UAAU,EAAE,MAAM,EAAE,CAAA;KAAE,CAAC;IAC5E,aAAa,IAAI,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CAC3C;AA0ID,wBAAgB,UAAU,CAAC,GAAG,GAAE,MAAM,CAAC,UAAwB,GAAG,MAAM,CAkHvE;AAED,eAAO,MAAM,MAAM,EAAE,MAAqB,CAAC"}
1
+ {"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../../src/config.ts"],"names":[],"mappings":"AAuBA,OAAO,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAChD,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,iCAAiC,CAAC;AAMpE,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,EAAE,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,SAAS,CAAA;KAAE,CAAC;IACnD,QAAQ,CAAC,OAAO,EAAE;QAAE,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7C,QAAQ,CAAC,KAAK,EAAE;QAAE,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAA;KAAE,CAAC;IACjD,QAAQ,CAAC,MAAM,EAAE;QACf,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;QAC1B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;QAC9B,QAAQ,CAAC,gBAAgB,EAAE,MAAM,CAAC;QAClC,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;QAC9B,QAAQ,CAAC,aAAa,EAAE,SAAS,MAAM,EAAE,GAAG,SAAS,CAAC;QACtD,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAC;KACjC,CAAC;IACF,QAAQ,CAAC,SAAS,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IACrE,QAAQ,CAAC,GAAG,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,OAAO,EAAE,MAAM,GAAG,SAAS,CAAA;KAAE,CAAC;IACrG,QAAQ,CAAC,OAAO,EAAE;QAAE,QAAQ,CAAC,UAAU,EAAE,MAAM,GAAG,SAAS,CAAC;QAAC,QAAQ,CAAC,OAAO,EAAE,SAAS,MAAM,EAAE,CAAA;KAAE,CAAC;CACpG;AAMD;;;;;GAKG;AACH,eAAO,MAAM,uBAAuB,EAAE,eAAe,CAAC,cAAc,CAsGnE,CAAC;AAMF,MAAM,MAAM,qBAAqB,GAAG;IAClC,QAAQ,EAAE,QAAQ,GAAG,WAAW,GAAG,WAAW,CAAC;IAC/C,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,GAAG,CAAC,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB,CAAC;AAEF,MAAM,MAAM,sBAAsB,GAAG;IACnC,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB,CAAC;AAyBF,iBAAS,iBAAiB,CAAC,IAAI,EAAE,qBAAqB,GAAG,sBAAsB,CAuB9E;AAED;;;;;;GAMG;AACH,iBAAS,aAAa,IAAI,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAM/C;AAMD,eAAO,MAAM,GAAG;;;CAGf,CAAC;AAEF;;;;;;;;GAQG;AACH,wBAAgB,cAAc,IAAI,YAAY,CAiB7C"}