solid_loop 0.0.4 → 0.0.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (56) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +445 -0
  3. data/README.md +305 -4
  4. data/Rakefile +5 -4
  5. data/app/controllers/solid_loop/application_controller.rb +6 -0
  6. data/app/controllers/solid_loop/dashboard_controller.rb +167 -22
  7. data/app/controllers/solid_loop/events_controller.rb +9 -1
  8. data/app/controllers/solid_loop/mcp_sessions_controller.rb +12 -0
  9. data/app/controllers/solid_loop/messages_controller.rb +11 -1
  10. data/app/controllers/solid_loop/tool_calls_controller.rb +32 -0
  11. data/app/helpers/solid_loop/application_helper.rb +4 -1
  12. data/app/helpers/solid_loop/metrics_helper.rb +232 -0
  13. data/app/jobs/solid_loop/janitor_job.rb +24 -0
  14. data/app/jobs/solid_loop/llm_completion_job.rb +2 -2
  15. data/app/models/solid_loop/base.rb +89 -6
  16. data/app/models/solid_loop/loop.rb +22 -0
  17. data/app/models/solid_loop/message.rb +33 -7
  18. data/app/models/solid_loop/tool_call.rb +7 -1
  19. data/app/services/solid_loop/adapters/native.rb +156 -23
  20. data/app/services/solid_loop/dialects/anthropic.rb +43 -10
  21. data/app/services/solid_loop/dialects/gemini.rb +55 -15
  22. data/app/services/solid_loop/dialects/open_ai.rb +17 -5
  23. data/app/services/solid_loop/dialects/reasoning_packer.rb +72 -7
  24. data/app/services/solid_loop/llm_usage_parser/llama.rb +18 -3
  25. data/app/services/solid_loop/mcp_session_initializer.rb +1 -0
  26. data/app/services/solid_loop/middlewares/agent_initialization.rb +1 -1
  27. data/app/services/solid_loop/middlewares/error_handling.rb +5 -1
  28. data/app/services/solid_loop/middlewares/event_logging.rb +3 -3
  29. data/app/services/solid_loop/middlewares/message_building.rb +38 -4
  30. data/app/services/solid_loop/middlewares/response_parsing.rb +19 -3
  31. data/app/views/layouts/solid_loop/admin.html.erb +134 -23
  32. data/app/views/solid_loop/dashboard/index.html.erb +330 -41
  33. data/app/views/solid_loop/events/index.html.erb +17 -1
  34. data/app/views/solid_loop/loops/index.html.erb +40 -3
  35. data/app/views/solid_loop/loops/show.html.erb +1 -1
  36. data/app/views/solid_loop/mcp_sessions/index.html.erb +40 -2
  37. data/app/views/solid_loop/messages/_message.html.erb +26 -35
  38. data/app/views/solid_loop/messages/index.html.erb +16 -2
  39. data/app/views/solid_loop/tool_calls/index.html.erb +18 -3
  40. data/db/migrate/20260819000100_solid_loop_add_retention_indexes.rb +22 -0
  41. data/docs/contributing/coverage.md +8 -8
  42. data/docs/decisions/mcp-server.md +4 -2
  43. data/docs/decisions/reasoning_persistence.md +120 -4
  44. data/docs/guides/dialects.md +1 -1
  45. data/docs/guides/mcp_transports.md +72 -0
  46. data/docs/validation.md +85 -0
  47. data/lib/solid_loop/configuration.rb +93 -0
  48. data/lib/solid_loop/engine.rb +8 -0
  49. data/lib/solid_loop/janitor.rb +110 -0
  50. data/lib/solid_loop/mcp/toolset.rb +119 -31
  51. data/lib/solid_loop/pipeline/builder.rb +31 -10
  52. data/lib/solid_loop/redaction.rb +26 -0
  53. data/lib/solid_loop/version.rb +1 -1
  54. data/lib/solid_loop.rb +47 -0
  55. metadata +7 -2
  56. data/lib/tasks/coverage.rake +0 -206
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e3bbfed94f57749cc453fa556177eb4aa339e304c41d5c6e2dd58f5f55cd7c55
4
- data.tar.gz: beda9759a9b2d1d576b007cc6782139b06c97922b7ce0bb0117685190d437f93
3
+ metadata.gz: bd93f901cbf4287ccdebd6779430f67cca0e4def0f73c159df3585f31c1644c8
4
+ data.tar.gz: 246d0fdda33059ed2c5a53d6356b7129f16c5c4c332d470966d366bc15eb659a
5
5
  SHA512:
6
- metadata.gz: b9447a7363162eac0dcc9c23964855897a427a09a50c7af701e07cbea1edd65967f72766b9e6d419a8312189d35492bf67c1a0253815c779b53e73aadb6e28a0
7
- data.tar.gz: 3477debd8e6e9fa6b8b047fb4e7b0cb2cd36aa53edeedfa37c99e05ab1c213a0fc2618f92a22b7edb3e0a014f8d49b48dbf4400728ba4bd4f73819bc048b8a59
6
+ metadata.gz: 2b742a5195cc53a0e5223b18eab5c18288fb87b8df7554c91cb7c55fd739f79d48dc18185688459fc9a422c333ebbc62ce5c1ae51d33a3d4e459e25d5348c8a6
7
+ data.tar.gz: 32d43a5bb2b7d2c9802ae12acd8212e6c6c3eaad7feffb2c2122775493071569bd077a5699218498bfd840aca15dd1c428f50e8d48ca102932c4bfe0f3af3cfa
data/CHANGELOG.md CHANGED
@@ -5,6 +5,451 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [0.0.5] - 2026-08-20
9
+
10
+ #### Fixed
11
+ - **A rejected tool call put `"name": null` on the wire and killed the loop
12
+ permanently on strict servers.** When a model sends tool-call arguments that
13
+ are not valid JSON, SolidLoop answers with a "Tool call rejected" tool message
14
+ asking it to resend — correct, but that call never becomes a `ToolCall` row,
15
+ so the `tool_call_id -> function_name` lookup for the next request resolved to
16
+ `nil` and the tool message went out with a null `name`. vLLM ignored it;
17
+ llama.cpp rejects the whole request with `Failed to parse messages: type must
18
+ be string, but is null`, and because the offending message stays in history
19
+ the loop then fails on **every** retry rather than recovering. The lookup is
20
+ now seeded from the assistant message's `tool_calls_raw` (where the name is
21
+ still present) before the persisted rows override it, and `name` is omitted
22
+ rather than sent as null when it cannot be resolved at all.
23
+ - **The `gemini` dialect read thought parts wrong, so on a thinking model the
24
+ reasoning and the answer swapped places.** `thought` on a Gemini part is a
25
+ *boolean flag* — the summary text sits in `text`, the same key an ordinary
26
+ answer uses. The dialect read `part["thought"]` as the reasoning (yielding
27
+ `true`) and then took the first part carrying a `text` as the answer, which on
28
+ a thinking response is the thought part. It also looked for `thoughtSignature`
29
+ on the candidate, where it never appears: signatures ride on a part — the
30
+ thought part, or the `functionCall` part on a tool-calling turn — so no
31
+ signature was ever captured and `:gemini_signed` could not work. Outbound
32
+ parts were built in the same wrong shape (`{ thought: "<text>" }`). The test
33
+ emulator encoded the same misreading, which is why the suite stayed green.
34
+ Verified against `ruby_llm`'s Gemini protocol, which handles all four cases.
35
+
36
+ #### Changed
37
+ - **Reasoning from earlier turns now goes back in each provider's own field
38
+ instead of being welded into `content`.** `reasoning_strategies` defaulted to
39
+ `[:xml]` for every dialect, which prepends `<think>…</think>` to the assistant
40
+ message body — the shape no provider can strip, and the shape every provider
41
+ bills. Because it lives in the body it is re-sent and re-charged on every
42
+ subsequent turn and compounds over a run; a long agent loop can end up
43
+ spending a large fraction of its context window on repacked reasoning, and
44
+ hit the window ceiling because of it.
45
+
46
+ Measured with vLLM's `/tokenize` on the Qwen3 chat template, one six-message
47
+ conversation cost **1438** prompt tokens welded, **108** in
48
+ `reasoning_content`, and **108** with the reasoning omitted entirely — the
49
+ template drops the field from history, as Qwen3's own guidance asks. The
50
+ equality is the point: welding is not a different encoding of the same
51
+ request, it re-shows the model its own past thoughts as ordinary text, and on
52
+ this template it costs ~13x the tokens to do something the model's authors
53
+ advise against.
54
+
55
+ `SolidLoop::Base#reasoning_strategies` now defaults to `nil`, meaning "ask the
56
+ dialect", and each dialect answers with the shape its provider actually has a
57
+ slot for:
58
+
59
+ | Dialect | Default | Wire shape |
60
+ | --- | --- | --- |
61
+ | `open_ai` | `["reasoning_content"]` | `reasoning_content` on the assistant message |
62
+ | `anthropic` | `[:anthropic_signed]` | `{ type: "thinking", thinking:, signature: }` block, first in `content` |
63
+ | `gemini` | `[:gemini_signed]` | `{ thought: true, text:, thoughtSignature: }` part |
64
+
65
+ The two signed shapes carry a provider-issued signature, which the dialects
66
+ now capture into `message.metadata["thought_signature"]` (`anthropic` did not
67
+ read it at all before). Both providers reject an unsigned block, so the signed
68
+ strategies are all-or-nothing: **messages written before this release have no
69
+ signature**, and a resumed older loop sends no reasoning for those turns
70
+ rather than a payload that would 400. Write `[:anthropic_signed, :xml]` to
71
+ keep the old welding as a fallback — `:xml` now fires exactly when a signed
72
+ strategy could not, instead of being suppressed by its mere presence.
73
+
74
+ **This changes the wire payload for hosts that never set
75
+ `reasoning_strategies` explicitly.** Restore the old behaviour with
76
+ `def reasoning_strategies = [:xml]`. A subclass that called `super` and
77
+ appended to the result must now handle `nil`. The token saving is specific to
78
+ templates that drop the field; re-run the comparison against a new model
79
+ before assuming it carries over.
80
+
81
+ #### Added
82
+ - **"Tools as typed output" is documented, with a supported way to end a loop
83
+ from the host.** Where tools are a typed output format rather than a
84
+ conversation — a batch turn of twenty atomic writes, one call per labelled
85
+ item, one per file in a codemod — the loop still schedules a further turn once
86
+ every call succeeds, and that turn is a paid request whose content is the word
87
+ "done". The README now shows a middleware that ends such a loop itself,
88
+ reaching a terminal state indistinguishable from a model-driven completion one
89
+ request sooner, and spells out the three details that are load-bearing:
90
+ placement before `NetworkCalling`, returning without calling downstream, and
91
+ keeping `expected_execution_token:`. No new API — it uses
92
+ `configure_llm_middlewares` and `Loop#transition_status`, both already public —
93
+ but the behaviour is now pinned by
94
+ `spec/integration/host_driven_completion_spec.rb` so it cannot break silently.
95
+ The section also says why raising `LimitExceededError` instead is a trap: it
96
+ ends in `paused`, which `resume!(from: %i[paused failed])` is documented to
97
+ pick back up.
98
+ - **`docs/validation.md` — what SolidLoop has been run against, and what it has
99
+ not.** 0.0.5 was exercised under sustained agentic load on local inference
100
+ (llama.cpp with `--jinja` / `--kv-unified` and vLLM, Qwen3 27B, two consumer
101
+ GPUs) through the `open_ai` dialect; that surface is stricter than the hosted
102
+ API and is where the `"name": null` defect above was found. The page is equally
103
+ explicit about the gaps: the native `anthropic` and `gemini` dialects have
104
+ never made a live call, hosted providers were last exercised in 0.0.4 through
105
+ the `open_ai` dialect, and the durability design has not had a long production
106
+ soak. Reports from other providers and scales are welcome.
107
+ - **The cost of carried reasoning is now a number on the message.** Each
108
+ assistant message's `metadata["reasoning_carried"]` records what that
109
+ request's prompt spent on earlier turns' reasoning — `messages`, `chars`,
110
+ `inline_chars` (the part welded into `content`, which every provider bills)
111
+ and `tokens_est`. Previously the only way to see it was to reconstruct it from
112
+ message lengths after the fact, which is why the 40% above went unnoticed.
113
+ - **`Base#resume!` takes `from:` to narrow which statuses it will re-enter.**
114
+ It has always resumed `completed` loops as well as `init`/`paused`/`failed`,
115
+ which is right for continuing a finished conversation and wrong wherever a
116
+ loop has already reported a final result: the resumed turn appends to a run
117
+ that was over. Hosts had no way to say "restart what stalled, leave finished
118
+ runs alone" short of checking the status themselves before calling. The
119
+ default is unchanged (`SolidLoop::Loop::RESUMABLE_STATUSES`), and passing a
120
+ status resume can never reach (`running`, `queued`) raises `ArgumentError`
121
+ instead of no-opping forever.
122
+ - **Automatic retry of transient LLM failures.** A 429 or 5xx from a hosted
123
+ provider killed the whole loop on first occurrence — including a shared-model
124
+ 429 that carried `Retry-After: 30`, i.e. the provider's own instruction to
125
+ wait was discarded and the run failed instead. `Adapters::Native` now re-sends
126
+ such a response up to `SolidLoop.config.llm_retries` times (default 2) with
127
+ exponential backoff, honouring `Retry-After` (delta-seconds and HTTP-date)
128
+ over its own curve and capping any single wait at `llm_retry_max_delay`.
129
+ Configurable via `llm_retries`, `llm_retry_base_delay`, `llm_retry_max_delay`
130
+ and `llm_retry_statuses` (default `[408, 409, 429, 500, 502, 503, 504]`;
131
+ 400/401/403/404/422 are never retried). Two boundaries are deliberate: a read
132
+ **timeout** is not retried (the full `read_timeout` already elapsed, the
133
+ request may have been served, and repeating it multiplies the turn's wall
134
+ clock), and a stream that has already handed a chunk to the caller is not
135
+ retried (the assistant shell holds partial content, so a re-send would
136
+ concatenate two generations — the latch is set where content is handed over,
137
+ not when bytes arrive, so a streamed *error body* stays retryable). Backoff
138
+ waits poll the cancellation check, so a pause/stop lands within a second
139
+ instead of after the full delay. Retries appear in the wire log; `ttft` and
140
+ `duration` describe the attempt that produced the answer.
141
+ - **Retention for events and inbound MCP sessions.** `solid_loop_events` (one
142
+ row per LLM turn, tool call and inbound MCP request) and
143
+ `solid_loop_mcp_inbound_sessions` grew without bound, with no way to trim
144
+ them. `SolidLoop.prune!` (via the new `SolidLoop::JanitorJob`, on the existing
145
+ `solid_loop_maintenance` queue) now applies `event_retention` and
146
+ `mcp_inbound_session_retention`. **Off by default** — `nil` means keep
147
+ forever, because silently deleting a host's audit trail on upgrade would be
148
+ the wrong default. Deletes in batches of `janitor_batch_size` (default 1000)
149
+ so a sweep never takes one long lock on a table being written to; idempotent
150
+ and safe to run concurrently. Sessions are pruned by *creation* age rather
151
+ than last use, so a client that never terminates and never returns cannot pin
152
+ its row forever, and a doomed session's events go with it even when
153
+ `event_retention` is unset. `SolidLoop.last_pruned_at` mirrors
154
+ `last_reaped_at` as a liveness signal. Additive migration
155
+ `SolidLoopAddRetentionIndexes` adds the `created_at` indexes the sweep needs,
156
+ built `CONCURRENTLY` so it does not lock an append-only hot table.
157
+ - **`Mcp::Toolset` answers `ping`.** The base protocol requires a prompt reply
158
+ with an empty result; the gem returned "method not found", which well-behaved
159
+ clients read as a dead server.
160
+ - **`Mcp::Toolset.server_version`.** `initialize` reported
161
+ `serverInfo.version = SolidLoop::VERSION`, so every mounted server announced
162
+ the *gem's* version — semantically wrong (this is the version of the host's
163
+ MCP server) and a needless disclosure of an internal detail. Declared next to
164
+ `server_name`, inherited by subclasses, and still defaulting to
165
+ `SolidLoop::VERSION`.
166
+ - **Dynamic tool and prompt catalogs.** The `tool` / `prompt` DSL registers at
167
+ class-load time, which is wrong for a host whose catalog lives in the database
168
+ and changes without a deploy — the only way out was to override `deliver` and
169
+ intercept `prompts/list` / `prompts/get`. Four overridable instance methods —
170
+ `available_tools`, `find_tool`, `available_prompts`, `find_prompt` — default
171
+ to the DSL and are consulted on every request and by `capabilities`, so a row
172
+ added a second ago is advertised on the next `prompts/list`.
173
+ - **`SolidLoop::FrozenLoopError`.** `Base#resume!` on an admin-frozen loop
174
+ raised a bare `StandardError` carrying "Unfreeze in admin UI before resuming
175
+ this loop", forcing hosts to match on the message text to tell an expected
176
+ user-facing case (a 422) from a real fault — so rewording the message would
177
+ silently turn their 422 into a 500. Now typed; still a `StandardError`, so
178
+ existing rescues keep working.
179
+ - **`SolidLoop::Base#reasoning_effort` — a first-class hook, translated per
180
+ dialect.** Passing effort through `llm_params` only ever worked on the OpenAI
181
+ dialect, because the Anthropic and Gemini dialects rebuild the payload from
182
+ scratch in `render_payload` and drop keys they do not know. The hook now sets a
183
+ canonical `reasoning_effort` on the payload (mirroring how `max_tokens` works)
184
+ and each dialect renders its own shape: OpenAI keeps the top-level
185
+ `reasoning_effort`, Anthropic emits `output_config: { effort: ... }`, and
186
+ Gemini emits `generationConfig.thinkingConfig.thinkingLevel` upper-cased.
187
+ Defaults to `nil`, sending nothing. The level vocabulary differs by provider
188
+ (OpenAI-compatible: minimal/low/medium/high; Anthropic adds xhigh and max;
189
+ Gemini: LOW/MEDIUM/HIGH), so the value is passed through rather than validated
190
+ — the provider is the authority on its own vocabulary. Note Gemini 2.5 takes a
191
+ numeric `thinkingBudget` instead, which this hook does not emit.
192
+ - **`SolidLoop::Base#llm_params` — host-supplied LLM payload keys.** The built
193
+ payload (`model`, `messages`, `stream`, `tools`) was closed, so a host could
194
+ not send anything the gem does not model itself. An agent may now return a
195
+ hash that `MessageBuilding#build_payload` deep-merges over the payload as its
196
+ last step — `reasoning_effort`, `temperature`, `top_p`, `max_tokens`, or any
197
+ provider extension. Merged last, so a host can also override a gem-set key
198
+ (e.g. force `stream: false` for one agent). String keys are symbolized so they
199
+ cannot produce a duplicate entry alongside a gem-set key. Defaults to `{}`,
200
+ leaving the payload byte-identical for agents that do not override it. Keys
201
+ reach the wire verbatim under the OpenAI dialect; the Anthropic and Gemini
202
+ dialects re-render the payload, so only keys those dialects read survive.
203
+
204
+ #### Changed
205
+ - **The `Mcp::Toolset` envelope builders are public API.** Overriding `deliver`
206
+ is the documented extension seam ("the toolset IS the transport"), yet the
207
+ helpers an override needs to answer a request — `success_envelope`,
208
+ `error_envelope`, `json_boundary`, `tool_result` — were private, so every
209
+ override either reached through `send` or depended on private methods a patch
210
+ release could rename. They are now public and covered by the compatibility
211
+ promise.
212
+ - **Admin list pages show latency, liveness and wire outcome.** Every list page
213
+ persisted the numbers that make a run diagnosable and then declined to display
214
+ them, so the admin could say *that* something happened but never *how long it
215
+ took* or *whether it worked* — a 608-second LLM read timeout had to be found by
216
+ querying the database by hand. Concretely:
217
+ - **Tool Calls** gains a DURATION column (`duration` was already persisted) and
218
+ a LOOP column linked like the Events page, so a 0.24s `solve_task` and a 40s
219
+ `fetch_url` are distinguishable and rows can be grouped by run without
220
+ opening each one. Slow calls are colour-flagged rather than left as a number
221
+ to be read. The FUNCTION filter was a free-text box the reader had to spell
222
+ from memory; it is now a select built from the discovered MCP tools (which is
223
+ the small, bounded side of that lookup — `DISTINCT function_name` over the
224
+ ever-growing tool_calls table is not), and a filter naming a tool no longer
225
+ advertised stays selectable instead of being silently reset to "All".
226
+ - **Events** gains STATUS and DURATION. There is no status column in the
227
+ schema — the outcome is buried in `response_data`, whose shape depends on
228
+ which middleware wrote it — so it is now derived once: a persisted
229
+ `http_status`, a transport/exception `error`, an MCP `isError` result (2xx
230
+ but the tool failed, flagged amber rather than red), or an implied 200. An
231
+ event with nothing recorded reads as "—", never as a success. Duration
232
+ thresholds are per-family: a minute-long local LLM completion is normal, a
233
+ minute-long MCP call is pathological, and one absolute threshold would have
234
+ painted the page red and hidden the outliers again. Event badges were all
235
+ the same blue; the LLM and MCP families now differ, and the MCP lifecycle
236
+ chatter (`mcp_initialize` / `mcp_list_tools`) recedes into a quiet outline.
237
+ - **Loops** gains WORK (`Loop#work_duration` — generation + tools, what
238
+ `max_duration` budgets against), TOKENS (`tokens_total`) and LAST ACTIVITY.
239
+ A loop that has worked 90 minutes used to look identical to one that was
240
+ created 90 minutes ago and is hung. Last activity is derived from the loop's
241
+ `updated_at`: the row is written at every turn commit AND by every lease
242
+ renewal (`LeaseRenewer` sets `updated_at` explicitly), so it advances *during*
243
+ a long generation, whereas messages are only written at turn boundaries and
244
+ streaming appends use `update_columns`, which skips timestamps — and it needs
245
+ no join. An active loop idle for minutes is flagged as likely stalled. COST
246
+ lost its column: under local inference it read $0.00 on every row, so it now
247
+ rides along under the token count and only when it is non-zero.
248
+ - **MCP Sessions** no longer spends half the table on a 48-character session
249
+ UUID: it is truncated head-and-tail with a copy-to-clipboard button carrying
250
+ the full value. The reclaimed width shows the owning loop (with its status)
251
+ and whether the session is still usable — a session is only ever reused while
252
+ its loop can still run, so a completed/failed loop's session is history —
253
+ plus how many tools discovery actually produced.
254
+ - **Messages** shows `tokens_total` with the in/out split beneath it instead of
255
+ the cryptic "P0+C0", falling back to the sum while a message is still
256
+ streaming and its total has not been written yet.
257
+
258
+ Associated data is fetched in bounded, narrow queries (one `pluck` per page for
259
+ the tool-call → loop mapping and the session → loop/tool counts) rather than
260
+ per row, and the Events and Messages lists now select only the columns they
261
+ render: `log` and `request_data` alone were tens of KB per event, so a 50-row
262
+ page was hauling megabytes of wire payload to draw a badge.
263
+ - **Admin message cards are compact and legibly bordered.** The card body was
264
+ `white-space: pre-wrap`, so the ERB template's own indentation and blank lines
265
+ rendered as content — the first line of every message came out indented and
266
+ large gaps opened above the footer. Each block now owns its own `<pre>` and the
267
+ container does not preserve whitespace. `--sl-border-color` was darker than the
268
+ card it outlined and read as no edge at all; it is now lighter than the card
269
+ surface. Padding and gaps are tightened throughout, and reasoning is collapsed
270
+ behind a `<details>` toggle — it routinely runs to thousands of characters and
271
+ used to push the per-turn metrics off-screen.
272
+ - **`max_duration` budgets accumulated WORK, not wall clock.** It was checked as
273
+ `loop.created_at <= max_duration.ago`, so every minute a loop spent paused,
274
+ failed, or waiting in a queue burned budget exactly like a minute of
275
+ generation. A loop paused overnight was dead on arrival the moment it resumed,
276
+ and one resumed after an outage could exhaust its budget having produced
277
+ nothing. It now checks `Loop#work_duration` — `duration_generation +
278
+ duration_tools`, both already accumulated on the success and failure paths —
279
+ which is derived rather than stored, so it cannot drift from its inputs. Work
280
+ lost to a hard kill is not counted, erring in the safe direction. Loops that
281
+ previously died on wall clock now get the budget they were meant to have;
282
+ hosts relying on `max_duration` as a hard wall-clock stop should add their own.
283
+ - **`Pipeline::Builder` positional edits raise instead of guessing.**
284
+ `insert_before` silently inserted at the FRONT when the target was not found,
285
+ `insert_after` silently APPENDED, and `replace` silently appended — so a typo,
286
+ a renamed middleware, or a stale constant quietly rebuilt the pipeline instead
287
+ of failing. The damage lands far from the cause: a middleware moved ahead of
288
+ `AgentInitialization` sees an unpopulated context (`env.model_config` is nil),
289
+ and one moved past the terminal link may never run. All three now raise
290
+ `Pipeline::Builder::UnknownMiddleware`, whose message lists the stack in order
291
+ and points at stale class objects as a likely cause. `delete` stays lenient:
292
+ "this must not run" is unambiguous whether or not it is currently present.
293
+ - **Anthropic dialect default `max_tokens`: 8096 → 64000** (now the named
294
+ `Dialects::Anthropic::DEFAULT_MAX_TOKENS`). Anthropic *requires* `max_tokens`
295
+ on every request, so this dialect must supply one when the agent does not —
296
+ unlike the OpenAI path, where an absent key lets the provider decide. The old
297
+ value was an unexplained magic number (apparently a typo for 8192) an order of
298
+ magnitude below the recommended default for streaming requests, which is what
299
+ SolidLoop does by default; current Claude models accept up to 128k output
300
+ tokens. A low cap truncates a turn mid-`tool_use`, costing a whole agent step
301
+ rather than a sentence. Agents that set `Base#max_tokens` are unaffected;
302
+ agents relying on the default will see longer completions and correspondingly
303
+ larger bills — set `max_tokens` explicitly to keep a tighter ceiling.
304
+
305
+ #### Fixed
306
+ - **llama.cpp prompt tokens were undercounted by the entire cached prefix.**
307
+ `LlmUsageParser::Llama` read `timings["prompt_n"]` as the prompt, but that
308
+ field counts only the tokens the request had to PROCESS; the reused prefix is
309
+ reported separately as `cache_n`. The prompt is their sum. The error compounds
310
+ across an agent loop: in a 30-turn run the recorded prompt fell to ~500 tokens
311
+ while the real context was ~37k, and 127 of 169 recorded turns stored
312
+ `tokens_prompt_cached > tokens_prompt` — impossible if `prompt_n` were the
313
+ whole prompt. Any capacity planning done from these figures (context per task,
314
+ how many turns fit a KV budget) was off by roughly an order of magnitude.
315
+ - **`tokens_prompt_cached` is clamped to the prompt at the persistence
316
+ boundary.** Cached tokens are a subset of the prompt; a provider reporting the
317
+ two on different bases could previously persist a cache figure larger than the
318
+ prompt it came from, so any derived hit rate exceeded 100%. Clamped in
319
+ `ResponseParsing` rather than in one parser, because the invariant belongs to
320
+ the row.
321
+ - **`Message#tps` is now persisted for every dialect, not just llama.cpp.** Only
322
+ the llama parser reported a rate; the OpenAI, Anthropic and Gemini parsers
323
+ hardcoded `0.0`, so the column stayed zero and every SQL aggregate over it read
324
+ as "zero throughput" rather than "not measured" — silently, since 0.0 looks
325
+ like a number. `ResponseParsing` now derives the rate from the decode window
326
+ when the provider does not report one, and `Message#tps` prefers the persisted
327
+ column so a Ruby read and a SQL aggregate cannot disagree (it still derives for
328
+ rows written before this change). Derivation is guarded by
329
+ `Message::MIN_DECODE_WINDOW`: when a whole response arrives in one chunk the
330
+ decode window collapses toward zero and the naive quotient reached ~9200 tok/s
331
+ on real traffic, so such turns store 0.0 and are excluded from averages.
332
+ - **`loop.error_message` is now redacted like the wire log.** `EventLogging`
333
+ ran its payloads through `redact_credentials` while `ErrorHandling` wrote the
334
+ raw exception message straight onto the loop. Harmless for header-auth
335
+ dialects, but the Gemini dialect carries the API key in the URL query
336
+ (`?key=…`) and a Faraday error message embeds the URL — so the key reached
337
+ `error_message`, the admin UI, and any host API surfacing it. The rules moved
338
+ to `SolidLoop::Redaction` and are now applied at every site that persists
339
+ error text (`ErrorHandling`, `ResponseParsing`, `LlmCompletionJob`,
340
+ `McpSessionInitializer`).
341
+ - **The gem's dev-only rake task no longer leaks into host applications.**
342
+ `lib/tasks/coverage.rake` was auto-loaded into every host's Rake by
343
+ `Rails::Engine`. A host with its own `coverage:report` got both bodies
344
+ executed (Rake *merges* same-named tasks), and because a `namespace` block is
345
+ not a Ruby module, `GROUPS` and `CoverageReporter` were defined on `Object` —
346
+ colliding with the host's constants on every `rake`/`rails` invocation. Moved
347
+ to `tasks/coverage.rake` (loaded explicitly by the gem's own Rakefile, and
348
+ outside the packaged file list) with its constants under `SolidLoopDev`.
349
+ - **README's OpenAI `base_url` example was wrong.** It showed
350
+ `https://api.openai.com/v1`, but `Dialects::OpenAi#completion_url` appends
351
+ `/v1/chat/completions` itself — following the README produced
352
+ `/v1/v1/chat/completions` and every request failed.
353
+ - **`subject_resolver`'s documented example could not work, and its links were
354
+ unreadable.** The doc comment showed `main_app.user_path(...)`, but the
355
+ resolver is invoked plainly as `resolver.call(loop)` and never `instance_exec`'d
356
+ in a view, so `main_app` is not in its binding — the example is replaced with a
357
+ runnable one using `Rails.application.routes.url_helpers`, plus the two
358
+ properties a resolver must have: return nil for subjects it does not handle,
359
+ and never raise. The rendered link also carried no class, so it fell back to
360
+ the browser's default link blue, which is nearly unreadable on the dark admin
361
+ background (the muted colour on the wrapping element does not cascade into an
362
+ anchor).
363
+
364
+ - **Admin dashboard: charts that told the truth badly.** Six defects, all in the
365
+ presentation layer, that together made the page harder to read than the raw
366
+ JSON behind it.
367
+
368
+ *Bar colour meant nothing.* `chartOptions.colors` carried three colours, and
369
+ Chartkick spreads a colour array across the BARS of a single-series chart, not
370
+ across series. So on LLM Calls 5 PM was blue, 6 PM green and 7 PM red — red,
371
+ the colour every other surface in this admin uses for failure, applied to an
372
+ hour purely because it was third in the list. Single-series charts now pin one
373
+ colour to the whole dataset and red is reserved for failure data.
374
+
375
+ *Time buckets were per-chart.* `date_trunc` only emits buckets that have rows,
376
+ so a quiet hour vanished from one panel and not from its neighbour: Tokens
377
+ showed 5/7/8 PM beside an LLM Calls showing 5–8 PM, and the two panels could
378
+ not be compared by eye at all. Every series is now zero-filled onto one
379
+ generated bucket list. That also surfaced a silent data bug: the window
380
+ `24.hours.ago` needs 25 hourly buckets, two of which format to the same label
381
+ ("9 PM"), and Chartkick keys a categorical series BY LABEL — the duplicate
382
+ collapsed and the current hour's bar was drawn on top of yesterday's, 24
383
+ columns from where it belonged. The window is now aligned down to a bucket
384
+ boundary: exactly 24/30/12 whole buckets, unique labels, and KPI totals that
385
+ equal the sum of the bars beneath them.
386
+
387
+ *Tokens was drawn sideways.* It was a horizontal `BarChart` between two
388
+ vertical `ColumnChart`s despite being the same value-per-time-bucket shape, it
389
+ printed raw six-figure counts on its axis ("111401"), and it carried a legend
390
+ for its one and only series. It is now a `ColumnChart` beside its siblings,
391
+ with compact axis ticks ("111.4k") and a tooltip that also gives the exact
392
+ count with thousands separators.
393
+
394
+ *Three panels were permanently $0.00.* Hosts running local inference
395
+ (llama.cpp) have no cost, so the Cost tile, the Cost line chart and Top Agents
396
+ by Cost were dead weight — the last with its axis helpfully scaled to $1
397
+ against three zero-length bars. When total cost for the window is zero those
398
+ panels are replaced by two that were never surfaced anywhere in the admin
399
+ despite the data being recorded all along: *Loops by status* (the first thing
400
+ an operator wants to know) and *Throughput* — average TPS and TTFT from
401
+ `solid_loop_messages.tps` / `.ttft`, the numbers that actually move when you
402
+ change model or hardware. Only rows that measured something are averaged, and
403
+ each metric reports its own sample count, because the two are not populated
404
+ together: `tps` comes from providers that report their own timings, `ttft`
405
+ only from the streaming path.
406
+
407
+ *There was no loading or empty state.* Until the XHR resolved the tiles read
408
+ "-" and four large boxes were blank — indistinguishable from a period with no
409
+ activity, and from a request that had failed. Loading, empty and error are now
410
+ three distinct, labelled states.
411
+
412
+ *And the KPI tiles never said what they counted.* "LLM CALLS 75" over what
413
+ window? The Hour/Day/Week toggle was invisible in the numbers it changed.
414
+ Every tile and panel now names its window, and the stats endpoint returns it.
415
+ - **Middleware stacks are rebuilt on reload (`SolidLoop.reset_middlewares!`).**
416
+ `llm_middlewares` / `tool_middlewares` are memoized on the `SolidLoop` module,
417
+ which lives in `lib/` and is never reloaded, while the middleware classes live
418
+ in the engine's `app/` and ARE reloaded in development. The memo therefore
419
+ kept class objects from an earlier load, and since `Array#index` compares by
420
+ `==`, every positional `Builder` call against a current constant missed its
421
+ target. Combined with the silent fallback above this was invisible and
422
+ destructive: host middleware landed at position 0 and crashed on a nil
423
+ context, a host `to_prepare` that re-inserted on every reload accumulated
424
+ duplicate copies, and a tool middleware could stop running entirely — silently
425
+ dropping the data it was there to record. The Engine now resets the memo on
426
+ `to_prepare`, before host initializers re-apply their own customization.
427
+ Development-only in effect; production never reloads.
428
+ - **The admin loop page showed a permanently-zero "Total Duration".** It read
429
+ `Loop#duration_total`, a column nothing has ever written (like `max_tokens`
430
+ above, a declared field with no producer). It now shows `work_duration`. The
431
+ vestigial column is left in place rather than dropped in a migration.
432
+ - **`resume!` sweeps orphaned `processing` assistant shells**, mirroring the
433
+ cleanup `pause!` already did. A shell is born hidden + `processing` at the
434
+ start of a streaming turn; the graceful-error path only finalizes the shell
435
+ its own attempt created, and its token-fenced CAS no-ops once a resume has
436
+ rotated the generation — so a turn that died while (or just before) the loop
437
+ was resumed left its shell `processing` forever. Nothing else swept it: the
438
+ reaper only sweeps shells off paused/terminal loops. The shell stays hidden,
439
+ so this never affected the payload sent to the provider — it accumulated
440
+ phantom "in flight" rows in the admin UI and made `processing` stop meaning
441
+ "a turn is actually running".
442
+ - **`max_tokens` reaches the provider, and only when you ask for it.**
443
+ `SolidLoop::Base#max_tokens` was a declared agent hook that nothing ever wrote
444
+ into the payload, so overriding it did nothing: the Anthropic dialect always
445
+ fell back to its own `8096` and the Gemini dialect dropped the key. It is now
446
+ written into the payload by `MessageBuilding#build_payload` — but only when
447
+ set. The default changed from `16 * 1024` to `nil`, meaning **no `max_tokens`
448
+ key is sent at all** unless an agent opts in, letting the provider apply its
449
+ own default rather than a silent gem-chosen cap. Agents that never overrode
450
+ the hook see byte-identical requests; agents that did now get the value they
451
+ asked for.
452
+
8
453
  ## [0.0.4] - 2026-07-16
9
454
 
10
455
  #### Fixed