solid_loop 0.0.4

data/docs/decisions/mcp-server.md

@@ -0,0 +1,223 @@
+# Decision: MCP Server (serving toolsets over Streamable HTTP)
+
+> **Status: implemented in 0.0.4.**
+
+**Date:** 2026-07-09
+**Builds on:** [mcp-only-tooling.md](mcp-only-tooling.md) — toolsets as logical MCP
+servers; this doc settles the open questions listed in its out-of-scope section.
+
+## Decision
+
+`SolidLoop::Mcp.server(toolset, auth:)` builds a mountable Rack endpoint that
+serves **one toolset** to external MCP clients over Streamable HTTP. Exposure
+is the mount itself — no registry, no global config:
+
+```ruby
+# config/routes.rb (host app)
+mount SolidLoop::Mcp.server(RpcTools,    auth: McpAuth.new)      => "/rpc"
+mount SolidLoop::Mcp.server(ReportTools, auth: ReportAuth.new)   => "/tools"
+# AdminTools is simply not mounted — it stays in-process only.
+```
+
+Each endpoint is the mirror image of `Mcp::HttpTransport` over the same port:
+the POST body is parsed and handed to `Toolset#deliver` — the exact method the
+in-process agent path calls. There is no second dispatch implementation, so
+local and remote behavior cannot drift. Define a toolset once; the durable
+agent loop consumes it in-process, external agents (Claude Code, other apps)
+consume the same tools remotely, with shared schemas, wire logs and admin.
+
+## Why mount-per-toolset (not one engine with a registry)
+
+An earlier sketch had a single `ServerEngine` mount plus a
+`config.mcp_server = { toolsets: [...] }` registry with path-per-`server_name`
+routing. Rejected in favor of one mount per toolset:
+
+- **Exposure is explicit and local.** What is served is exactly what is
+  mounted, readable in `routes.rb`; keeping a toolset internal means *not
+  writing a line*, instead of remembering to exclude it from a registry.
+- **The host owns the paths.** `/rpc`, `/tools`, `/api/mcp/search` — any
+  path, independent of `server_name`; no coupling between a class-level name
+  and public URL layout.
+- **Auth is per-server by construction.** Different toolsets can carry
+  different authorizers (or the same instance) without inventing per-entry
+  config syntax.
+- **Less machinery.** No registry, no boot-time duplicate-`server_name`
+  validation, no `<mount>/<server_name>` routing layer — route uniqueness is
+  the router's job.
+
+The endpoint is a plain Rack app (not a `Rails::Engine`): it serves a single
+path with POST/GET/DELETE semantics and needs no routes, views or isolation.
+It also keeps the auth posture structurally separate from the admin engine,
+which is intentionally unauthenticated inside the gem (host-protected, like
+Sidekiq Web) — the opposite of this endpoint (see Auth).
+
+## Auth: fail-closed, authorizer object, principal
+
+- Transport: `Authorization: Bearer <token>` — symmetric with what
+  `HttpTransport` sends as a client.
+- `auth:` is a **required argument** — `Mcp.server(toolset)` without it raises
+  at boot. Fail-closed is structural: an endpoint without an authorizer cannot
+  exist, in any environment. There is no anonymous mode and no development
+  bypass; serving without real auth must be written explicitly
+  (`auth: ->(_token, _request) { :anonymous }`).
+- The authorizer is any object responding to
+  `call(token, request) -> principal | nil` — a lambda for trivial cases, a
+  host class when the logic deserves a home:
+
+  ```ruby
+  class McpAuth
+    def call(token, request)
+      client = ApiClient.active.find_by(token: token)
+      return nil unless client
+      return nil unless client.ip_allowed?(request.remote_ip)
+
+      client
+    end
+  end
+  ```
+
+  It receives the raw bearer token (`nil` if the header is absent or
+  malformed) and the `ActionDispatch::Request`. Returning `nil` → `401`,
+  empty body; the wire log records the rejection. Any other value becomes the
+  **principal** for the call.
+- No token model / storage in the gem. Host apps already own API-token
+  persistence and rotation; an authorizer object subsumes bearer lists, DB
+  lookups and per-token IP allowlists without the gem designing any of them.
+
+## CallContext gains `principal`
+
+```ruby
+CallContext = Struct.new(:agent, :loop, :principal, keyword_init: true)
+```
+
+The principal is the answer to "*who is calling*". In-process, that answer is
+already structural — the agent and its loop. An external caller has neither;
+the principal (whatever the authorizer returned: an `ApiClient` record, a
+user, a symbol) is the caller's identity for authorization decisions inside
+handlers (tenant scoping, per-client capability checks), for the audit trail
+(events/admin show who did what), and for the future HITL policy layer.
+
+| Caller | `agent` / `loop` | `principal` |
+|---|---|---|
+| In-process agent (`toolset:` entry) | set | `Base#mcp_principal` (subject by default) |
+| External client via `Mcp.server` | `nil` | authorizer's return value |
+
+This is a **contract change for every toolset handler**: `ctx.agent` and
+`ctx.loop` are no longer guaranteed. A toolset that reads loop state must
+either guard (`ctx.loop or raise`) or simply never be mounted — serving is
+opt-in per mount, so agent-only toolsets are unaffected by default. The
+dummy-app toolsets are the canary; the guide documents the pattern.
+
+`keyword_init` construction keeps every existing
+`CallContext.new(agent:, loop:)` call site source-compatible.
+
+## Inbound sessions: dedicated model
+
+A new table (new migration — the init migration is frozen since 0.0.3
+published), not a reuse of `McpSession`:
+
+```ruby
+create_table :solid_loop_mcp_inbound_sessions do |t|
+  t.string :server_name, null: false
+  t.string :session_id,  null: false, index: { unique: true }
+  t.string :principal    # human-readable label for admin (principal.to_s)
+  t.datetime :last_used_at
+  t.datetime :terminated_at
+  t.timestamps
+end
+```
+
+`McpSession` is shaped around the loop (`belongs_to :loop`, uniqueness scoped
+to `loop_id`, `transport_kind`, tool sync) — none of which applies inbound.
+Overloading it with a nullable loop and a `direction` flag would poison every
+existing per-loop invariant for a model that shares only a name.
+
+**Why persist inbound sessions at all** (toolsets are stateless by default,
+and Streamable HTTP even allows running session-less):
+
+1. **Observability anchor.** `events.eventable` is `NOT NULL` polymorphic;
+   inbound wire logs need a record to hang off, and "this client's
+   connection" is the natural one — all calls of one client session group
+   under one row.
+2. **Admin visibility.** Who is connected to which server, as which
+   principal, since when, last activity — the shared-admin story is the
+   product's differentiator versus commodity MCP-server gems.
+3. **Session-id honoring.** Toolsets minting real ids via `on_initialize`
+   (same hook the in-process path honors) need the server side to recognize
+   them on subsequent requests and `404` unknown/expired ids
+   (spec-compliant re-initialize signal).
+
+The session-less alternative (never emit `Mcp-Session-Id`, relax
+`events.eventable`) was considered and rejected: it saves one small table but
+loses all three points above.
+
+Lifecycle:
+
+- `initialize` → endpoint calls the toolset's `on_initialize`; `nil` → a UUID
+  is minted. Row created, id returned in the `Mcp-Session-Id` header.
+- Subsequent requests carry `Mcp-Session-Id`; missing header → `400`,
+  unknown or terminated id → `404`; `last_used_at` touched on use.
+- `DELETE` with a session header → session **terminated, row kept**
+  (`terminated_at`), `204`. Destroying the row would cascade into its events —
+  a client must not be able to erase its own audit trail with one request.
+- No TTL/eviction in 0.0.4; rows are cheap and admin-visible. Retention can
+  ride the 0.0.5 storage work.
+
+## Observability
+
+Every inbound request writes a `SolidLoop::Event`, same envelope shape as
+outbound MCP events: `eventable` = the inbound session, `loop_id` = `nil`
+(column is already nullable), `request_data`/`response_data` = the JSON-RPC
+envelopes, plus a wire log. Admin: inbound sessions get index/show pages next
+to outbound `McpSessions`; events appear in the existing Events UI.
+
+## Protocol scope (deliberately narrow)
+
+| Feature | 0.0.4 answer |
+|---|---|
+| `POST` single JSON-RPC request | ✅ single `application/json` response |
+| JSON-RPC notification (no `id`) | ✅ `202 Accepted`, empty body |
+| `initialize` / `tools/list` / `tools/call` | ✅ via `Toolset#deliver` (unchanged) |
+| `prompts/list` / `prompts/get` | ✅ new `prompt` DSL on `Toolset` (benefits the in-process path too); `prompts` capability advertised only when defined |
+| `DELETE` (session termination) | ✅ `204` |
+| **`GET` (SSE stream)** | ❌ **`405 Method Not Allowed`** — the spec's no-stream option; no half-working stream, revisit only if a real client cannot connect without it |
+| JSON-RPC batch (array body) | ❌ `-32600` (removed from the MCP spec anyway) |
+| `resources/*` (server side) | ❌ not in 0.0.4 (client-side *consumption* is a separate 0.0.4 workstream) |
+| Server-initiated notifications / streamed `tools/call` | ❌ out of scope (unchanged from 0.0.3 decision) |
+| Malformed JSON | `-32700` envelope |
+
+Protocol errors ride HTTP 200 with a JSON-RPC `error` object (matching
+`Toolset#deliver` today); HTTP status codes are reserved for transport-level
+outcomes (401/404/405/202/204).
+
+## Positioning guardrail
+
+`fast-mcp` and `action_mcp` already serve MCP from Rails — the endpoint is a
+commodity. The differentiator is one toolset consumed by the durable agent
+loop in-process *and* served externally, with shared schemas, wire logs, admin
+and (future) HITL policy. Every feature request that exists to match dedicated
+server gems is a scope trap; the table above is the tie-breaker.
+
+## Known trade-offs
+
+- **`405` on GET locks out SSE-requiring clients.** Accepted: Streamable HTTP
+  explicitly permits servers that never open a stream, and the clients we
+  target (Claude Code, `McpClient` itself) operate request/response.
+- **Object-based auth means no out-of-the-box token admin.** Accepted: the gem
+  avoids owning credential storage; hosts wire their existing API-auth into a
+  small authorizer class.
+- **N toolsets → N mount lines.** Accepted: that verbosity *is* the exposure
+  audit trail.
+- **Stateless bias.** Sessions minted by `on_initialize` are persisted, but
+  the endpoint offers no affinity or server-side state storage beyond the
+  row; a toolset holding real per-session state in memory will break under
+  multiple app processes — same restriction the in-process path already has,
+  now documented for the mount.
+
+## Out of scope (future work)
+
+- Per-tool execution policy (`auto` / `requires_approval`) — the principal is
+  deliberately available to a future HITL layer, but 0.0.4 does not define it.
+- Server-side `resources/*` DSL, SSE, rate limiting, inbound-session TTL.
+- OAuth flows — bearer-token authorizer only; hosts needing OAuth terminate it
+  in front of the mount.

data/docs/decisions/reasoning_persistence.md

@@ -0,0 +1,51 @@
+# Reasoning Persistence & Feedback Strategies
+
+## 1. The "Reasoning Amnesia" Problem
+
+In multi-turn agentic workflows, Large Language Models (LLMs) generate "Chain of Thought" or "Reasoning" blocks. While these blocks are crucial for the model to arrive at a correct conclusion, most current APIs treat them as **output-only** fields.
+
+**The Issue:**
+If the reasoning block from Turn N is not sent back to the model in Turn N+1, the model loses the logical context of its own previous decisions. This often leads to:
+*   Repetitive errors.
+*   Logical inconsistencies.
+*   Loss of plan coherence (the model forgets decisions it just made).
+
+## 2. Multi-Strategy Approach (Array-based)
+
+`SolidLoop` implements a flexible **Reasoning Protocol** to handle the inconsistent landscape of LLM providers. Instead of a single "hack," we use an **array of strategies** configured per agent/model. This allows combining multiple methods to ensure the model receives its thoughts regardless of how strict or fragmented the provider's API is.
+
+### Supported Strategies:
+
+*   **`:xml` (The DeepSeek/Reliability Standard):**
+    Wraps the reasoning content in `<think>...</think>` tags and prepends it to the main `content` string. This is the most robust method as it bypasses API field stripping.
+    *Note: In `SolidLoop`, this is a "Smart" strategy. It will automatically suppress itself if a native strategy (like `:gemini_signed`) is also present, avoiding redundant context.*
+*   **`:xml!` (Forced XML):**
+    Same as `:xml`, but **ignores** native strategy suppression. Use this if you want to ensure thoughts are visible in the main context even if the provider supports native fields.
+*   **`:gemini_signed` (The Google Standard):**
+    Passes both the `thought` text and the `thought_signature` (stored in JSONB metadata) back in specific fields required by Gemini 2.0+ for multi-turn reasoning.
+*   **`"field_name"` (Explicit Mapping):**
+    Allows copying the reasoning content into a specific JSON key (e.g., `"reasoning_content"`, `"thought"`, or `"internal_thinking"`) to satisfy various provider requirements (OpenRouter, vLLM, Groq).
+
+### Example Combinations:
+
+*   **`[:xml]`**: Maximum reliability for DeepSeek or OpenRouter (when field stripping is suspected).
+*   **`["reasoning", "reasoning_content"]`**: Populating multiple fields in parallel — useful when the provider's accepted field name is uncertain.
+*   **`[:gemini_signed, "thought", "reasoning_content"]`**: A comprehensive strategy for Gemini-compatible providers that might also look for OpenAI-style fields.
+*   **`[]` (Empty Array)**: Standard for OpenAI o1/o3 (reasoning is stripped to avoid 400 errors).
+
+## 3. The Reasoning Verifier *(not implemented)*
+
+To ensure an agent is actually "hearing its own thoughts," a **Reasoning Verification Test** can be written:
+
+1.  **Inject a Secret:** Create a mock Turn 1 where the `reasoning` block contains a specific "Thought Secret" (e.g., *"The hidden password is 'SOLID-TANK'"*), while the `content` is generic.
+2.  **Ask for Recall:** In Turn 2, ask the model: *"What was the hidden password you were thinking about in the previous step?"*.
+3.  **Validate Strategy:**
+    *   If the model answers correctly, the current `reasoning_strategies` are working.
+    *   If it fails, the provider is stripping the thoughts, and a more aggressive strategy (like `:xml`) is required.
+
+Worth doing — it would give a cheap, repeatable way to verify that `ReasoningPacker` is actually working end-to-end with a given provider/strategy combination, rather than assuming it does.
+
+## 4. Data Storage
+
+*   **Reasoning Text:** Stored in the `reasoning_content` column of `solid_loop_messages`.
+*   **Signatures & Metadata:** Stored in the `metadata` JSONB column. This ensures we can support future tokens like `thought_signature` without schema migrations.

data/docs/decisions/ruby_llm_rejected.md

@@ -0,0 +1,35 @@
+# Architectural Decision Record: Why we rejected ruby_llm as an Adapter
+
+**Date:** 2026-03-18  
+**Status:** Rejected  
+**Context:** Investigated integrating `ruby_llm` (v1.14) as a high-level adapter for the `SolidLoop` gem.
+
+## The Problem
+`SolidLoop` is designed around three constraints that every agent step must satisfy:
+1. **Persistent:** Stored in the database (`ActiveRecord`).
+2. **Asynchronous:** Executed via background workers (`ActiveJob`).
+3. **Observable:** Raw wire-level logs available for debugging.
+
+## Why `ruby_llm` Failed to Align
+
+### 1. The "In-Memory Loop" vs. "Database Orchestration"
+`ruby_llm` is optimized for interactive, in-process workflows where development speed is the priority — a design that serves its goals well. Its `Chat` object manages its own **in-memory execution loop**:
+* **Automatic Execution:** When a model returns a `tool_call`, `ruby_llm` **immediately executes the tool's code** and triggers the next LLM completion in the same process.
+* **The conflict:** `SolidLoop` requires that the process **must STOP** after an LLM response. We must persist the `tool_call` to the database and delegate execution to a separate, asynchronous `ActiveJob`. 
+* **Risk:** Using an in-memory loop makes the agent fragile—if the process crashes during a tool call, the entire state is lost. `SolidLoop` solves this by making the DB the "source of truth" for every single turn.
+
+### 2. Loss of Observability (Wire Logs)
+`SolidLoop` exposes raw Faraday-based HTTP logs via `env.log_buffer`, which is important for production debugging.
+* **The conflict:** `ruby_llm` encapsulates Faraday inside a private `Connection` class. It does not expose an easy way to inject custom Faraday middleware and uses its own logging format.
+
+### 3. Tight Coupling & High Abstraction
+`ruby_llm` expects tools to be registered as its own `Tool` objects. `SolidLoop` uses raw OpenAI-compatible JSON schemas (often coming from MCP servers). 
+* **The conflict:** Converting between schemas and mocking internal tool objects (`params_schema` errors) added unnecessary complexity and maintenance burden.
+
+## Final Decision
+We decided to double down on our **Native Adapter** approach. 
+
+### Benefits of the Native Path:
+* Direct use of Faraday gives full control over headers, timeouts, and logs.
+* Straightforward integration with the `ActiveRecord` + `ActiveJob` cycle.
+* Adding provider-specific payload mapping (e.g. Anthropic, Gemini) is simpler than adapting to a high-level framework's abstractions.

data/docs/guides/dialects.md

@@ -0,0 +1,55 @@
+# LLM Dialects in SolidLoop
+
+## 1. Concept
+
+While the `Adapter` handles the **Network Transport** (Faraday, Logging, SSE), the `Dialect` handles the **Language of the Model**. 
+
+Each LLM provider (OpenAI, Gemini, Anthropic) has its own JSON structure for requests and responses. Instead of embedding `if/else` blocks in the adapter, `SolidLoop` uses Dialects to map the internal standard to each provider's specific format.
+
+## 2. Default Dialects
+
+### `SolidLoop::Dialects::OpenAi` (Default)
+The standard engine for OpenAI, Ollama, vLLM, and most OpenAI-compatible APIs.
+*   **URL:** `/v1/chat/completions`
+*   **Payload:** Standard `messages` and `tools` arrays.
+*   **Parsing:** Looks for data in `choices[0].message`.
+
+### `SolidLoop::Dialects::Gemini`
+A native dialect for Google Gemini (AI Studio / Vertex AI).
+*   **URL:** `/v1beta/models/...`
+*   **Payload:** Translates OpenAI-style messages to Gemini's `contents/parts` format.
+*   **Roles:** Automatically maps `assistant` role to `model`.
+*   **Parsing:** Extracts data from Gemini's specific candidate structure and handles `thought_signature`.
+
+## 3. Configuration in Agents
+
+An agent defines its preferred adapter and dialect through methods. This allows the same adapter (e.g., `Native`) to speak different languages.
+
+```ruby
+class MyCustomAgent < SolidLoop::Base
+  # Which network engine to use
+  def llm_adapter_name
+    :native
+  end
+
+  # Which JSON dialect to speak
+  def llm_dialect_name
+    :gemini
+  end
+
+  # How to return thoughts to the model
+  def reasoning_strategies
+    [:gemini_signed, "reasoning_content"]
+  end
+end
+```
+
+## 4. Dialect Interface
+
+A Dialect object must implement the following methods to be compatible with the `Native` adapter:
+
+1.  **`completion_url(base_url)`**: Returns the specific API endpoint.
+2.  **`render_payload(universal_payload)`**: (Optional) Transforms the payload before sending.
+3.  **`apply_reasoning_strategies!(messages, strategies)`**: Formats message history according to the [Reasoning Protocol](./reasoning_persistence.md).
+4.  **`extract_message_data(raw_json)`**: Extracts content and tools from a streaming chunk or partial response.
+5.  **`normalize_response(raw_json)`**: Converts a full API response into the SolidLoop universal format (including token usage and metadata).

data/docs/guides/llm_middlewares.md

@@ -0,0 +1,294 @@
+# LLM Middlewares Architecture in SolidLoop
+
+## 1. Middleware Principles
+
+In `SolidLoop`, the generation of LLM responses (the `LlmCompletionJob` class) is built upon the classic **Chain of Responsibility (Middleware)** pattern, widely used in Rack (Ruby) or Express.js (Node).
+
+**Why is this needed?**
+Instead of having one massive, monolithic `perform` method, the logic is broken down into isolated layers (Middlewares). Each layer is responsible for exactly one specific task (e.g., error handling, payload building, network requests).
+
+### The Middleware Stack (Layered View)
+
+`SolidLoop` processes LLM interactions through a robust, layered stack:
+
+```text
+┌──────────────────────────────────────────────────────────┐
+│ 1. ErrorHandling (Guard)                                 │ ← Catch-all for stability
+├──────────────────────────────────────────────────────────┤
+│ 2. AgentInitialization (Context)                         │ ← Agent logic & MCP setup
+├──────────────────────────────────────────────────────────┤
+│ 3. MessageBuilding (Assembler)                           │ ← DB to OpenAI-style payload
+├──────────────────────────────────────────────────────────┤
+│ 4. EventLogging (Auditor)                                │ ← Full persistence of logs
+├──────────────────────────────────────────────────────────┤
+│ 5. NetworkCalling (Orchestrator)                         │ ← Orchestrates LLM Adapters
+│    │                                                     │
+│    │   ┌─────────────────────────────────────────────┐   │
+│    └───┤ Adapter: Native (The Default Engine)        ├───┘
+│        │ - Clean Faraday Transport                   │
+│        │ - Raw Wire Logging                          │
+│        │ - Shopify EventStreamParser (SSE)           │
+│        └─────────────────────────────────────────────┘
+├──────────────────────────────────────────────────────────┤
+│ 6. ResponseParsing (Finalizer)                           │ ← Final DB commit & metrics
+└──────────────────────────────────────────────────────────┘
+```
+
+**How it works:**
+- Middlewares are arranged in a strict **chain (Pipeline)**.
+- They pass a shared state object to each other — the **`Context`** (`SolidLoop::Pipeline::Context`).
+- There are two phases of data flow:
+  - **"Downstream" (Request phase):** Execution flows top-to-bottom. A layer prepares data (e.g., formats messages) and passes control to the next layer (`@app.call(env)`).
+  - **"Upstream" (Response phase):** Execution returns bottom-to-top. A layer receives the response from nested layers (e.g., the HTTP network response) and can modify it or save results to the database.
+- Any layer can halt the chain simply by not calling `@app.call(env)` (for example, if a rate limit is hit or the loop was cancelled by the user).
+
+**Extensibility (Custom Middlewares):**
+Host applications can inject their own custom layers into this chain without modifying the core gem's source code.
+
+There are two levels of customization:
+- **Global:** configure `config.llm_middlewares` in the SolidLoop initializer using the `Pipeline::Builder` DSL (`insert_before`, `insert_after`, `use`, `replace`, `delete`).
+- **Per-agent:** implement `configure_llm_middlewares(builder)` on your agent instance. This runs for each `LlmCompletionJob`, after the default stack is copied into a fresh builder and before the pipeline executes.
+
+For example, suppose you want to inject a custom `SummarizationMiddleware` that compresses older messages to save tokens before sending the payload to the LLM:
+
+```ruby
+# config/initializers/solid_loop.rb
+SolidLoop.configure do |config|
+  config.llm_middlewares.insert_after(SolidLoop::Middlewares::AgentInitialization, SummarizationMiddleware)
+end
+
+# app/services/summarization_middleware.rb
+class SummarizationMiddleware
+  def initialize(app)
+    @app = app
+  end
+
+  def call(env)
+    # REQUEST PHASE: We can intercept and modify the state BEFORE the payload is built.
+    # For example, let's keep only the last 10 messages if there are too many.
+    if env.loop.messages.count > 10
+      # Abstract summarization logic:
+      # env.loop.messages = CustomSummarizer.compress(env.loop.messages)
+      Rails.logger.info "SummarizationMiddleware: Compressed context for loop #{env.loop.id}"
+    end
+
+    # Pass control down the chain (to MessageBuilding, NetworkCalling, etc.)
+    @app.call(env)
+
+    # RESPONSE PHASE: Execution returns here after the LLM responds.
+    # (Optional: do something with env.response if needed)
+  end
+end
+```
+
+Another example is **Domain Object Rewriting**. Some legacy or local models return tool calls as raw `<tool_call>` XML tags in the text `content` rather than a standard JSON array. `SolidLoop` ships with a built-in (but disabled by default) `ToolCallXmlParser` middleware.
+
+It should be inserted **before** `ResponseParsing` in the builder. That way it wraps `ResponseParsing`, lets the core parser create the assistant message first, and then runs on the response path to inspect `env.assistant_message`, extract XML tool calls, and create the `tool_calls` records in the database.
+
+If a specific agent always needs this legacy behavior, prefer the per-agent hook:
+
+```ruby
+# app/agents/my_agent.rb
+class MyAgent < SolidLoop::Base
+  def configure_llm_middlewares(builder)
+    builder.insert_before(
+      SolidLoop::Middlewares::ResponseParsing,
+      SolidLoop::Middlewares::ToolCallXmlParser
+    )
+  end
+end
+```
+
+If every loop for an application needs the same behavior, configure it globally instead:
+
+```ruby
+# config/initializers/solid_loop.rb
+SolidLoop.configure do |config|
+  config.llm_middlewares.insert_before(
+    SolidLoop::Middlewares::ResponseParsing,
+    SolidLoop::Middlewares::ToolCallXmlParser
+  )
+end
+```
+
+The core `ResponseParsing` layer never needs to know XML was involved, and the legacy compatibility stays isolated to a dedicated middleware.
+
+---
+
+## 2. Default LLM Middlewares Stack
+
+When `LlmCompletionJob` is executed, the following layers run in strict order by default:
+
+### 1. `ErrorHandling`
+* **What it does:** Wraps the entire context-gathering and networking process in a `begin/rescue` block.
+* **Request Phase:** Simply passes control down the chain.
+* **Response Phase:** Catches any exceptions (`StandardError`) thrown by deeper layers. It logs the error with a backtrace and correctly transitions the `SolidLoop::Loop` and its `SolidLoop::Message` to a `failed` status. An exception is `CancellationError`, which quietly stops the thread and marks the status as `success`.
+
+### 2. `AgentInitialization`
+* **What it does:** Prepares the AI Agent instance and initializes connections with the MCP servers assigned to the current loop.
+* **Request Phase:**
+  1. Reads the agent class name from the DB and instantiates it.
+  2. Runs `McpSessionInitializer` to establish connections (TCP/IP or stdio) with all tools before building the formatted text block or JSON structure of the prompts.
+
+### 3. `MessageBuilding`
+* **What it does:** Prepares the data payload for the LLM provider.
+* **Request Phase:**
+  1. Retrieves all previous `Loop.messages` from the DB.
+  2. Formats them into an array of JSON hashes (`[{role: "user", content: "..."}]`).
+  3. Attaches reasoning data from previous turns as internal keys (`_sl_reasoning`, `_sl_signature`) — the adapter will apply the configured `reasoning_strategies` and remove these keys before sending.
+  4. Forms the final `payload` hash that will be sent via HTTP.
+
+### 4. `EventLogging`
+* **What it does:** Responsible for logging requests and responses for debugging purposes (saves records to the `SolidLoop::Event` table).
+* **Request Phase:** Creates an `Event` record in the database with a draft of the request (URL, Headers, assembled Payload). Records the start time.
+* **Response Phase:** Updates the created `Event`, writing the received JSON network response, raw stream chunks (if applicable), and total execution duration.
+
+### 5. `NetworkCalling` & The Adapter Layer
+* **What it does:** The orchestrator of network communication. It doesn't perform HTTP calls directly but delegates them to an **Adapter** (e.g., `SolidLoop::Adapters::Native`).
+* **Request Phase:** 
+  1. Instantiates `SolidLoop::Adapters::Native` (the only built-in adapter).
+  2. If **Streaming** is enabled, it creates a `SolidLoop::Message` in the DB and subscribes to the adapter's chunk callbacks to deliver incremental content in real-time.
+  3. Executes the adapter's `call` method.
+* **Adapter (Native):** The built-in engine that uses **Faraday** for transport and provides raw wire logs. It integrates **Shopify's `event_stream_parser`** for SSE streaming.
+* **Response Phase:** Captures the response, duration, and logs (merged back into `env.log_buffer`) from the adapter and saves them into the `Context`.
+
+### 6. `ResponseParsing`
+* **What it does:** Parses the network result and saves the assistant's message to the database. **Operates exclusively on the return path (Response Phase).**
+* **Response Phase:**
+  1. Reads `Context.response` or the aggregator's result.
+  2. Extracts the response text (`content`) and separates reasoning blocks (`reasoning_content`, `thought`).
+  3. Parses tool calls (`tool_calls`) from the normalized response data.
+  4. Extracts meta-token statistics (`usage`) using **`SolidLoop::LlmMetrics`** (calculating TTFT, TPS, and duration).
+  5. Saves (or updates) the final `SolidLoop::Message` record with the `assistant` role in the database.
+  6. If the LLM invoked tools, this layer enqueues `SolidLoop::ToolExecutionJob` jobs. If no tools were called, it marks the `Loop` as `completed`.
+
+---
+
+## 3. Real-World Recipes
+
+### Recipe: Provider Payload Transformation (MCP tool names with `/`)
+
+**The problem.** MCP tools are named with `/` by protocol convention: `files/write_file`, `search/get_results`. OpenAI rejects these with a 400 error — it validates function names against `^[a-zA-Z0-9_-]+$`. The MCP server can't be changed, and forking the gem to patch it is not an option.
+
+**The solution.** Two middlewares in a symmetric pair: one encodes before the request, one decodes after.
+
+```
+[payload: files/write_file]
+        ↓
+  ToolNameEncoder   →  files__write_file   (insert_before NetworkCalling)
+        ↓
+  NetworkCalling    →  OpenAI sees a valid name
+        ↓
+  ToolNameDecoder   →  files/write_file    (insert_after NetworkCalling)
+        ↓
+  ResponseParsing   →  original name saved to DB, MCP routing works correctly
+```
+
+**Key detail:** the encoder must patch two places in the payload, not just the tool list:
+1. `tools[].function.name` — the tool definitions sent to the LLM
+2. `messages[].tool_calls[].function.name` — the conversation history, because OpenAI validates past tool_call names in the context window too
+
+The decoder only touches `env.normalized_data` (the LLM response) — exactly what needs to be restored before `ResponseParsing` writes to the database.
+
+```ruby
+# app/services/solid_loop/middlewares/tool_name_encoder.rb
+module SolidLoop
+  module Middlewares
+    class ToolNameEncoder
+      def initialize(app) = @app = app
+
+      def call(env)
+        if env.payload
+          encode_tools!(env.payload)
+          encode_messages!(env.payload)
+        end
+        @app.call(env)
+      end
+
+      private
+
+      def encode(name) = name.to_s.gsub("/", "__")
+
+      def encode_tools!(payload)
+        return unless payload[:tools]
+        payload[:tools] = payload[:tools].map do |tool|
+          next tool unless tool.dig(:function, :name)
+          tool.merge(function: tool[:function].merge(name: encode(tool[:function][:name])))
+        end
+      end
+
+      def encode_messages!(payload)
+        return unless payload[:messages]
+        payload[:messages] = payload[:messages].map do |msg|
+          msg = msg.dup
+          if msg[:tool_calls]
+            msg[:tool_calls] = msg[:tool_calls].map do |tc|
+              next tc unless tc.dig(:function, :name)
+              tc.merge(function: tc[:function].merge(name: encode(tc.dig(:function, :name))))
+            end
+          end
+          msg[:name] = encode(msg[:name]) if msg[:name]
+          msg
+        end
+      end
+    end
+  end
+end
+```
+
+```ruby
+# app/services/solid_loop/middlewares/tool_name_decoder.rb
+module SolidLoop
+  module Middlewares
+    class ToolNameDecoder
+      def initialize(app) = @app = app
+
+      def call(env)
+        decode_normalized_data!(env)
+        @app.call(env)
+      end
+
+      private
+
+      def decode(name) = name.to_s.gsub("__", "/")
+
+      def decode_normalized_data!(env)
+        return unless env.normalized_data&.dig(:tool_calls)
+        env.normalized_data[:tool_calls].each do |tc|
+          next unless tc.dig("function", "name")
+          tc["function"]["name"] = decode(tc["function"]["name"])
+        end
+      end
+    end
+  end
+end
+```
+
+```ruby
+# config/initializers/solid_loop.rb
+SolidLoop.configure do |config|
+  config.llm_middlewares.insert_before(SolidLoop::Middlewares::NetworkCalling,
+                                       SolidLoop::Middlewares::ToolNameEncoder)
+  config.llm_middlewares.insert_after(SolidLoop::Middlewares::NetworkCalling,
+                                       SolidLoop::Middlewares::ToolNameDecoder)
+end
+```
+
+**The takeaway.** Provider-specific constraints are a moving target. The middleware pipeline lets you adapt to them in your application code, without forking the gem, without modifying the MCP server, and without leaking provider quirks into your agent logic. The fix is a small, self-contained middleware pair.
+
+---
+
+### General pattern: symmetric encoder/decoder around `NetworkCalling`
+
+Any transformation that must be invisible to the LLM provider but transparent to the rest of the system follows this shape:
+
+```
+insert_before(NetworkCalling) → encode / normalize / adapt
+insert_after(NetworkCalling)  → decode / restore / revert
+```
+
+Examples where this applies:
+- Tool name character restrictions (shown above)
+- Payload size limits: compress large tool results before sending, decompress in response phase
+- Provider-specific authentication headers injected per-request
+- Request signing for self-hosted models behind an API gateway