solid_loop 0.0.4

data/docs/guides/mcp_transports.md

@@ -0,0 +1,374 @@
+# MCP Transports — port, adapters, local toolset, server endpoint
+
+**Companions:** [mcp-only-tooling.md](../decisions/mcp-only-tooling.md) — the decision and
+migration guide; [mcp-server.md](../decisions/mcp-server.md) — serving toolsets to
+external clients.
+
+Since 0.0.3 every tool in SolidLoop is an MCP tool. MCP is the *interface*;
+where and how the server runs is a *transport* concern. This guide documents
+the transport port, the built-in adapters, and the in-process toolset.
+
+## The port: `SolidLoop::Mcp::Transport`
+
+A transport is any object that can deliver a JSON-RPC envelope to an MCP server
+and return the response:
+
+```ruby
+module SolidLoop
+  module Mcp
+    module Transport
+      # payload    — raw JSON-RPC request Hash ({jsonrpc:, id:, method:, params:})
+      # session_id — nil for `initialize`; afterwards whatever initialize returned
+      # context    — SolidLoop::Mcp::CallContext (agent:, loop:), may be nil.
+      #              HTTP transports typically ignore it (auth travels in
+      #              headers); in-process transports may use it to reach
+      #              domain state without a network hop.
+      # returns    — SolidLoop::Mcp::Result
+      def deliver(payload, session_id: nil, context: nil)
+        raise NotImplementedError
+      end
+    end
+
+    Result = Struct.new(
+      :body,         # JSON-RPC response Hash (result or error envelope)
+      :session_id,   # set by `initialize`; echoed back otherwise; nil => stateless
+      :raw_request,  # exact envelope sent   — recorded to wire logs / Events
+      :raw_response, # exact envelope received — recorded to wire logs / Events
+      :status,       # transport status code (HTTP status or synthetic 200/500)
+      :stderr,       # optional transport diagnostics for the wire log (stdio stderr)
+      keyword_init: true
+    )
+  end
+end
+```
+
+Contract rules:
+
+1. **Session identity is explicit.** `initialize` may return a `session_id`
+   (`Result#session_id`); every subsequent request receives it as a keyword.
+   A transport that returns `nil` is *stateless* — SolidLoop synthesizes a
+   session id for the `McpSession` row (as the current `stateless: true` client
+   mode does) and the transport must not rely on server-side continuity.
+2. **Errors follow MCP semantics.** Tool-level failures are a *successful*
+   `tools/call` response with `isError: true` content — the model sees them and
+   can self-correct. Protocol-level failures are a JSON-RPC error envelope in
+   `Result#body`. Transport-level failures (process died, connection refused,
+   timeout) raise `SolidLoop::McpError`. Custom transports must not leak raw
+   exceptions for the first two categories.
+3. **Transports are stateless and thread-safe.** Concurrent `ToolExecutionJob`s
+   share the transport instance; all per-session state must be keyed by
+   `session_id`, never held in transport ivars.
+4. **Constructible anywhere.** The admin MCP Inspector calls tools from the web
+   process; a transport must be buildable from the agent's `mcps` config alone,
+   with no job-runtime state.
+5. **Wire logs are mandatory.** `raw_request` / `raw_response` feed
+   `env.log_buffer`, `Event` records and the admin UI identically for every
+   transport. An adapter that returns empty raw payloads forfeits the main
+   reason SolidLoop owns its transport layer.
+
+`McpClient` keeps its public API (`initialize_session`, `list_tools`,
+`call_tool`) and becomes a thin protocol layer that builds envelopes and
+delegates I/O to the transport.
+
+## Agent configuration
+
+`Base#mcps` accepts three entry forms; `name:` is required in all of them.
+
+```ruby
+def mcps
+  [
+    # 1. HTTP sugar (today's form) → Mcp::HttpTransport
+    { name: :main,
+      url: ENV["MCP_URL"],
+      api_token: ENV["MCP_TOKEN"],
+      custom_headers: { "X-Workspace-Id" => WORKSPACE_ID },
+      tools: %w[list_files read_file],          # optional whitelist (unchanged)
+      required_tools: %w[list_files] },          # optional validation (unchanged)
+
+    # 2. In-process toolset → the toolset class IS the server; local transport
+    { name: :search, toolset: SearchTools },
+
+    # 3. General form — any object implementing Mcp::Transport
+    { name: :fs,
+      transport: SolidLoop::Mcp::StdioTransport.new(
+        command: ["npx", "-y", "@modelcontextprotocol/server-filesystem", "/data"],
+        timeout: 30
+      ),
+      prefix: "fs" }
+  ]
+end
+```
+
+### Session recovery: `session_recovery: :auto | :fail`
+
+On session loss (`McpSessionInvalidError`) the default `:auto` re-initializes
+the session once, transparently, and retries the tool call — today's behavior.
+`:fail` is for servers where the session is expensive server-side state (e.g.
+a sandbox container per session): a silent re-initialize would destroy that
+state mid-loop, so the tool call fails loudly instead and recreation stays
+explicit.
+
+### Tool-name collisions and `prefix:`
+
+Tool resolution is `loop.mcp_tools.find_by(name:)` across *all* servers of the
+loop, so names must be loop-unique. With several domains this is no longer
+guaranteed:
+
+- `prefix: "search"` advertises tools to the LLM as `search__fetch_url` and
+  strips the prefix before issuing `tools/call` to the server.
+- Without prefixes, a name collision across servers **raises at session
+  initialization** with a message naming both servers. Collisions never resolve
+  silently by insertion order.
+
+## Built-in adapters
+
+### `Mcp::HttpTransport` (reference adapter)
+
+Today's Faraday implementation extracted verbatim: JSON request/response,
+`Authorization: Bearer`, `Mcp-Session-Id` header mapped to the port's
+`session_id`, custom headers, timeout. Everything downstream (session rows,
+tool sync, wire logs, events) is transport-agnostic and unchanged.
+
+### `Mcp::Toolset` (in-process server — the home of former native tools)
+
+A toolset *is* both the server and its transport: it implements `deliver` by
+dispatching `initialize` / `tools/list` / `tools/call` in-process, no HTTP.
+
+```ruby
+class SearchTools < SolidLoop::Mcp::Toolset
+  server_name "search"
+
+  tool "web_search",
+       description: "Search the web and return the top results",
+       input_schema: {
+         type: "object",
+         properties: { query: { type: "string" } },
+         required: %w[query]
+       } do |args, ctx|
+    Search::Web.call(loop: ctx.loop, **args.symbolize_keys)
+  end
+
+  tool "fetch_url",
+       description: "Fetch a URL and return a text summary",
+       input_schema: {
+         type: "object",
+         properties: { url: { type: "string" } },
+         required: %w[url]
+       } do |args, ctx|
+    Search::Fetch.call(loop: ctx.loop, **args.symbolize_keys).to_json
+  end
+end
+```
+
+Semantics:
+
+- `tools/list` is generated from the declarations; `to_tool_spec` conversion,
+  whitelists, `required_tools`, prefixes and the admin Inspector all work as
+  for any MCP server.
+- The block receives parsed `arguments` (string keys, exactly as they would
+  arrive over HTTP — the JSON boundary is preserved so local and remote
+  behavior cannot drift) and a `CallContext` (`ctx.agent`, `ctx.loop`,
+  `ctx.principal`). **Do not assume `ctx.agent`/`ctx.loop` are present**: for
+  a toolset served externally via `SolidLoop::Mcp.server` they are `nil` and
+  `ctx.principal` carries the caller's identity instead (see the server
+  section below). A handler that needs loop state should guard
+  (`ctx.loop or raise`) — or its toolset should simply never be mounted.
+- Return value: a String becomes text content; a Hash is serialized to JSON
+  text content **and** exposed as `structuredContent` (MCP spec rev 2025-06);
+  raising inside the block produces an `isError: true` tool result with the
+  exception message — the standard self-correction path for the model.
+  **Sanitize those messages when the toolset is mounted**: they go to the
+  external client verbatim, so a raw `ActiveRecord::RecordNotFound` (or any
+  driver error) can leak scoping clauses and schema details. Rescue expected
+  failures and re-raise an opaque message (`raise "Generation not found"`).
+- Toolsets are stateless by default (`session_id` = nil → synthetic). A toolset
+  may override `on_initialize(params, context)` to mint and later receive a
+  real session id.
+- **Transactions:** the block runs on the caller's thread and shares the job's
+  ActiveRecord connection. That makes a tool atomic with SolidLoop's own writes
+  — and makes `transaction do ... end` inside a tool a *nested* transaction.
+  Both are by design; the boundary is the toolset author's responsibility.
+- **Ordering & concurrency:** by default SolidLoop enqueues every tool call from
+  one assistant turn at once, so they run **in parallel** across workers with no
+  order guarantee. An agent that returns `sequential_tool_calls? == true` runs
+  them one at a time in model order instead. Either way different loops run
+  concurrently, so a toolset guarding shared invariants (running counters,
+  sequences, append-only timelines) must serialize per resource itself
+  (`record.with_lock`, advisory lock) — and under the parallel default that also
+  applies to two calls from the *same* turn touching the same resource.
+- **Principal:** in-process calls receive `ctx.principal` from
+  `Base#mcp_principal`, which defaults to the loop subject and can be overridden
+  when the subject is not the actor. Mounted calls continue to use the
+  authorizer's return value.
+- **Idempotency:** every tool invocation carries a stable per-`ToolCall` key.
+  In-process handlers read it from `ctx.idempotency_key`; on the wire it is sent
+  as MCP `_meta` on the `tools/call` params:
+
+  ```json
+  { "method": "tools/call",
+    "params": { "name": "web_search", "arguments": { "...": "..." },
+                "_meta": { "solidloop/idempotencyKey": "solid_loop:tool_call:42" } } }
+  ```
+
+  SolidLoop's `executed_at` checkpoint stops it from re-invoking a tool whose
+  result committed, but a crash between a remote side effect and that commit
+  re-delivers the invocation with the **same** key. **Irreversible tools
+  (sends, posts, deletes) should dedupe on this key** — e.g. a unique index on
+  it next to the side effect. SolidLoop does not (and cannot) make external
+  effects exactly-once itself; it only guarantees the key is stable per tool
+  call across retries. (A toolset mounted via `Mcp::Server` also surfaces an
+  inbound client's `_meta` key as `ctx.idempotency_key`.)
+- `refresh_mcp_tools?` note: toolset schemas live in code, so discovery is free;
+  agents using toolsets should refresh on every loop start (otherwise schemas
+  persisted in `mcp_tools` go stale across deploys).
+
+#### Prompts
+
+A toolset can also declare prompts — reusable, parameterized message templates
+served via `prompts/list` / `prompts/get`. The `prompts` capability is
+advertised only when prompts are declared.
+
+```ruby
+class SearchTools < SolidLoop::Mcp::Toolset
+  prompt "research_brief",
+         description: "Skeleton for a topic research brief",
+         arguments: [ { name: "topic", description: "e.g. renewable energy", required: true } ] do |args, ctx|
+    "Research the #{args['topic']} topic and summarize the key findings."
+  end
+end
+```
+
+The block returns a String (wrapped into a single user message) or a
+ready-made messages array. Missing required arguments and unknown prompt names
+are `-32602` protocol errors; unlike `tools/call` there is no `isError` shape
+for prompts, so handler exceptions become `-32603` instead of propagating.
+
+### `Mcp::StdioTransport` (restricted contract)
+
+Runs a child-process MCP server over stdin/stdout. Because SolidLoop executes
+each tool call in an independent job (any worker, any machine), there is no
+process that could own a long-lived child across rounds. Therefore:
+
+- **Lifecycle is spawn-per-call:** spawn → `initialize` handshake → request →
+  graceful shutdown (kill on timeout), all inside one `deliver`.
+- **Stateless servers only.** `session_id` is synthetic and implies no
+  server-side continuity. Stateful stdio servers (a running shell, a browser
+  session, a REPL sandbox) are unsupported — their in-memory state would be
+  lost at every job boundary. This is a deliberate consequence of the
+  durable-async core, not a transport bug.
+- Options: `command:` (argv array, never a shell string), `env:`, `cwd:`,
+  `timeout:` (per call, includes spawn+handshake). stderr is captured into the
+  wire log.
+- Cost note: spawn+handshake per call is typically 50–500 ms for Node/Python
+  servers. Prefer HTTP or a toolset for hot paths.
+
+## Custom transports & conformance
+
+Any object implementing the port can be passed via `transport:`. The gem ships
+RSpec shared examples so adapter authors can verify conformance:
+
+```ruby
+RSpec.describe MyTransport do
+  it_behaves_like "a solid_loop mcp transport" do
+    let(:transport) { described_class.new(...) }
+  end
+end
+```
+
+The shared examples cover: initialize returns/omits `session_id` coherently,
+session id round-trip, `tools/list` shape, `isError` tool failures vs JSON-RPC
+protocol errors vs raised `McpError`, raw request/response presence, and
+thread-safety of concurrent `deliver` calls.
+
+## Serving toolsets over HTTP — `SolidLoop::Mcp.server`
+
+The mirror image of `HttpTransport` over the same port: a mountable Rack
+endpoint that serves **one toolset** to external MCP clients (Claude Code,
+other apps) over Streamable HTTP. The POST body goes straight into
+`Toolset#deliver` — the same method the in-process agent path calls — so a
+toolset defined once is consumed both locally and remotely with shared
+schemas, wire logs and admin. Decision doc: [mcp-server.md](../decisions/mcp-server.md).
+
+```ruby
+# config/routes.rb (host app)
+mount SolidLoop::Mcp.server(RpcTools,    auth: McpAuth.new)    => "/rpc"
+mount SolidLoop::Mcp.server(ReportTools, auth: ReportAuth.new) => "/tools"
+# AdminTools is simply not mounted — it stays in-process only.
+```
+
+### Auth (fail-closed)
+
+`auth:` is a **required argument**; the endpoint cannot be constructed without
+it. The authorizer is any object responding to
+`call(token, request) -> principal | nil` — a lambda for trivial cases, a host
+class when the logic deserves a home:
+
+```ruby
+class McpAuth
+  def call(token, request)
+    client = ApiClient.active.find_by(token: token)
+    return nil unless client
+    return nil unless client.ip_allowed?(request.remote_ip)
+
+    client
+  end
+end
+```
+
+It receives the raw bearer token from `Authorization: Bearer ...` (`nil` if
+absent or a different scheme) and the `ActionDispatch::Request`. Returning
+`nil` → `401` with an empty body. There is no anonymous mode and no
+development bypass; serving without real auth must be written explicitly
+(`auth: ->(_token, _request) { :anonymous }`). The gem stores no tokens —
+bearer lists, DB lookups and IP allowlists live in the host's authorizer.
+
+Any other return value becomes the **principal**: handlers receive
+`CallContext(agent: nil, loop: nil, principal: <that value>)` and can use it
+for authorization decisions and tenant scoping.
+
+### Sessions & protocol scope
+
+- `initialize` creates a `McpInboundSession` row (honoring `on_initialize`
+  minting, else a UUID) and returns it in the `Mcp-Session-Id` header. Every
+  other request must carry that header: `400` without it, `404` for an
+  unknown or terminated session (the client's signal to re-initialize).
+- `DELETE` terminates the session but keeps the row — a client must not be
+  able to erase its own audit trail.
+- Notifications (requests without an `id`) → `202`, not dispatched. Batch
+  bodies → `-32600`. Malformed JSON → `-32700`. **No SSE:** `GET` → `405`
+  (the spec's no-stream option); POST always answers with a single
+  `application/json` response.
+- Every request that reaches a session writes a `SolidLoop::Event`
+  (`mcp_inbound_*`, loop-less, eventable = the inbound session) with the
+  JSON-RPC envelopes and a wire log. The admin UI lists inbound sessions
+  under **Inbound MCP** with per-session event history.
+
+## Client breadth: prompts & resources
+
+`McpClient` consumes more than tools:
+
+- `list_prompts(session_id)` / `get_prompt(session_id, name, arguments = {})`
+- `list_resources(session_id)` / `read_resource(session_id, uri)`
+- `server_capabilities` — the capabilities object from the last `initialize`.
+
+At session initialization SolidLoop snapshots the server's capabilities into
+`McpSession#metadata`, and — when the server advertises `prompts` /
+`resources` — the discovered lists as well (best-effort: a failing
+prompts/resources endpoint does not take tool availability down with it). The
+admin session page shows the discovered prompts and resources; the fetches
+leave `mcp_list_prompts` / `mcp_list_resources` events.
+
+## Persistence & admin
+
+- `McpSession` rows are created per loop per server for **every** transport,
+  with `transport_kind` (`http` / `local` / `stdio` / `custom`) shown in the
+  admin UI. Stateless transports get a synthetic `session_id`.
+- `McpTool` rows, wire logs, `Event` records, the Inspector (including
+  `run_tool` and replay) behave identically across transports.
+- `McpInboundSession` rows are created per external client connection by
+  `SolidLoop::Mcp.server` (server name, principal label, last-used and
+  terminated timestamps) and listed under **Inbound MCP** in the admin UI;
+  inbound `Event` records hang off them with `loop_id` nil.
+- Schema change: `transport_kind` (`string`, default `"http"`, `null: false`)
+  on `solid_loop_mcp_sessions` — added to the init migration (nothing is
+  published yet, so it is still editable in place).

data/docs/guides/tool_middlewares.md

@@ -0,0 +1,148 @@
+# Tools Middlewares Architecture in SolidLoop
+
+## 1. Middleware Principles
+
+The execution of agent-side tools (`ToolExecutionJob`) in `SolidLoop` also utilizes the **Chain of Responsibility (Middleware)** pattern to encapsulate functionality and provide extensibility.
+
+**Why is this needed?**
+Monolithic background jobs are notoriously difficult to maintain:
+1. Every tool uses the MCP interface, either through an in-process `Toolset` or another MCP transport. Middlewares separate policy, telemetry, execution and response creation.
+2. The logic for saving telemetry Events, calculating elapsed duration, and creating the new `tool` message in the loop must be reusable and isolated.
+
+**How it works:**
+- Execution begins by initializing a shared `SolidLoop::Pipeline::ToolContext` containing the `tool_call_id`.
+- Middlewares are executed sequentially through an array (`Pipeline.new(middlewares).call(context)`).
+- The stack guarantees that even if a tool throws an error within Ruby code or an MCP server is unreachable (Connection Timeout), the system gracefully logs the error, updates `is_success: false`, and transitions the conversational loop to the `failed` state without crashing the ActiveJob thread.
+
+**Extensibility (Custom Middlewares):**
+Unlike the global LLM initializer, tool execution configuration is delegated to the **specific agent instance** handling the loop.
+An agent can implement `configure_tool_middlewares(builder)` and explicitly insert custom layers into the framework's baseline stack using the `Pipeline::Builder` DSL (`insert_before`, `insert_after`, `use`, `replace`, `delete`).
+
+For example, if you need to trigger your own business logic immediately after a tool finishes executing:
+
+```ruby
+# app/agents/benchmark_agent.rb
+class BenchmarkAgent < SolidLoop::Base
+  def configure_tool_middlewares(builder)
+    builder.insert_after(SolidLoop::ToolMiddlewares::ToolExecution, BenchmarkEvaluatorMiddleware)
+  end
+end
+
+# app/services/benchmark_evaluator_middleware.rb
+class BenchmarkEvaluatorMiddleware
+  def initialize(app)
+    @app = app
+  end
+
+  def call(env)
+    @app.call(env) # Pass control downstream (the tool is logged and actually executed)
+
+    # RESPONSE PHASE: After the tool acts, we run our custom evaluation:
+    BenchmarkToolEvaluator.call(env.tool_call)
+  end
+end
+```
+
+---
+
+## 2. Default Tool Middlewares Stack
+
+During a `ToolExecutionJob` (which is typically enqueued by `LlmCompletionJob` when `tool_calls` are present), the layers execute in the following order:
+
+### 1. `ErrorHandling`
+* **What it does:** A global catch-all error handler for exceptional situations. Wraps all nested layers.
+* **Request Phase:** Passes control down the pipeline.
+* **Response Phase:** If an unhandled exception (`StandardError`) propagates up from inner layers, it is logged to the Rails console with a backtrace. The `SolidLoop::ToolCall` record is updated with `is_success: false` and the `error_message`. The parent `SolidLoop::Loop` gracefully assumes a `failed` status. Exception: `CancellationError`, which safely aborts execution.
+
+### 2. `AgentInitialization`
+* **What it does:** Instantiates the AI agent class.
+* **Request Phase:**
+  1. Relies on the pre-fetched ActiveRecord context (`env.loop`) populated by the worker.
+  2. Initializes an instance of the required AI Agent class (e.g., `BenchmarkAgent` or whatever is defined in `loop.agent_class_name`).
+  3. Stores the agent back into the state object (`env.agent`).
+
+### 3. `EventLogging`
+* **What it does:** Responsible for logging telemetry (requests and responses to external tools) into the `SolidLoop::Event` table.
+* **Request Phase:** Passes control down the chain.
+* **Response Phase:** On the way back up, the layer inspects `Context.metadata`. If lower layers (like `ToolExecution`) executed an MCP server call and populated network parameters (TCP/IP logs, `mcp_request_data`, `mcp_response_data`), `EventLogging` will save or update a record of this interaction in the database for the debug event log.
+
+### 4. `ToolExecution`
+* **What it does:** Resolves the persisted call against the loop's MCP tools,
+  invokes it and commits a terminal `executed_at` checkpoint with the result.
+* Duplicate jobs serialize on the `ToolCall` row. Once `executed_at` is set,
+  later jobs reuse the persisted result and do not invoke the tool again.
+* MCP `isError` results are persisted with `is_success: false`; they remain
+  normal tool responses so the model can self-correct.
+* A policy middleware can call
+  `env.skip_execution!(result:, success: false)` and continue downstream. The
+  supplied result is checkpointed and emitted without invoking MCP. Returning
+  without calling downstream still holds the step for approval.
+* The layer always invokes `@app.call(env)`, allowing `ResponseCreation` to use
+  the same path for executed, rejected and replayed results.
+
+### 5. `ResponseCreation`
+* **What it does:** The layer responsible for advancing the LLM dialogue. It converts the tool's output into a format suitable for the neural network.
+* **Response Phase:**
+  1. Creates a new `SolidLoop::Message` with the role `"tool"`, linking it via `tool_call_id` to the tool we just executed, and assigns the final result (output payload) to the message's content.
+  2. Creates the response idempotently; replaying the same job cannot create a
+     duplicate `tool` message.
+  3. Advances the turn according to the agent's execution mode (see below). Only
+     after *all* results exist does it conditionally queue the next LLM turn, and
+     that completion gate is fenced by the execution token so exactly one job
+     queues the next turn.
+
+## 2b. Parallel vs. sequential tool execution
+
+An assistant turn can return several tool calls. `SolidLoop::Base#sequential_tool_calls?`
+selects how they run; the single flag gates both emission and ordering:
+
+| | `sequential_tool_calls? == false` (default) | `sequential_tool_calls? == true` |
+|---|---|---|
+| **Emission** (`ResponseParsing`) | enqueues **every** call at once → they run in parallel across workers | enqueues **only the first** call |
+| **Ordering guard** (`ToolExecutionJob`) | not applied; any call may run first | rejects any job that is not the first unresolved call |
+| **Continuation** (`ResponseCreation`) | does not chain; each job only fires the completion gate once every result exists | chains the next unresolved call in model order after each result |
+| **Completion gate** | identical in both modes — the job that commits the *last* result queues the next LLM turn |
+
+Safety is orthogonal to ordering and holds in both modes: execution-token
+fencing rejects stale-turn jobs, and the `executed_at` checkpoint makes a
+re-delivered job replay its persisted result instead of re-invoking the tool. In
+parallel mode the N jobs serialize their response creation on the assistant
+message row, so exactly one of them observes "all resolved" and queues the next
+turn.
+
+`resume!` mirrors emission: it re-enters every unresolved call for a parallel
+agent (so a half-finished multi-call turn cannot stall) and only the first for a
+sequential agent.
+
+## 3. Pause, resume and HITL
+
+`Base#resume!` inspects the latest assistant tool turn. If a tool result is
+missing, it re-enqueues that exact step; it starts a new LLM turn only when no
+tool step is unresolved. This is the same re-entry primitive an approval action
+uses after changing its host-owned decision to approved.
+
+Loop transitions are conditional and fenced by an execution token. A worker
+from before a pause/resume cycle cannot complete or fail the new generation.
+Pausing between tool execution and response creation is safe: resume replays the
+persisted checkpoint without repeating the tool.
+
+`executed_at` prevents duplicates after a result is committed. It cannot provide
+exactly-once semantics if a process dies after an external side effect but before
+the database checkpoint commits. For that window SolidLoop supplies a stable
+per-`ToolCall` idempotency key with every invocation — `ctx.idempotency_key` for
+in-process handlers, `_meta["solidloop/idempotencyKey"]` on the wire (see
+`docs/guides/mcp_transports.md`). Irreversible remote tools should dedupe on it.
+
+Approval records, approvers, reasons and deadlines remain host concerns. A
+generic pause TTL is intentionally not provided because `paused` also represents
+manual pauses and safety limits. Hosts should put `expires_at` on the approval,
+resolve approval/timeout races under a lock and use `skip_execution!` to emit an
+expired or rejected result.
+
+**HITL is cleanest under `sequential_tool_calls? == true`.** With sequential
+execution there is a single in-flight tool at a time, so an approval gate has one
+unambiguous step to hold, approve or reject before the next call is dispatched.
+`skip_execution!` still works in either mode, but the parallel default puts every
+call from the turn in flight simultaneously, so per-tool approval must gate each
+of the concurrent jobs independently. Agents that gate tools on human approval
+should opt into sequential execution.