mistri 0.5.0 → 0.6.0

data/README.md

@@ -7,189 +7,189 @@
 
 <p align="center"><strong>mistri</strong>, the agent harness for Ruby applications.</p>
 
-<p align="center"><a href="https://mistri.sh">mistri.sh</a> · <a href="https://mistri.sh/docs/getting-started/">docs</a></p>
+<p align="center"><a href="https://mistri.sh">mistri.sh</a> · <a href="docs/README.md">documentation</a> · <a href="UPGRADING.md">upgrading</a></p>
 
 <p align="center">
   <a href="https://rubygems.org/gems/mistri"><img alt="Gem Version" src="https://img.shields.io/gem/v/mistri"></a>
   <a href="https://github.com/mcheemaa/mistri/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/mcheemaa/mistri/actions/workflows/ci.yml/badge.svg"></a>
   <a href="https://codecov.io/gh/mcheemaa/mistri"><img alt="Coverage" src="https://img.shields.io/codecov/c/github/mcheemaa/mistri"></a>
   <a href="mistri.gemspec"><img alt="Ruby >= 3.2" src="https://img.shields.io/badge/ruby-%3E%3D%203.2-CC342D"></a>
-  <a href="Gemfile"><img alt="Runtime dependencies: zero" src="https://img.shields.io/badge/runtime_deps-0-brightgreen"></a>
+  <a href="mistri.gemspec"><img alt="Runtime dependencies: zero" src="https://img.shields.io/badge/runtime_deps-0-brightgreen"></a>
   <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-blue.svg"></a>
 </p>
 
 A mistri (Urdu: مستری) is the fixer: the skilled tradesperson who actually
-gets it done. This one lives inside your app, not in a terminal. It runs the
-model loop, executes tools, streams every event, persists sessions to your
-own database, and pauses for a human when a tool needs approval, all with
-zero runtime gem dependencies.
+gets it done. Mistri is an application-embedded agent runtime for Ruby. It owns
+the model loop, safe tool boundary, event stream, and append-only session
+record. Your application owns authorization, policy, presentation, and the
+semantics of its side effects.
 
-```ruby
-require "mistri"
-
-weather = Mistri::Tool.define(
-  "get_weather", "Current weather for a city.",
-  schema: -> { string :city, "City name", required: true },
-) do |args|
-  Weather.for(args["city"])
-end
-
-agent = Mistri.agent("claude-opus-4-8", tools: [weather]) # reads ANTHROPIC_API_KEY
-
-agent.run("What should I wear in Lahore today?") do |event|
-  print event.delta if event.type == :text_delta
-end
-```
-
-## Why Mistri
+Mistri works in plain Ruby applications, including Rails, Sinatra, Hanami, jobs,
+and services. Rails integrations are optional. The gem declares zero runtime
+dependencies.
 
-- **Built for applications.** Sessions are durable, append-only records in
-  your own store. Runs stop, resume, steer, and compact from any process.
-- **Fire-and-forget human approval.** A gated tool suspends the run and
-  returns immediately. The approval can arrive two days later from a bare
-  web request; nothing sleeps waiting.
-- **Three providers, frontier-deep.** Anthropic, OpenAI, and Gemini, each
-  streamed natively with thinking, prompt caching, parallel tool calls, and
-  constrained JSON output. One message model across all three.
-- **Zero runtime dependencies.** Plain Ruby all the way down.
-- **Verified against real APIs.** A live integration harness runs every
-  feature end to end on every provider (`rake integration`).
+## Start in sixty seconds
 
-## Install
+Add Mistri to your bundle:
 
 ```ruby
 gem "mistri"
 ```
 
-## Sixty-second start
+Then define a tool and run an agent:
 
 ```ruby
-agent = Mistri.agent("claude-opus-4-8")
-result = agent.run("Name three Ruby web frameworks.")
-puts result.text
-```
-
-`Mistri.agent` infers the provider from the model id (`claude-*`, `gpt-*`,
-`gemini-*`) and reads the matching key (`ANTHROPIC_API_KEY`,
-`OPENAI_API_KEY`, `GEMINI_API_KEY`); pass `api_key:` to set it explicitly.
-Every run returns a `Result`: `completed?`, `awaiting_approval?`,
-`aborted?`, `errored?`, with `text` and (for tasks) `output`.
-
-## Tools
-
-A tool is a name, a description, an argument schema, and a block. The block
-returns a String, a Hash (sent as JSON), or content such as an image. The
-agent calls tools, feeds results back, and loops until the model answers;
-independent calls in a turn run in parallel.
+require "mistri"
 
-```ruby
-weather = Mistri::Tool.define("get_weather", "Current weather for a city.", schema: lambda {
-  string :city, "City name", required: true
-  string :units, "Temperature units", enum: %w[celsius fahrenheit]
-}) do |args|
-  Weather.for(args["city"], units: args["units"] || "celsius")
+weather = Mistri::Tool.define(
+  "get_weather", "Current weather for a city.",
+  schema: -> { string :city, "City name", required: true },
+) do |args|
+  { city: args.fetch("city"), forecast: "34 C and clear" }
 end
-```
 
-A tool can speak on two channels: `content` for the model, `ui` for your
-interface. The `ui` payload rides the `:tool_result` event and persists with
-the session, but never reaches a provider:
+agent = Mistri.agent("claude-opus-4-8", tools: [weather])
 
-```ruby
-Mistri::Tool.define("edit_page", "Applies a page edit.") do |args|
-  page = apply(args)
-  Mistri::ToolResult.new(content: "Saved.", ui: { "html" => page })
+result = agent.run("What should I wear in Lahore today? One sentence.") do |event|
+  print event.delta if event.type == :text_delta
 end
+puts
 ```
 
-Handlers and hooks can take the run's context as a second argument, and
-`context.app` carries whatever object you pass as `Mistri.agent(context:)`
-(the acting user, a tenant, a request), so tools stay authorization-aware
-without closure gymnastics:
+`Mistri.agent` infers the provider from the model ID and reads
+`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, or `GEMINI_API_KEY`. Pass `api_key:`
+explicitly when the host resolves credentials another way. A normally
+completed invocation returns a `Mistri::Result` with status
+predicates, final text, usage, and structured task output when requested.
+Host-contract failures raise as documented in [Reliability](docs/reliability.md).
 
-```ruby
-agent = Mistri.agent("claude-opus-4-8", tools: tools,
-                     context: { traveler: current_traveler })
+Runnable examples live in [`examples/`](examples).
 
-Mistri::Tool.define("book_hotel", "Books the chosen hotel.") do |args, context|
-  Bookings.create!(args, traveler: context.app[:traveler])
-end
-```
+## Why Mistri
 
-A tool can also be the last word of its turn. `ends_turn: true` makes the
-loop end the run once the tool executes, instead of prompting the model
-again: an `ask_user` tool hands the floor to a human structurally, no
-"remember to stop after asking" prompt required. The result says it
-happened (`result.handed_off?`), and the answer arrives as the next run's
-input:
+- **One session model.** The same append-only log runs over memory, JSONL,
+  Active Record, or a host store. With a persistent store, runs can suspend,
+  reload, resume, steer, and compact without inventing a second state model. See
+  [Sessions and control](docs/sessions.md).
+- **Human approval does not hold a thread.** A gated tool parks the request and
+  returns. With a shared store, another process can record the decision later;
+  `resume` revalidates the exact call before execution.
+- **Provider semantics survive the abstraction.** Anthropic, OpenAI, and
+  Gemini stream through direct implementations that retain each provider's
+  reasoning, tool-pairing, and replay requirements. Unknown model IDs still
+  pass through when their provider can be inferred.
+- **Tool input crosses a real boundary.** Completed model arguments are owned,
+  bounded, validated, and deeply frozen before hooks, approval predicates, MCP,
+  or handlers see them. See [Tool contracts](docs/tool-contracts.md).
+- **The runtime stays small.** The [gemspec](mistri.gemspec) declares no runtime
+  gem dependencies. Optional Rails adapters load only when required.
+- **Real APIs are part of the test strategy.** The
+  [live integration suite](CONTRIBUTING.md#setup)
+  exercises critical control, persistence, delegation, MCP, and
+  structured-output paths across all three providers when their keys are
+  present. The default CI suite remains hermetic.
+
+## Choose the right layer
+
+These projects solve different problems:
+
+| Need | Start with |
+| --- | --- |
+| A durable tool loop inside any Ruby application, with append-only sessions, approval, steering, compaction, linked workers, and provider-neutral MCP | **Mistri** |
+| Broad model access plus media, embeddings, image generation, and Rails-backed chat | [RubyLLM](https://github.com/crmne/ruby_llm) |
+| Agents expressed through Rails controllers, actions, callbacks, views, and Active Job | [Active Agent](https://github.com/activeagents/activeagent) |
+
+Mistri is deliberately not a terminal UI, hosted agent service, broad media
+client, policy engine, or exactly-once job system. The host application keeps
+its users, authorization rules, queues, database, UI, idempotency keys, and
+reconciliation policy. Mistri supplies the execution mechanism that connects
+them.
+
+## How the loop works
+
+1. The input is appended to the session before the provider is called.
+2. Provider events stream to the subscriber as the response arrives.
+3. Completed tool calls cross Mistri's ownership and validation boundary.
+4. Current host policy and approval rules run before a handler can start.
+5. Tool results append to the same session, then the model continues until it
+   answers, hands off the turn, suspends for approval, aborts, or reaches a
+   budget.
+
+Every completed message is appended to the chosen Session store before the next
+model-visible turn. Opaque reasoning and pairing metadata replay only to the
+provider that created them; the shared transcript stays provider-neutral.
+
+## Tools and approval
+
+A tool has a name, description, JSON Schema object, and handler. Independent
+calls in one model turn execute concurrently up to `max_concurrency`.
 
 ```ruby
-ask_user = Mistri::Tool.define("ask_user", "Ask the human and wait.",
-                               ends_turn: true,
-                               schema: -> { string :question, "The question", required: true }) do |args|
-  "Question presented to the user."
+book_hotel = Mistri::Tool.define(
+  "book_hotel", "Books the selected hotel.",
+  schema: lambda {
+    string :hotel_id, "Hotel ID", required: true
+    number :total_usd, "Quoted total", required: true
+  },
+  needs_approval: ->(args) { args.fetch("total_usd") > 500 },
+) do |args, context|
+  Bookings.create!(
+    hotel_id: args.fetch("hotel_id"),
+    traveler: context.app.fetch(:traveler),
+  )
 end
+
+agent = Mistri.agent(
+  "claude-opus-4-8",
+  tools: [book_hotel],
+  context: { traveler: current_traveler },
+)
 ```
 
-## Human approval
+Validation runs before the approval predicate, so policy never receives a
+string where the schema promised a number. Invalid calls return a bounded,
+model-readable error and valid siblings still run. Mistri never coerces model
+input implicitly.
 
-Mark a tool `needs_approval: true` (or a predicate on its arguments) and the
-run suspends instead of executing it, instantly, with no thread waiting.
-The decision is a one-line session write from any process, any time later;
-`resume` settles it and carries on.
+When a gated call arrives:
 
 ```ruby
-book_hotel = Mistri::Tool.define("book_hotel", "Books the chosen hotel.",
-                                 needs_approval: ->(args) { args["total_usd"].to_i > 500 }) do |args|
-  Bookings.create!(args)
-end
-
 result = agent.run("Book the corner suite for the Lisbon trip")
-result.awaiting_approval?   # => true; nothing executed
+result.awaiting_approval? # => true; the handler did not run
 
-# Days later, in a controller:
-Mistri::Session.new(store:, id: session_id).approve(call_id)   # or .deny(call_id, note: "...")
-
-# Then, in a worker:
-Mistri.agent("claude-opus-4-8", tools: tools, session: reloaded).resume
+call = result.pending.fetch(0)
+agent.session.approve(call.id, note: "Approved by finance")
+result = agent.resume
 ```
 
-The harness renders nothing: it emits an `:approval_needed` event and your
-app draws the UI.
+That is the smallest in-process flow. With a durable store, a controller,
+route, mutation, job, or console can append the decision using only the Session
+ID and call ID; a later process rebuilds current tools and policy before
+resuming. See [Sessions and control](docs/sessions.md#human-approval).
 
-## Steering
+Tool results can also carry host-only UI data or an explicit failure fact. See
+[Tool contracts](docs/tool-contracts.md) for schemas, validators, normalizers,
+`Mistri::ToolResult`, hooks, and `ends_turn`.
 
-Queue a message into a running exchange from any process. It folds into the
-conversation at the next turn boundary; one that arrives as the model
-finishes cleanly extends the run so it gets answered.
+## Sessions and control
 
-```ruby
-Mistri::Session.new(store:, id: session_id).steer("Make the headline blue instead.")
-```
-
-Background workers' reports arrive through the same inbox (see Sub-agents).
-A host that wakes an idle session when a steer lands should watch
-`session.pending_inbox`, which holds both, in arrival order.
-
-## Sessions
-
-A session is the durable record of a run: an append-only entry log over a
-pluggable store (memory, JSONL files, or your database).
+The default memory store is useful for one process. JSONL persists the complete
+append-only record on local disk without a database:
 
 ```ruby
-store = Mistri::Stores::JSONL.new("tmp/sessions")
-session = Mistri::Session.new(store:)
+store = Mistri::Stores::JSONL.new("tmp/mistri-sessions")
+session = Mistri::Session.new(store: store)
 
-agent = Mistri.agent("claude-opus-4-8", session:)
+agent = Mistri.agent("claude-opus-4-8", session: session)
 agent.run("Start a haiku about the sea.")
 
-# Later, even in another process: reload by id and continue.
-resumed = Mistri.agent("claude-opus-4-8", session: Mistri::Session.new(store:, id: session.id))
-resumed.run("Now finish it.")
+# A later process can reload the same session by ID.
+reloaded = Mistri::Session.new(store: store, id: session.id)
+Mistri.agent("claude-opus-4-8", session: reloaded).run("Now finish it.")
 ```
 
-In Rails, generate a model (name it whatever you like) and use the
-ActiveRecord store:
+For a database-backed store, implement `append(id, entry)` and `load(id)`, or
+use the optional Active Record adapter:
 
 ```console
 $ bin/rails generate mistri:install AgentEntry
@@ -197,317 +197,221 @@ $ bin/rails generate mistri:install AgentEntry
 
 ```ruby
 require "mistri/stores/active_record"
+
 store = Mistri::Stores::ActiveRecord.new(AgentEntry)
 ```
 
-## Compaction
-
-Long sessions survive their context window: when the conversation grows into
-the reserve headroom, the provider writes a visible structured summary and
-replay continues from it. The full history stays in your store for
-transcript views. On by default whenever the model's window is known;
-`compaction: false` disables it.
+Steering is another append, so a request handler or worker can redirect a live
+exchange without sharing its Agent object:
 
 ```ruby
-agent.context_usage   # => { tokens: 141_000, window: 200_000, fraction: 0.705 }
-agent.compact         # the manual button
+Mistri::Session.new(store: store, id: session_id).steer(
+  "Make the headline blue instead."
+)
 ```
 
-`:compacting` and `:compaction` events carry the summary, so users see
-exactly what the model still remembers.
+Only one Agent may actively call `run` or `resume` for a session at a time.
+Approvals, steering, and worker reports may append concurrently. Read the
+[reliability contract](docs/reliability.md) before connecting tools with
+external side effects.
 
-## Task mode
+## Streaming into any Ruby application
 
-A run that must end in JSON matching a schema. Tools run as usual; providers
-constrain the final answer natively where they can, and the answer is
-validated client-side everywhere. A violation goes back to the model once,
-then raises. You get a guaranteed shape or a loud error, never silence.
+Every run accepts a block. Handle events directly, or compose a sink:
 
 ```ruby
-schema = {
-  type: "object",
-  properties: { "tiers" => { type: "array", items: { type: "string" } } },
-  required: ["tiers"],
-}
+sse = Mistri::Sinks::SSE.new(stream)          # any IO-like object
+sink = Mistri::Sinks::Coalesced.new(sse)      # merge token bursts to UI speed
 
-result = agent.task("Extract the pricing tiers from this page.", schema: schema)
-result.output # => { "tiers" => [...] }, parsed and validated
+agent.run(input, &sink)
 ```
 
-A run that suspends for approval, or that an `ends_turn` tool ended,
-returns as-is: validation applies to answers, not handoffs. Route on
-`result.handed_off?` and ask again once the human's answer arrives.
+`Mistri::Sinks::ActionCable` is available for Rails, but no Railtie is required.
+The same event stream works in Sinatra, Rack, a WebSocket server, a background
+job, or a test. Event types form an extensible union; consumers should handle
+the types they use and ignore the rest.
 
-## Skills
+## Long conversations and structured tasks
 
-Expert playbooks with progressive disclosure: each skill costs one line in
-the system prompt until the model decides it is relevant and pulls the full
-body through an auto-provided `read_skill` tool.
+Compaction is on by default for catalogued models. When a session enters the
+model's safe headroom, Mistri asks the provider for a visible structured
+summary, appends it, and continues from the summary plus a recent tail. The
+exact history remains in the store for transcripts.
 
 ```ruby
-agent = Mistri.agent("claude-opus-4-8", skills: "app/skills")   # or an array of Mistri::Skill
-```
-
-A skill is a `SKILL.md` (or flat `.md`) with `name:`/`description:`
-frontmatter, or built from database rows with
-`Mistri::Skill.new(name:, description:, body:)`.
-
-## Definitions
+agent.context_usage
+# => { tokens: 141_000, window: 1_000_000, fraction: 0.141 }
 
-An agent as a markdown file: YAML frontmatter for config, the body as the
-prompt, `{placeholders}` filled at build time (unfilled ones raise). Tool
-names and any extra keys stay your vocabulary; the gem only reads the
-file.
-
-```ruby
-definition = Mistri::Definition.load("app/agents/trip_planner.md")
-agent = Mistri.agent(definition.model,
-                     system: definition.render(first_name: traveler.first_name),
-                     tools: registry.build(definition.tools, traveler))
+agent.compact # explicit checkpoint; returns nil when nothing should compact
 ```
 
-## Sub-agents
-
-Delegate to a child agent with a clean context: exploration fills the
-child's window, and only the final answer returns. Children run on their own
-sessions in your store, linked in the parent transcript; their events stream
-into the parent tagged with an `origin`. Each run of a specialist can carry
-its own name, so two parallel researchers read as Corgi and Beagle in your
-UI instead of "researcher" twice.
+Task mode requires a final JSON value matching a schema. Providers use native
+constraints only when they can represent the contract; Mistri validates the
+answer locally in every case.
 
 ```ruby
-researcher = Mistri::SubAgent.new(
-  name: "researcher", description: "Reads pages and answers factual questions.",
-  provider: Mistri.provider("claude-haiku-4-5-20251001"),   # cheaper model for grunt work
-  system: "Research. Report findings only.", tools: [fetch_page],
-)
-agent = Mistri.agent("claude-opus-4-8", tools: [researcher.tool])
-```
-
-Or hand the model an open spawn tool and let it compose its own workers:
-a name, instructions, a tool subset, and a host-allowlisted model per
-child. Several spawns in one turn fan out in parallel. `pack` is the whole
-kit: the spawn tool plus a management console (`list_agents`, `read_agent`,
-`steer_agent`, `stop_agent`), with curated types and an optional
-dispatcher:
-
-```ruby
-spawn = Mistri::SubAgent.spawner(provider: provider, tools: [fetch_page, search])
-
-tools = Mistri::SubAgent.pack(
-  provider: provider, tools: [fetch_page, search],
-  types: { "researcher" => Mistri::Definition.load("agents/researcher.md") },
-  models: ["claude-haiku-4-5-20251001"],
-  dispatcher: Mistri::Dispatchers::Thread.new,      # or one lambda onto your queue
-)
-```
-
-A typed worker takes its prompt, tools, and model from the host's
-definition; `general-purpose` stays open for the model to compose.
-`max_children` (default 4) caps live workers, and every policy violation
-answers the model in band.
-
-Everything cross-process (stopping a worker from another process, the
-`:interrupted` liveness read, and the lease fence below) rides a lock
-adapter; configure one at boot. Without it, workers still run, but stops
-need the parent's own signal, a crashed child reads `:running`, and
-dispatched retries are unfenced:
+schema = {
+  type: "object",
+  properties: {
+    "tiers" => { type: "array", items: { type: "string" } },
+  },
+  required: ["tiers"],
+}
 
-```ruby
-Mistri.locks = Mistri::Locks::RailsCache.new   # Locks::Memory for a single process
+result = agent.task("Extract the pricing tiers.", schema: schema)
+result.output # parsed and validated
 ```
 
-With a dispatcher, `spawn_agent` takes `mode: "background"`: the model gets
-a truthful receipt at once and keeps working while the child runs. The
-console manages the roster with the same functions a host UI calls, so
-agent and user control stay structurally equal: either can read a worker's
-transcript, steer it mid-run, or stop it, from any process. When a worker
-finishes, its report delivers itself: a typed entry queues in the parent's
-inbox and folds at the next turn boundary as `[Corgi finished] <report>`
-(failures carry the error), while a `:subagent_report` event settles the
-worker's lane in whatever UI watched the spawn. In a queue host, the job
-rebuilds tools from the serializable spec and calls
-`SubAgent.run_dispatched`, which fences on the child's lease: a redelivered
-job leaves the running owner alone, a retry of a finished child is a no-op,
-and a retry of a crashed one runs it again.
-
-Everything about a child derives from the store and reads the same from
-any process, while it runs and forever after:
+Task validation applies when that invocation completes normally. An approval
+suspension or `ends_turn` handoff returns with `output` nil; after settlement or
+the human answer, call `task` again if structured output is still required.
 
-```ruby
-session.children               # => [#<Mistri::Child Corgi running>, ...]
-child.status                   # :queued, :running, :done, :stopped, :failed
-child.report                   # the terminal entry's report, once finished
-child.say("Check pricing too") # folds at the child's next turn boundary
-child.stop                     # cooperative, cross-process, within a tick
-
-session.transcript(include_children: true)
-# the whole conversation, each child's log spliced at its link entry and
-# origin-tagged like the live stream: a reloading UI rebuilds its lanes
-```
+See [Sessions and control](docs/sessions.md) for compaction boundaries,
+cross-provider replay, stores, approvals, and transcripts.
 
-## Editing documents
+## Sub-agents
 
-The document tools (`read_file`, `edit_file`, `write_file`, `find_in_file`,
-`list_files`) work over a workspace: a directory, memory, ActiveRecord, or
-a single value anywhere, like one database column holding a page:
+Delegate work into a child session so exploration does not consume the parent's
+context. Only the report returns to the parent, while the full child transcript
+remains linked and inspectable.
 
 ```ruby
-workspace = Mistri::Workspace::Single.new(
-  read: -> { page.html },
-  write: ->(html) { page.update!(html: html) },
-  path: "hero.html",
+researcher = Mistri::SubAgent.new(
+  name: "researcher",
+  description: "Reads pages and answers factual questions.",
+  provider: Mistri.provider("claude-haiku-4-5"),
+  system: "Research the question. Return findings and sources.",
+  tools: [fetch_page],
 )
-agent = Mistri.agent("claude-opus-4-8", tools: Mistri::Tools.files(workspace))
-```
 
-The edit engine matches exactly, then whitespace-tolerantly; an ambiguous
-match refuses (never silently edits the wrong place), and a near-miss error
-names the closest region so the model's retry is one-shot.
-
-## MCP
-
-Bridge any Model Context Protocol server's tools into an agent. The client
-speaks Streamable HTTP with zero new dependencies; auth is a token string
-or a lambda that re-resolves once on 401, so refresh logic lives in one
-place. Approval gates compose: a third-party write tool can require a
-human.
-
-```ruby
-client = Mistri::MCP::Client.new(url: "https://mcp.linear.app/mcp",
-                                 token: -> { connection.bearer_token })
-tools = Mistri::MCP.tools(client, prefix: "linear",
-                          gates: { "create_issue" => true })
-
-agent = Mistri.agent("claude-opus-4-8", tools: tools)
-```
-
-The bridge lists the server's tools once, at build time; `client.tools(refresh: true)`
-re-lists when a host wants a changed toolset. `prefix:` namespaces local
-names (`linear__create_issue`) because duplicate tool names raise at
-`Agent.new`: collisions fail loud instead of one server's tool silently
-shadowing another's.
-
-Local stdio servers spawn as child processes, credentials in their
-environment. That is also the whole "give the agent a browser" story:
-
-```ruby
-browser = Mistri::MCP::Client.new(
-  command: ["npx", "-y", "@playwright/mcp@latest", "--browser", "chrome", "--headless"],
-)
-agent = Mistri.agent("claude-opus-4-8",
-                     tools: Mistri::MCP.tools(browser, allow: %w[browser_navigate browser_snapshot]))
+agent = Mistri.agent("claude-opus-4-8", tools: [researcher.tool])
 ```
 
-For the full connect-your-tools story in Rails, generate a connection model
-(name it whatever you like):
+Mistri also provides a host-bounded spawn tool, named worker types, background
+dispatch with worker-side runtime reconstruction, exact capability grants,
+reports, steering, stopping, and a management console. See
+[Sub-agents](docs/sub-agents.md), including the runtime, concurrency, and lease
+limits a production dispatcher must understand.
 
-```console
-$ bin/rails generate mistri:mcp McpConnection
-```
+## Model Context Protocol
 
-Each row is one server connection carrying its own OAuth flow state and
-encrypted tokens. The OAuth services underneath (`Mistri::MCP::OAuth.start`,
-`.complete`, `.refresh`) are storage-agnostic, so the same flow works from a
-controller, a GraphQL mutation, or a job. Registration happens as your
-application: `client_name:` is yours to set.
+Bridge tools over Streamable HTTP or stdio, then apply the same local approval
+and validation boundary:
 
 ```ruby
-connection, authorize_url = McpConnection.connect(
-  name: "Linear", url: params[:url],
-  client_name: "YourApp", redirect_uri: mcp_callback_url,
+client = Mistri::MCP::Client.new(
+  url: "https://mcp.linear.app/mcp",
+  token: -> { connection.bearer_token },
 )
-# redirect the user to authorize_url; then, in the callback:
-connection = McpConnection.complete(state: params[:state], code: params[:code])
-
-agent = Mistri.agent("claude-opus-4-8", tools: connection.tools(prefix: "linear"))
-```
-
-## Streaming into Rails
 
-Sinks bridge the event stream to a transport, and compose as blocks:
-
-```ruby
-cable = Mistri::Sinks::ActionCable.new("agent_#{session.id}")
-sink = Mistri::Sinks::Coalesced.new(cable) # merges token bursts to UI speed
-
-agent.run(input, &sink)
-```
-
-`Mistri::Sinks::SSE.new(response.stream)` does the same for
-`ActionController::Live`. There is no Railtie and nothing to configure;
-the generator and stores duck-type into any app.
-
-## Stopping, budgets, reliability
-
-```ruby
-# Trip the signal from anywhere; the partial turn persists, resume is clean.
-signal = Mistri::AbortSignal.new
-agent.run("Draft a long essay.", signal: signal)
-
-# Ceilings are opt-in and off by default.
-budget = Mistri::Budget.new(turns: 20, cost_usd: 2.00)
-
-# Transient failures (429, 5xx, timeouts) retry with backoff, invisibly to
-# the model. On by default; retries: false disables.
-policy = Mistri::RetryPolicy.new(attempts: 3)
-```
-
-Retries are invisible to the model but not to your UI: each backoff emits a
-`:retry` event carrying `attempt`, `max_attempts`, and `delay`, so a sink can
-show a live "reconnecting" state instead of a silent spinner. Terminal events
-are loop-owned: `:done` and `:error` reach the subscriber only for the
-accepted attempt, so a recovered retry never flashes an error it then walks
-back.
-
-```ruby
-agent.run("Plan the itinerary.") do |event|
-  case event.type
-  when :text_delta then stream(event.delta)
-  when :retry then banner("Retrying (#{event.attempt}/#{event.max_attempts}) in #{event.delay}s")
-  when :done, :error then clear_banner
-  end
-end
+tools = Mistri::MCP.tools(
+  client,
+  prefix: "linear",
+  gates: { "create_issue" => true },
+)
 ```
 
-## Images and provider options
-
-```ruby
-photo = Mistri::Content::Image.from_bytes(File.binread("chart.png"), mime_type: "image/png")
-photo = Mistri::Content::Image.from_data_uri(params[:image])   # canvases and uploads
-agent.run("What trend does this chart show?", images: [photo])
+Remote URLs default to public HTTPS with validated, pinned DNS answers. Inbound
+records are size-bounded. An unconfirmed `tools/call` is never replayed
+automatically because the server may already have committed its side effect.
+OAuth services are storage-agnostic; the Rails connection generator is one
+optional host adapter.
+
+Read [MCP](docs/mcp.md) for stdio, OAuth, private-network policy, schema
+authority, response limits, and large-result resource links.
+
+## Skills, definitions, and workspaces
+
+- **Skills** expose one-line descriptions until the model requests the full
+  `SKILL.md`, keeping the system prompt small.
+- **Definitions** load an agent's model, tool names, and prompt from Markdown
+  with YAML frontmatter and checked placeholders.
+- **Workspaces** give the built-in read, write, edit, find, and list tools a
+  directory, memory value, Active Record table, or single host-owned value.
+
+The edit engine refuses ambiguous matches and, when a workspace supports
+conditional writes, rebases anchored edits without losing unrelated concurrent
+changes. A Directory workspace rejects lexical escape and traversal through
+existing symlinks in a stable, host-controlled tree; it is not an
+operating-system sandbox. See
+[Context and workspaces](docs/context-and-workspaces.md) and the runnable
+[`page_editor.rb`](examples/page_editor.rb) example.
+
+## Reliability contract
+
+Mistri makes failures explicit, but it does not pretend distributed side
+effects are exactly once:
+
+- Provider requests retry transient failures with backoff. The MCP client never
+  replays an ambiguous tool call; the Agent bridge returns the uncertainty as
+  an error-marked tool result.
+- A process can crash after a handler commits an external write but before the
+  result reaches the session. Tools with side effects must be idempotent or
+  reconcilable.
+- Budgets are checked between model-visible turns, so they are soft ceilings.
+- Provider and MCP records default to an 8 MiB per-record limit; streaming has
+  no total-size limit while each record remains safe.
+- `Mistri::AbortSignal` is an in-process cooperative signal. Cross-process
+  worker control uses the configured lock adapter and is also cooperative.
+
+The complete operational contract, event lifecycle, retry behavior, pricing
+rules, and sink guidance live in [Reliability](docs/reliability.md).
+
+## Documentation
+
+All primary documentation is versioned with the code and renders directly on
+GitHub:
+
+| Guide | What it covers |
+| --- | --- |
+| [Documentation index](docs/README.md) | The complete task-oriented map |
+| [Tool contracts](docs/tool-contracts.md) | Schemas, validation, approval, hooks, results, and handoff |
+| [Sessions and control](docs/sessions.md) | Stores, replay, approvals, steering, compaction, and transcripts |
+| [Context and workspaces](docs/context-and-workspaces.md) | Skills, definitions, context transforms, host-owned memory, and editable documents |
+| [Sub-agents](docs/sub-agents.md) | Specialists, spawning, dispatch, locks, reports, and control |
+| [MCP](docs/mcp.md) | HTTP and stdio, OAuth, egress policy, limits, and schema handling |
+| [Reliability](docs/reliability.md) | Retries, events, budgets, stopping, concurrency, and side effects |
+| [Upgrade guide](UPGRADING.md) | Required host changes between releases |
+| [Changelog](CHANGELOG.md) | Complete release history and exact behavior changes |
+
+## Testing and contributing
+
+The default suite is hermetic and needs no API keys:
 
-Mistri.agent("gpt-5.5", provider_options: { reasoning: { effort: "high" } })
-Mistri.agent("claude-opus-4-8", provider_options: { cache: false })
+```console
+$ bundle exec rake test
+$ bundle exec rubocop
 ```
 
-## Testing
-
-`rake test` is hermetic and fast. The Fake provider streams like the real
-ones, tool-call arguments included: each delta's partial carries the
-in-progress call with arguments parsed so far, so a UI that renders tool
-input as it arrives tests headless. `rake integration` runs every feature
-end to end against real provider APIs, once per model in the matrix: an
-Anthropic, an OpenAI, and a Gemini model by default. Scenarios assert that
-coined codenames (a ghost of a word like `Wraithowyn` exists in no training
-data) flowed through tool results, summaries, and child agents: proof of
-information flow, not model knowledge.
+Live tests read provider keys from the gitignored
+`.env.development.local` file:
 
 ```console
+$ MISTRI_LIVE=1 bundle exec rake test
 $ bundle exec rake integration
 $ MISTRI_INTEGRATION_MODELS=claude-opus-4-8 bundle exec rake integration
 ```
 
-## Roadmap
+The integration matrix exercises critical end-to-end scenarios against one
+Anthropic, OpenAI, and Gemini model by default when the corresponding keys are
+present. See
+[CONTRIBUTING.md](CONTRIBUTING.md) for the project contract.
+
+## Compatibility and support
+
+Mistri supports Ruby 3.2 and newer. It is pre-1.0: minor releases may include
+intentional contract changes with an explicit migration path. Read
+[UPGRADING.md](UPGRADING.md) and [CHANGELOG.md](CHANGELOG.md) before upgrading.
 
-Next up: strict tool schemas, provider-native MCP passthrough, and the
-hardening that falls out of the first production applications.
+Use [GitHub issues](https://github.com/mcheemaa/mistri/issues) for reproducible
+bugs and focused feature proposals. Report vulnerabilities privately as
+described in [SECURITY.md](SECURITY.md).
 
 ## Credits
 
 Mistri's architecture is informed by [pi](https://github.com/badlogic/pi-mono)
-by Mario Zechner. See NOTICE.
+by Mario Zechner. See [NOTICE](NOTICE).
 
 ## License
 
-MIT. See LICENSE.
+MIT. See [LICENSE](LICENSE).