mistri 0.1.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5b5526cf23888f4a5872797ce5a6ebe245c0ff1a857ba8bc717681930f8dd4b8
-  data.tar.gz: fa3f6dee2638cbd1e9c4b60378c40b4ac5a894d3639afff157be6bb70e5fd00f
+  metadata.gz: e553e955f2a1263afed1b26889dbf04773916dbd6f33f9229e6db2b729dd655b
+  data.tar.gz: 98b95079d9f91b068580ac2708c362b1317452378eba99fe7643e301660ad89d
 SHA512:
-  metadata.gz: 8cb353d464264b7d44b8c335cfa938b72136f5273f674d4517632326929216a1a93db9577fb9c75ce66f46a70fc862726ad4444dad6e0a47ba0c8d74f3356ae5
-  data.tar.gz: e5114ba9c5404ee9698c4878599d9f2a7895e3d9c470e1874b0c377b2ac6b59ef62b541940c5798b5cb769df148f0e5a02d835b78bc75a5655905a2fd4bdfcdb
+  metadata.gz: cb690580fe35c2226bfa0d86d23f77c9236e8dcb10ba3af0a7753a1d52b96c11e951d41ee647d0cf947dae1d3c82ca834105070099dec0c4bac728bcd296698b
+  data.tar.gz: 9e5be1a7fde00c351abc5f123ddfa82db8cb0ae34f14b0c6aa474b12338c587958c547d68e70280de7cda780ea4e674953e0f28681d33c17b37edd8ea1672d8e

data/CHANGELOG.md

@@ -5,6 +5,64 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [0.2.1] - 2026-07-05
+
+- The gem homepage and documentation links now point at
+  [mistri.sh](https://mistri.sh), the project site with full docs at
+  [mistri.sh/docs](https://mistri.sh/docs/getting-started/).
+- README polish: wordmark, testing section.
+
+## [0.2.0] - 2026-07-05
+
+- Repository hygiene: coverage floor enforced in CI (simplecov, 90% line),
+  contributing/security/conduct docs, issue and PR templates, Dependabot,
+  and rubygems documentation and bug tracker links.
+
+- Per-tool timeouts: Tool.define(..., timeout: 30) answers in band when a
+  handler stalls, so one hung tool cannot stall the run.
+- :tool_result events carry duration (seconds) for executed tools, feeding
+  latency metrics straight from any sink.
+
+- Mistri::Reminder.every(3, text): a periodic tail reminder for long runs,
+  riding transform_context; due by completed assistant turns, fresh on the
+  wire each time, never persisted.
+
+- Tool hooks: before_tool(call, context) blocks a call by returning the
+  reason as a String, answered to the model in band; it outranks the
+  approval gate and screens approved calls again at settle time, so an
+  aged approval never beats current policy. after_tool(call, result,
+  context) may replace a result (both channels), nil keeps it. Hooks that
+  raise fail safe: before blocks, after answers in band.
+- transform_context accepts an array of transforms, applied in order.
+
+- Result#usage: every run reports its own token and cost accounting,
+  summing persisted turns and compaction calls; task sums across its fix
+  passes. Hosts meter a run without walking the session.
+
+- MCP stdio wire: Client.new(command: [...], env: {...}) spawns a local
+  server as a child process speaking line-delimited JSON-RPC, credentials
+  in its environment per spec. Dying servers and non-protocol stdout fail
+  loudly; close terminates the child.
+
+- MCP connections out of the box: Mistri::MCP::OAuth.start/.complete/
+  .refresh are storage-agnostic services implementing the spec's OAuth 2.1
+  subset (challenge and well-known discovery, RFC 8414 metadata with an
+  OpenID fallback, dynamic client registration as the host application,
+  PKCE with resource indicators, rotating refresh). `rails generate
+  mistri:mcp YourModel` creates a host-named connection model whose rows
+  carry their own flow state and encrypted tokens, with connection.tools
+  bridging straight into an agent and refreshing ahead of expiry.
+
+- MCP bridge: Mistri::MCP::Client speaks Streamable HTTP (initialize
+  handshake, tools/list with pagination, tools/call, sessions with
+  transparent expiry recovery, JSON or SSE responses) with zero new
+  dependencies. Auth is a headers hash or a token string-or-lambda; a
+  lambda re-resolves once on 401, so host refresh logic lives in one place.
+  Mistri::MCP.tools bridges any server (or any duck-typed client, the
+  official mcp gem included) into Mistri tools with allow/deny lists, name
+  prefixing, and per-tool approval gates, so a third-party write tool can
+  ride the human-approval arc.
+
 ## [0.1.0] - 2026-07-05
 
 - Live integration harness: `rake integration` runs every feature end to
@@ -175,3 +233,5 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 ## [0.0.1] - 2026-07-04
 
 - Reserved the gem name.
+
+[0.1.0]: https://github.com/mcheemaa/mistri/releases/tag/v0.1.0

data/README.md

@@ -1,10 +1,16 @@
-<h1 align="center">مستری</h1>
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="assets/logo-dark.svg">
+    <img src="assets/logo-light.svg" alt="مستری" width="360">
+  </picture>
+</p>
 
 <p align="center"><strong>mistri</strong>, the agent harness for Ruby applications.</p>
 
 <p align="center">
   <a href="https://rubygems.org/gems/mistri"><img alt="Gem Version" src="https://img.shields.io/gem/v/mistri"></a>
   <a href="https://github.com/mcheemaa/mistri/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/mcheemaa/mistri/actions/workflows/ci.yml/badge.svg"></a>
+  <a href="https://codecov.io/gh/mcheemaa/mistri"><img alt="Coverage" src="https://codecov.io/gh/mcheemaa/mistri/graph/badge.svg"></a>
   <a href="mistri.gemspec"><img alt="Ruby >= 3.2" src="https://img.shields.io/badge/ruby-%3E%3D%203.2-CC342D"></a>
   <a href="Gemfile"><img alt="Runtime dependencies: zero" src="https://img.shields.io/badge/runtime_deps-0-brightgreen"></a>
   <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-blue.svg"></a>
@@ -250,6 +256,58 @@ The edit engine matches exactly, then whitespace-tolerantly; an ambiguous
 match refuses (never silently edits the wrong place), and a near-miss error
 names the closest region so the model's retry is one-shot.
 
+## MCP
+
+Bridge any Model Context Protocol server's tools into an agent. The client
+speaks Streamable HTTP with zero new dependencies; auth is a token string
+or a lambda that re-resolves once on 401, so refresh logic lives in one
+place. Approval gates compose: a third-party write tool can require a
+human.
+
+```ruby
+client = Mistri::MCP::Client.new(url: "https://mcp.linear.app/mcp",
+                                 token: -> { connection.bearer_token })
+tools = Mistri::MCP.tools(client, prefix: "linear",
+                          gates: { "create_issue" => true })
+
+agent = Mistri.agent("claude-opus-4-8", tools: tools)
+```
+
+Local stdio servers spawn as child processes, credentials in their
+environment. That is also the whole "give the agent a browser" story:
+
+```ruby
+browser = Mistri::MCP::Client.new(
+  command: ["npx", "-y", "@playwright/mcp@latest", "--browser", "chrome", "--headless"],
+)
+agent = Mistri.agent("claude-opus-4-8",
+                     tools: Mistri::MCP.tools(browser, allow: %w[browser_navigate browser_snapshot]))
+```
+
+For the full connect-your-tools story in Rails, generate a connection model
+(name it whatever you like):
+
+```console
+$ bin/rails generate mistri:mcp McpConnection
+```
+
+Each row is one server connection carrying its own OAuth flow state and
+encrypted tokens. The OAuth services underneath (`Mistri::MCP::OAuth.start`,
+`.complete`, `.refresh`) are storage-agnostic, so the same flow works from a
+controller, a GraphQL mutation, or a job. Registration happens as your
+application: `client_name:` is yours to set.
+
+```ruby
+connection, authorize_url = McpConnection.connect(
+  name: "Linear", url: params[:url],
+  client_name: "YourApp", redirect_uri: mcp_callback_url,
+)
+# redirect the user to authorize_url; then, in the callback:
+connection = McpConnection.complete(state: params[:state], code: params[:code])
+
+agent = Mistri.agent("claude-opus-4-8", tools: connection.tools(prefix: "linear"))
+```
+
 ## Streaming into Rails
 
 Sinks bridge the event stream to a transport, and compose as blocks:
@@ -290,7 +348,7 @@ Mistri.agent("gpt-5.5", provider_options: { reasoning: { effort: "high" } })
 Mistri.agent("claude-opus-4-8", provider_options: { cache: false })
 ```
 
-## Verified for real
+## Testing
 
 `rake test` is hermetic and fast. `rake integration` runs every feature end
 to end against real provider APIs, once per model in the matrix. Scenarios

data/lib/generators/mistri/mcp/mcp_generator.rb

@@ -0,0 +1,57 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/named_base"
+require "rails/generators/active_record"
+
+module Mistri
+  module Generators
+    # bin/rails generate mistri:mcp McpConnection
+    #
+    # Creates a host-named MCP connection model and migration: each row is
+    # one server connection and carries its own OAuth flow state, so the
+    # connect/callback pair works from a controller, a GraphQL mutation, or
+    # anywhere else the host prefers.
+    class McpGenerator < Rails::Generators::NamedBase
+      include ActiveRecord::Generators::Migration
+
+      source_root File.expand_path("templates", __dir__)
+
+      argument :name, type: :string, default: "McpConnection"
+
+      def create_model
+        template "model.rb.tt", File.join("app/models", class_path, "#{file_name}.rb")
+      end
+
+      def create_migration_file
+        migration_template "migration.rb.tt",
+                           File.join(db_migrate_path, "create_#{table_name}.rb")
+      end
+
+      def show_wiring
+        say <<~NOTE
+
+          Connect (from a controller, GraphQL mutation, wherever):
+
+            connection, authorize_url = #{class_name}.connect(
+              name: "Linear", url: params[:url],
+              client_name: "YourApp", redirect_uri: mcp_callback_url,
+            )
+            redirect_to authorize_url, allow_other_host: true
+
+          Callback:
+
+            connection = #{class_name}.complete(state: params[:state], code: params[:code])
+
+          Then hand its tools to an agent:
+
+            agent = Mistri.agent("claude-opus-4-8", tools: connection.tools(prefix: "linear"))
+
+          Tokens are encrypted; run bin/rails db:encryption:init if you have
+          not set up Active Record encryption.
+
+        NOTE
+      end
+    end
+  end
+end

data/lib/generators/mistri/mcp/templates/migration.rb.tt

@@ -0,0 +1,27 @@
+class Create<%= table_name.camelize %> < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    create_table :<%= table_name %> do |t|
+      t.string :name, null: false
+      t.string :url, null: false
+      t.string :status, null: false, default: "pending"
+
+      # OAuth flow state; cleared once connected.
+      t.string :state
+      t.string :code_verifier
+      t.string :redirect_uri
+
+      t.string :client_id
+      t.string :client_secret
+      t.string :token_endpoint
+      t.string :token_auth_method
+      t.text :access_token
+      t.text :refresh_token
+      t.datetime :expires_at
+      t.string :scope
+      t.timestamps
+    end
+
+    # The callback finds its pending row by state.
+    add_index :<%= table_name %>, :state, unique: true
+  end
+end

data/lib/generators/mistri/mcp/templates/model.rb.tt

@@ -0,0 +1,63 @@
+# One MCP server connection: its own OAuth flow state, its tokens, and the
+# bridge into agent tools. Rows with a manually supplied access_token (API
+# key servers) work the same way without the OAuth columns.
+class <%= class_name %> < ApplicationRecord
+  encrypts :access_token, :refresh_token, :client_secret
+
+  # Begin the OAuth flow: persists a pending connection and returns it with
+  # the URL to send the user to.
+  def self.connect(name:, url:, client_name:, redirect_uri:, scope: nil)
+    flow = Mistri::MCP::OAuth.start(url: url, client_name: client_name,
+                                    redirect_uri: redirect_uri, scope: scope)
+    connection = create!(name: name, url: url, status: "pending",
+                         state: flow["state"], code_verifier: flow["code_verifier"],
+                         client_id: flow["client_id"], client_secret: flow["client_secret"],
+                         token_endpoint: flow["token_endpoint"],
+                         token_auth_method: flow["token_auth_method"],
+                         redirect_uri: flow["redirect_uri"])
+    [connection, flow["authorize_url"]]
+  end
+
+  # Finish the flow from the OAuth callback.
+  def self.complete(state:, code:)
+    connection = find_by!(state: state)
+    tokens = Mistri::MCP::OAuth.complete(code: code,
+                                         code_verifier: connection.code_verifier,
+                                         client_id: connection.client_id,
+                                         client_secret: connection.client_secret,
+                                         token_endpoint: connection.token_endpoint,
+                                         token_auth_method: connection.token_auth_method,
+                                         resource: connection.url,
+                                         redirect_uri: connection.redirect_uri)
+    connection.update!(status: "connected", state: nil, code_verifier: nil,
+                       access_token: tokens["access_token"],
+                       refresh_token: tokens["refresh_token"],
+                       expires_at: tokens["expires_at"], scope: tokens["scope"])
+    connection
+  end
+
+  def client
+    Mistri::MCP::Client.new(url: url, token: -> { bearer_token })
+  end
+
+  def tools(**options)
+    Mistri::MCP.tools(client, **options)
+  end
+
+  # A fresh bearer for every request; refreshes ahead of expiry and persists
+  # the rotated refresh token.
+  def bearer_token
+    refresh! if refresh_token.present? && expires_at && expires_at <= 1.minute.from_now
+    access_token
+  end
+
+  def refresh!
+    tokens = Mistri::MCP::OAuth.refresh(refresh_token: refresh_token,
+                                        client_id: client_id, client_secret: client_secret,
+                                        token_endpoint: token_endpoint,
+                                        token_auth_method: token_auth_method, resource: url)
+    update!(access_token: tokens["access_token"],
+            refresh_token: tokens["refresh_token"] || refresh_token,
+            expires_at: tokens["expires_at"], scope: tokens["scope"] || scope)
+  end
+end

data/lib/mistri/agent.rb

@@ -22,9 +22,15 @@ module Mistri
     # skills: an array of Skill (or a directory path for Skills.load). Their
     # descriptions join the system prompt and a read_skill tool serves full
     # bodies on demand.
+    # before_tool and after_tool are the programmatic gates around every
+    # execution: before_tool(call, context) blocks a call by returning the
+    # reason as a String, which answers the model in band (and it runs again
+    # when an approved call finally executes, so a decision that aged days
+    # still passes current policy); after_tool(call, result, context) may
+    # return a replacement result, or nil to keep the original.
     def initialize(provider:, session: nil, system: nil, tools: [], budget: nil,
                    max_concurrency: 4, transform_context: nil, compaction: Compaction.new,
-                   retries: RetryPolicy.new, skills: [])
+                   retries: RetryPolicy.new, skills: [], before_tool: nil, after_tool: nil)
       @provider = provider
       @session = session || Session.new(store: Stores::Memory.new)
       skills = skills.is_a?(String) ? Skills.load(skills) : Array(skills)
@@ -35,9 +41,11 @@ module Mistri
 
       @budget = budget || Budget.new
       @max_concurrency = max_concurrency
-      @transform_context = transform_context
+      @transform_context = Array(transform_context)
       @compaction = compaction || nil
       @retries = retries || nil
+      @before_tool = before_tool
+      @after_tool = after_tool
     end
 
     attr_reader :session
@@ -70,7 +78,8 @@ module Mistri
       pending = open.select { |approval| approval[:decision].nil? }
       if pending.any?
         return Result.new(message: nil, status: :awaiting_approval,
-                          pending: pending.map { |approval| approval[:call] })
+                          pending: pending.map { |approval| approval[:call] },
+                          usage: Usage.zero)
       end
 
       settle(open, signal, &emit)
@@ -89,7 +98,9 @@ module Mistri
     def task(input, schema:, images: [], signal: nil, fixes: 1, &emit)
       result = run(task_input(input, schema), images: images, signal: signal,
                                               output_schema: schema, &emit)
+      spent = result.usage
       fixes.downto(0) do |remaining|
+        result = result.with(usage: spent)
         return result unless result.completed?
 
         value = parse_output(result.text)
@@ -98,6 +109,7 @@ module Mistri
         raise SchemaError, "task output failed validation: #{errors.join("; ")}" if remaining.zero?
 
         result = run(fix_prompt(errors), signal: signal, output_schema: schema, &emit)
+        spent += result.usage
       end
     end
 
@@ -126,7 +138,7 @@ module Mistri
       started = monotonic_now
       loop do
         reason = @budget.exceeded(turns: turns, usage: usage, elapsed: monotonic_now - started)
-        return stop_for_budget(reason, &emit) if reason
+        return stop_for_budget(reason, usage, &emit) if reason
 
         fold_steers
         compacted = auto_compact(&emit)
@@ -138,8 +150,8 @@ module Mistri
         # Any tool call the turn made must be answered or parked, or the
         # transcript is unpairable and replay fails.
         parked = last.tool_calls? ? run_tools(last, signal, &emit) : []
-        return suspended(last, parked) if parked.any?
-        return finished(last) if done?(last, signal)
+        return suspended(last, parked, usage) if parked.any?
+        return finished(last, usage) if done?(last, signal)
       end
     end
 
@@ -191,8 +203,9 @@ module Mistri
     # attempt is recorded as a retry entry, never as a message, so retries
     # stay invisible to the model. Only the final outcome persists.
     def run_turn(signal, output_schema = nil, &emit)
-      history = @session.messages
-      history = @transform_context.call(history) if @transform_context
+      history = @transform_context.reduce(@session.messages) do |messages, transform|
+        transform.call(messages)
+      end
       attempt = 0
       loop do
         message = @provider.stream(messages: history, system: @system,
@@ -243,7 +256,7 @@ module Mistri
         return []
       end
 
-      parked, free = calls.partition { |call| gated?(call) }
+      parked, free = screen(calls, signal, &emit).partition { |call| gated?(call) }
       execute(free, signal, &emit)
       parked.each do |call|
         @session.append("approval_request", "call" => call.to_h)
@@ -254,7 +267,8 @@ module Mistri
 
     def settle(open, signal, &emit)
       approved, denied = open.partition { |approval| approval[:decision]["approved"] }
-      execute(approved.map { |approval| approval[:call] }, signal, &emit)
+      cleared = screen(approved.map { |approval| approval[:call] }, signal, &emit)
+      execute(cleared, signal, &emit)
       denied.each do |approval|
         note = approval[:decision]["note"]
         text = "The user denied this tool call#{note ? ": #{note}" : "."}"
@@ -268,17 +282,52 @@ module Mistri
       results = ToolExecutor.call(calls, @tools_by_name, signal: signal,
                                                          max_concurrency: @max_concurrency,
                                                          session: @session, emit: emit)
-      results.each { |call, result| answer(call, result, &emit) }
+      context = hook_context(signal, emit)
+      results.each do |call, result, seconds|
+        result = rewrite(call, result, context) if @after_tool
+        answer(call, result, duration: seconds, &emit)
+      end
+    end
+
+    # A blocked call answers in band, so the model reads the reason and
+    # reacts; a hook that raises blocks conservatively rather than letting
+    # an unpoliced call through.
+    def screen(calls, signal, &emit)
+      return calls unless @before_tool
+
+      context = hook_context(signal, emit)
+      calls.reject do |call|
+        reason = begin
+          @before_tool.call(call, context)
+        rescue StandardError => e
+          "the before_tool hook failed: #{e.class}: #{e.message}"
+        end
+        next false unless reason.is_a?(String)
+
+        answer(call, "Blocked: #{reason}", &emit)
+        true
+      end
+    end
+
+    def rewrite(call, result, context)
+      @after_tool.call(call, result, context) || result
+    rescue StandardError => e
+      "Error in after_tool hook: #{e.class}: #{e.message}"
+    end
+
+    def hook_context(signal, emit)
+      ToolContext.new(session: @session, signal: signal, emit: emit)
     end
 
     # The tool message carries both channels; the :tool_result event exposes
     # it whole so hosts read event.message.ui for their side of the result.
-    def answer(call, result, &emit)
+    def answer(call, result, duration: nil, &emit)
       content, ui = result.is_a?(ToolResult) ? [result.content, result.ui] : [result, nil]
       message = @session.append_message(Message.tool(content: content, tool_call_id: call.id,
                                                      tool_name: call.name, ui: ui))
       text = content.is_a?(String) ? content : "[content]"
-      emit&.call(Event.new(type: :tool_result, tool_call: call, content: text, message: message))
+      emit&.call(Event.new(type: :tool_result, tool_call: call, content: text,
+                           message: message, duration: duration))
     end
 
     def gated?(call)
@@ -286,24 +335,24 @@ module Mistri
       tool ? tool.needs_approval?(call.arguments) : false
     end
 
-    def finished(message)
+    def finished(message, usage)
       status = { StopReason::ABORTED => :aborted, StopReason::BUDGET => :budget,
                  StopReason::ERROR => :error }.fetch(message.stop_reason, :completed)
-      Result.new(message: message, status: status)
+      Result.new(message: message, status: status, usage: usage)
     end
 
-    def suspended(message, parked)
-      Result.new(message: message, status: :awaiting_approval, pending: parked)
+    def suspended(message, parked, usage)
+      Result.new(message: message, status: :awaiting_approval, pending: parked, usage: usage)
     end
 
-    def stop_for_budget(reason, &emit)
+    def stop_for_budget(reason, usage, &emit)
       message = Message.assistant(content: "Run stopped: #{reason} budget reached.",
                                   stop_reason: StopReason::BUDGET,
                                   error_message: "budget_#{reason}")
       @session.append_message(message)
       emit&.call(Event.new(type: :error, reason: StopReason::BUDGET, message: message,
                            error_message: "budget_#{reason}"))
-      Result.new(message: message, status: :budget)
+      Result.new(message: message, status: :budget, usage: usage)
     end
 
     # Distinguishable from a parsed nil: JSON "null" is a valid value.

data/lib/mistri/event.rb

@@ -11,8 +11,11 @@ module Mistri
   # message's content list.
   # origin names the sub-agent an event came from: nil for this agent's own
   # turns, and nesting joins names left to right ("researcher>writer").
+  # duration is the tool's execution time in seconds on :tool_result
+  # events; nil where nothing ran (denials, interruptions).
   class Event < Data.define(:type, :content_index, :delta, :content, :tool_call,
-                            :reason, :message, :error_message, :partial, :origin)
+                            :reason, :message, :error_message, :partial, :origin,
+                            :duration)
     # The stream types come from a provider mid-turn; the loop adds
     # :tool_result after it runs each tool, :approval_needed when a gated
     # call parks for a human, and :compacting/:compaction around a context
@@ -29,7 +32,8 @@ module Mistri
     ].freeze
 
     def initialize(type:, content_index: nil, delta: nil, content: nil, tool_call: nil,
-                   reason: nil, message: nil, error_message: nil, partial: nil, origin: nil)
+                   reason: nil, message: nil, error_message: nil, partial: nil, origin: nil,
+                   duration: nil)
       raise ArgumentError, "unknown event type #{type.inspect}" unless TYPES.include?(type)
 
       super
@@ -44,7 +48,7 @@ module Mistri
     # Partials are ephemeral streaming state and stay out of serialization.
     def to_h
       { type:, content_index:, delta:, content:, tool_call: tool_call&.to_h,
-        reason:, message: message&.to_h, error_message:, origin: }.compact
+        reason:, message: message&.to_h, error_message:, origin:, duration: }.compact
     end
   end
 end