mistri 0.5.0 → 0.6.0

data/docs/tool-contracts.md

@@ -0,0 +1,324 @@
+# Tool contracts
+
+A Mistri tool is application code exposed to model output. Treat that boundary
+like an HTTP endpoint: declare what it accepts, authorize the acting context,
+and make writes idempotent or reconcilable.
+
+This guide covers local tools. Remote MCP tools cross the same execution
+boundary; see [MCP](mcp.md) for the additional server contract.
+
+## Define a tool
+
+```ruby
+charge_card = Mistri::Tool.define(
+  "charge_card", "Charges a saved payment method.",
+  schema: lambda {
+    string :payment_method_id, "Saved payment method ID", required: true
+    number :amount_usd, "Amount in US dollars", required: true
+  },
+  needs_approval: ->(args) { args.fetch("amount_usd") >= 500 },
+) do |args, context|
+  Payments.charge!(
+    customer: context.app.fetch(:customer),
+    payment_method_id: args.fetch("payment_method_id"),
+    amount_usd: args.fetch("amount_usd"),
+  )
+end
+```
+
+The handler may accept only `args`, or `args` and `context`. `context.app` is
+the object passed as `Mistri.agent(context:)`; it is the usual place for the
+acting user, tenant, or request-scoped authorization context.
+
+Handlers may return:
+
+- a String, delivered to the model as text;
+- a Hash, number, boolean, nil, or data Array, serialized as JSON or empty text;
+- an Array made only of Strings and Mistri content blocks, delivered as
+  multiple content blocks, including images;
+- `Mistri::ToolResult`, which separates model content, host-only UI, and an
+  explicit failure fact.
+
+## The execution boundary
+
+A completed model call moves through these phases in order:
+
+1. Mistri verifies the provider's call envelope and pairing metadata.
+2. The argument value is copied into bounded, immutable JSON owned by Mistri.
+3. The tool's explicit `argument_normalizer`, if any, runs once.
+4. Core schema validation and the tool's host validator run.
+5. `before_tool` evaluates current application policy.
+6. `needs_approval` decides whether the prepared call must park.
+7. A free call emits `:tool_started` and invokes the handler.
+8. `after_tool` may rewrite the result, which then persists and emits
+   `:tool_result`.
+
+Malformed or invalid calls do not reach policy or execution. The model receives
+a bounded error result it can correct, while valid sibling calls in the same
+turn continue. Results settle in the model's call order even when handlers run
+concurrently.
+
+An approved call is revalidated against the current tool definition and passes
+through `before_tool` again on `resume`. Its normalizer and approval predicate
+do not run again: the human approved the already prepared arguments.
+
+## Schema forms
+
+The DSL builds a JSON Schema 2020-12 object:
+
+```ruby
+search = Mistri::Tool.define("search", "Searches products.", schema: lambda {
+  string :query, "Search text", required: true
+  integer :limit, "Maximum result count"
+  array :tags, "Required tags", items: { type: "string" }
+  object :filters, "Structured filters" do
+    string :country, "ISO country code"
+    boolean :in_stock, "Only products in stock"
+  end
+}) do |args|
+  filters = args.fetch("filters", {})
+  Catalog.search(
+    query: args.fetch("query"),
+    limit: args["limit"],
+    tags: args.fetch("tags", []),
+    country: filters["country"],
+    in_stock: filters["in_stock"],
+  )
+end
+```
+
+Use `input_schema:` for a raw schema:
+
+```ruby
+schema = {
+  type: "object",
+  properties: {
+    "invoice_id" => { type: "string" },
+  },
+  required: ["invoice_id"],
+  additionalProperties: false,
+}
+
+Mistri::Tool.define("pay_invoice", "Pays one invoice.", input_schema: schema) do |args|
+  Invoices.pay!(args.fetch("invoice_id"))
+end
+```
+
+The root must be an object schema. `Tool.define` without `schema:` or
+`input_schema:` creates a closed no-argument tool; model-supplied fields are
+rejected.
+
+### Open and closed objects
+
+Supplied object schemas follow JSON Schema's default-open semantics. Declared
+properties do not automatically reject additional keys. Extract the fields you
+intend to use rather than mass-assigning the entire model hash.
+
+Set `additionalProperties: false` in a raw schema when extra keys must fail.
+
+A nested object declared without a block is intentionally freeform:
+
+```ruby
+object :config, "Chart-library configuration", required: true
+```
+
+It accepts any JSON object, including `{}`. This is useful when Mistri should
+not prescribe keys owned by another library. A bare top-level `Schema.build`
+raises because it communicates no argument contract.
+
+Freeform objects cannot become strict constrained output without changing their
+meaning. `Schema.strict` and task planning reject that shape instead of silently
+narrowing it to `{}`.
+
+### Portable validation subset
+
+Mistri's zero-dependency validator enforces:
+
+- JSON `type`, including type unions;
+- `enum`;
+- `required` and declared `properties`;
+- one-schema array `items`;
+- tuple array `prefixItems`;
+- boolean schemas;
+- boolean or schema-valued `additionalProperties`.
+
+Other standard keywords remain provider-facing generation guidance unless a
+complete validator is supplied. Examples include `minimum`, `pattern`,
+`format`, length constraints, and applicators such as `anyOf`.
+
+An argument-applicable, non-empty `patternProperties` requires complete
+authority because approximating its key and value interactions would be
+unsafe. External references are not resolved.
+
+Task output is stricter than tool input: task planning rejects assertions that
+local validation cannot guarantee. A task promises a validated value; a tool
+server can still enforce its own broader contract after Mistri's local subset.
+
+## Host validation
+
+Use `argument_validator:` for domain rules that supplement core validation:
+
+```ruby
+invoice = Mistri::Tool.define(
+  "pay_invoice", "Pays an approved invoice.",
+  input_schema: invoice_schema,
+  argument_validator: lambda { |args, _schema|
+    if args.fetch("invoice_id").start_with?("inv_")
+      []
+    else
+      ["$.invoice_id must identify an invoice"]
+    end
+  },
+) do |args|
+  Invoices.pay!(args.fetch("invoice_id"))
+end
+```
+
+The validator receives the deeply frozen arguments and canonical schema. It
+must return an Array of Strings. Core validation runs first and cannot be
+weakened.
+
+Use `complete_argument_validator:` only when the host validator implements the
+whole schema, including interactions outside Mistri's portable subset. It has
+the same `(args, schema)` signature and is mutually exclusive with
+`argument_validator:`.
+
+Validator errors are sent back to the model. Describe the expected contract;
+do not echo received secrets or field values.
+
+## Explicit normalization
+
+Mistri does not coerce model input. If a tool intentionally accepts a legacy
+alias or representation, declare that compatibility policy on the tool:
+
+```ruby
+normalizer = lambda do |args|
+  next args unless args.key?("account")
+  raise ArgumentError, "choose account or account_id" if args.key?("account_id")
+
+  normalized = args.dup
+  normalized["account_id"] = normalized.delete("account")
+  normalized
+end
+
+tool = Mistri::Tool.define(
+  "lookup_account", "Looks up one account.",
+  schema: -> { string :account_id, "Account ID", required: true },
+  argument_normalizer: normalizer,
+) do |args|
+  Accounts.find(args.fetch("account_id"))
+end
+```
+
+The normalizer must return a Hash and should be pure, deterministic, and
+idempotent. Its output is what validation, policy, approval, and the handler
+see.
+
+Direct `Tool#call` is a trusted host invocation. It applies the normalizer for
+compatibility, but the Agent's model-input ownership and validation boundary is
+not involved.
+
+## Policy hooks
+
+`before_tool` can block a call with a model-readable reason:
+
+```ruby
+before_tool = lambda do |call, context|
+  next "customer cannot perform this action" unless context.app.fetch(:customer).active?
+  next "currency is not enabled" unless call.arguments.fetch("currency") == "USD"
+end
+
+agent = Mistri.agent("claude-opus-4-8", tools: tools, before_tool: before_tool)
+```
+
+The hook also runs when an approved call resumes, so long-lived approval does
+not bypass current authorization. A raised hook error blocks conservatively.
+
+`after_tool` receives `(call, result, context)` and may return a replacement
+result. Returning nil keeps the original. If the original result is an error,
+rewriting its text cannot relabel it as success.
+
+## Results and UI
+
+Use `Mistri::ToolResult` when the model and host need different views:
+
+```ruby
+Mistri::Tool.define("edit_page", "Applies a page edit.", schema: lambda {
+  object :changes, "Page changes", required: true
+}) do |args|
+  page = Pages.apply(args.fetch("changes"))
+  Mistri::ToolResult.new(
+    content: "Saved the page.",
+    ui: { "html" => page.html, "revision" => page.lock_version },
+  )
+end
+```
+
+`ui` persists on the tool message and appears on the `:tool_result` event, but
+is never sent to a provider.
+
+Non-text tool-result content remains provider-neutral in the Session. Anthropic
+can receive image blocks in a tool result. The current OpenAI and Gemini
+function-result serializers have no such encoding and send an explicit
+non-text-omission marker alongside the text instead.
+
+Return `error: true` for an expected failure the model should handle:
+
+```ruby
+account || Mistri::ToolResult.new(
+  content: "Account not found.",
+  error: true,
+)
+```
+
+Mistri also marks unknown tools, validation and policy rejection, handler
+exceptions, timeouts, pre-invocation interruption, failed result hooks,
+crash-healed calls, and MCP `isError` results. Human denial is an expected
+control outcome, not an execution failure.
+
+`event.tool_error` and `event.message.tool_error?` expose the fact without
+parsing prose. A legacy persisted result without the field remains unknown;
+Mistri does not infer historical success from text.
+
+An error flag does not make a replay safe. A handler may have committed a write
+before raising or timing out. Mistri never mechanically replays the same call
+because it is marked failed, but the model may issue another call. The host
+still owns idempotency and reconciliation.
+
+## Hand off the turn
+
+`ends_turn: true` ends the run after the tool executes instead of asking the
+model for another response. This is useful for `ask_user`, escalation, or a
+structural handoff:
+
+```ruby
+ask_user = Mistri::Tool.define(
+  "ask_user", "Asks the human and waits.",
+  ends_turn: true,
+  schema: -> { string :question, "Question", required: true },
+) do |_args|
+  "Question presented to the user."
+end
+```
+
+`result.handed_off?` reports that the model's floor moved away. It does not
+claim the tool succeeded; inspect the tool result separately. The human answer
+arrives as the next `run` input.
+
+## Resource limits
+
+Completed arguments are bounded at 8 MiB, 64 levels, 10,000 JSON nodes, and 64
+KiB per numeric token. Encoded Anthropic and OpenAI arguments receive a linear
+lexical pass before JSON parsing. Gemini's enclosing SSE record receives the
+same structural preflight before its argument object is canonicalized.
+
+These are safety boundaries, not business validation. Keep tool schemas narrow,
+validate domain rules explicitly, and return references rather than embedding
+unbounded data in a tool call.
+
+## Related guides
+
+- [Sessions and control](sessions.md)
+- [MCP](mcp.md)
+- [Reliability](reliability.md)
+- [Upgrading](../UPGRADING.md)

data/examples/approval.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+# Fire-and-forget human approval. The run suspends without executing the
+# gated tool; the decision is a session write (here immediate, in production
+# a controller action days later); resume settles it and finishes. Sessions
+# live in JSONL files so every step could happen in a different process.
+# Needs ANTHROPIC_API_KEY.
+#
+#   ruby examples/approval.rb
+
+require "mistri"
+require "tmpdir"
+
+Dir.mktmpdir do |dir|
+  store = Mistri::Stores::JSONL.new(dir)
+  session = Mistri::Session.new(store:)
+
+  send_gift = Mistri::Tool.define(
+    "send_gift", "Sends a real gift.", needs_approval: true,
+                                       schema: -> { string :to, "Recipient", required: true }
+  ) do |args|
+    "gift queued for #{args["to"]}"
+  end
+
+  agent = Mistri.agent("claude-opus-4-8", tools: [send_gift], session:)
+  result = agent.run("Send a welcome gift to Ana.")
+  puts "suspended: #{result.awaiting_approval?} (nothing executed)"
+
+  # Any process can decide: a bare session is enough.
+  Mistri::Session.new(store:, id: session.id).approve(result.pending.first.id)
+
+  reloaded = Mistri::Session.new(store:, id: session.id)
+  resumed = Mistri.agent("claude-opus-4-8", tools: [send_gift], session: reloaded).resume
+  puts "resumed: #{resumed.status}"
+  puts resumed.text
+end

data/examples/browser.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+# Give an agent a real browser in one line: Playwright's official MCP
+# server over the stdio wire. The allowlist grants only what this agent
+# needs, and needs_approval gates could ride any of it. Needs
+# ANTHROPIC_API_KEY, node, and Chrome.
+#
+#   ruby examples/browser.rb
+
+require "mistri"
+
+browser = Mistri::MCP::Client.new(
+  command: ["npx", "-y", "@playwright/mcp@latest", "--browser", "chrome", "--headless"],
+  read_timeout: 180
+)
+
+begin
+  tools = Mistri::MCP.tools(browser, prefix: "web",
+                                     allow: %w[browser_navigate browser_snapshot])
+
+  agent = Mistri.agent("claude-opus-4-8", tools: tools,
+                                          system: "Use the browser tools to answer. Be brief.")
+
+  puts agent.run("Open https://example.com and report the main heading.").text
+ensure
+  browser.close
+end

data/examples/page_editor.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+# The landing-page shape: one value (a database column in production, a
+# string here) edited in place through the document tools. A host can wrap the
+# result in ToolResult when it also needs a separate UI payload.
+# Needs ANTHROPIC_API_KEY.
+#
+#   ruby examples/page_editor.rb
+
+require "mistri"
+
+page = +<<~HTML
+  <header>
+    <h1>Placeholder Headline</h1>
+    <p class="tagline">Ship faster.</p>
+  </header>
+HTML
+
+workspace = Mistri::Workspace::Single.new(
+  read: -> { page.dup },
+  write: ->(html) { page.replace(html) },
+  path: "hero.html"
+)
+
+agent = Mistri.agent("claude-opus-4-8",
+                     tools: Mistri::Tools.files(workspace),
+                     system: "Edit the page with the document tools. Read before editing.")
+
+agent.run("Change the h1 headline to: Gifts that land.")
+
+puts page

data/examples/quickstart.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+# A tool-using agent, streamed. Needs ANTHROPIC_API_KEY.
+#
+#   ruby examples/quickstart.rb
+
+require "mistri"
+
+weather = Mistri::Tool.define(
+  "get_weather", "Current weather for a city.",
+  schema: -> { string :city, "City name", required: true }
+) do |args|
+  "34C and clear in #{args["city"]}"
+end
+
+agent = Mistri.agent("claude-opus-4-8", tools: [weather])
+
+agent.run("What should I wear in Lahore today? One sentence.") do |event|
+  print event.delta if event.type == :text_delta
+end
+puts

data/lib/generators/mistri/install/install_generator.rb

@@ -41,14 +41,18 @@ module Mistri
 
       private
 
-      # MySQL TEXT caps at 64KB, which a single large tool result can blow
-      # through; MEDIUMTEXT holds 16MB. Postgres text is unbounded.
+      # A parallel tool turn can legitimately exceed MySQL MEDIUMTEXT even
+      # though each call stays inside its own input limit. Postgres text is unbounded.
       def payload_size
         adapter = ::ActiveRecord::Base.connection_db_config.adapter.to_s
-        adapter.match?(/mysql|trilogy/) ? ", size: :medium" : ""
+        payload_size_for(adapter)
       rescue StandardError
         ""
       end
+
+      def payload_size_for(adapter)
+        adapter.to_s.match?(/mysql|trilogy/) ? ", size: :long" : ""
+      end
     end
   end
 end

data/lib/generators/mistri/install/templates/migration.rb.tt

@@ -7,8 +7,8 @@ class Create<%= table_name.camelize %> < ActiveRecord::Migration[<%= ActiveRecor
       t.timestamps
     end
 
-    # One session has one writer; a colliding append must raise, never
-    # silently reorder entries.
+    # The unique position is the optimistic concurrency boundary for
+    # independent appenders; a collision retries at the next position.
     add_index :<%= table_name %>, [:session_id, :position], unique: true
   end
 end

data/lib/generators/mistri/mcp/templates/migration.rb.tt

@@ -12,7 +12,7 @@ class Create<%= table_name.camelize %> < ActiveRecord::Migration[<%= ActiveRecor
 
       t.string :client_id
       t.string :client_secret
-      t.string :token_endpoint
+      t.string :issuer
       t.string :token_auth_method
       t.text :access_token
       t.text :refresh_token

data/lib/generators/mistri/mcp/templates/model.rb.tt

@@ -4,15 +4,19 @@
 class <%= class_name %> < ApplicationRecord
   encrypts :access_token, :refresh_token, :client_secret
 
+  def self.mcp_allow_non_public = nil
+
   # Begin the OAuth flow: persists a pending connection and returns it with
   # the URL to send the user to.
-  def self.connect(name:, url:, client_name:, redirect_uri:, scope: nil)
+  def self.connect(name:, url:, client_name:, redirect_uri:, scope: nil, issuer: nil)
     flow = Mistri::MCP::OAuth.start(url: url, client_name: client_name,
-                                    redirect_uri: redirect_uri, scope: scope)
+                                    redirect_uri: redirect_uri, scope: scope,
+                                    issuer: issuer,
+                                    allow_non_public: mcp_allow_non_public)
     connection = create!(name: name, url: url, status: "pending",
                          state: flow["state"], code_verifier: flow["code_verifier"],
                          client_id: flow["client_id"], client_secret: flow["client_secret"],
-                         token_endpoint: flow["token_endpoint"],
+                         issuer: flow["issuer"],
                          token_auth_method: flow["token_auth_method"],
                          redirect_uri: flow["redirect_uri"])
     [connection, flow["authorize_url"]]
@@ -25,10 +29,11 @@ class <%= class_name %> < ApplicationRecord
                                          code_verifier: connection.code_verifier,
                                          client_id: connection.client_id,
                                          client_secret: connection.client_secret,
-                                         token_endpoint: connection.token_endpoint,
+                                         issuer: connection.issuer,
                                          token_auth_method: connection.token_auth_method,
                                          resource: connection.url,
-                                         redirect_uri: connection.redirect_uri)
+                                         redirect_uri: connection.redirect_uri,
+                                         allow_non_public: mcp_allow_non_public)
     connection.update!(status: "connected", state: nil, code_verifier: nil,
                        access_token: tokens["access_token"],
                        refresh_token: tokens["refresh_token"],
@@ -37,7 +42,8 @@ class <%= class_name %> < ApplicationRecord
   end
 
   def client
-    Mistri::MCP::Client.new(url: url, token: -> { bearer_token })
+    Mistri::MCP::Client.new(url: url, token: -> { bearer_token },
+                            allow_non_public: self.class.mcp_allow_non_public)
   end
 
   def tools(**options)
@@ -54,8 +60,9 @@ class <%= class_name %> < ApplicationRecord
   def refresh!
     tokens = Mistri::MCP::OAuth.refresh(refresh_token: refresh_token,
                                         client_id: client_id, client_secret: client_secret,
-                                        token_endpoint: token_endpoint,
-                                        token_auth_method: token_auth_method, resource: url)
+                                        issuer: issuer, token_auth_method: token_auth_method,
+                                        resource: url,
+                                        allow_non_public: self.class.mcp_allow_non_public)
     update!(access_token: tokens["access_token"],
             refresh_token: tokens["refresh_token"] || refresh_token,
             expires_at: tokens["expires_at"], scope: tokens["scope"] || scope)