ruby_llm-contract 0.10.1 → 0.10.3

data/docs/guide/output_schema.md

@@ -0,0 +1,93 @@
+# Output Schema
+
+> Read this as a reference for the schema DSL — every constraint, nested objects, arrays of objects, the full pattern table.
+
+Declare the expected output structure using [ruby_llm-schema](https://github.com/danielfriis/ruby_llm-schema) DSL. The schema serves **two purposes**:
+
+1. **Output validation** — replaces type and shape checks (enums, ranges, required fields). One declaration instead of many.
+2. **Provider-side request** — with the RubyLLM adapter, the schema is sent to the LLM provider via `chat.with_schema(...)`, asking the model to return JSON matching the shape. Cheaper models sometimes ignore the request, which is why client-side validation (point 1) still matters.
+
+> **Same DSL `RubyLLM::Agent.schema` accepts.** `output_schema do ... end` here is a wrapper around `RubyLLM::Schema.create(&block)` plus a client-side validation step. `RubyLLM::Agent.schema` accepts the same block — choosing one over the other does not change the schema language. The difference: `Agent.schema` lets you pass a `Proc` evaluated in runtime context (dynamic per-call schema); `Step.output_schema` is eager-compiled at class load and additionally drives `output_type` inference and `Validator.validate`. Both can coexist.
+
+All examples below extend the `SummarizeArticle` step from the [README](../../README.md).
+
+## Schema replaces type and shape checks
+
+```ruby
+# WITHOUT schema — many validates:
+validate("tldr must be a string")          { |o| o[:tldr].is_a?(String) }
+validate("takeaways must be an array")     { |o| o[:takeaways].is_a?(Array) }
+validate("takeaways 3 to 5")               { |o| (3..5).cover?(o[:takeaways].size) }
+ALLOWED_TONES = %w[neutral positive negative analytical].freeze
+validate("tone must be an allowed label") { |o| ALLOWED_TONES.include?(o[:tone]) }
+
+# WITH schema — one declaration:
+output_schema do
+  string :tldr
+  array  :takeaways, of: :string, min_items: 3, max_items: 5
+  string :tone, enum: %w[neutral positive negative analytical]
+end
+```
+
+## Nested objects in arrays
+
+Use `object do...end` inside `array` when you need more than a primitive per element. Concrete scenario: the UI card grows a "confidence bar" next to each takeaway so editors can see which points the model was sure about vs guessing. That requires `confidence` paired with `text`, not two parallel arrays that could desync. Nested objects make the pairing a schema invariant:
+
+```ruby
+output_schema do
+  string :tldr
+  array :takeaways, min_items: 3, max_items: 5 do
+    object do
+      string :text
+      number :confidence, minimum: 0.0, maximum: 1.0
+    end
+  end
+  string :tone, enum: %w[neutral positive negative analytical]
+end
+```
+
+## Schema pattern reference
+
+| Your output looks like | Schema pattern | Example |
+|---|---|---|
+| `{"tldr": "...", "tone": "positive"}` | Flat fields | `string :tldr; string :tone, enum: [...]` |
+| `{"takeaways": ["...", "..."]}` | Array of primitives | `array :takeaways, of: :string, min_items: 3, max_items: 5` |
+| `{"takeaways": [{"text": "...", "confidence": 0.9}]}` | Array of objects | `array :takeaways do; object do; string :text; number :confidence; end; end` |
+
+Without `object do...end`, `array :takeaways do; string :text; end` tells the provider "takeaways is an array of strings" — not objects. That's what you get back.
+
+## Why schema alone is not enough
+
+Schema validates **shape** — correct types, allowed values, field presence. But LLMs can return structurally valid JSON that is **logically wrong**. Validates catch what schema can't:
+
+```ruby
+output_schema do
+  string :tldr
+  array  :takeaways, of: :string, min_items: 3, max_items: 5
+  string :tone, enum: %w[neutral positive negative analytical]
+end
+
+# Schema allows any string for :tldr — but a 500-char "summary" breaks the UI card.
+validate("TL;DR fits the card") { |o, _| o[:tldr].length <= 200 }
+
+# Schema enforces 3–5 takeaways — but says nothing about them being distinct.
+validate("takeaways are unique") { |o, _| o[:takeaways] == o[:takeaways].uniq }
+
+# Schema can't express cross-field rules.
+validate("critical tone requires at least one concrete risk") do |o, _|
+  next true unless o[:tone] == "negative"
+  o[:takeaways].any? { |t| t.match?(/fail|break|crash|outage|vulnerab/i) }
+end
+```
+
+## Supported constraints
+
+| Constraint | Types | Example |
+|---|---|---|
+| `enum` | string, integer | `string :tone, enum: %w[neutral positive negative analytical]` |
+| `minimum` / `maximum` | number, integer | `number :confidence, minimum: 0.0, maximum: 1.0` |
+| `min_length` / `max_length` | string | `string :tldr, min_length: 1, max_length: 200` |
+| `min_items` / `max_items` | array | `array :takeaways, of: :string, min_items: 3, max_items: 5` |
+| `additional_properties` | object | Set to `false` in the schema to reject extra keys |
+
+Keyword args use Ruby snake_case (`min_length`, `min_items`). The DSL converts them internally to JSON Schema's camelCase (`minLength`, `minItems`) before sending the schema to the provider — you don't need to write camelCase in Ruby.

data/docs/guide/pipeline.md

@@ -0,0 +1,154 @@
+# Pipeline
+
+> Read this when one step isn't enough — you need multi-step with fail-fast, automatic data threading, and per-step models.
+
+Chain multiple steps with automatic data threading, fail-fast, per-step models, trace, and timeout.
+
+A pipeline needs more than one step to be interesting. This guide grows the `SummarizeArticle` step from the [README](../../README.md) into a three-step content pipeline that tags and routes the summary to a UI card.
+
+## Full example: article → summary → hashtags → card
+
+```ruby
+# Step 1 — the flagship step from README, unchanged.
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  prompt <<~PROMPT
+    Summarize this article for a UI card. Return a short TL;DR,
+    3 to 5 key takeaways, and a tone label.
+
+    {input}
+  PROMPT
+
+  output_schema do
+    string :tldr
+    array  :takeaways, of: :string, min_items: 3, max_items: 5
+    string :tone, enum: %w[neutral positive negative analytical]
+  end
+
+  validate("TL;DR fits the card") { |o, _| o[:tldr].length <= 200 }
+end
+
+# Step 2 — reads SummarizeArticle's output, produces hashtags suitable for social posts.
+class GenerateHashtags < RubyLLM::Contract::Step::Base
+  input_type Hash
+
+  output_schema do
+    # Carry through the summary fields downstream consumers (and the next step) need.
+    string :tldr
+    array  :takeaways, of: :string
+    string :tone, enum: %w[neutral positive negative analytical]
+    # Add new field.
+    array  :hashtags, of: :string, min_items: 2, max_items: 5
+  end
+
+  prompt do
+    rule "Preserve tldr / takeaways / tone exactly as given."
+    user "Article summary: {tldr}\nTone: {tone}\nGenerate 2 to 5 concise hashtags."
+  end
+
+  validate("tone preserved") { |o, input| o[:tone] == input[:tone] }
+end
+
+# Step 3 — final shape the UI card consumes.
+class BuildArticleCard < RubyLLM::Contract::Step::Base
+  input_type Hash
+
+  output_schema do
+    string :headline
+    string :summary
+    array  :hashtags, of: :string
+    string :sentiment_icon, enum: %w[😐 🙂 ⚠️ 🧠]
+  end
+
+  prompt do
+    rule "Headline <= 70 chars. Summary is the incoming tldr reprinted verbatim."
+    rule "Pick sentiment_icon from: 😐 neutral, 🙂 positive, ⚠️ negative, 🧠 analytical."
+    user "TL;DR: {tldr}\nTone: {tone}\nHashtags: {hashtags}"
+  end
+
+  validate("summary is the tldr verbatim") { |o, input| o[:summary] == input[:tldr] }
+end
+
+# Pipeline: summarize → hashtags → card
+class ArticleCardPipeline < RubyLLM::Contract::Pipeline::Base
+  step SummarizeArticle, as: :summarize
+  step GenerateHashtags, as: :tag
+  step BuildArticleCard, as: :card
+end
+```
+
+## Running and inspecting
+
+```ruby
+result = ArticleCardPipeline.run(article_text, context: { adapter: adapter })
+result.ok?                          # => true
+result.outputs_by_step[:summarize]  # => { tldr: "...", takeaways: [...], tone: "analytical" }
+result.outputs_by_step[:card]       # => { headline: "...", summary: "...", ... }
+result.trace.total_cost             # => 0.000128 (all steps combined)
+result.trace.total_latency_ms       # => 2340
+```
+
+## Fail-fast behavior
+
+When a step's schema, validate, or preflight check rejects the output, the pipeline stops there — downstream steps never run:
+
+```ruby
+# Summarize returns a TL;DR over 200 chars → the "TL;DR fits the card" validate fails
+adapter = RubyLLM::Contract::Adapters::Test.new(response: {
+  tldr: "x" * 500,
+  takeaways: %w[one two three],
+  tone: "neutral"
+})
+
+result = ArticleCardPipeline.run("article text", context: { adapter: adapter })
+result.failed?        # => true
+result.failed_step    # => :summarize (validate rejected; retries exhausted)
+# tag and card never run — no downstream tokens spent on garbage
+```
+
+## Per-step model override
+
+```ruby
+class ArticleCardPipeline < RubyLLM::Contract::Pipeline::Base
+  step SummarizeArticle, as: :summarize, model: "gpt-4.1-mini"
+  step GenerateHashtags, as: :tag,       model: "gpt-4.1-nano"
+  step BuildArticleCard, as: :card,      model: "gpt-4.1-nano"
+end
+```
+
+## Timeout
+
+```ruby
+result = ArticleCardPipeline.run(article_text, timeout_ms: 30_000)
+```
+
+## Pipeline eval
+
+```ruby
+ArticleCardPipeline.define_eval("e2e") do
+  add_case "ruby 3.4 release",
+    input: "Ruby 3.4 ships with frozen string literals by default and better YJIT...",
+    expected: { sentiment_icon: "🧠" }
+end
+
+report = ArticleCardPipeline.run_eval("e2e", context: { model: "gpt-4.1-mini" })
+report.print_summary
+```
+
+## Pretty print
+
+```ruby
+puts result
+# Pipeline: ok  3 steps  1234ms  450+120 tokens  trace=abc12345
+
+result.pretty_print
+# Full ASCII table with per-step outputs (Pipeline::Result)
+
+# For eval reports, use print_summary instead:
+report.print_summary
+# Tabular pass/fail breakdown (Eval::Report)
+```
+
+## See also
+
+- [Testing](testing.md) — `ArticleCardPipeline.test(..., responses: { summarize: ..., tag: ..., card: ... })` for pipeline-level spec adapters.
+- [Optimizing retry_policy](optimizing_retry_policy.md) — `optimize_retry_policy` runs per-step; pipelines benchmark one step at a time.

data/docs/guide/prompt_ast.md

@@ -0,0 +1,76 @@
+# Prompt AST
+
+> Read this when your prompt has more than one shape (per-tenant, per-language, per-audience) and string concatenation is starting to drift.
+
+Prompts are structured data, not strings. That matters the moment `SummarizeArticle` has to ship in more than one shape — a different audience per tenant, a different language per region, a different tone template for a B2B vs consumer card. Building those variants by string-concatenating a monolithic prompt leads to silent drift across environments. The AST gives you typed nodes (`system`, `rule`, `section`, `example`, `user`) that compose, diff, and snapshot-test cleanly.
+
+Available node types:
+
+```ruby
+prompt do
+  system  "You summarize articles for a UI card."     # system message
+  rule    "Return valid JSON only."                   # appended as separate system message
+  section "AUDIENCE", "Rails developers"              # labeled system message: [AUDIENCE]\n...
+  example input:  "Ruby 3.4 ships frozen strings...", # user/assistant few-shot pair
+          output: '{"tldr":"...","takeaways":[...],"tone":"analytical"}'
+  user    "{input}"                                   # user message with interpolation
+end
+```
+
+Or just a plain string (wraps as a single user message):
+
+```ruby
+prompt "Summarize this article for a UI card. {input}"
+```
+
+The AST is immutable, diffable, and hashable. Useful for snapshot testing and auditing prompt changes.
+
+## Hash inputs with variable interpolation
+
+When input is a Hash, each key becomes a template variable. Concrete scenario: a multi-language newsletter product where the same article has to be summarised in Polish for EU subscribers, English for US, with different audiences per tier (Rails developers vs engineering managers). Hash inputs let one step cover all of these without forking the class:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  input_type RubyLLM::Contract::Types::Hash.schema(
+    article:  RubyLLM::Contract::Types::String,
+    audience: RubyLLM::Contract::Types::String,
+    language: RubyLLM::Contract::Types::String
+  )
+
+  prompt do
+    system  "You summarize articles for a UI card."
+    rule    "Write the TL;DR and takeaways in {language}."
+    section "AUDIENCE", "{audience}"
+    user    "{article}"
+  end
+
+  output_schema do
+    string :tldr
+    array  :takeaways, of: :string, min_items: 3, max_items: 5
+    string :tone, enum: %w[neutral positive negative analytical]
+  end
+end
+```
+
+Every `{key}` in a prompt node is pulled from the input hash at run time. Missing keys raise — making wire-up bugs loud, not silent.
+
+> **Not the same as `RubyLLM::Agent.inputs`.** `Step.input_type` is a *runtime type check* on the positional argument passed to `run(input)` — it raises `TypeError` if the input violates the declared shape. `RubyLLM::Agent.inputs` is a list of *named template locals* injected into ERB instructions. They solve different problems (validation vs templating) and can coexist on the same project.
+
+> **Not the same as `RubyLLM::Agent` ERB templates either.** The `prompt do ... end` DSL above builds a multi-role message list (`system` / `user` / `assistant` / `example` nodes) into a node-AST. `Agent.instructions :name` loads a single-string ERB file from `app/prompts/<agent_path>/<name>.txt.erb` for the system prompt only. Different output shape, different scope. Variable interpolation here uses `{key}` substitution; ERB uses full `<%= ruby %>`.
+
+## Cross-validating output against input
+
+Validate blocks support 2-arity `|output, input|` so you can check that the model's answer stays faithful to the request:
+
+```ruby
+validate("tldr is not just the article reprinted") do |output, input|
+  # Guard against lazy models that return the input verbatim.
+  output[:tldr].length < input[:article].length / 2
+end
+
+validate("no takeaway repeats the TL;DR") do |output, _input|
+  output[:takeaways].none? { |t| t == output[:tldr] }
+end
+```
+
+The first example uses `input`; the second ignores it. Both are legal 2-arity signatures — Ruby accepts the unused `_input` parameter naming convention.

data/docs/guide/rails_integration.md

@@ -0,0 +1,218 @@
+# Rails integration
+
+> Read this when you've seen the `SummarizeArticle` example and want to know where contract steps fit in an actual Rails app — directory, initializer, jobs, logging, tests, CI. Skip if you're writing a non-Rails script.
+
+Seven pre-emptive answers to the questions that come up first.
+
+## 1. Where do step classes live?
+
+**Recommended: `app/contracts/`.** The gem's Railtie auto-reloads eval files under `app/contracts/eval/` and `app/steps/eval/` in development, so picking `app/contracts/` aligns with the default reload paths.
+
+```
+app/contracts/summarize_article.rb   # class SummarizeArticle
+```
+
+Any autoloaded directory works (`app/llm_steps/`, `app/services/llm/`, etc.) — Rails 7/8 autoloading resolves them all, and the step class itself does not depend on the path. Pick the default if you have no stronger convention.
+
+Keep evals in the same file as the step (`define_eval` block at the bottom of the class) — one source of truth per contract. If your evals grow too large for the class file, move them to `app/contracts/eval/summarize_article_eval.rb` — the Railtie reloads that directory explicitly in development.
+
+## 2. Initializer configuration
+
+```ruby
+# config/initializers/ruby_llm_contract.rb
+RubyLLM.configure do |c|
+  c.openai_api_key    = ENV.fetch("OPENAI_API_KEY", nil)
+  c.anthropic_api_key = ENV.fetch("ANTHROPIC_API_KEY", nil)
+end
+
+RubyLLM::Contract.configure do |c|
+  c.default_model   = Rails.env.production? ? "gpt-5-mini" : "gpt-5-nano"
+  c.default_adapter = RubyLLM::Contract::Adapters::RubyLLM.new
+end
+```
+
+In specs, override the default adapter to `Adapters::Test` in `spec_helper.rb` (or use `stub_step` per-example — see §5).
+
+Evals defined inside a step class (the recommended pattern) are picked up as soon as Rails autoloads the class — you do not need the eval file in any special directory. If you move evals into separate files under `app/contracts/eval/` or `app/steps/eval/`, the gem's Railtie reloads those two directories explicitly on each request in development; other directories follow standard Rails autoloading rules.
+
+## 3. Background jobs — never call LLMs inline in a controller
+
+LLM calls take 0.8–5 seconds and can fail. Wrap every step invocation in an ActiveJob:
+
+```ruby
+class SummarizeArticleJob < ApplicationJob
+  queue_as :llm
+
+  def perform(article_id)
+    article = Article.find(article_id)
+    result  = SummarizeArticle.run(article.body)
+
+    if result.ok?
+      # parsed_output uses symbol keys in memory. jsonb/json columns round-trip
+      # as strings on reload, so either use deep_stringify_keys before write or
+      # access downstream with string keys — pick one convention and stick to it.
+      article.update!(summary: result.parsed_output.deep_stringify_keys)
+    else
+      article.update!(summary_error: result.validation_errors.join("; "))
+    end
+  end
+end
+```
+
+`SummarizeArticleJob.perform_later(article.id)` returns in milliseconds; the controller stays responsive. If you use Sidekiq, pair `queue_as :llm` with a dedicated concurrency cap in `sidekiq.yml` so long-running LLM calls do not starve other job queues (mailers, webhooks, cleanups).
+
+## 4. Logging and observability
+
+`around_call` runs once per `run()` with the final `Result` (after all retries). Use it to write one row per LLM call:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  # ... prompt, schema, validates ...
+
+  around_call do |step, input, result|
+    AiCallLog.create!(
+      step: step.name,
+      model: result.trace[:model],
+      status: result.status.to_s,
+      latency_ms: result.trace[:latency_ms],
+      input_tokens: result.trace[:usage]&.dig(:input_tokens),
+      output_tokens: result.trace[:usage]&.dig(:output_tokens),
+      cost: result.trace[:cost],
+      validation_errors: result.validation_errors
+    )
+  end
+end
+```
+
+The `AiCallLog` model assumed above is a thin audit record. One possible migration:
+
+```ruby
+# rails g model AiCallLog step:string model:string status:string ...
+create_table :ai_call_logs do |t|
+  t.string  :step, null: false
+  t.string  :model
+  t.string  :status, null: false
+  t.integer :latency_ms
+  t.integer :input_tokens
+  t.integer :output_tokens
+  t.decimal :cost, precision: 10, scale: 6
+  t.jsonb   :validation_errors, default: []
+  t.timestamps
+end
+add_index :ai_call_logs, :step
+add_index :ai_call_logs, :status
+```
+
+For Appsignal / Honeybadger / Datadog, emit an `ActiveSupport::Notifications` event from inside the same `around_call` and subscribe in an initializer:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  # ... prompt, schema, validates ...
+
+  around_call do |step, _input, result|
+    ActiveSupport::Notifications.instrument(
+      "ruby_llm_contract.run",
+      step: step.name, model: result.trace[:model], status: result.status
+    )
+  end
+end
+
+# config/initializers/observability.rb
+ActiveSupport::Notifications.subscribe("ruby_llm_contract.run") do |*, payload|
+  Appsignal.increment_counter("llm.run.#{payload[:status]}", 1, step: payload[:step])
+end
+```
+
+Trace inspection in an admin UI: `result.trace[:attempts]` gives you per-attempt model, status, cost, latency — render it in a partial to debug production failures without re-running.
+
+## 5. Testing — RSpec and Minitest
+
+Add to `spec/spec_helper.rb` (or `test_helper.rb`):
+
+```ruby
+require "ruby_llm/contract/rspec"    # or ruby_llm/contract/minitest
+```
+
+Then in specs:
+
+```ruby
+RSpec.describe ArticlesController do
+  it "saves the summary when the step passes" do
+    stub_step(SummarizeArticle, response: {
+      tldr: "...", takeaways: %w[a b c], tone: "analytical"
+    })
+
+    post :summarize, params: { id: article.id }
+
+    # NOTE: jsonb/json column round-trips as string keys on reload.
+    expect(article.reload.summary["tldr"]).to eq("...")
+  end
+end
+```
+
+For the step itself, use the `satisfy_contract` and `pass_eval` matchers — details in the [Testing guide](testing.md).
+
+## 6. Error handling in controllers
+
+Never raise on `result.failed?` — that crashes the request. Branch instead:
+
+```ruby
+class ArticlesController < ApplicationController
+  def summarize
+    SummarizeArticleJob.perform_later(params[:id])
+    head :accepted
+  end
+
+  # For synchronous cases (admin tools, small content):
+  def preview
+    result = SummarizeArticle.run(@article.body)
+
+    if result.ok?
+      render json: result.parsed_output
+    else
+      Rails.logger.warn "[llm] #{SummarizeArticle.name} failed: #{result.status}"
+      render json: { error: "Could not summarize; try again shortly." }, status: :service_unavailable
+    end
+  end
+end
+```
+
+When `retry_policy` exhausts and all models fail, `result.failed?` is true but `result.parsed_output` still contains the last attempt's output — useful for logging what the model *did* return before the validate rejected it.
+
+## 7. CI gate — block regressions before merge
+
+Add to your `Rakefile`:
+
+```ruby
+require "ruby_llm/contract/rake_task"
+
+RubyLLM::Contract::RakeTask.new do |t|
+  t.minimum_score      = 0.8
+  t.maximum_cost       = 0.05
+  t.fail_on_regression = true
+  t.save_baseline      = false # read-only in CI; refresh baselines manually
+end
+```
+
+Then wire it in GitHub Actions:
+
+```yaml
+- name: LLM contract evals
+  env:
+    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+  run: bundle exec rake ruby_llm_contract:eval
+```
+
+The job fails when a previously-passing eval case now fails, when the average score drops below the threshold, or when total cost exceeds the cap. That is the signal that blocks a prompt regression or an accidental model upgrade from shipping.
+
+**Two practical notes:**
+
+- **Live evals spend real money on every run** — provider tokens per case × number of cases × every merge. Keep the dataset small and targeted (5–15 high-value cases), use cheap models where quality allows, and rely on offline `sample_response` smoke tests in the bulk of CI runs. Live evals belong on merge-candidate branches and scheduled nightly runs, not on every commit.
+- **Baselines are checkout-managed** — commit them to git under `.eval_baselines/`. Refresh them in a separate manual workflow (or locally + a dedicated PR) rather than from the merge gate, which would dirty the checkout and race with the regression check it is supposed to run.
+
+## See also
+
+- [Getting Started](getting_started.md) — the feature walkthrough the step above is built on
+- [Migration](migration.md) — before/after for replacing a raw `LlmClient.new.call` service with a contract
+- [Eval-First](eval_first.md) — the workflow behind the CI gate above
+- [Testing](testing.md) — `satisfy_contract` and `pass_eval` matcher chains

data/docs/guide/relation_to_agent.md

@@ -0,0 +1,52 @@
+# Relation to `RubyLLM::Agent`
+
+> Read this when you already use `RubyLLM::Agent` (or are about to) and want to understand where `ruby_llm-contract` sits in the same project.
+
+`RubyLLM::Agent` shipped in RubyLLM 1.12. `Step::Base` from this gem and `Agent` target the **same niche**: reusable, class-based prompts. They are **siblings**, not foundation-and-roof.
+
+## Feature mapping
+
+| What you write | Where it lives |
+|---|---|
+| `model`, `temperature`, `schema`, `instructions`, `tools`, `thinking` | covered by both — same idea, different DSL surface |
+| `validate :rule do ... end` business invariants on output | only in `ruby_llm-contract` |
+| `retry_policy escalate(...)` model escalation on validation failure | only here (different from RubyLLM's network-level retry) |
+| `max_cost` / `max_input` / `max_output` pre-flight refusal | only here |
+| `define_eval` + baseline regression + `compare_models` + `optimize_retry_policy` | only here (RubyLLM does not ship an evaluation framework) |
+| Pipeline composition with `step SomeStep, as: :alias` | only here (RubyLLM intentionally leaves workflows as plain Ruby) |
+| `around_call`, named `observe` hooks with pass/fail recorded in trace | only here |
+
+## Runtime relationship
+
+`Step::Base` does **not** use `Agent` internally today. The actual call path is:
+
+```
+Step.run(input)
+  → Runner
+  → Adapters::RubyLLM
+  → RubyLLM.chat(model:, ...)
+  → ... .ask(prompt)
+```
+
+`Agent` is a sibling abstraction calling into `RubyLLM::Chat` through its own `apply_configuration` path. Both end up at `Chat`. They do not share the macro-storage layer.
+
+This may change in a future release if upstream APIs make a layered design natural. The decision is not committed; it depends on adopter signal.
+
+## Coexistence on the same project
+
+The two abstractions can live in the same Rails (or non-Rails) project. Pick one per use case:
+
+- **`RubyLLM::Agent`** when you want a reusable prompt with `model` + `instructions` + `schema` + `tools` and that is enough — no retry-on-validation-failure, no business invariants, no eval framework, no budget gating.
+- **`ruby_llm-contract`'s `Step::Base`** when you need any of: invariants (`validate`), retry with model escalation on validation failure, pre-flight cost ceilings, an evaluation framework with baseline regression, or pipeline composition.
+
+A common pattern: simple ad-hoc prompts as `Agent`, contracts on the LLM features that touch production behaviour or money as `Step`.
+
+## On retry strategies
+
+The retry-strategy framing in this gem favours `retry_policy escalate(model_2, ...)` (model escalation, addresses model bias) over same-model `retry_policy attempts: N` (variance retry).
+
+This is grounded in empirical comparison across PDF quiz generation, GSM8K math (n=30 + n=120), and multi-constraint schedule generation: same-model retry produced no useful lift for nano-class models on tasks with clear correctness criteria. Model escalation did move the needle when same-model retry could not.
+
+`attempts: N` stays in the gem API (backward compat + niche cases like subjective-criteria tasks, multi-step pipelines, weaker open-source models) but is not marketed as a default retry strategy.
+
+See [Optimize retry policy](optimizing_retry_policy.md) for the empirical tooling.