ruby_llm-contract 0.8.0 → 0.10.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 33f6a8a686f7f20791904c4fbfacd19f6ea5b8bad428c374ec14b7e33521354d
-  data.tar.gz: a2b2f7d9ff1e6cd69b39d55a3809b1babbd82a5d42afe3c567733076f03fa317
+  metadata.gz: ac8c19463285a3e2c0f9050e835125454b21d2b1500dcb5bad2dd4a114666bd0
+  data.tar.gz: 339c0609e3dccbf55da649c3470ed234c5abafa55c8649aee5d8dd41f9bd82ec
 SHA512:
-  metadata.gz: '0895986db9cde7d26d2e91ffc7c6469b7d34df299a361148c9ee339dbf1dc61539e44adf250c00c06383717ed2e47ff250d4490d1cce43c3cdf8c3169529fba5'
-  data.tar.gz: ed35a4b4cc9ab1afd46c427468dcb33844d8d54207531f98a9f1d775004efc5a1e19d64fb22c64b20da81750165a3a979f4de4ecaa46aff4121eb7ba80a27ed2
+  metadata.gz: c04fea66393868fbdb369f5a75eaeb66a28172c8f9ddabd0a8c053fd55e5daa06174aeeb00c4bb7e9aabaf613bd1ef4ea354ff7d0f4b817f2ec41af1a4c61667
+  data.tar.gz: 71f502ebe497ede22df508d3b56396660fbd1eba109066885d5e905724bd82d30a0d46fddfe4f1c1e7efe47f95ea41b4f61341a4698c12d3f3a397cab2510068

data/CHANGELOG.md

@@ -1,5 +1,83 @@
 # Changelog
 
+## 0.10.2 (2026-06-10)
+
+Patch release: ship the `docs/guide/` directory inside the gem so adopters and LLM integration agents can read the manuals locally (via `bundle show ruby_llm-contract` or `gem unpack`) without an internet round-trip to GitHub. No code behavior change.
+
+### Added
+
+- **`docs/guide/*` is now packaged with the gem** (14 files, ~120 KB). Previously the README's "See also" links pointed at `docs/guide/getting_started.md`, `docs/guide/optimizing_retry_policy.md`, etc., but those files were stripped from the published gem - LLM integration agents (Cursor, Claude Code, Copilot) reported "no documentation in the gem" because the links 404'd locally. `docs/ideas/` and `doc/decisions/` remain excluded.
+
+### Fixed
+
+- **`models:` keyword form documented + pinned for hash configs.** `retry_policy models: ["gpt-5-nano", { model: "gpt-5-mini", reasoning_effort: "high" }]` is now covered by a spec and shown in [getting_started.md](docs/guide/getting_started.md). The block form (`escalate(...)`) and the keyword form share the same `@configs` storage; both forms accept config hashes.
+- **`reasoning_effort` examples corrected to gpt-5 family.** Pre-0.10.2 docs and specs paired `reasoning_effort` with `gpt-4.1-*` model names, which is incorrect: `gpt-4.1` is not a reasoning model. Updated across `docs/guide/getting_started.md`, `docs/guide/optimizing_retry_policy.md`, `CHANGELOG.md`, and 7 spec files. Non-reasoning `gpt-4.1` examples (model fallback chains without `reasoning_effort`) are unchanged.
+- **Version mentions corrected in README and multimodal guide.** README FAQ ("Upgraded to 0.9.0 - why?") now reads "Upgraded to 0.10.0 from 0.8.x"; `docs/guide/multimodal_input.md` references `0.10.0+` and `0.10.x` instead of `0.9.0`. 0.9.0 and 0.9.1 were tagged but never published to rubygems; adopters jump from 0.8.0 directly to 0.10.x.
+
+## 0.10.1 (2026-06-01)
+
+Patch release fixing gem packaging. 0.10.0 was yanked from rubygems.org due to the issue documented below; 0.10.1 is the recommended upgrade target. No code behavior change vs 0.10.0.
+
+### Fixed
+
+- **Gem no longer ships internal tracker / dev configs.** Excluded from `spec.files`: `TODO.md`, `.rspec`, `.rubycritic.yml`, `.simplecov`, and the `.revive/` directory. Pre-0.10.1 the published gem contained these files; adopters who already extracted 0.10.0 can safely delete them.
+
+## 0.10.0 (2026-06-01)
+
+First published release since 0.8.0. Consolidates work originally tagged as 0.9.0 (multimodal input) and 0.9.1 (internal quality refactor), neither of which was pushed to rubygems. Adopters upgrading from 0.8.0 should read the **Behavioural change** and **Breaking changes** sections below before installing.
+
+### Breaking changes
+
+- **`validate(description, &block)` and `Definition#invariant(description, &block)` now raise `ArgumentError` when `description` is `nil` or empty.** Pre-0.10.0 the empty descriptor was silently accepted and produced `""` entries in `result.validation_errors`, making debugging impossible. Codex audit found zero production use sites across `lib/`, `examples/`, `README` — only the regression-marker test certifying the bug.
+
+### Migration
+
+Ensure every `validate` / `invariant` call has a non-empty descriptor (this is already how every README example writes them):
+
+```ruby
+# Before (silently accepted, produced "" in validation_errors):
+validate("") { |o| o[:score].between?(0, 100) }
+
+# After (required):
+validate("score in range 0-100") { |o| o[:score].between?(0, 100) }
+```
+
+### Added
+
+- **Multimodal input via `context: { attachment: ... }`** — pass a file/IO/URL through `Step.run(input, context: { attachment: path })`; the adapter forwards it to `RubyLLM::Chat#ask(content, with: attachment)`. RubyLLM normalises wire format per provider (Anthropic url/base64, OpenAI `image_url`/`file`, Gemini `inline_data`). Multi-attachment supported natively (`with: [pdf1, pdf2]` or `with: { images: [...], pdfs: [...] }`). See [multimodal input guide](docs/guide/multimodal_input.md) and [ADR-0022](doc/decisions/ADR-0022-v09-multimodal-input.md).
+- **`attachment_token_estimate(n)` class macro** — adopter-declared conservative estimate of attachment input tokens. Applied to BOTH runtime (`limit_checker`) and pre-flight (`estimate_cost`) — same source of truth, no estimate/runtime drift.
+- **`on_unknown_attachment_size(:refuse | :warn)` class macro** — mirrors `on_unknown_pricing` opt-out semantics. Defaults to `:refuse`. Never settable as global default — same invariant as `max_cost` fail-closed.
+
+### Behavioural change (READ BEFORE UPGRADING)
+
+- **Contracts with `max_cost` or `max_input` AND `context[:attachment]` set AND no `attachment_token_estimate` declared → REFUSE with `:limit_exceeded`.** This is fail-closed semantics: the gem cannot bound vision/PDF token cost without an adopter-declared estimate. Opt out per-step with `on_unknown_attachment_size :warn`. Text-only contracts and contracts without `max_cost`/`max_input` are unaffected.
+
+### Changed
+
+- **`run_eval` (no args) return shape pinned to `Hash<String, Report>` keyed by eval name.** Documents the existing contract used by `RubyLLM::Contract::RakeTask#collect_host_reports` and adopters. No runtime change vs 0.8.0 — only the spec assertion now locks the shape.
+- **`Parser.parse(text, strategy: :json)` first-bracket-wins boundary documented.** Extraction commits to the first balanced `{` or `[` structure and does NOT retry on later candidates. Empty `{}` followed by real JSON parses as the empty Hash; non-JSON `{braces}` before real JSON raises `ParseError`. No runtime change — this codifies long-standing behavior with explicit boundary tests.
+
+### Fixed
+
+- **`with_retry_disabled` no longer mutates the step class's singleton method.** The optimizer now passes `retry_policy_override: nil` through `context:` to `compare_models`, which `Step::Base#runtime_settings` already honours. Removes a concurrency hazard where two parallel `optimize_retry_policy` calls on the same step would race on the singleton restore in `ensure`.
+- **`CostCalculator.find_model` exposed as a public class method.** Removes two `CostCalculator.send(:find_model, ...)` workarounds in `Step::Base#estimate_cost`. The `estimated_cost_for` helper is gone — `estimate_cost` now routes through the existing public `CostCalculator.calculate(model_name:, usage:)`.
+- **`stub_step` unified on a single storage path.** Both block and non-block forms now write to `RubyLLM::Contract.step_adapter_overrides` (thread-local). The `around(:each)` hook in `rspec.rb` handles cleanup between examples. Removes the prior `allow(step).to receive(:run)` branch.
+
+### Internal
+
+- **Anti-facade audit complete: 89/89 spec files under per-test 17-mode walk** (Phase A: 26 specs, Phase C: 63 specs via parallel Codex fan-out). Net +30 strengthened tests against mutation-blind assertions, zero public API change beyond the breaking entry above.
+- **Dead `ObjectSpace.each_object(Class)` fallback removed** in `concerns/eval_host.rb#register_subclasses`. The gemspec requires Ruby `>= 3.2.0`, so `Class#subclasses` (Ruby ≥ 3.1) is always available; the legacy fallback was unreachable code that would have iterated all loaded classes O(n) and was not thread-safe.
+
+### Deferred (not in 0.10.x)
+
+- `add_history` multi-turn replay of prior attachments — single-turn multimodal supported; follow-up questions on the same document deferred to a later release.
+- Streaming + attachment — contract steps remain synchronous.
+- Provider-specific attachment size caps — surface only via `attachment_token_estimate` calibration; consult provider docs.
+
+### Tests
+
+- Suite: 1401 examples / 0 failures / 7 pending (was 1346/0/8 at 0.8.0).
+
 ## 0.8.0 (2026-04-26)
 
 Narrative repositioning + small API additions. Internal architecture unchanged: no `Step::Base` refactor, no breaking changes to existing DSL.
@@ -190,7 +268,7 @@ end
 - **`Step.recommend`** — `ClassifyTicket.recommend("eval", candidates: [...], min_score: 0.95)` runs eval on all candidates and returns a `Recommendation` with optimal model, retry chain, rationale, savings vs current config, and `to_dsl` code output.
 - **Candidates as configurations** — `candidates:` accepts `{ model:, reasoning_effort: }` hashes, not just model name strings. `gpt-5-mini` with `reasoning_effort: "low"` is a different candidate than with `"high"`.
 - **`compare_models` extended** — new `candidates:` parameter alongside existing `models:` (backward compatible). Candidate labels include reasoning effort in output table.
-- **Per-attempt `reasoning_effort` in retry policies** — `escalate` accepts config hashes: `escalate({ model: "gpt-4.1-nano" }, { model: "gpt-5-mini", reasoning_effort: "high" })`. Each attempt gets its own reasoning_effort forwarded to the provider.
+- **Per-attempt `reasoning_effort` in retry policies** — `escalate` accepts config hashes: `escalate({ model: "gpt-5-nano" }, { model: "gpt-5-mini", reasoning_effort: "high" })`. Each attempt gets its own reasoning_effort forwarded to the provider.
 - **`pass_rate_ratio`** — numeric float (0.0–1.0) on `Report` and `ReportStats`, complementing the string `pass_rate` (`"3/5"`).
 - **History entries enriched** — `save_history!` accepts `reasoning_effort:` and stores `model`, `reasoning_effort`, `pass_rate_ratio` in JSONL entries.
 

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    ruby_llm-contract (0.8.0)
+    ruby_llm-contract (0.10.2)
       dry-types (~> 1.7)
       ruby_llm (~> 1.12)
       ruby_llm-schema (~> 0.3)
@@ -258,7 +258,7 @@ CHECKSUMS
   rubocop-ast (1.49.1) sha256=4412f3ee70f6fe4546cc489548e0f6fcf76cafcfa80fa03af67098ffed755035
   ruby-progressbar (1.13.0) sha256=80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33
   ruby_llm (1.14.0) sha256=57c6f7034fc4a44504ea137d70f853b07824f1c1cdbe774ab3ab3522e7098deb
-  ruby_llm-contract (0.8.0)
+  ruby_llm-contract (0.10.2)
   ruby_llm-schema (0.3.0) sha256=a591edc5ca1b7f0304f0e2261de61ba4b3bea17be09f5cf7558153adfda3dec6
   ruby_parser (3.22.0) sha256=1eb4937cd9eb220aa2d194e352a24dba90aef00751e24c8dfffdb14000f15d23
   rubycritic (4.12.0) sha256=024fed90fe656fa939f6ea80aab17569699ac3863d0b52fd72cb99892247abc8

data/README.md

@@ -2,9 +2,9 @@
 
 **Contracts + Evals for [ruby_llm](https://github.com/crmne/ruby_llm).**
 
-Your eval passed. Prod broke anyway? This gem wraps `RubyLLM::Chat` with input/output contracts, business-rule validation, retry with model escalation on validation failure, pre-flight cost ceilings, and an evaluation framework — so a flaky cheap-model call escalates to a stronger model instead of shipping garbage to your user.
+Your eval passed. Prod broke anyway? This gem wraps `RubyLLM::Chat` with input/output contracts, business-rule validation, retry with model escalation on validation failure, pre-flight cost ceilings, and a regression-eval framework — so a flaky cheap-model call escalates to a stronger model instead of shipping garbage to your user.
 
-`ruby_llm` handles the HTTP side (rate limits, timeouts, streaming, tool calls, embeddings). This gem handles what the model *returned*: schema validation, business rules, model escalation on failed validation, datasets, regression tests.
+`ruby_llm` handles the HTTP side (rate limits, timeouts, streaming, tool calls, embeddings). This gem handles what the model *returned* at **runtime**: schema validation, business rules, model escalation on failed validation, regression datasets that gate prompt/model changes in CI.
 
 ## Install
 
@@ -13,23 +13,24 @@ gem "ruby_llm-contract"
 ```
 
 ```ruby
-RubyLLM.configure { |c| c.openai_api_key = ENV["OPENAI_API_KEY"] }
-RubyLLM::Contract.configure { |c| c.default_model = "gpt-4.1-mini" }
-```
-
-Works with any `ruby_llm` provider (OpenAI, Anthropic, Gemini, etc).
-
-## Do I need this?
+RubyLLM.configure do |c|
+  c.openai_api_key = ENV["OPENAI_API_KEY"]
+  c.default_model  = "gpt-4.1-mini"   # used when a Step has no explicit model
+end
 
-Use this if LLM output affects production behaviour, money, user trust, or downstream code. You probably don't need it if you have one low-risk prompt, manually inspect every result, or only generate best-effort prose.
+# Required: boots the gem so `Step.run` knows how to talk to your LLM.
+# Empty block is fine. Pass options here if you need them (e.g. `c.logger`).
+RubyLLM::Contract.configure { }
+```
 
-Already using structured outputs from your provider? This gem adds business-rule validation, retry with model fallback, evals, regression gating, and test stubs on top of them — the layer that stops schema-valid-but-wrong output from reaching users. See [Why contracts?](docs/guide/why.md) for the four production failure modes the gem exists for, or run `ruby examples/01_fallback_showcase.rb` to see the fallback loop in 30 seconds (no API key needed).
+Works with any `ruby_llm` provider (OpenAI, Anthropic, Gemini, etc). Requires `ruby_llm ~> 1.12` and Ruby ≥ 3.2.
 
 ## Example
 
 A Rails app takes article text extracted from a user-submitted URL and wants to show a summary card: a short TL;DR, 3–5 key takeaways, and a tone label. The output has to fit the UI (TL;DR under 200 chars) and the schema has to be strict enough to render without conditionals.
 
 ```ruby
+# app/contracts/summarize_article.rb
 class SummarizeArticle < RubyLLM::Contract::Step::Base
   prompt <<~PROMPT
     Summarize this article for a UI card. Return a short TL;DR,
@@ -45,48 +46,93 @@ class SummarizeArticle < RubyLLM::Contract::Step::Base
   end
 
   validate("TL;DR fits the card")  { |o, _| o[:tldr].length <= 200 }
-  validate("takeaways are unique") { |o, _| o[:takeaways].uniq.size == o[:takeaways].size }
+  validate("takeaways are unique") { |o, _| o[:takeaways] == o[:takeaways].uniq }
 
-  retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1]
+  # Cheapest first; last step adds a reasoning model with more thinking.
+  retry_policy do
+    escalate "gpt-4.1-nano",
+             "gpt-4.1-mini",
+             { model: "gpt-5", reasoning_effort: "high" }
+  end
 end
 
 result = SummarizeArticle.run(article_text)
-result.parsed_output    # => { tldr: "...", takeaways: [...], tone: "analytical" }
-result.trace[:model]    # => "gpt-4.1-nano"  (first model that passed)
-result.trace[:cost]     # => 0.000032
+result.status           # => :ok  (or :validation_failed if all steps fail)
+result.parsed_output    # => { tldr: "...", takeaways: [...], tone: "..." }
+result.trace[:model]    # => "gpt-4.1-mini"  (winning step)
+result.trace[:cost]     # => 0.000520        (total across all attempts)
+
+result.trace[:attempts]
+# => [
+#      {
+#        attempt: 1,
+#        model: "gpt-4.1-nano",
+#        status: :validation_failed,
+#        usage: { input_tokens: 256, output_tokens: 84 },
+#        latency_ms: 45,
+#        cost: 0.000100
+#      },
+#      {
+#        attempt: 2,
+#        model: "gpt-4.1-mini",
+#        status: :ok,
+#        usage: { input_tokens: 256, output_tokens: 92 },
+#        latency_ms: 92,
+#        cost: 0.000420
+#      }
+#    ]
+```
+
+If the response is malformed, the TL;DR overflows the card, or the takeaway count is off, the gem moves to the next step. This is model **escalation**, not a fallback list — each step is an independent config (`model`, `reasoning_effort`), so the retry policy spends more compute only when the cheaper one couldn't satisfy the contract.
+
+### Add a CI gate in 6 lines
+
+The contract above already runs in production. The same `Step` doubles as the unit your regression eval runs against:
+
+```ruby
+SummarizeArticle.define_eval("regression") do
+  # `expected:` is a partial hash match — only listed keys check parsed_output.
+  add_case "neutral release",
+           input: "Ruby 3.4 shipped frozen string literals...",
+           expected: { tone: "analytical" }
+  add_case "outage post",
+           input: "Service was down for 4 hours...",
+           expected: { tone: "negative" }
+end
+
+# in CI (RSpec):
+expect(SummarizeArticle).to pass_eval("regression").without_regressions
 ```
 
-The model returns JSON matching the schema. If the response is malformed, the TL;DR overflows the card, or the takeaway count is off, the gem retries — moving to the next model in `models:` only when the cheaper one can't satisfy the rules. In this setup cheaper models are tried first and the expensive ones are used only when cheaper models fail.
+A bad prompt edit or model swap that drops accuracy on the frozen dataset → red CI, blocked merge. The first CI run records a baseline; subsequent runs compare against it. Every production miss should become the next `add_case`. See [Prevent silent prompt regressions](docs/guide/eval_first.md) for the full flywheel.
+
+## Do I need this?
+
+Use this if LLM output affects production behaviour, money, user trust, or downstream code. You probably don't need it if you have one low-risk prompt, manually inspect every result, or only generate best-effort prose.
 
-You could write this loop yourself once. The gem gives you the loop, a trace of every attempt (model, status, cost, latency), fallback policy, evals, baselines, and CI checks as one contract object — tracked per-step so adding a new LLM feature to your app is one class, not one-off scaffolding.
+Already using structured outputs from your provider? This gem adds business-rule validation, retry with model escalation, evals, regression gating, and test stubs on top of them — the layer that stops schema-valid-but-wrong output from reaching users. See [Why contracts?](docs/guide/why.md) for the four production failure modes the gem exists for.
 
 ## Most useful next
 
 Everything below is optional — the example above is a complete step. Reach for these when one step isn't enough.
 
-- **[CI regression gates](docs/guide/getting_started.md)** — `define_eval` + `save_baseline!` + `pass_eval(...).without_regressions` blocks CI when accuracy drops on a model update or prompt tweak.
-- **[Find the cheapest viable fallback list](docs/guide/optimizing_retry_policy.md)** — `Step.recommend("regression", candidates: [...], min_score: 0.95)` returns the cheapest list of models that still passes your evals. `production_mode:` measures retry-aware cost.
-- **[A/B test prompts](docs/guide/eval_first.md)** — `SummarizeArticleV2.compare_with(SummarizeArticleV1, eval: "regression")` reports whether the new prompt is safe to ship.
-- **[Budget caps](docs/guide/getting_started.md)** — `max_cost`, `max_input`, `max_output` refuse the request before calling the API when a heuristic estimate (~±30% accuracy) exceeds the limit.
-- **[Reasoning effort / thinking config](docs/guide/optimizing_retry_policy.md)** — `thinking effort: :low` (or alias `reasoning_effort :low`) on the Step class; mirrors `RubyLLM::Agent.thinking` and forwards through `Chat#with_thinking`.
+- **[CI regression gates](docs/guide/getting_started.md)** — block CI when accuracy drops on a model update or prompt tweak.
+- **[Find the cheapest viable fallback list](docs/guide/optimizing_retry_policy.md)** — empirically pick the cheapest model chain that still passes your evals.
+- **[A/B test prompts](docs/guide/eval_first.md)** — measure whether a new prompt is safe to ship before merging.
+- **[Budget caps](docs/guide/getting_started.md)** — refuse the request pre-flight when an estimate exceeds the limit.
+- **[Reasoning effort / thinking config](docs/guide/optimizing_retry_policy.md)** — Anthropic / OpenAI thinking configuration on the Step class.
 
-Also supports [multi-step pipelines](docs/guide/pipeline.md) with fail-fast and `retry_policy attempts: N` for niche cases (we measured this empirically — for `gpt-4.1-nano` / `gpt-5-nano` on tasks with clear correctness criteria, same-model retry rarely helps; `escalate(model_2)` is the strategy that moves the needle, see [optimizing_retry_policy.md](docs/guide/optimizing_retry_policy.md)).
+Also supports [multi-step pipelines](docs/guide/pipeline.md) with fail-fast and per-step models.
 
 ## Relation to `RubyLLM::Agent`
 
-`RubyLLM::Agent` (since RubyLLM 1.12) and `Step::Base` here target the **same niche**: reusable, class-based prompts. They are siblings, not foundation-and-roof.
+`Step::Base` and `RubyLLM::Agent` (since RubyLLM 1.12) are **siblings** targeting the same niche: reusable, class-based prompts. Both call into `RubyLLM::Chat` directly — Step does not wrap Agent. Step adds the contract layer: `validate` (business invariants), `retry_policy escalate(...)` (model escalation on validation failure), `max_cost` pre-flight refusal, regression-eval framework, pipeline composition. **[Full feature mapping →](docs/guide/relation_to_agent.md)**
 
-| What you write | Where it lives |
-|---|---|
-| `model`, `temperature`, `schema`, `instructions`, `tools`, `thinking` | covered by both — same idea, different DSL surface |
-| `validate :rule do |out| ... end` business invariants | only here |
-| `retry_policy escalate(...)` model escalation on validation failure | only here (different from RubyLLM's network-level retry) |
-| `max_cost` / `max_input` / `max_output` pre-flight refusal | only here |
-| `define_eval` + baseline regression + `compare_models` + `optimize_retry_policy` | only here (RubyLLM does not ship an eval framework) |
-| Pipeline composition with `step SomeStep, as: :alias` | only here (RubyLLM intentionally leaves workflows as plain Ruby) |
-| `around_call`, named `observe` hooks with pass/fail in trace | only here |
+## Relation to `ruby_llm-tribunal`
 
-`Step::Base` does NOT use `Agent` internally today — both call into `RubyLLM::Chat` directly. The two abstractions can coexist on the same project: use `Agent` for prompt-only reuse, use `Step` when you need any of the contract-layer features above. The retry-strategy framing here (favouring `escalate(...)` over same-model `attempts: N`) is grounded in an empirical comparison; `attempts: N` stays in the API for niche cases.
+Different layers, complementary. [`ruby_llm-tribunal`](https://github.com/Alqemist-labs/ruby_llm-tribunal) is a **test framework** that grades outputs **after they've reached your code**, typically in a spec. `ruby_llm-contract` is **runtime** — schema + `validate` rules gate the call **before the output reaches your code**, retry/escalate attempts to recover from failed outputs, `max_cost` refuses pre-flight. Our `define_eval` is *regression* (does this prompt/model still pass on a frozen dataset?), not *grading*.
+
+**One-liner:** Tribunal answers *"is this output good?"* (fail → red test in CI). Contract answers *"what do we do when it isn't?"* (fail → retry/escalate, or fail closed). **[Visual flows + coexistence patterns →](docs/guide/relation_to_tribunal.md)**
 
 ## Docs
 
@@ -95,6 +141,8 @@ Also supports [multi-step pipelines](docs/guide/pipeline.md) with fail-fast and
 | Guide | What it does for your app |
 |-------|---------------------------|
 | [Why contracts?](docs/guide/why.md) | Recognise the four production failures the gem exists for |
+| [Relation to RubyLLM::Agent](docs/guide/relation_to_agent.md) | Sibling abstractions; what each adds; runtime call path; coexistence patterns |
+| [Relation to ruby_llm-tribunal](docs/guide/relation_to_tribunal.md) | Different layers (test framework vs runtime contract); visual flows; integration recipes |
 | [Getting Started](docs/guide/getting_started.md) | Walk the full feature set on one concrete step |
 | [Rails integration](docs/guide/rails_integration.md) | Directory, initializer, jobs, logging, specs, CI gate — 7 FAQs for Rails devs |
 | [Adopt in an existing Rails app](docs/guide/migration.md) | Replace raw `LlmClient.call` with a contract, Before/After |
@@ -103,12 +151,23 @@ Also supports [multi-step pipelines](docs/guide/pipeline.md) with fail-fast and
 | [Write validate rules that catch real bugs](docs/guide/best_practices.md) | Patterns for cross-input checks and content-quality rules |
 | [Stub LLM calls in tests](docs/guide/testing.md) | Deterministic specs, RSpec + Minitest matchers |
 | [Chain LLM calls into a pipeline](docs/guide/pipeline.md) | Multi-step with fail-fast and per-step models |
+| [Multimodal input (PDF / image / audio)](docs/guide/multimodal_input.md) | Route attachments through the contract; `attachment_token_estimate`, fail-closed cost, calibration table |
 | [Schema DSL reference](docs/guide/output_schema.md) | Every constraint, nested objects, pattern table |
 | [Prompt DSL reference](docs/guide/prompt_ast.md) | `system` / `rule` / `section` / `example` / `user` nodes |
 
-## Roadmap
+## Status & versioning
+
+Pre-1.0 (currently **0.10.2**). Semver tracked; breaking changes flagged in [CHANGELOG](CHANGELOG.md). Pin `~> 0.10.2` until 1.0 ships.
+
+## FAQ
+
+**Thread-safe / Sidekiq?** Yes. Each `Step.run` builds an isolated `RubyLLM::Chat`; class-level state (`output_schema`, `validate`, `retry_policy`) is set up once at class load and read-only afterwards. Safe to run from concurrent jobs/threads.
+
+**How do I stub `Step.run` in specs?** Include `RubyLLM::Contract::RSpec::Helpers` and use `stub_step(MyStep, response: { ... })`. The block form scopes the stub to one `it`. See [testing guide](docs/guide/testing.md).
+
+**Where in a Rails app?** Default `app/contracts/`. The Railtie reloads `app/contracts/eval/` and `app/steps/eval/` in development; any autoloaded directory also works. See [Rails integration](docs/guide/rails_integration.md).
 
-Latest: **v0.8.0** — tagline + narrative repositioning around "Contracts + Evals for RubyLLM", `thinking` / `reasoning_effort` class macro, TokenEstimator labelled as heuristic, CostCalculator repositioned. See [CHANGELOG](CHANGELOG.md) for history.
+**Upgraded to 0.10.0 from 0.8.x and my contract started refusing — why?** 0.10.0 added multimodal input. If your contract has `max_cost` or `max_input` set AND now receives `context: { attachment: ... }`, you must declare `attachment_token_estimate(n)` (conservative input-token budget for the attachment) — otherwise the call fails closed with `:limit_exceeded`. The gem cannot bound vision/PDF cost without your estimate. Opt out per-step with `on_unknown_attachment_size :warn`. Text-only contracts are unaffected. See [multimodal input guide](docs/guide/multimodal_input.md).
 
 ## License
 

data/docs/architecture.md

@@ -0,0 +1,50 @@
+# Architecture
+
+```
+RubyLLM::Contract::Pipeline::Base   # optional: compose steps
+  ├── Pipeline::Runner           # sequential execution, fail-fast, trace, timeout
+  └── Pipeline::Result           # per-step outputs + aggregated trace
+
+RubyLLM::Contract::Step::Base       # single contracted step
+  ├── Step::Dsl                  # DSL macros (prompt, validate, output_schema, etc.)
+  ├── Step::RetryPolicy          # attempts, models, reasoning_effort, retry_on
+  ├── Step::RetryExecutor        # retry loop driven by RetryPolicy
+  ├── Step::LimitChecker         # preflight cost / input / output checks
+  ├── Step::Runner               # runtime flow (single attempt)
+  ├── Step::Result               # status + outputs + errors + trace
+  ├── Step::Trace                # model, latency, tokens, cost, attempts
+  ├── Prompt::AST                # structured prompt (immutable)
+  │     ├── Prompt::Builder      # DSL: system, rule, example, user, section
+  │     └── Prompt::Renderer     # AST → messages array
+  ├── Contract::Definition       # parse strategy + validates
+  │     ├── Contract::Parser     # :json / :text (auto-inferred from output type)
+  │     ├── Contract::Validator  # runs parse + schema + validates + observations
+  │     └── Contract::SchemaValidator  # JSON Schema validation (nested)
+  ├── CostCalculator             # per-step cost estimation from model pricing
+  ├── TokenEstimator             # input-token count estimation for limit checks
+  ├── estimate_cost              # single-call cost estimate (class method)
+  ├── estimate_eval_cost         # cost estimate for a full eval across models
+  └── Adapters::Base             # provider interface
+        ├── Adapters::RubyLLM    # real LLM calls via ruby_llm (any provider)
+        └── Adapters::Test       # canned responses for specs and examples
+
+RubyLLM::Contract::Eval             # quality measurement
+  ├── Eval::EvalDefinition       # define_eval DSL (verify, add_case, default_input, sample_response)
+  ├── Eval::Dataset              # test cases
+  ├── Eval::Runner               # sequential or concurrent execution
+  ├── Eval::Report               # score, pass_rate, per-case results
+  ├── Eval::AggregatedReport     # merged reports across models or runs
+  ├── Eval::CaseResult           # value object (name, passed?, output, expected, mismatches, cost)
+  ├── Eval::ExpectationEvaluator # expected / expected_traits / evaluator proc
+  ├── Eval::ModelComparison      # compare_models result (table, best_for, candidate configs)
+  ├── Eval::Recommender          # model recommendation algorithm (candidates → optimal config)
+  ├── Eval::Recommendation       # recommendation result (best, retry_chain, savings, to_dsl)
+  ├── Eval::RetryOptimizer       # optimize_retry_policy result (per-eval breakdown, fallback list)
+  ├── Eval::BaselineDiff         # save_baseline! + without_regressions comparison
+  ├── Eval::PromptDiffComparator # compare_with prompt A/B diff
+  └── Eval::EvalHistory          # time-series view across saved reports
+
+RubyLLM::Contract::RakeTask         # rake ruby_llm_contract:eval
+RubyLLM::Contract::OptimizeRakeTask # rake ruby_llm_contract:optimize
+RubyLLM::Contract::Railtie          # auto-loads eval files in Rails
+```

data/docs/guide/best_practices.md

@@ -0,0 +1,136 @@
+# Best Practices
+
+> Read this when writing your first `validate` rules and you want patterns that catch real bugs instead of restating schema.
+
+Schema guarantees valid JSON structure. An LLM can still return structurally perfect JSON that is **semantically wrong**. Schema handles _shape_, validates handle _meaning_.
+
+All examples extend the `SummarizeArticle` step from the [README](../../README.md).
+
+## 1. Guard against empty / placeholder output
+
+**Why it matters:** a cheap model that answers `{"tldr": "This article discusses...", "takeaways": ["This article discusses X", ...]}` passes the schema but renders a broken UI card that tells the user nothing. The validate catches it before `Article.update!` persists it.
+
+```ruby
+output_schema do
+  string :tldr
+  array  :takeaways, of: :string, min_items: 3, max_items: 5
+  string :tone, enum: %w[neutral positive negative analytical]
+end
+
+validate("tldr is substantive") do |o, _|
+  o[:tldr].to_s.split.length >= 5   # at least five words
+end
+
+validate("no boilerplate takeaways") do |o, _|
+  o[:takeaways].none? { |t| t.downcase.start_with?("this article") }
+end
+```
+
+## 2. Cross-validate output against input
+
+**Why it matters:** a lazy model will return the article text verbatim as the "summary", or invent takeaways about topics the article never mentions. The 2-arity form is how you catch answers that are internally consistent but unfaithful to the actual input.
+
+`validate` blocks with 2-arity `|output, input|` compare the model's answer against what was asked:
+
+```ruby
+validate("tldr is shorter than the article") do |output, input|
+  output[:tldr].length < input.length / 2
+end
+
+validate("every takeaway appears, in spirit, in the article") do |output, input|
+  output[:takeaways].all? { |t|
+    # cheap keyword overlap heuristic
+    t.downcase.split.any? { |w| input.downcase.include?(w) && w.length > 4 }
+  }
+end
+```
+
+## 3. Conditional logic schema can't express
+
+**Why it matters:** customer success filters on `tone == "negative"` to route angry users to a human. If the model labels an outage complaint "negative" but the takeaways are all positive-sounding, the filter runs on a label that doesn't match the content — the routing breaks silently.
+
+```ruby
+validate("negative tone requires at least one concrete concern") do |output, _input|
+  next true unless output[:tone] == "negative"
+  output[:takeaways].any? { |t| t.match?(/fail|break|crash|outage|vulnerab|risk/i) }
+end
+```
+
+A model that picks `tone: "negative"` but gives three upbeat takeaways fails this check. Schema can't catch it because each takeaway is, individually, a valid string.
+
+## 4. Content quality
+
+**Why it matters:** a TL;DR with `## Summary` leaks markdown into a plain-text card. A one-word takeaway ("Fast.") wastes a UI slot. A leaked `{article}` placeholder reveals the prompt template to end users. All pass schema; all embarrass you in front of customers.
+
+```ruby
+validate("no markdown headings in the TL;DR") do |o, _|
+  !o[:tldr].match?(/^\#{1,6}\s/)
+end
+
+validate("takeaways aren't single words") do |o, _|
+  o[:takeaways].all? { |t| t.split.length >= 3 }
+end
+
+validate("no template placeholders leaked") do |o, _|
+  !(o[:tldr] + o[:takeaways].join(" ")).include?("{")
+end
+```
+
+## 5. Pipeline: preserve data between steps
+
+In a pipeline, each step only sees the previous step's output. If a later step needs original article metadata, an intermediate step must carry it through. Suppose a pipeline `SummarizeArticle → GenerateHashtags`, where `GenerateHashtags` needs the `tone` from the summary:
+
+```ruby
+class GenerateHashtags < RubyLLM::Contract::Step::Base
+  input_type Hash
+
+  output_schema do
+    # Carry through the fields a downstream step (or the caller) might need
+    string :tone, enum: %w[neutral positive negative analytical]
+    array  :hashtags, of: :string, min_items: 2, max_items: 5
+  end
+
+  prompt do
+    rule "Preserve the tone label from the input unchanged."
+    user "Summary: {tldr}\nTone: {tone}\nTakeaways: {takeaways}"
+  end
+
+  validate("tone preserved") { |o, input| o[:tone] == input[:tone] }
+end
+```
+
+The explicit `validate("tone preserved")` catches the case where the model silently rewrites the tone during a downstream transform.
+
+## 6. Model fallback
+
+**Why it matters:** 80% of production articles are short and simple — `gpt-4.1-nano` handles them for ~$0.0001. The remaining 20% are dense, critical, or multi-topic — those need `gpt-4.1-mini` or `gpt-4.1`. Paying `gpt-4.1` rates for every call when nano is enough for most is throwing money away. Contracts tell you when nano wasn't enough, so fallback is cost-aware, not hope-based.
+
+Small models are cheap but hallucinate. Big models are accurate but expensive. Start cheap, fall back only when validates catch a failure:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  output_schema do
+    string :tldr
+    array  :takeaways, of: :string, min_items: 3, max_items: 5
+    string :tone, enum: %w[neutral positive negative analytical]
+  end
+
+  validate("TL;DR fits the card")  { |o, _| o[:tldr].length <= 200 }
+  validate("takeaways are unique") { |o, _| o[:takeaways] == o[:takeaways].uniq }
+
+  retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1]
+end
+```
+
+**Key insight:** without contracts, you can't do model fallback — you'd have no way to know if the cheap model's output is good enough. Validates are the quality gate that makes cost optimization possible. See [Optimizing retry_policy](optimizing_retry_policy.md) for how to find the cheapest viable fallback list for your step.
+
+## Summary
+
+| What to validate | Use |
+|---|---|
+| Field types, enums, ranges, required fields | `output_schema` |
+| Output makes sense given the input | `validate` (2-arity `\|output, input\|`) |
+| Conditional business rules | `validate` |
+| Content quality (not empty, not template) | `validate` |
+| Data preserved across pipeline steps | `validate` + schema carry-through |
+| Cost optimization via model fallback | `retry_policy` + `validate` as quality gate |