ruby_llm-contract 0.7.1 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a165b7d8436d99e564fd5e774c601bae38562c0827c226995f1922e75f9987cf
-  data.tar.gz: 4124a3a13caba843448529a55eb1420601cbde70873f5924c792d9ed0c97161b
+  metadata.gz: 33f6a8a686f7f20791904c4fbfacd19f6ea5b8bad428c374ec14b7e33521354d
+  data.tar.gz: a2b2f7d9ff1e6cd69b39d55a3809b1babbd82a5d42afe3c567733076f03fa317
 SHA512:
-  metadata.gz: e5f17ef5241d9f7ddc047a16608b607d5fefedeff207cc6c3fb969c937192527c5b49c4213b3a3635db4e6aa68784f10f2363a5d99ed8528740b0d4ecebf3790
-  data.tar.gz: e58336f998f4df510b707534fd5fecdd87c8e4d04187f6ed9512e2b391dd359e3c9c4cfab106022a9a34ed8dfc27669156f5fc4214320ffe9dec3a0176aab181
+  metadata.gz: '0895986db9cde7d26d2e91ffc7c6469b7d34df299a361148c9ee339dbf1dc61539e44adf250c00c06383717ed2e47ff250d4490d1cce43c3cdf8c3169529fba5'
+  data.tar.gz: ed35a4b4cc9ab1afd46c427468dcb33844d8d54207531f98a9f1d775004efc5a1e19d64fb22c64b20da81750165a3a979f4de4ecaa46aff4121eb7ba80a27ed2

data/CHANGELOG.md

@@ -1,5 +1,101 @@
 # Changelog
 
+## 0.8.0 (2026-04-26)
+
+Narrative repositioning + small API additions. Internal architecture unchanged: no `Step::Base` refactor, no breaking changes to existing DSL.
+
+### Added
+
+- **`thinking(effort:, budget:)` class macro on `Step::Base`** — mirrors `RubyLLM::Agent.thinking` signature exactly. Stored as `{ effort:, budget: }` hash; reader returns the hash; supports `:default` reset semantics; superclass inheritance like `model`/`temperature`. The convenience alias `reasoning_effort(:low)` is implemented as `thinking(effort: :low)` — single normalized state, not separate ivar.
+- **Adapter wiring for `with_thinking`** — when `thinking` is set on the Step class, OR when `reasoning_effort:` is passed through context, OR when an attempt config in `retry_policy escalate(...)` carries `reasoning_effort:`, the RubyLLM adapter resolves the effective `{ effort:, budget: }` hash and forwards it via `chat.with_thinking(**)` — provider-agnostic (supports OpenAI `reasoning_effort` AND Anthropic extended-thinking budget). Precedence: per-attempt / context `reasoning_effort` overrides class-level `thinking[:effort]`; budget is taken from class-level `thinking[:budget]`. **Behavioural change vs 0.7.x**: `reasoning_effort` is now forwarded via `with_thinking` instead of `with_params`. Same wire-level OpenAI parameter; provider-agnostic Anthropic support is now automatic.
+
+### Dependencies
+
+- **`ruby_llm` constraint bumped from `~> 1.0` to `~> 1.12`** — `Chat#with_thinking` is the canonical path for reasoning effort + extended thinking; it shipped in RubyLLM 1.12. Adopters on `ruby_llm < 1.12` need to bump RubyLLM before upgrading this gem to 0.8.0.
+
+### Changed
+
+- **Tagline + README opening** — repositioned around "Contracts + Evals for RubyLLM". New "Relation to RubyLLM::Agent" section explicitly frames Step as a sibling abstraction (same niche as Agent, wider contract), not an alternative or foundation. README does not claim "Step uses Agent under the hood" — current call path is `Step → Runner → Adapters::RubyLLM → RubyLLM.chat` directly.
+- **`TokenEstimator` documented as heuristic** — module docstring expanded with explicit "±30% accuracy" framing. Refusal messages from `LimitChecker` now include `(heuristic ±30%)` suffix so adopters know the pre-flight number is estimated, not measured. RubyLLM 1.14 also has no pre-flight tokenizer; `RubyLLM::Tokens` is post-hoc only.
+- **`CostCalculator` repositioned in docs** — module narrative reframed from "cost calculator" to "fine-tune pricing registry + lookup with fallback chain". Math methods (`compute_cost`, `token_cost`, etc.) were already private; this release makes the docs match. Public API surface unchanged: `register_model`, `unregister_model`, `reset_custom_models!`, `calculate`.
+- **`output_schema` reframed in docs** — described as "wrapper around `RubyLLM::Schema` + client-side validation step", not a standalone feature. The schema language is identical to what `RubyLLM::Agent.schema` accepts; the difference is what wraps it.
+- **README retry framing** — `retry_policy escalate(...)` (model escalation on validation failure) is the marketed default. `retry_policy attempts: N` (same-model retry) stays in the API for backward compat and niche cases (subjective criteria, multi-step pipelines, weaker models) but is no longer marketed as a recommended default. Empirical basis: four small experiments across PDF quiz generation, GSM8K math (n=30 + n=120), and multi-constraint schedule generation found no useful lift for nano-class models on tasks with clear correctness criteria.
+
+### Documentation
+
+- **New disambiguation paragraphs** in `prompt_ast.md` (`Step.input_type` vs `RubyLLM::Agent.inputs`; `Prompt::Builder` multi-role DSL vs Agent ERB single-string template loader), `testing.md` (`Step.observe` vs `Chat#on_end_message` / `on_tool_call`), `output_schema.md` (relation to `Agent.schema`), and `optimizing_retry_policy.md` (orthogonal model + thinking dimensions).
+- **`getting_started.md` refusal message example** updated to include the new `(heuristic ±30%)` suffix.
+
+### Issues closed
+
+- **#11** (Optimizer is blind to same-model attempts) — closed after empirical experiments. `attempts: N` retry stays in API; not marketed as a default.
+- **#6** (Production cost reporting) — already implemented in 0.7.x; close confirmed.
+
+### Not in this release (deferred)
+
+- `output_schema` Proc form for runtime-input-aware schemas (parity with `Agent.schema` Proc form). Additive, low-risk; deferred to 0.9 to keep 0.8 scope tight.
+- H4 (Step composing `RubyLLM::Agent` internally as config holder) — verified feasible but ROI insufficient for current adopter base; trigger-based revisit, no calendar commitment.
+
+## 0.7.3 (2026-04-24)
+
+Adoption-friction release. No runtime behavior changes — every delta is in `docs/`, `examples/`, or `spec/integration/` (plus the `version.rb` / Gemfile.lock bumps). Upgrading from 0.7.2 picks up the expanded guide set, the new runnable showcases, and one extra integration spec.
+
+### Documentation
+
+- **New guide: `docs/guide/why.md`** — four production failure modes the gem exists for (schema-valid logically wrong, silent prompt regression, sampling variance on fixed-temperature models, runaway cost). Opens from a concrete incident each time; designed for readers who have not yet felt the pain the gem relieves.
+- **New guide: `docs/guide/rails_integration.md`** — seven Rails-specific FAQs with runnable snippets: where step classes live (`app/contracts/`), initializer setup, background jobs, `around_call` observability, RSpec/Minitest stubs, error handling in controllers, CI gate wiring.
+- **README adoption-friction pass** — added a short "Do I need this?" block after Install, a reading-order hint (`README → why.md → getting_started.md`), and outcome-based labels in the docs index ("Prevent silent prompt regressions" instead of "Eval-First", etc.).
+- **TL;DR box at the top of every guide** — single-sentence orientation for readers who land via search; "Skip if" clause added where real confusion exists (`eval_first.md`, `testing.md`, `migration.md`).
+- **API coverage gaps closed** — `estimate_cost` / `estimate_eval_cost`, `max_cost on_unknown_pricing: :warn`, `run_eval(..., concurrency:)`, `around_call` testing patterns now documented in `getting_started.md`, `eval_first.md`, `testing.md`.
+- **Industry-standard terminology** — `temperature-locked` → `fixed-temperature`, `variance-induced` → `sampling variance`, `severity signals` → `severity keywords`, `takeaway drift` → `tone/takeaways mismatch`.
+- **`docs/architecture.md` refresh** — diagram now reflects the current class layout: added `Step::RetryPolicy`, `Pipeline::Result`, `Eval::AggregatedReport`, `Eval::BaselineDiff`, `Eval::PromptDiffComparator`, `Eval::EvalHistory`, `Eval::RetryOptimizer`, `OptimizeRakeTask`. Replaced the outdated `Eval::TraitEvaluator` entry with `Eval::ExpectationEvaluator`.
+- **Business framing added to guides** — every guide opens with a concrete production scenario or "why it matters" hook before the API reference.
+
+### Examples — consolidated on `SummarizeArticle`, renumbered 00-06
+
+The previous 12-file set mixed a private Reddit promo planner, customer support, meetings, keyword extraction, and translation. The new set is seven runnable files, each answering one adopter question on the README's `SummarizeArticle` case.
+
+| # | File | Answers |
+|---|------|---------|
+| 00 | `00_basics.rb` | How do I start? (seven incremental layers + real-LLM pointer) |
+| 01 | `01_fallback_showcase.rb` | Show me the gem in 30 seconds (zero API keys) |
+| 02 | `02_real_llm_minimal.rb` | How do I plug in a real LLM? (~30 lines) |
+| 03 | `03_summarize_with_keywords.rb` | How does the contract evolve? (growing prompt) |
+| 04 | `04_summarize_and_translate.rb` | Pipeline composition + pipeline-level `run_eval` |
+| 05 | `05_eval_dataset.rb` | How do I stop silent prompt regressions? |
+| 06 | `06_retry_variants.rb` | `attempts: 3`, `reasoning_effort` escalation, cross-provider (Ollama → Anthropic → OpenAI) |
+
+Every file carries an "Expected output" block in its header so readers see the result without running the script. The `docs/ideas/` directory is now fully untracked (already in `.gitignore`; one stray file removed from version control).
+
+### Examples — bug fixes carried along
+
+- **Schema pitfall fixed in 5 files** — `array :x do; string :y; ...; end` silently produces `items: string` and drops every declaration after the first, matching the documented pitfall in `spec/ruby_llm/contract/nested_schema_spec.rb:71`. Every affected array block is now wrapped in `object do...end`.
+- **`examples/05_eval_dataset.rb` (pre-renumber: `09_eval_dataset.rb`) `result[:passed]` → `result.passed?`** — the previous code called `[]` on an `Eval::CaseResult` and raised `NoMethodError` at runtime.
+
+### Testing
+
+- **New `spec/integration/pipeline_eval_spec.rb`** — three cases guaranteeing pipeline-level `run_eval` stays functional: happy path, final-step mismatch, and fail-fast propagation when an intermediate `validate` rejects. Closes the "09 STEP 5 pipeline evaluation" known issue flagged in the 0.7.2 release. The fail-fast case asserts `step_status == :validation_failed` and the validate's label in `details`, so a regression that short-circuits on schema instead of validate would fail loudly.
+
+### Deleted (private-project cleanup)
+
+- `examples/01_classify_threads.rb`, `02_generate_comment.rb`, `03_target_audience.rb`, `10_reddit_full_showcase.rb`, `spec/integration/reddit_pipeline_spec.rb` — Reddit Promo Planner was a separate private project; its examples do not belong in the gem's public repo.
+- `examples/02_output_schema.rb` — fully covered by `docs/guide/output_schema.md`; deleting avoids duplication.
+
+## 0.7.2 (2026-04-22)
+
+### Changed
+
+- **Terminal output labels renamed for consistency with README narrative.** `print_summary` now prints `Hardest eval` (was `Constraining eval`), `Suggested fallback list` (was `Suggested chain`), and the production-mode table uses `first-attempt` / `fallback %` as column headers (was `single-shot` / `escalation`). Programmatic metric names unchanged: `single_shot_cost`, `single_shot_latency_ms`, `escalation_rate`. `RetryOptimizer::Result` exposes `hardest_eval` as an alias for `constraining_eval`.
+
+### Documentation
+
+- **`docs/guide/optimizing_retry_policy.md` rewritten** — 17.7k → 6.4k characters. Continues the `SummarizeArticle` narrative from README. Offline mode clearly positioned as wiring-check; real optimization runs via `LIVE=1 RUNS=3`. Output samples match actual `print_summary` format.
+- **`docs/guide/getting_started.md` rewritten** — 8.7k → 6.1k. Every example uses `SummarizeArticle`. Evals + CI gates section moved before Budget caps. Structured Prompts / Dynamic Prompts / "Already using ruby_llm?" / Reasoning effort sections removed; content delegated to `prompt_ast.md` and README.
+- **`docs/guide/eval_first.md` refined** — 6.3k → 5.0k. Switched to `SummarizeArticle` case. Team workflow section compressed with links back to `getting_started.md` for the matcher chain.
+- **`docs/guide/testing.md` refined** — 10.7k → 7.4k. Switched to `SummarizeArticle` case. Threshold gating / Rake task / baseline walkthrough / prompt A/B sections delegated back to `getting_started.md` and `eval_first.md`.
+- **`docs/guide/output_schema.md` DSL bug fix** — the Supported constraints table documented JSON Schema camelCase keys (`minLength`, `minItems`, `additionalProperties`) that are not valid DSL arguments. Every copy-paste from the previous table would have raised `ArgumentError`. Switched to snake_case (`min_length`, `min_items`, `additional_properties`) as the DSL actually expects; added a short note on the internal camelCase conversion.
+- **`docs/guide/best_practices.md`, `pipeline.md`, `migration.md` sanity pass** — terminology alignment (model escalation → model fallback where narrative; `escalate` DSL method unchanged) and `SummarizeArticle` case where the guide is not inherently multi-step.
+
 ## 0.7.1 (2026-04-22)
 
 ### Changed (behavioral, follow-up to v0.7.0)

data/Gemfile.lock

@@ -1,9 +1,9 @@
 PATH
   remote: .
   specs:
-    ruby_llm-contract (0.7.1)
+    ruby_llm-contract (0.8.0)
       dry-types (~> 1.7)
-      ruby_llm (~> 1.0)
+      ruby_llm (~> 1.12)
       ruby_llm-schema (~> 0.3)
 
 GEM
@@ -258,7 +258,7 @@ CHECKSUMS
   rubocop-ast (1.49.1) sha256=4412f3ee70f6fe4546cc489548e0f6fcf76cafcfa80fa03af67098ffed755035
   ruby-progressbar (1.13.0) sha256=80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33
   ruby_llm (1.14.0) sha256=57c6f7034fc4a44504ea137d70f853b07824f1c1cdbe774ab3ab3522e7098deb
-  ruby_llm-contract (0.7.1)
+  ruby_llm-contract (0.8.0)
   ruby_llm-schema (0.3.0) sha256=a591edc5ca1b7f0304f0e2261de61ba4b3bea17be09f5cf7558153adfda3dec6
   ruby_parser (3.22.0) sha256=1eb4937cd9eb220aa2d194e352a24dba90aef00751e24c8dfffdb14000f15d23
   rubycritic (4.12.0) sha256=024fed90fe656fa939f6ea80aab17569699ac3863d0b52fd72cb99892247abc8

data/README.md

@@ -1,121 +1,10 @@
 # ruby_llm-contract
 
-**Handle LLM output variance for [ruby_llm](https://github.com/crmne/ruby_llm).** Transport is a solved problem — ruby_llm already retries rate limits, timeouts, and server errors at the Faraday layer. What it can't do: retry when the model returns malformed JSON or a wrong answer, escalate to a smarter model when the cheap one fails the rules, measure variance on your dataset, and gate CI on regressions. That's what this gem adds.
+**Contracts + Evals for [ruby_llm](https://github.com/crmne/ruby_llm).**
 
-## Where the boundary sits
+Your eval passed. Prod broke anyway? This gem wraps `RubyLLM::Chat` with input/output contracts, business-rule validation, retry with model escalation on validation failure, pre-flight cost ceilings, and an evaluation framework — so a flaky cheap-model call escalates to a stronger model instead of shipping garbage to your user.
 
-| Concern | Handled by |
-|---|---|
-| Rate limits, timeouts, 5xx, connection errors | `ruby_llm` (Faraday retry middleware) |
-| Streaming, tool calls, embeddings, images, transcription | `ruby_llm` |
-| Chat history persistence (`acts_as_chat`) | `ruby_llm` |
-| **Malformed JSON / parse errors** | **`ruby_llm-contract`** |
-| **Business rule violations (invariants, schema)** | **`ruby_llm-contract`** |
-| **Retry with model escalation on bad output** | **`ruby_llm-contract`** |
-| **Measuring output variance on datasets** | **`ruby_llm-contract`** |
-| **Regression detection + CI gates** | **`ruby_llm-contract`** |
-
-Put together: `ruby_llm` owns the wire, this gem owns what the model *said*.
-
-```
-  YOU WRITE                       THE GEM HANDLES                 YOU GET
-  ─────────                       ───────────────                 ───────
-
-  validate { |o| ... }            catch bad answers — combined     Zero garbage
-                                  with retry_policy, auto-retry   in production
-
-  retry_policy                    start cheap, escalate only      Pay for the cheapest
-  models: %w[nano mini full]      when validation fails           model that works
-
-  max_cost 0.01                   estimate tokens, check price,   No surprise bills
-                                  refuse before calling LLM
-
-  output_schema { ... }           send JSON schema to provider,   Zero parsing code
-                                  validate response client-side
-
-  define_eval { ... }             test cases + baselines,          Regressions caught
-                                  run in CI with real LLM          before deploy
-
-  recommend(candidates: [...])    evaluate all configs, pick      Optimal model +
-                                  cheapest that passes            retry chain
-```
-
-## Before and after
-
-```
-  ┌─────────────────────────────────────────────────────────────────┐
-  │ BEFORE: pick one model, hope for the best                      │
-  │                                                                 │
-  │   expensive model → accurate, but you overpay on every call     │
-  │   cheap model     → fast, but wrong answers slip to production  │
-  │   prompt change   → "looks good to me" → deploy → users suffer │
-  └─────────────────────────────────────────────────────────────────┘
-
-                         ⬇  add ruby_llm-contract
-
-  ┌─────────────────────────────────────────────────────────────────┐
-  │ YOU DEFINE A CONTRACT                                            │
-  │                                                                 │
-  │   output_schema { string :priority }       ← valid structure   │
-  │   validate("valid priority") { |o| ... }   ← business rules    │
-  │   retry_policy models: %w[nano mini full]  ← escalation chain  │
-  │   max_cost 0.01                            ← budget cap         │
-  └───────────────────────────┬─────────────────────────────────────┘
-                              │
-                              ▼
-  ┌─────────────────────────────────────────────────────────────────┐
-  │ THE GEM HANDLES THE REST                                        │
-  │                                                                 │
-  │   request ──→ ┌──────┐   ┌──────────┐                           │
-  │               │ nano │─→ │ contract │──→ ✓ pass → done         │
-  │               └──────┘   └────┬─────┘                           │
-  │                               │ ✗ fail                          │
-  │                               ▼                                 │
-  │               ┌──────┐   ┌──────────┐                           │
-  │               │ mini │─→ │ contract │──→ ✓ pass → done         │
-  │               └──────┘   └────┬─────┘                           │
-  │                               │ ✗ fail                          │
-  │                               ▼                                 │
-  │               ┌──────┐   ┌──────────┐                           │
-  │               │ full │─→ │ contract │──→ ✓ pass → done         │
-  │               └──────┘   └──────────┘                           │
-  └───────────────────────────┬─────────────────────────────────────┘
-                              │
-                              ▼
-  ┌─────────────────────────────────────────────────────────────────┐
-  │ YOU GET                                                         │
-  │                                                                 │
-  │   ✓ Valid output guaranteed — schema + business rules enforced  │
-  │   ✓ Cheapest model that works — most requests stay on nano     │
-  │   ✓ Cost, latency, tokens — tracked on every call              │
-  │   ✓ Eval scores per model — data instead of gut feeling        │
-  │   ✓ Regressions caught — before deploy, not after              │
-  │   ✓ Recommendation — "use nano+mini, drop full, save $X/mo"   │
-  └─────────────────────────────────────────────────────────────────┘
-```
-
-## 30-second version
-
-```ruby
-class ClassifyTicket < RubyLLM::Contract::Step::Base
-  prompt "Classify this support ticket by priority and category.\n\n{input}"
-
-  output_schema do
-    string :priority, enum: %w[low medium high urgent]
-    string :category
-  end
-
-  validate("urgent needs justification") { |o, input| o[:priority] != "urgent" || input.length > 20 }
-  retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1]
-end
-
-result = ClassifyTicket.run("I was charged twice")
-result.parsed_output  # => {priority: "high", category: "billing"}
-result.trace[:model]  # => "gpt-4.1-nano" (first model that passed)
-result.trace[:cost]   # => 0.000032
-```
-
-Bad JSON? Retried automatically. Wrong answer? Escalated to a smarter model. Schema violated? Caught client-side. The contract guarantees every response meets your rules — you pay for the cheapest model that passes.
+`ruby_llm` handles the HTTP side (rate limits, timeouts, streaming, tool calls, embeddings). This gem handles what the model *returned*: schema validation, business rules, model escalation on failed validation, datasets, regression tests.
 
 ## Install
 
@@ -128,239 +17,98 @@ RubyLLM.configure { |c| c.openai_api_key = ENV["OPENAI_API_KEY"] }
 RubyLLM::Contract.configure { |c| c.default_model = "gpt-4.1-mini" }
 ```
 
-Works with any ruby_llm provider (OpenAI, Anthropic, Gemini, etc).
-
-## Handle output variance with model escalation
-
-Models are non-deterministic. A prompt that works on 95% of inputs can break on the edge case sitting in your production traffic right now. The defensive response is to pick the strongest model and pay for it on every call. The measured response is to define a contract and let the gem escalate only when the cheaper model's output actually fails the rules:
-
-```ruby
-retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1]
-```
+Works with any `ruby_llm` provider (OpenAI, Anthropic, Gemini, etc).
 
-```
-Attempt 1: gpt-4.1-nano  → contract failed  ($0.0001)
-Attempt 2: gpt-4.1-mini  → contract passed  ($0.0004)
-           gpt-4.1       → never called      ($0.00)
-```
+## Do I need this?
 
-Most requests succeed on the cheapest model. The expensive ones fire only when output variance demands it. The cost win is a consequence of measuring variance correctly — not the primary goal. Want to know how often each tier triggers? Run `compare_models` against your dataset.
+Use this if LLM output affects production behaviour, money, user trust, or downstream code. You probably don't need it if you have one low-risk prompt, manually inspect every result, or only generate best-effort prose.
 
-Default retry statuses (since 0.7.0) are `:validation_failed` and `:parse_error` — the two flavors of LLM output variance. Transport errors (rate limits, timeouts, 5xx) are retried by ruby_llm at the HTTP layer and intentionally not duplicated here. If you want `:adapter_error` in retry too, opt in explicitly — it's meaningful paired with an escalation chain.
+Already using structured outputs from your provider? This gem adds business-rule validation, retry with model fallback, evals, regression gating, and test stubs on top of them — the layer that stops schema-valid-but-wrong output from reaching users. See [Why contracts?](docs/guide/why.md) for the four production failure modes the gem exists for, or run `ruby examples/01_fallback_showcase.rb` to see the fallback loop in 30 seconds (no API key needed).
 
-## Soft delivery: retry for variance, ship the last attempt
+## Example
 
-Sometimes validation is a **soft quality check** — "options balanced", "style consistent", "tone friendly" — and a partial output is better than none. The same model generating the same prompt produces different samples run-to-run (OpenAI forces `temperature=1.0` on gpt-5/o-series), so a single unlucky draw shouldn't fail the user. Use `attempts:` to retry on the SAME model — no escalation — and get the last attempt back even if it still failed the contract:
+A Rails app takes article text extracted from a user-submitted URL and wants to show a summary card: a short TL;DR, 3–5 key takeaways, and a tone label. The output has to fit the UI (TL;DR under 200 chars) and the schema has to be strict enough to render without conditionals.
 
 ```ruby
-class GenerateQuiz < RubyLLM::Contract::Step::Base
-  output_schema do
-    # ... 15 questions × 4 options ...
-  end
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  prompt <<~PROMPT
+    Summarize this article for a UI card. Return a short TL;DR,
+    3 to 5 key takeaways, and a tone label.
 
-  validate("answer options balanced") do |out, _|
-    out[:questions].all? do |q|
-      lens = q[:answer_options].map(&:length)
-      next false if lens.empty? || lens.min.zero?
+    {input}
+  PROMPT
 
-      lens.min >= 15 && (lens.max.to_f / lens.min) <= 1.7
-    end
+  output_schema do
+    string :tldr
+    array  :takeaways, of: :string, min_items: 3, max_items: 5
+    string :tone, enum: %w[neutral positive negative analytical]
   end
 
-  retry_policy attempts: 3
-end
-
-result = GenerateQuiz.run(document)
-if result.ok?
-  publish(result.parsed_output)
-else
-  # Three unlucky draws in a row — ship the last one anyway, log the miss.
-  Rails.logger.warn "Quiz delivered with soft-validation miss: #{result.validation_errors.join('; ')}"
-  publish(result.parsed_output)
-end
-```
-
-How this differs from the escalation chain:
-
-- `retry_policy models: %w[nano mini full]` — **document hardness.** Retry means "the cheap model isn't enough, use a smarter one."
-- `retry_policy attempts: 3` — **sampling variance.** Retry means "same model, different random seed — the model can do better on a second try."
-
-After all `attempts` are exhausted (`attempts: 3` means 3 total tries, not 3 retries on top of the first), the Result carries `status: :validation_failed` plus the last attempt's `parsed_output`. The caller decides: ship anyway, fall back to a template, or surface an error. The gem does not raise on exhaustion — your application policy, your choice.
-
-Combine both when helpful:
-
-```ruby
-retry_policy do
-  escalate({ model: "gpt-4.1-mini" }, { model: "gpt-4.1-mini" }, { model: "gpt-4.1" })
-end
-```
+  validate("TL;DR fits the card")  { |o, _| o[:tldr].length <= 200 }
+  validate("takeaways are unique") { |o, _| o[:takeaways].uniq.size == o[:takeaways].size }
 
-Two tries on mini (variance retry) before paying for full-fat gpt-4.1.
-
-## Know which model to use — with data
-
-Don't guess. Define test cases, compare models, get numbers:
-
-```ruby
-ClassifyTicket.define_eval("regression") do
-  add_case "billing", input: "I was charged twice", expected: { priority: "high" }
-  add_case "feature", input: "Add dark mode please", expected: { priority: "low" }
-  add_case "outage",  input: "Database is down",    expected: { priority: "urgent" }
+  retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1]
 end
 
-comparison = ClassifyTicket.compare_models("regression",
-  models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1])
-```
-
-```
-Candidate                  Score       Cost  Avg Latency
----------------------------------------------------------
-gpt-4.1-nano                0.67    $0.0001         48ms
-gpt-4.1-mini                1.00    $0.0004         92ms
-gpt-4.1                     1.00    $0.0021        210ms
-
-Cheapest at 100%: gpt-4.1-mini
-```
-
-Nano fails on edge cases. Mini and full both score 100% — but mini is **5x cheaper**. Now you know.
-
-Running live against gpt-5 / o-series? Pass `runs: 3` to average out sampling variance (OpenAI forces `temperature=1.0` server-side, so one unlucky run can misclassify a viable candidate). See [Reducing variance with `runs:`](docs/guide/optimizing_retry_policy.md#reducing-variance-with-runs).
-
-Want the *effective* cost — first-attempt plus retries — rather than the single-shot headline number? Pass `production_mode: { fallback: "gpt-5-mini" }` and the table gains `escalation`, `effective cost`, and a `Chain` column. See [Production-mode cost measurement](docs/guide/optimizing_retry_policy.md#production-mode-cost-measurement).
-
-## Let the gem tell you what to do
-
-Don't read tables — get a recommendation. Supports `model + reasoning_effort` combinations:
-
-```ruby
-rec = ClassifyTicket.recommend("regression",
-  candidates: [
-    { model: "gpt-4.1-nano" },
-    { model: "gpt-4.1-mini" },
-    { model: "gpt-5-mini", reasoning_effort: "low" },
-    { model: "gpt-5-mini", reasoning_effort: "high" },
-  ],
-  min_score: 0.95
-)
-
-rec.best           # => { model: "gpt-4.1-mini" }
-rec.retry_chain    # => [{ model: "gpt-4.1-nano" }, { model: "gpt-4.1-mini" }]
-rec.to_dsl         # => "retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini]"
-rec.savings        # => savings vs your current model (if configured)
-```
-
-Copy `rec.to_dsl` into your step. Done.
-
-## Catch regressions before users do
-
-A model update silently dropped your accuracy? A prompt tweak broke an edge case? You'll know before deploying:
-
-```ruby
-# Save a baseline once:
-report = ClassifyTicket.run_eval("regression", context: { model: "gpt-4.1-nano" })
-report.save_baseline!(model: "gpt-4.1-nano")
-
-# In CI — block merge if anything regressed:
-expect(ClassifyTicket).to pass_eval("regression")
-  .with_context(model: "gpt-4.1-nano")
-  .without_regressions
-```
-
-```ruby
-diff = report.compare_with_baseline(model: "gpt-4.1-nano")
-diff.regressed?    # => true
-diff.regressions   # => [{case: "outage", baseline: {passed: true}, current: {passed: false}}]
-diff.score_delta   # => -0.33
-```
-
-No more "it worked in the playground". Regressions are caught in CI, not production.
-
-## A/B test your prompts
-
-Changed a prompt? Compare old vs new on the same dataset with regression safety:
-
-```ruby
-diff = ClassifyTicketV2.compare_with(ClassifyTicketV1,
-  eval: "regression", model: "gpt-4.1-mini")
-
-diff.safe_to_switch?  # => true (no regressions)
-diff.improvements     # => [{case: "outage", ...}]
-diff.score_delta      # => +0.33
-```
-
-```ruby
-# CI gate:
-expect(ClassifyTicketV2).to pass_eval("regression")
-  .compared_with(ClassifyTicketV1)
-  .with_minimum_score(0.8)
+result = SummarizeArticle.run(article_text)
+result.parsed_output    # => { tldr: "...", takeaways: [...], tone: "analytical" }
+result.trace[:model]    # => "gpt-4.1-nano"  (first model that passed)
+result.trace[:cost]     # => 0.000032
 ```
 
-## Chain steps with fail-fast
+The model returns JSON matching the schema. If the response is malformed, the TL;DR overflows the card, or the takeaway count is off, the gem retries — moving to the next model in `models:` only when the cheaper one can't satisfy the rules. In this setup cheaper models are tried first and the expensive ones are used only when cheaper models fail.
 
-Pipeline stops at the first contract failure — no wasted tokens on downstream steps:
+You could write this loop yourself once. The gem gives you the loop, a trace of every attempt (model, status, cost, latency), fallback policy, evals, baselines, and CI checks as one contract object — tracked per-step so adding a new LLM feature to your app is one class, not one-off scaffolding.
 
-```ruby
-class TicketPipeline < RubyLLM::Contract::Pipeline::Base
-  step ClassifyTicket,  as: :classify
-  step RouteToTeam,     as: :route
-  step DraftResponse,   as: :draft
-end
+## Most useful next
 
-result = TicketPipeline.run("I was charged twice")
-result.outputs_by_step[:classify]   # => {priority: "high", category: "billing"}
-result.trace.total_cost             # => $0.000128
-```
+Everything below is optional — the example above is a complete step. Reach for these when one step isn't enough.
 
-## Gate merges on quality and cost
+- **[CI regression gates](docs/guide/getting_started.md)** — `define_eval` + `save_baseline!` + `pass_eval(...).without_regressions` blocks CI when accuracy drops on a model update or prompt tweak.
+- **[Find the cheapest viable fallback list](docs/guide/optimizing_retry_policy.md)** — `Step.recommend("regression", candidates: [...], min_score: 0.95)` returns the cheapest list of models that still passes your evals. `production_mode:` measures retry-aware cost.
+- **[A/B test prompts](docs/guide/eval_first.md)** — `SummarizeArticleV2.compare_with(SummarizeArticleV1, eval: "regression")` reports whether the new prompt is safe to ship.
+- **[Budget caps](docs/guide/getting_started.md)** — `max_cost`, `max_input`, `max_output` refuse the request before calling the API when a heuristic estimate (~±30% accuracy) exceeds the limit.
+- **[Reasoning effort / thinking config](docs/guide/optimizing_retry_policy.md)** — `thinking effort: :low` (or alias `reasoning_effort :low`) on the Step class; mirrors `RubyLLM::Agent.thinking` and forwards through `Chat#with_thinking`.
 
-```ruby
-# RSpec — block merge if accuracy drops or cost spikes
-expect(ClassifyTicket).to pass_eval("regression")
-  .with_minimum_score(0.8)
-  .with_maximum_cost(0.01)
-
-# Rake — run all evals across all steps
-RubyLLM::Contract::RakeTask.new do |t|
-  t.minimum_score = 0.8
-  t.maximum_cost = 0.05
-end
-# bundle exec rake ruby_llm_contract:eval
-```
+Also supports [multi-step pipelines](docs/guide/pipeline.md) with fail-fast and `retry_policy attempts: N` for niche cases (we measured this empirically — for `gpt-4.1-nano` / `gpt-5-nano` on tasks with clear correctness criteria, same-model retry rarely helps; `escalate(model_2)` is the strategy that moves the needle, see [optimizing_retry_policy.md](docs/guide/optimizing_retry_policy.md)).
 
-## Full power: data-driven retry chains
+## Relation to `RubyLLM::Agent`
 
-The pieces above — evals, compare_models, recommend — combine into a workflow that replaces guesswork with measured optimization. You define evals for your step, run `recommend` against all of them, find the eval that actually needs the strongest model, and build a retry chain where each attempt is as cheap as the data allows.
+`RubyLLM::Agent` (since RubyLLM 1.12) and `Step::Base` here target the **same niche**: reusable, class-based prompts. They are siblings, not foundation-and-roof.
 
-The difference: instead of "gpt-5-mini seems to work, let's use it everywhere", you get "nano handles 4/6 scenarios, mini@low catches the 5th, full mini only fires on the hardest edge case — first attempt is 4× cheaper."
+| What you write | Where it lives |
+|---|---|
+| `model`, `temperature`, `schema`, `instructions`, `tools`, `thinking` | covered by both — same idea, different DSL surface |
+| `validate :rule do |out| ... end` business invariants | only here |
+| `retry_policy escalate(...)` model escalation on validation failure | only here (different from RubyLLM's network-level retry) |
+| `max_cost` / `max_input` / `max_output` pre-flight refusal | only here |
+| `define_eval` + baseline regression + `compare_models` + `optimize_retry_policy` | only here (RubyLLM does not ship an eval framework) |
+| Pipeline composition with `step SomeStep, as: :alias` | only here (RubyLLM intentionally leaves workflows as plain Ruby) |
+| `around_call`, named `observe` hooks with pass/fail in trace | only here |
 
-Full procedure with examples: **[Optimizing retry_policy](docs/guide/optimizing_retry_policy.md)**
+`Step::Base` does NOT use `Agent` internally today — both call into `RubyLLM::Chat` directly. The two abstractions can coexist on the same project: use `Agent` for prompt-only reuse, use `Step` when you need any of the contract-layer features above. The retry-strategy framing here (favouring `escalate(...)` over same-model `attempts: N`) is grounded in an empirical comparison; `attempts: N` stays in the API for niche cases.
 
 ## Docs
 
-| Guide | |
-|-------|-|
-| [Getting Started](docs/guide/getting_started.md) | Features walkthrough, model escalation, eval |
-| [Eval-First](docs/guide/eval_first.md) | Practical workflow for prompt engineering with datasets, baselines, and A/B gates |
-| [Optimizing retry_policy](docs/guide/optimizing_retry_policy.md) | Find the cheapest retry chain that passes all your evals |
-| [Best Practices](docs/guide/best_practices.md) | 6 patterns for bulletproof validates |
-| [Output Schema](docs/guide/output_schema.md) | Full schema reference + constraints |
-| [Pipeline](docs/guide/pipeline.md) | Multi-step composition, timeout, fail-fast |
-| [Testing](docs/guide/testing.md) | Test adapter, RSpec matchers |
-| [Migration](docs/guide/migration.md) | Adopting the gem in existing Rails apps |
+**New here?** Read in order: this README → [Why contracts?](docs/guide/why.md) → [Getting Started](docs/guide/getting_started.md).
+
+| Guide | What it does for your app |
+|-------|---------------------------|
+| [Why contracts?](docs/guide/why.md) | Recognise the four production failures the gem exists for |
+| [Getting Started](docs/guide/getting_started.md) | Walk the full feature set on one concrete step |
+| [Rails integration](docs/guide/rails_integration.md) | Directory, initializer, jobs, logging, specs, CI gate — 7 FAQs for Rails devs |
+| [Adopt in an existing Rails app](docs/guide/migration.md) | Replace raw `LlmClient.call` with a contract, Before/After |
+| [Prevent silent prompt regressions](docs/guide/eval_first.md) | Evals, baselines, CI gates that block quality drift |
+| [Control retry cost and fallback behaviour](docs/guide/optimizing_retry_policy.md) | Find the cheapest viable fallback list empirically |
+| [Write validate rules that catch real bugs](docs/guide/best_practices.md) | Patterns for cross-input checks and content-quality rules |
+| [Stub LLM calls in tests](docs/guide/testing.md) | Deterministic specs, RSpec + Minitest matchers |
+| [Chain LLM calls into a pipeline](docs/guide/pipeline.md) | Multi-step with fail-fast and per-step models |
+| [Schema DSL reference](docs/guide/output_schema.md) | Every constraint, nested objects, pattern table |
+| [Prompt DSL reference](docs/guide/prompt_ast.md) | `system` / `rule` / `section` / `example` / `user` nodes |
 
 ## Roadmap
 
-**v0.7.1 (current):** Follow-up — `Step::Base#run_once` no longer masks adapter-phase `ArgumentError` as `:input_error`. Programmer bugs in adapter code now propagate; DSL misconfiguration still becomes `:input_error` via narrower rescue.
-
-**v0.7.0:** Sharpened retry semantics. `DEFAULT_RETRY_ON` now targets LLM output variance only (`:validation_failed`, `:parse_error`); transport errors are delegated to ruby_llm's Faraday retry. `AdapterCaller` narrowed to let programmer errors propagate instead of masking them as retries. Breaking change — see [CHANGELOG](CHANGELOG.md) for migration.
-
-**v0.6:** "What should I do?" — `Step.recommend` returns optimal model, reasoning effort, and retry chain. Per-attempt `reasoning_effort` in retry policies.
-
-**v0.5:** Prompt A/B testing with `compare_with`. Soft observations with `observe`.
-
-**v0.4:** Eval history, batch concurrency, pipeline per-step eval, Minitest, structured logging.
-
-**v0.3:** Baseline regression detection, migration guide.
+Latest: **v0.8.0** — tagline + narrative repositioning around "Contracts + Evals for RubyLLM", `thinking` / `reasoning_effort` class macro, TokenEstimator labelled as heuristic, CostCalculator repositioned. See [CHANGELOG](CHANGELOG.md) for history.
 
 ## License