ruby_llm-contract 0.10.1 → 0.10.2

data/docs/guide/pipeline.md

@@ -0,0 +1,154 @@
+# Pipeline
+
+> Read this when one step isn't enough — you need multi-step with fail-fast, automatic data threading, and per-step models.
+
+Chain multiple steps with automatic data threading, fail-fast, per-step models, trace, and timeout.
+
+A pipeline needs more than one step to be interesting. This guide grows the `SummarizeArticle` step from the [README](../../README.md) into a three-step content pipeline that tags and routes the summary to a UI card.
+
+## Full example: article → summary → hashtags → card
+
+```ruby
+# Step 1 — the flagship step from README, unchanged.
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  prompt <<~PROMPT
+    Summarize this article for a UI card. Return a short TL;DR,
+    3 to 5 key takeaways, and a tone label.
+
+    {input}
+  PROMPT
+
+  output_schema do
+    string :tldr
+    array  :takeaways, of: :string, min_items: 3, max_items: 5
+    string :tone, enum: %w[neutral positive negative analytical]
+  end
+
+  validate("TL;DR fits the card") { |o, _| o[:tldr].length <= 200 }
+end
+
+# Step 2 — reads SummarizeArticle's output, produces hashtags suitable for social posts.
+class GenerateHashtags < RubyLLM::Contract::Step::Base
+  input_type Hash
+
+  output_schema do
+    # Carry through the summary fields downstream consumers (and the next step) need.
+    string :tldr
+    array  :takeaways, of: :string
+    string :tone, enum: %w[neutral positive negative analytical]
+    # Add new field.
+    array  :hashtags, of: :string, min_items: 2, max_items: 5
+  end
+
+  prompt do
+    rule "Preserve tldr / takeaways / tone exactly as given."
+    user "Article summary: {tldr}\nTone: {tone}\nGenerate 2 to 5 concise hashtags."
+  end
+
+  validate("tone preserved") { |o, input| o[:tone] == input[:tone] }
+end
+
+# Step 3 — final shape the UI card consumes.
+class BuildArticleCard < RubyLLM::Contract::Step::Base
+  input_type Hash
+
+  output_schema do
+    string :headline
+    string :summary
+    array  :hashtags, of: :string
+    string :sentiment_icon, enum: %w[😐 🙂 ⚠️ 🧠]
+  end
+
+  prompt do
+    rule "Headline <= 70 chars. Summary is the incoming tldr reprinted verbatim."
+    rule "Pick sentiment_icon from: 😐 neutral, 🙂 positive, ⚠️ negative, 🧠 analytical."
+    user "TL;DR: {tldr}\nTone: {tone}\nHashtags: {hashtags}"
+  end
+
+  validate("summary is the tldr verbatim") { |o, input| o[:summary] == input[:tldr] }
+end
+
+# Pipeline: summarize → hashtags → card
+class ArticleCardPipeline < RubyLLM::Contract::Pipeline::Base
+  step SummarizeArticle, as: :summarize
+  step GenerateHashtags, as: :tag
+  step BuildArticleCard, as: :card
+end
+```
+
+## Running and inspecting
+
+```ruby
+result = ArticleCardPipeline.run(article_text, context: { adapter: adapter })
+result.ok?                          # => true
+result.outputs_by_step[:summarize]  # => { tldr: "...", takeaways: [...], tone: "analytical" }
+result.outputs_by_step[:card]       # => { headline: "...", summary: "...", ... }
+result.trace.total_cost             # => 0.000128 (all steps combined)
+result.trace.total_latency_ms       # => 2340
+```
+
+## Fail-fast behavior
+
+When a step's schema, validate, or preflight check rejects the output, the pipeline stops there — downstream steps never run:
+
+```ruby
+# Summarize returns a TL;DR over 200 chars → the "TL;DR fits the card" validate fails
+adapter = RubyLLM::Contract::Adapters::Test.new(response: {
+  tldr: "x" * 500,
+  takeaways: %w[one two three],
+  tone: "neutral"
+})
+
+result = ArticleCardPipeline.run("article text", context: { adapter: adapter })
+result.failed?        # => true
+result.failed_step    # => :summarize (validate rejected; retries exhausted)
+# tag and card never run — no downstream tokens spent on garbage
+```
+
+## Per-step model override
+
+```ruby
+class ArticleCardPipeline < RubyLLM::Contract::Pipeline::Base
+  step SummarizeArticle, as: :summarize, model: "gpt-4.1-mini"
+  step GenerateHashtags, as: :tag,       model: "gpt-4.1-nano"
+  step BuildArticleCard, as: :card,      model: "gpt-4.1-nano"
+end
+```
+
+## Timeout
+
+```ruby
+result = ArticleCardPipeline.run(article_text, timeout_ms: 30_000)
+```
+
+## Pipeline eval
+
+```ruby
+ArticleCardPipeline.define_eval("e2e") do
+  add_case "ruby 3.4 release",
+    input: "Ruby 3.4 ships with frozen string literals by default and better YJIT...",
+    expected: { sentiment_icon: "🧠" }
+end
+
+report = ArticleCardPipeline.run_eval("e2e", context: { model: "gpt-4.1-mini" })
+report.print_summary
+```
+
+## Pretty print
+
+```ruby
+puts result
+# Pipeline: ok  3 steps  1234ms  450+120 tokens  trace=abc12345
+
+result.pretty_print
+# Full ASCII table with per-step outputs (Pipeline::Result)
+
+# For eval reports, use print_summary instead:
+report.print_summary
+# Tabular pass/fail breakdown (Eval::Report)
+```
+
+## See also
+
+- [Testing](testing.md) — `ArticleCardPipeline.test(..., responses: { summarize: ..., tag: ..., card: ... })` for pipeline-level spec adapters.
+- [Optimizing retry_policy](optimizing_retry_policy.md) — `optimize_retry_policy` runs per-step; pipelines benchmark one step at a time.

data/docs/guide/prompt_ast.md

@@ -0,0 +1,76 @@
+# Prompt AST
+
+> Read this when your prompt has more than one shape (per-tenant, per-language, per-audience) and string concatenation is starting to drift.
+
+Prompts are structured data, not strings. That matters the moment `SummarizeArticle` has to ship in more than one shape — a different audience per tenant, a different language per region, a different tone template for a B2B vs consumer card. Building those variants by string-concatenating a monolithic prompt leads to silent drift across environments. The AST gives you typed nodes (`system`, `rule`, `section`, `example`, `user`) that compose, diff, and snapshot-test cleanly.
+
+Available node types:
+
+```ruby
+prompt do
+  system  "You summarize articles for a UI card."     # system message
+  rule    "Return valid JSON only."                   # appended as separate system message
+  section "AUDIENCE", "Rails developers"              # labeled system message: [AUDIENCE]\n...
+  example input:  "Ruby 3.4 ships frozen strings...", # user/assistant few-shot pair
+          output: '{"tldr":"...","takeaways":[...],"tone":"analytical"}'
+  user    "{input}"                                   # user message with interpolation
+end
+```
+
+Or just a plain string (wraps as a single user message):
+
+```ruby
+prompt "Summarize this article for a UI card. {input}"
+```
+
+The AST is immutable, diffable, and hashable. Useful for snapshot testing and auditing prompt changes.
+
+## Hash inputs with variable interpolation
+
+When input is a Hash, each key becomes a template variable. Concrete scenario: a multi-language newsletter product where the same article has to be summarised in Polish for EU subscribers, English for US, with different audiences per tier (Rails developers vs engineering managers). Hash inputs let one step cover all of these without forking the class:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  input_type RubyLLM::Contract::Types::Hash.schema(
+    article:  RubyLLM::Contract::Types::String,
+    audience: RubyLLM::Contract::Types::String,
+    language: RubyLLM::Contract::Types::String
+  )
+
+  prompt do
+    system  "You summarize articles for a UI card."
+    rule    "Write the TL;DR and takeaways in {language}."
+    section "AUDIENCE", "{audience}"
+    user    "{article}"
+  end
+
+  output_schema do
+    string :tldr
+    array  :takeaways, of: :string, min_items: 3, max_items: 5
+    string :tone, enum: %w[neutral positive negative analytical]
+  end
+end
+```
+
+Every `{key}` in a prompt node is pulled from the input hash at run time. Missing keys raise — making wire-up bugs loud, not silent.
+
+> **Not the same as `RubyLLM::Agent.inputs`.** `Step.input_type` is a *runtime type check* on the positional argument passed to `run(input)` — it raises `TypeError` if the input violates the declared shape. `RubyLLM::Agent.inputs` is a list of *named template locals* injected into ERB instructions. They solve different problems (validation vs templating) and can coexist on the same project.
+
+> **Not the same as `RubyLLM::Agent` ERB templates either.** The `prompt do ... end` DSL above builds a multi-role message list (`system` / `user` / `assistant` / `example` nodes) into a node-AST. `Agent.instructions :name` loads a single-string ERB file from `app/prompts/<agent_path>/<name>.txt.erb` for the system prompt only. Different output shape, different scope. Variable interpolation here uses `{key}` substitution; ERB uses full `<%= ruby %>`.
+
+## Cross-validating output against input
+
+Validate blocks support 2-arity `|output, input|` so you can check that the model's answer stays faithful to the request:
+
+```ruby
+validate("tldr is not just the article reprinted") do |output, input|
+  # Guard against lazy models that return the input verbatim.
+  output[:tldr].length < input[:article].length / 2
+end
+
+validate("no takeaway repeats the TL;DR") do |output, _input|
+  output[:takeaways].none? { |t| t == output[:tldr] }
+end
+```
+
+The first example uses `input`; the second ignores it. Both are legal 2-arity signatures — Ruby accepts the unused `_input` parameter naming convention.

data/docs/guide/rails_integration.md

@@ -0,0 +1,218 @@
+# Rails integration
+
+> Read this when you've seen the `SummarizeArticle` example and want to know where contract steps fit in an actual Rails app — directory, initializer, jobs, logging, tests, CI. Skip if you're writing a non-Rails script.
+
+Seven pre-emptive answers to the questions that come up first.
+
+## 1. Where do step classes live?
+
+**Recommended: `app/contracts/`.** The gem's Railtie auto-reloads eval files under `app/contracts/eval/` and `app/steps/eval/` in development, so picking `app/contracts/` aligns with the default reload paths.
+
+```
+app/contracts/summarize_article.rb   # class SummarizeArticle
+```
+
+Any autoloaded directory works (`app/llm_steps/`, `app/services/llm/`, etc.) — Rails 7/8 autoloading resolves them all, and the step class itself does not depend on the path. Pick the default if you have no stronger convention.
+
+Keep evals in the same file as the step (`define_eval` block at the bottom of the class) — one source of truth per contract. If your evals grow too large for the class file, move them to `app/contracts/eval/summarize_article_eval.rb` — the Railtie reloads that directory explicitly in development.
+
+## 2. Initializer configuration
+
+```ruby
+# config/initializers/ruby_llm_contract.rb
+RubyLLM.configure do |c|
+  c.openai_api_key    = ENV.fetch("OPENAI_API_KEY", nil)
+  c.anthropic_api_key = ENV.fetch("ANTHROPIC_API_KEY", nil)
+end
+
+RubyLLM::Contract.configure do |c|
+  c.default_model   = Rails.env.production? ? "gpt-5-mini" : "gpt-5-nano"
+  c.default_adapter = RubyLLM::Contract::Adapters::RubyLLM.new
+end
+```
+
+In specs, override the default adapter to `Adapters::Test` in `spec_helper.rb` (or use `stub_step` per-example — see §5).
+
+Evals defined inside a step class (the recommended pattern) are picked up as soon as Rails autoloads the class — you do not need the eval file in any special directory. If you move evals into separate files under `app/contracts/eval/` or `app/steps/eval/`, the gem's Railtie reloads those two directories explicitly on each request in development; other directories follow standard Rails autoloading rules.
+
+## 3. Background jobs — never call LLMs inline in a controller
+
+LLM calls take 0.8–5 seconds and can fail. Wrap every step invocation in an ActiveJob:
+
+```ruby
+class SummarizeArticleJob < ApplicationJob
+  queue_as :llm
+
+  def perform(article_id)
+    article = Article.find(article_id)
+    result  = SummarizeArticle.run(article.body)
+
+    if result.ok?
+      # parsed_output uses symbol keys in memory. jsonb/json columns round-trip
+      # as strings on reload, so either use deep_stringify_keys before write or
+      # access downstream with string keys — pick one convention and stick to it.
+      article.update!(summary: result.parsed_output.deep_stringify_keys)
+    else
+      article.update!(summary_error: result.validation_errors.join("; "))
+    end
+  end
+end
+```
+
+`SummarizeArticleJob.perform_later(article.id)` returns in milliseconds; the controller stays responsive. If you use Sidekiq, pair `queue_as :llm` with a dedicated concurrency cap in `sidekiq.yml` so long-running LLM calls do not starve other job queues (mailers, webhooks, cleanups).
+
+## 4. Logging and observability
+
+`around_call` runs once per `run()` with the final `Result` (after all retries). Use it to write one row per LLM call:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  # ... prompt, schema, validates ...
+
+  around_call do |step, input, result|
+    AiCallLog.create!(
+      step: step.name,
+      model: result.trace[:model],
+      status: result.status.to_s,
+      latency_ms: result.trace[:latency_ms],
+      input_tokens: result.trace[:usage]&.dig(:input_tokens),
+      output_tokens: result.trace[:usage]&.dig(:output_tokens),
+      cost: result.trace[:cost],
+      validation_errors: result.validation_errors
+    )
+  end
+end
+```
+
+The `AiCallLog` model assumed above is a thin audit record. One possible migration:
+
+```ruby
+# rails g model AiCallLog step:string model:string status:string ...
+create_table :ai_call_logs do |t|
+  t.string  :step, null: false
+  t.string  :model
+  t.string  :status, null: false
+  t.integer :latency_ms
+  t.integer :input_tokens
+  t.integer :output_tokens
+  t.decimal :cost, precision: 10, scale: 6
+  t.jsonb   :validation_errors, default: []
+  t.timestamps
+end
+add_index :ai_call_logs, :step
+add_index :ai_call_logs, :status
+```
+
+For Appsignal / Honeybadger / Datadog, emit an `ActiveSupport::Notifications` event from inside the same `around_call` and subscribe in an initializer:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  # ... prompt, schema, validates ...
+
+  around_call do |step, _input, result|
+    ActiveSupport::Notifications.instrument(
+      "ruby_llm_contract.run",
+      step: step.name, model: result.trace[:model], status: result.status
+    )
+  end
+end
+
+# config/initializers/observability.rb
+ActiveSupport::Notifications.subscribe("ruby_llm_contract.run") do |*, payload|
+  Appsignal.increment_counter("llm.run.#{payload[:status]}", 1, step: payload[:step])
+end
+```
+
+Trace inspection in an admin UI: `result.trace[:attempts]` gives you per-attempt model, status, cost, latency — render it in a partial to debug production failures without re-running.
+
+## 5. Testing — RSpec and Minitest
+
+Add to `spec/spec_helper.rb` (or `test_helper.rb`):
+
+```ruby
+require "ruby_llm/contract/rspec"    # or ruby_llm/contract/minitest
+```
+
+Then in specs:
+
+```ruby
+RSpec.describe ArticlesController do
+  it "saves the summary when the step passes" do
+    stub_step(SummarizeArticle, response: {
+      tldr: "...", takeaways: %w[a b c], tone: "analytical"
+    })
+
+    post :summarize, params: { id: article.id }
+
+    # NOTE: jsonb/json column round-trips as string keys on reload.
+    expect(article.reload.summary["tldr"]).to eq("...")
+  end
+end
+```
+
+For the step itself, use the `satisfy_contract` and `pass_eval` matchers — details in the [Testing guide](testing.md).
+
+## 6. Error handling in controllers
+
+Never raise on `result.failed?` — that crashes the request. Branch instead:
+
+```ruby
+class ArticlesController < ApplicationController
+  def summarize
+    SummarizeArticleJob.perform_later(params[:id])
+    head :accepted
+  end
+
+  # For synchronous cases (admin tools, small content):
+  def preview
+    result = SummarizeArticle.run(@article.body)
+
+    if result.ok?
+      render json: result.parsed_output
+    else
+      Rails.logger.warn "[llm] #{SummarizeArticle.name} failed: #{result.status}"
+      render json: { error: "Could not summarize; try again shortly." }, status: :service_unavailable
+    end
+  end
+end
+```
+
+When `retry_policy` exhausts and all models fail, `result.failed?` is true but `result.parsed_output` still contains the last attempt's output — useful for logging what the model *did* return before the validate rejected it.
+
+## 7. CI gate — block regressions before merge
+
+Add to your `Rakefile`:
+
+```ruby
+require "ruby_llm/contract/rake_task"
+
+RubyLLM::Contract::RakeTask.new do |t|
+  t.minimum_score      = 0.8
+  t.maximum_cost       = 0.05
+  t.fail_on_regression = true
+  t.save_baseline      = false # read-only in CI; refresh baselines manually
+end
+```
+
+Then wire it in GitHub Actions:
+
+```yaml
+- name: LLM contract evals
+  env:
+    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+  run: bundle exec rake ruby_llm_contract:eval
+```
+
+The job fails when a previously-passing eval case now fails, when the average score drops below the threshold, or when total cost exceeds the cap. That is the signal that blocks a prompt regression or an accidental model upgrade from shipping.
+
+**Two practical notes:**
+
+- **Live evals spend real money on every run** — provider tokens per case × number of cases × every merge. Keep the dataset small and targeted (5–15 high-value cases), use cheap models where quality allows, and rely on offline `sample_response` smoke tests in the bulk of CI runs. Live evals belong on merge-candidate branches and scheduled nightly runs, not on every commit.
+- **Baselines are checkout-managed** — commit them to git under `.eval_baselines/`. Refresh them in a separate manual workflow (or locally + a dedicated PR) rather than from the merge gate, which would dirty the checkout and race with the regression check it is supposed to run.
+
+## See also
+
+- [Getting Started](getting_started.md) — the feature walkthrough the step above is built on
+- [Migration](migration.md) — before/after for replacing a raw `LlmClient.new.call` service with a contract
+- [Eval-First](eval_first.md) — the workflow behind the CI gate above
+- [Testing](testing.md) — `satisfy_contract` and `pass_eval` matcher chains

data/docs/guide/relation_to_agent.md

@@ -0,0 +1,52 @@
+# Relation to `RubyLLM::Agent`
+
+> Read this when you already use `RubyLLM::Agent` (or are about to) and want to understand where `ruby_llm-contract` sits in the same project.
+
+`RubyLLM::Agent` shipped in RubyLLM 1.12. `Step::Base` from this gem and `Agent` target the **same niche**: reusable, class-based prompts. They are **siblings**, not foundation-and-roof.
+
+## Feature mapping
+
+| What you write | Where it lives |
+|---|---|
+| `model`, `temperature`, `schema`, `instructions`, `tools`, `thinking` | covered by both — same idea, different DSL surface |
+| `validate :rule do ... end` business invariants on output | only in `ruby_llm-contract` |
+| `retry_policy escalate(...)` model escalation on validation failure | only here (different from RubyLLM's network-level retry) |
+| `max_cost` / `max_input` / `max_output` pre-flight refusal | only here |
+| `define_eval` + baseline regression + `compare_models` + `optimize_retry_policy` | only here (RubyLLM does not ship an evaluation framework) |
+| Pipeline composition with `step SomeStep, as: :alias` | only here (RubyLLM intentionally leaves workflows as plain Ruby) |
+| `around_call`, named `observe` hooks with pass/fail recorded in trace | only here |
+
+## Runtime relationship
+
+`Step::Base` does **not** use `Agent` internally today. The actual call path is:
+
+```
+Step.run(input)
+  → Runner
+  → Adapters::RubyLLM
+  → RubyLLM.chat(model:, ...)
+  → ... .ask(prompt)
+```
+
+`Agent` is a sibling abstraction calling into `RubyLLM::Chat` through its own `apply_configuration` path. Both end up at `Chat`. They do not share the macro-storage layer.
+
+This may change in a future release if upstream APIs make a layered design natural. The decision is not committed; it depends on adopter signal.
+
+## Coexistence on the same project
+
+The two abstractions can live in the same Rails (or non-Rails) project. Pick one per use case:
+
+- **`RubyLLM::Agent`** when you want a reusable prompt with `model` + `instructions` + `schema` + `tools` and that is enough — no retry-on-validation-failure, no business invariants, no eval framework, no budget gating.
+- **`ruby_llm-contract`'s `Step::Base`** when you need any of: invariants (`validate`), retry with model escalation on validation failure, pre-flight cost ceilings, an evaluation framework with baseline regression, or pipeline composition.
+
+A common pattern: simple ad-hoc prompts as `Agent`, contracts on the LLM features that touch production behaviour or money as `Step`.
+
+## On retry strategies
+
+The retry-strategy framing in this gem favours `retry_policy escalate(model_2, ...)` (model escalation, addresses model bias) over same-model `retry_policy attempts: N` (variance retry).
+
+This is grounded in empirical comparison across PDF quiz generation, GSM8K math (n=30 + n=120), and multi-constraint schedule generation: same-model retry produced no useful lift for nano-class models on tasks with clear correctness criteria. Model escalation did move the needle when same-model retry could not.
+
+`attempts: N` stays in the gem API (backward compat + niche cases like subjective-criteria tasks, multi-step pipelines, weaker open-source models) but is not marketed as a default retry strategy.
+
+See [Optimize retry policy](optimizing_retry_policy.md) for the empirical tooling.

data/docs/guide/relation_to_tribunal.md

@@ -0,0 +1,135 @@
+# Relation to `ruby_llm-tribunal`
+
+> Read this when you've seen [`ruby_llm-tribunal`](https://github.com/Alqemist-labs/ruby_llm-tribunal) and want to know how it relates to `ruby_llm-contract` — and which one (or both) you need.
+
+Both gems sit on top of `ruby_llm`. The space they cover overlaps in vocabulary (both talk about "evals") but they live in different layers and answer different questions. They are not alternatives — they compose.
+
+## The core distinction
+
+| | `ruby_llm-tribunal` | `ruby_llm-contract` |
+|---|---|---|
+| Layer | **Test framework** | **Runtime contract** |
+| When it runs | After the LLM call returns, typically in a spec | Before the LLM result reaches your code |
+| Where the output lives at evaluation time | Already in your variable, returned to caller | Still inside the gem's runner, not yet released |
+| What "fail" means | Red test in CI | Trigger retry/escalate on a stronger model, or fail closed |
+| Strongest features | Rich LLM-as-judge (faithful, relevant, hallucination, refusal, bias, toxicity, jailbreak, PII), red-team adversarial prompts, deterministic helpers (`assert_contains`, `assert_levenshtein`, …), RSpec/Minitest matchers, HTML/JUnit/GH reporters | Schema DSL with constraints, `validate` business rules, `retry_policy escalate(...)` model escalation, `max_cost` pre-flight refusal, regression-eval framework (frozen dataset + baseline + min_score gate), pipeline composition |
+| What it does NOT cover | No retry, no model escalation, no pre-flight cost cap, no contract layer between LLM and your code | No 10-judge LLM-as-judge catalog, no red-team generation, no rich deterministic assertion library, no test-framework matchers |
+
+## Visual: where each gem sits in your call
+
+### Tribunal alone (test-time, in CI)
+
+```
+your code ──► LLM ──► output ──► variable ──► [Tribunal assert_*] ──► ✅ / ❌ red test
+                                                       ▲
+                                              runs in your spec, not in prod
+```
+
+The output **already exists in your code** by the time Tribunal sees it. Tribunal grades it after the fact. A failed grade is a red test — production is unaffected, you fix the prompt or model and re-run.
+
+### Contract alone (runtime, in prod)
+
+```
+your code ──► Step.run ──► LLM ──► [schema + validate]
+                                          │
+                                          ├── valid ────────────► output ──► your code
+                                          │
+                                          └── invalid ──► retry/escalate ──► next model
+                                                                                 │
+                                                              all attempts fail ─┘
+                                                                                 ▼
+                                                                  Result(:validation_error)
+```
+
+The output **never reaches your code** until the contract passes. A failed validation either fixes itself (the retry policy escalates to a stronger model) or fails closed with `Result(:validation_error)` — your call site sees a deterministic failure status, never schema-invalid data.
+
+### Both together (Tribunal in CI → then Contract in prod)
+
+```
+CI (before merge):
+  define_eval(frozen dataset) ──► run Step ──► [Tribunal grades each case]
+                                                       │
+                                                       ▼
+                                                regression gate
+                                           (prevents quality drift over time)
+                                                       │
+                                                       ▼
+                                              ✅ merge allowed
+                                                       │
+                                                       ▼
+PROD (every request):
+  your code ──► Step.run ──► LLM ──► [contract] ──► output ──► your code
+                                         ▲
+                                         │ keeps bad outputs out of prod
+```
+
+Tribunal grades **a fixed set of cases on every PR** to catch quality regressions before merge. Once merged, Contract gates **every production call** to keep bad outputs from reaching users. Each gem owns the layer it is best at — and they run in the order developers experience them.
+
+## When to use which
+
+**Just Contract.** You ship LLM features whose output drives downstream code, money, or user trust. You need the bad-output-doesn't-reach-prod guarantee, retry escalation, and budget refusal. You are happy to write your own `validate` blocks for content checks; you don't need a 10-judge catalog yet.
+
+**Just Tribunal.** You have a stable production path you don't want to wrap, but you want a CI safety net that grades LLM output for faithfulness, hallucination, PII, jailbreak resistance, etc. You're testing a RAG pipeline or chatbot whose runtime is owned by other code.
+
+**Both.** You ship contracts in prod (Contract) AND want stronger CI signal beyond schema regression — judge-quality grading on a frozen dataset, plus adversarial red-team probes. Use Contract's `Step` to make the call, run it in `define_eval` over your dataset, and grade each case with Tribunal helpers in your spec or via the dataset's `evaluator:` proc.
+
+## Integration patterns
+
+These work today without any code changes in either gem — both use plain Ruby blocks/procs as extension points.
+
+### Tribunal helpers inside Contract `validate`
+
+```ruby
+class ChatReply < RubyLLM::Contract::Step::Base
+  prompt "Answer this question grounded in the docs:\n{input}"
+
+  validate("no PII in answer") do |output, _ctx|
+    test_case = RubyLLM::Tribunal::TestCase.new(actual_output: output[:answer])
+    RubyLLM::Tribunal::Assertions.evaluate(:pii, test_case, {}).first == :pass
+  end
+end
+```
+
+A failed Tribunal grade triggers Contract's retry/escalate just like any other validation failure. You get LLM-as-judge **runtime gating**, not just CI testing.
+
+### Tribunal as `evaluator:` in a Contract dataset
+
+```ruby
+ChatReply.define_eval "rag_regression" do
+  add_case "policy",
+    input: "What is the return policy?",
+    evaluator: ->(output, _expected, _input) {
+      tc = RubyLLM::Tribunal::TestCase.new(
+        actual_output: output[:answer],
+        context: ["Returns accepted within 30 days with receipt."]
+      )
+      result = RubyLLM::Tribunal::Assertions.evaluate(:faithful, tc, {})
+      score = result.last[:score] || 0.0
+      RubyLLM::Contract::Eval::EvaluationResult.new(score: score, passed: result.first == :pass)
+    }
+end
+```
+
+Each case is graded by a Tribunal judge; baseline + min_score gate then fails the build on regression. You write the judge once, get the regression gate for free.
+
+### Contract `Step` as Tribunal's `opts[:llm]` injection
+
+Tribunal's built-in judges call `RubyLLM.chat(...).ask(...)` and naively `JSON.parse` the result. If you want **schema-validated, retried, cost-capped judge calls**, inject a Contract `Step` as the LLM caller via `opts[:llm]`. This is an advanced pattern; sketch it from `Tribunal::Assertions::Judge#run_judge`'s injection point and your own judge wrapping a `Step.run`.
+
+This is a recipe, not a shipped adapter. Tribunal's `opts[:llm]` API is at v0.x — recipes survive minor changes; a shipped adapter would not.
+
+## What we are NOT doing
+
+- **No `Contract::ContainsAssertion` or similar 16-helper deterministic library.** Tribunal owns that layer well. Contract's evaluator surface is intentionally minimal (`Exact`, `Regex`, `JsonIncludes`, `ProcEvaluator`, `TraitEvaluator`); for richer deterministic checks, drop a Tribunal helper into your `evaluator:` proc.
+- **No built-in LLM-as-judge catalog.** `Faithful`, `Hallucination`, `Refusal`, etc. are Tribunal's domain. We provide the runtime contract; they provide the grading vocabulary.
+- **No Tribunal as a hard or soft dependency.** Both gems work standalone. Recipes above are documentation, not code in this gem.
+
+## Summary
+
+Three questions, three owners:
+
+- **"Is this output good?"** — Tribunal, in CI, on outputs you already hold.
+- **"What do we do when it isn't?"** — Contract, at runtime, before outputs reach your code (retry/escalate, or fail-closed with `Result(:validation_failed)`).
+- **"What do we do when it _is_ good?"** — your application code. Once Contract returns `:ok`, you persist, render, hand off downstream. The gem deliberately doesn't touch the happy path; it owns failure semantics, not domain logic.
+
+Use Tribunal, Contract, or both — whichever questions your application needs to answer.