ruby_llm-contract 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: dc8b1278c5464978cfc50d87ac90cbde94c2a5920b00996365a2e366bb27f1e6
+  data.tar.gz: daf51e9b66472464137d371439f503317b84706f167fa74c2118635ae82823b1
+SHA512:
+  metadata.gz: 7899b4c2df5e7824a5104c24698b728b017e996cced82022c26d81167e8876085fcec93396d13ce67aed7034cfbd0cfbde2e0ebd76376a8dd198d6a561d273d7
+  data.tar.gz: 7ca10ee16ea71eda609439546b9f02b20e184e9c6a8861d88a172c0e75ca611cf3c2bddf8827b820b31aa9cc5baf88bdaf9b708732b78f3e6321438030186ef8

data/.rspec

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

data/.rubocop.yml

@@ -0,0 +1,55 @@
+AllCops:
+  TargetRubyVersion: 3.2
+  NewCops: enable
+  SuggestExtensions: false
+
+Style/Documentation:
+  Enabled: false
+
+Style/StringLiterals:
+  EnforcedStyle: double_quotes
+
+Style/StringLiteralsInInterpolation:
+  EnforcedStyle: double_quotes
+
+Metrics/BlockLength:
+  Exclude:
+    - 'spec/**/*'
+    - '*.gemspec'
+
+Metrics/MethodLength:
+  Max: 25
+
+Layout/LineLength:
+  Max: 120
+
+Style/OneClassPerFile:
+  Exclude:
+    - 'spec/**/*'
+    - 'examples/**/*'
+
+Lint/UnusedBlockArgument:
+  Exclude:
+    - 'spec/**/*'
+
+Naming/VariableNumber:
+  Exclude:
+    - 'spec/**/*'
+    - 'examples/**/*'
+
+AllCops:
+  Exclude:
+    - 'internal/**/*'
+
+Metrics/ClassLength:
+  Max: 130
+
+Metrics/AbcSize:
+  Max: 30
+
+Metrics/ParameterLists:
+  Max: 11
+  MaxOptionalParameters: 9
+
+Style/FormatStringToken:
+  Enabled: false

data/CHANGELOG.md

@@ -0,0 +1,76 @@
+# Changelog
+
+## 0.2.0 (2026-03-23)
+
+Contracts for LLM quality. Know which model to use, what it costs, and when accuracy drops.
+
+### Breaking changes
+
+- **`report.results` returns `CaseResult` objects** instead of hashes. Use `result.name`, `result.passed?`, `result.score` instead of `result[:case_name]`, `result[:passed]`. `CaseResult#to_h` for backward compat.
+- **`report.print_summary`** replaces `report.pretty_print` (avoids shadowing `Kernel#pretty_print`).
+
+### Features
+
+- **`add_case` in `define_eval`** — `add_case "billing", input: "...", expected: { priority: "high" }` with partial matching. Supports `expected_traits:` for regex/range matching.
+- **`CaseResult` value objects** — `result.name`, `result.passed?`, `result.output`, `result.expected`, `result.mismatches` (structured diff), `result.cost`, `result.duration_ms`.
+- **`report.failures`** — returns only failed cases. `report.skipped` counts skipped (offline) cases.
+- **Model comparison** — `Step.compare_models("eval", models: %w[nano mini full])` runs same eval across models. Returns table with score/cost/latency per model. `comparison.best_for(min_score: 0.95)` returns cheapest model meeting threshold.
+- **Cost tracking** — `report.total_cost`, `report.avg_latency_ms`, per-case `result.cost`. Pipeline eval uses total pipeline cost, not just last step.
+- **Cost prediction** — `Step.estimate_cost(input:, model:)` and `Step.estimate_eval_cost("eval", models: [...])` predict spend before API calls.
+- **CI gating** — `pass_eval("regression").with_minimum_score(0.8).with_maximum_cost(0.01)`. RakeTask with suite-level `minimum_score` and `maximum_cost`.
+- **`RubyLLM::Contract.run_all_evals`** — discovers all Steps/Pipelines with evals, runs them all. Includes inherited evals.
+- **`RubyLLM::Contract::RakeTask`** — `rake ruby_llm_contract:eval` with `minimum_score`, `maximum_cost`, `fail_on_empty`, `eval_dirs`.
+- **Rails Railtie** — auto-loads eval files via `config.after_initialize` + `config.to_prepare` (supports development reload).
+- **Offline mode** — cases without adapter return `:skipped` instead of crashing. Skipped cases excluded from score/passed.
+- **Safe `define_eval`** — warns on duplicate name; suppressed during reload.
+
+### Fixes
+
+- **P1: Eval files not autoloaded by Rails** — Railtie uses `load` (not Zeitwerk). Hooks into reloader for dev.
+- **P2: report.results returns raw Hashes** — now returns `CaseResult` objects.
+- **P3: No way to run all evals at once** — `Contract.run_all_evals` + Rake task.
+- **P4: String vs symbol key mismatch** — warns when `validate` or `verify` proc returns nil.
+- **Pipeline eval cost** — uses `Pipeline::Trace#total_cost` (all steps), not just last step.
+- **Reload lifecycle** — `load_evals!` clears definitions before re-loading. Registry filters stale hosts.
+- **Adapter isolation** — `compare_models` and `run_all_own_evals` deep-dup context per run.
+
+### Verified with real API
+
+```
+Model                      Score       Cost  Avg Latency
+---------------------------------------------------------
+gpt-4.1-nano                0.67    $0.000032      687ms
+gpt-4.1-mini                1.00    $0.000102     1070ms
+```
+
+### Stats
+
+- 1077 tests, 0 failures
+- 3 architecture review rounds, 32 findings fixed
+- Verified with real OpenAI API (gpt-4.1-nano, gpt-4.1-mini)
+
+## 0.1.0 (2026-03-20)
+
+Initial release.
+
+### Features
+
+- **Step abstraction** — `RubyLLM::Contract::Step::Base` with prompt DSL, typed input/output
+- **Output schema** — declarative structure via ruby_llm-schema, sent to provider for enforcement
+- **Validate** — business logic checks (1-arity and 2-arity with input cross-validation)
+- **Retry with model escalation** — start cheap, auto-escalate on contract failure or network error
+- **Preflight limits** — `max_input`, `max_cost`, `max_output` refuse before calling the LLM
+- **Pipeline** — multi-step composition with fail-fast, timeout, token budget
+- **Eval** — offline contract verification with `define_eval`, `run_eval`, zero-verify auto-case
+- **Adapters** — RubyLLM (production), Test (deterministic specs)
+- **RSpec matchers** — `satisfy_contract`, `pass_eval`
+- **Structured trace** — model, latency, tokens, cost, attempt log per step
+
+### Robustness
+
+- 1005 tests, 0 failures
+- 42 bugs found and fixed via 10 rounds of adversarial testing
+- 0 RuboCop offenses
+- Parser handles: markdown code fences, UTF-8 BOM, JSON extraction from prose
+- SchemaValidator: full nested validation, additionalProperties, minItems/maxItems, minLength/maxLength
+- Deep-frozen parsed_output prevents mutation via shared references

data/Gemfile

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gemspec
+
+group :development, :test do
+  gem "rake", "~> 13.0"
+  gem "rspec", "~> 3.13"
+  gem "rubocop", "~> 1.75"
+end

data/Gemfile.lock

@@ -0,0 +1,176 @@
+PATH
+  remote: .
+  specs:
+    ruby_llm-contract (0.2.0)
+      dry-types (~> 1.7)
+      ruby_llm (~> 1.0)
+      ruby_llm-schema (~> 0.3)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    addressable (2.8.9)
+      public_suffix (>= 2.0.2, < 8.0)
+    ast (2.4.3)
+    base64 (0.3.0)
+    bigdecimal (4.0.1)
+    concurrent-ruby (1.3.6)
+    diff-lcs (1.6.2)
+    dry-core (1.2.0)
+      concurrent-ruby (~> 1.0)
+      logger
+      zeitwerk (~> 2.6)
+    dry-inflector (1.3.1)
+    dry-logic (1.6.0)
+      bigdecimal
+      concurrent-ruby (~> 1.0)
+      dry-core (~> 1.1)
+      zeitwerk (~> 2.6)
+    dry-types (1.9.1)
+      bigdecimal (>= 3.0)
+      concurrent-ruby (~> 1.0)
+      dry-core (~> 1.0)
+      dry-inflector (~> 1.0)
+      dry-logic (~> 1.4)
+      zeitwerk (~> 2.6)
+    event_stream_parser (1.0.0)
+    faraday (2.14.1)
+      faraday-net_http (>= 2.0, < 3.5)
+      json
+      logger
+    faraday-multipart (1.2.0)
+      multipart-post (~> 2.0)
+    faraday-net_http (3.4.2)
+      net-http (~> 0.5)
+    faraday-retry (2.4.0)
+      faraday (~> 2.0)
+    json (2.19.2)
+    json-schema (6.2.0)
+      addressable (~> 2.8)
+      bigdecimal (>= 3.1, < 5)
+    language_server-protocol (3.17.0.5)
+    lint_roller (1.1.0)
+    logger (1.7.0)
+    marcel (1.1.0)
+    mcp (0.9.0)
+      json-schema (>= 4.1)
+    multipart-post (2.4.1)
+    net-http (0.9.1)
+      uri (>= 0.11.1)
+    parallel (1.27.0)
+    parser (3.3.10.2)
+      ast (~> 2.4.1)
+      racc
+    prism (1.9.0)
+    public_suffix (7.0.5)
+    racc (1.8.1)
+    rainbow (3.1.1)
+    rake (13.3.1)
+    regexp_parser (2.11.3)
+    rspec (3.13.2)
+      rspec-core (~> 3.13.0)
+      rspec-expectations (~> 3.13.0)
+      rspec-mocks (~> 3.13.0)
+    rspec-core (3.13.6)
+      rspec-support (~> 3.13.0)
+    rspec-expectations (3.13.5)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.13.0)
+    rspec-mocks (3.13.8)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.13.0)
+    rspec-support (3.13.7)
+    rubocop (1.85.1)
+      json (~> 2.3)
+      language_server-protocol (~> 3.17.0.2)
+      lint_roller (~> 1.1.0)
+      mcp (~> 0.6)
+      parallel (~> 1.10)
+      parser (>= 3.3.0.2)
+      rainbow (>= 2.2.2, < 4.0)
+      regexp_parser (>= 2.9.3, < 3.0)
+      rubocop-ast (>= 1.49.0, < 2.0)
+      ruby-progressbar (~> 1.7)
+      unicode-display_width (>= 2.4.0, < 4.0)
+    rubocop-ast (1.49.1)
+      parser (>= 3.3.7.2)
+      prism (~> 1.7)
+    ruby-progressbar (1.13.0)
+    ruby_llm (1.14.0)
+      base64
+      event_stream_parser (~> 1)
+      faraday (>= 1.10.0)
+      faraday-multipart (>= 1)
+      faraday-net_http (>= 1)
+      faraday-retry (>= 1)
+      marcel (~> 1)
+      ruby_llm-schema (~> 0)
+      zeitwerk (~> 2)
+    ruby_llm-schema (0.3.0)
+    unicode-display_width (3.2.0)
+      unicode-emoji (~> 4.1)
+    unicode-emoji (4.2.0)
+    uri (1.1.1)
+    zeitwerk (2.7.5)
+
+PLATFORMS
+  arm64-darwin-25
+  ruby
+
+DEPENDENCIES
+  rake (~> 13.0)
+  rspec (~> 3.13)
+  rubocop (~> 1.75)
+  ruby_llm-contract!
+
+CHECKSUMS
+  addressable (2.8.9) sha256=cc154fcbe689711808a43601dee7b980238ce54368d23e127421753e46895485
+  ast (2.4.3) sha256=954615157c1d6a382bc27d690d973195e79db7f55e9765ac7c481c60bdb4d383
+  base64 (0.3.0) sha256=27337aeabad6ffae05c265c450490628ef3ebd4b67be58257393227588f5a97b
+  bigdecimal (4.0.1) sha256=8b07d3d065a9f921c80ceaea7c9d4ae596697295b584c296fe599dd0ad01c4a7
+  concurrent-ruby (1.3.6) sha256=6b56837e1e7e5292f9864f34b69c5a2cbc75c0cf5338f1ce9903d10fa762d5ab
+  diff-lcs (1.6.2) sha256=9ae0d2cba7d4df3075fe8cd8602a8604993efc0dfa934cff568969efb1909962
+  dry-core (1.2.0) sha256=0cc5a7da88df397f153947eeeae42e876e999c1e30900f3c536fb173854e96a1
+  dry-inflector (1.3.1) sha256=7fb0c2bb04f67638f25c52e7ba39ab435d922a3a5c3cd196120f63accb682dcc
+  dry-logic (1.6.0) sha256=da6fedbc0f90fc41f9b0cc7e6f05f5d529d1efaef6c8dcc8e0733f685745cea2
+  dry-types (1.9.1) sha256=baebeecdb9f8395d6c9d227b62011279440943e3ef2468fe8ccc1ba11467f178
+  event_stream_parser (1.0.0) sha256=a2683bab70126286f8184dc88f7968ffc4028f813161fb073ec90d171f7de3c8
+  faraday (2.14.1) sha256=a43cceedc1e39d188f4d2cdd360a8aaa6a11da0c407052e426ba8d3fb42ef61c
+  faraday-multipart (1.2.0) sha256=7d89a949693714176f612323ca13746a2ded204031a6ba528adee788694ef757
+  faraday-net_http (3.4.2) sha256=f147758260d3526939bf57ecf911682f94926a3666502e24c69992765875906c
+  faraday-retry (2.4.0) sha256=7b79c48fb7e56526faf247b12d94a680071ff40c9fda7cf1ec1549439ad11ebe
+  json (2.19.2) sha256=e7e1bd318b2c37c4ceee2444841c86539bc462e81f40d134cf97826cb14e83cf
+  json-schema (6.2.0) sha256=e8bff46ed845a22c1ab2bd0d7eccf831c01fe23bb3920caa4c74db4306813666
+  language_server-protocol (3.17.0.5) sha256=fd1e39a51a28bf3eec959379985a72e296e9f9acfce46f6a79d31ca8760803cc
+  lint_roller (1.1.0) sha256=2c0c845b632a7d172cb849cc90c1bce937a28c5c8ccccb50dfd46a485003cc87
+  logger (1.7.0) sha256=196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203
+  marcel (1.1.0) sha256=fdcfcfa33cc52e93c4308d40e4090a5d4ea279e160a7f6af988260fa970e0bee
+  mcp (0.9.0) sha256=a0a3737b0ac9df0772f4ef7e2b013c260ddbcf217a5d50a66bff0baeddf03e47
+  multipart-post (2.4.1) sha256=9872d03a8e552020ca096adadbf5e3cb1cd1cdd6acd3c161136b8a5737cdb4a8
+  net-http (0.9.1) sha256=25ba0b67c63e89df626ed8fac771d0ad24ad151a858af2cc8e6a716ca4336996
+  parallel (1.27.0) sha256=4ac151e1806b755fb4e2dc2332cbf0e54f2e24ba821ff2d3dcf86bf6dc4ae130
+  parser (3.3.10.2) sha256=6f60c84aa4bdcedb6d1a2434b738fe8a8136807b6adc8f7f53b97da9bc4e9357
+  prism (1.9.0) sha256=7b530c6a9f92c24300014919c9dcbc055bf4cdf51ec30aed099b06cd6674ef85
+  public_suffix (7.0.5) sha256=1a8bb08f1bbea19228d3bed6e5ed908d1cb4f7c2726d18bd9cadf60bc676f623
+  racc (1.8.1) sha256=4a7f6929691dbec8b5209a0b373bc2614882b55fc5d2e447a21aaa691303d62f
+  rainbow (3.1.1) sha256=039491aa3a89f42efa1d6dec2fc4e62ede96eb6acd95e52f1ad581182b79bc6a
+  rake (13.3.1) sha256=8c9e89d09f66a26a01264e7e3480ec0607f0c497a861ef16063604b1b08eb19c
+  regexp_parser (2.11.3) sha256=ca13f381a173b7a93450e53459075c9b76a10433caadcb2f1180f2c741fc55a4
+  rspec (3.13.2) sha256=206284a08ad798e61f86d7ca3e376718d52c0bc944626b2349266f239f820587
+  rspec-core (3.13.6) sha256=a8823c6411667b60a8bca135364351dda34cd55e44ff94c4be4633b37d828b2d
+  rspec-expectations (3.13.5) sha256=33a4d3a1d95060aea4c94e9f237030a8f9eae5615e9bd85718fe3a09e4b58836
+  rspec-mocks (3.13.8) sha256=086ad3d3d17533f4237643de0b5c42f04b66348c28bf6b9c2d3f4a3b01af1d47
+  rspec-support (3.13.7) sha256=0640e5570872aafefd79867901deeeeb40b0c9875a36b983d85f54fb7381c47c
+  rubocop (1.85.1) sha256=3dbcf9e961baa4c376eeeb2a03913dca5e3987033b04d38fa538aa1e7406cc77
+  rubocop-ast (1.49.1) sha256=4412f3ee70f6fe4546cc489548e0f6fcf76cafcfa80fa03af67098ffed755035
+  ruby-progressbar (1.13.0) sha256=80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33
+  ruby_llm (1.14.0) sha256=57c6f7034fc4a44504ea137d70f853b07824f1c1cdbe774ab3ab3522e7098deb
+  ruby_llm-contract (0.2.0)
+  ruby_llm-schema (0.3.0) sha256=a591edc5ca1b7f0304f0e2261de61ba4b3bea17be09f5cf7558153adfda3dec6
+  unicode-display_width (3.2.0) sha256=0cdd96b5681a5949cdbc2c55e7b420facae74c4aaf9a9815eee1087cb1853c42
+  unicode-emoji (4.2.0) sha256=519e69150f75652e40bf736106cfbc8f0f73aa3fb6a65afe62fefa7f80b0f80f
+  uri (1.1.1) sha256=379fa58d27ffb1387eaada68c749d1426738bd0f654d812fcc07e7568f5c57c6
+  zeitwerk (2.7.5) sha256=d8da92128c09ea6ec62c949011b00ed4a20242b255293dd66bf41545398f73dd
+
+BUNDLED WITH
+  4.0.3

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Justyna
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,154 @@
+# ruby_llm-contract
+
+Contracts for LLM quality. Know which model to use, what it costs, and when accuracy drops.
+
+Companion gem for [ruby_llm](https://github.com/crmne/ruby_llm).
+
+## The problem
+
+You call an LLM. It returns bad JSON, wrong values, or costs 4x more than it should. You switch models and quality drops silently. You have no data to decide which model to use.
+
+## The fix
+
+```ruby
+class ClassifyTicket < RubyLLM::Contract::Step::Base
+  prompt do
+    system "You are a support ticket classifier."
+    rule "Return valid JSON only, no markdown."
+    rule "Use exactly one priority: low, medium, high, urgent."
+    example input: "My invoice is wrong", output: '{"priority": "high"}'
+    user "{input}"
+  end
+
+  output_schema do
+    string :priority, enum: %w[low medium high urgent]
+    string :category
+  end
+
+  validate("urgent needs justification") { |o, input| o[:priority] != "urgent" || input.length > 20 }
+  retry_policy models: %w[gpt-4.1-nano gpt-4.1-mini gpt-4.1]
+end
+
+result = ClassifyTicket.run("I was charged twice")
+result.ok?               # => true
+result.parsed_output     # => {priority: "high", category: "billing"}
+result.trace[:cost]      # => 0.000032
+result.trace[:model]     # => "gpt-4.1-nano"
+```
+
+Bad JSON? Auto-retry. Wrong value? Escalate to a smarter model. Schema violated? Caught client-side even if the provider ignores it. All with cost tracking.
+
+## Which model should I use?
+
+Define test cases. Compare models. Get data.
+
+```ruby
+ClassifyTicket.define_eval("regression") do
+  add_case "billing", input: "I was charged twice", expected: { priority: "high" }
+  add_case "feature", input: "Add dark mode please", expected: { priority: "low" }
+  add_case "outage",  input: "Database is down",    expected: { priority: "urgent" }
+end
+
+comparison = ClassifyTicket.compare_models("regression",
+  models: %w[gpt-4.1-nano gpt-4.1-mini])
+```
+
+Real output from real API calls:
+
+```
+Model                      Score       Cost  Avg Latency
+---------------------------------------------------------
+gpt-4.1-nano                0.67    $0.000032      687ms
+gpt-4.1-mini                1.00    $0.000102     1070ms
+
+Cheapest at 100%: gpt-4.1-mini
+```
+
+```ruby
+comparison.best_for(min_score: 0.95)  # => "gpt-4.1-mini"
+
+# Inspect failures
+comparison.reports["gpt-4.1-nano"].failures.each do |f|
+  puts "#{f.name}: expected #{f.expected}, got #{f.output}"
+  puts "  mismatches: #{f.mismatches}"
+  # => outage: expected {priority: "urgent"}, got {priority: "high"}
+  #      mismatches: {priority: {expected: "urgent", got: "high"}}
+end
+```
+
+## Pipeline
+
+Chain steps with fail-fast. Hallucination in step 1 stops before step 2 spends tokens.
+
+```ruby
+class TicketPipeline < RubyLLM::Contract::Pipeline::Base
+  step ClassifyTicket,  as: :classify
+  step RouteToTeam,     as: :route
+  step DraftResponse,   as: :draft
+end
+
+result = TicketPipeline.run("I was charged twice")
+result.ok?                          # => true
+result.outputs_by_step[:classify]   # => {priority: "high", category: "billing"}
+result.trace.total_cost             # => 0.000128
+```
+
+## CI gate
+
+```ruby
+# RSpec — block merge if accuracy drops or cost spikes
+expect(ClassifyTicket).to pass_eval("regression")
+  .with_context(model: "gpt-4.1-mini")
+  .with_minimum_score(0.8)
+  .with_maximum_cost(0.01)
+
+# Rake — run all evals across all steps
+require "ruby_llm/contract/rake_task"
+RubyLLM::Contract::RakeTask.new do |t|
+  t.minimum_score = 0.8
+  t.maximum_cost = 0.05
+end
+# bundle exec rake ruby_llm_contract:eval
+```
+
+## Predict cost before running
+
+```ruby
+ClassifyTicket.estimate_eval_cost("regression", models: %w[gpt-4.1-nano gpt-4.1-mini])
+# => { "gpt-4.1-nano" => 0.000024, "gpt-4.1-mini" => 0.000096 }
+```
+
+## Install
+
+```ruby
+gem "ruby_llm-contract"
+```
+
+```ruby
+RubyLLM.configure { |c| c.openai_api_key = ENV["OPENAI_API_KEY"] }
+RubyLLM::Contract.configure { |c| c.default_model = "gpt-4.1-mini" }
+```
+
+Works with any ruby_llm provider (OpenAI, Anthropic, Gemini, etc).
+
+## Docs
+
+| Guide | |
+|-------|-|
+| [Getting Started](docs/guide/getting_started.md) | Features walkthrough, model escalation, eval |
+| [Best Practices](docs/guide/best_practices.md) | 6 patterns for bulletproof validates |
+| [Output Schema](docs/guide/output_schema.md) | Full schema reference + constraints |
+| [Pipeline](docs/guide/pipeline.md) | Multi-step composition, timeout, fail-fast |
+| [Testing](docs/guide/testing.md) | Test adapter, RSpec matchers |
+
+## Roadmap
+
+**v0.2 (current):** Model comparison, cost tracking, eval with `add_case`, CI gating, Rails Railtie.
+
+**v0.3:** Regression baselines — compare eval results with previous run, detect quality drift.
+
+**v0.4:** Auto-routing — learn which model works for which input pattern.
+
+## License
+
+MIT

data/Rakefile

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec