squishling 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (71) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +133 -1
  3. data/README.md +63 -15
  4. data/docs/adapters.md +189 -0
  5. data/docs/configuration.md +89 -24
  6. data/docs/coolhand.md +66 -0
  7. data/docs/failures.md +129 -20
  8. data/docs/harnesses.md +186 -48
  9. data/docs/measuring-tokens.md +36 -42
  10. data/docs/observers.md +80 -0
  11. data/docs/routing.md +7 -2
  12. data/docs/schemas.md +33 -6
  13. data/docs/testing.md +183 -0
  14. data/docs/verifying.md +84 -0
  15. data/lib/squishling/adapter/error_body.rb +47 -0
  16. data/lib/squishling/adapter.rb +275 -0
  17. data/lib/squishling/adapters/openai_compatible.rb +304 -0
  18. data/lib/squishling/adapters/openai_options.rb +72 -0
  19. data/lib/squishling/adapters/ruby_llm.rb +193 -0
  20. data/lib/squishling/budget.rb +71 -0
  21. data/lib/squishling/class_methods.rb +43 -7
  22. data/lib/squishling/class_registry.rb +63 -0
  23. data/lib/squishling/configuration.rb +72 -8
  24. data/lib/squishling/definition.rb +95 -17
  25. data/lib/squishling/errors.rb +97 -12
  26. data/lib/squishling/harness/base.rb +136 -0
  27. data/lib/squishling/harness/escalation.rb +22 -0
  28. data/lib/squishling/harness/judged_squishsum.rb +83 -0
  29. data/lib/squishling/harness/option_types.rb +94 -0
  30. data/lib/squishling/harness/registry.rb +41 -0
  31. data/lib/squishling/harness/squishsum.rb +22 -0
  32. data/lib/squishling/harness.rb +25 -99
  33. data/lib/squishling/invoker.rb +28 -233
  34. data/lib/squishling/judge.rb +78 -21
  35. data/lib/squishling/llm_client.rb +73 -109
  36. data/lib/squishling/model_path.rb +117 -27
  37. data/lib/squishling/named_registry.rb +64 -0
  38. data/lib/squishling/observer.rb +109 -0
  39. data/lib/squishling/observers/coolhand.rb +86 -0
  40. data/lib/squishling/output_check.rb +13 -9
  41. data/lib/squishling/params.rb +15 -27
  42. data/lib/squishling/railtie.rb +10 -0
  43. data/lib/squishling/result.rb +37 -16
  44. data/lib/squishling/rotation.rb +31 -0
  45. data/lib/squishling/rspec.rb +18 -0
  46. data/lib/squishling/run/messages.rb +79 -0
  47. data/lib/squishling/run/rejections.rb +43 -0
  48. data/lib/squishling/run.rb +320 -0
  49. data/lib/squishling/schema.rb +10 -9
  50. data/lib/squishling/squawk.rb +33 -3
  51. data/lib/squishling/tasks/squishling.rake +18 -0
  52. data/lib/squishling/testing/adapter.rb +194 -0
  53. data/lib/squishling/testing/checks.rb +39 -0
  54. data/lib/squishling/testing/direct.rb +41 -0
  55. data/lib/squishling/testing/errors.rb +18 -0
  56. data/lib/squishling/testing/fixtures.rb +206 -0
  57. data/lib/squishling/testing/helpers.rb +71 -0
  58. data/lib/squishling/testing/minitest.rb +32 -0
  59. data/lib/squishling/testing/observer.rb +31 -0
  60. data/lib/squishling/testing/reply.rb +60 -0
  61. data/lib/squishling/testing/request.rb +75 -0
  62. data/lib/squishling/testing/rspec.rb +30 -0
  63. data/lib/squishling/testing/script.rb +189 -0
  64. data/lib/squishling/testing/state.rb +73 -0
  65. data/lib/squishling/testing/stubs.rb +112 -0
  66. data/lib/squishling/testing.rb +126 -0
  67. data/lib/squishling/verification.rb +39 -0
  68. data/lib/squishling/verifier.rb +179 -0
  69. data/lib/squishling/version.rb +1 -1
  70. data/lib/squishling.rb +89 -4
  71. metadata +53 -21
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: '0748f552c985521890d365ebc0818ec59c4336f93fb183d1769f49ed8cd32349'
4
- data.tar.gz: '05128ecf8d7156e0c8ca084bf8a1860b754596017df2cdfdcc35e2242138fa7b'
3
+ metadata.gz: 2c50560e08f93a644589fb8ab529a41eca9ae201b81b9dd9233ecc9ea86873c5
4
+ data.tar.gz: f8e2e5bca400137917567449ce299b59a05b40af787631154144f6b986ba8dbb
5
5
  SHA512:
6
- metadata.gz: 2be443b51d2dc1460a410fbf0850bed09e33fff5f7ad894d4bd92e3f47d428d22df7bfc0371036c090a6b806f466b1d930de3a9caf8d1b044185fd4000b765bf
7
- data.tar.gz: 3a18d6e041f5807c966677fea5499de8af5d0f28146cb98bbe205141598ee6a01a2d598b1326df3b78227455dcc124c69cf95afacabf9a1689a1cc382a652414
6
+ metadata.gz: fa16bccb992c1a7b040254e6c909bb1a36588c4e9fa068ce8eb7d6b5bcf4f2e0394429c15b5d4ab7e90ac39f2cf286367eb44475b7abfce9b9a2f177e1388b3f
7
+ data.tar.gz: 97998785808aae5c1182fa6e0780990f512e2ea653856b853e89fbb330e09621f653355aabc6be2e294d2a13c031df5f94143e3bad85fa3f2315609d445bda0d
data/CHANGELOG.md CHANGED
@@ -7,7 +7,139 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
- ## [0.3.0] - 2026-10-09
10
+ ## [0.5.0] - 2026-10-28
11
+
12
+ ### Added
13
+
14
+ - Pluggable adapters: provider requests go through an adapter (`Squishling::Adapter::Base`, a stateless
15
+ `complete(request)`), set with `config.adapter` or registered with `Squishling.register_adapter`. Two ship:
16
+ `:ruby_llm` (the previous behavior) and `:openai_compatible`, plain `Net::HTTP` against any `/chat/completions`
17
+ endpoint with retries and strict `json_schema` output, which keeps keys and provider text out of errors, logs, and
18
+ `inspect`. See [Adapters](docs/adapters.md). (#47)
19
+ - `include_when:` on an escalation step: a Proc called like `squish_when` (the method's inputs as keywords, on the
20
+ instance) that leaves the step out of a call when falsy, for example a model without a BAA. Steps are filtered before
21
+ rotation, and a call left with no steps raises `ConfigurationError`. (#46)
22
+ - `squishling/testing` (opt-in, `require "squishling/testing"`): a scripted adapter that records requests and
23
+ conversations, `assert_no_llm_call`/`forbid_provider_calls!`, schema-validated `stub_result`/`stub_error`,
24
+ `validate`/`fallback_for`/`preview`, schema-driven `valid_reply_for`/`invalid_reply_for`, squawk capture, and
25
+ Minitest and RSpec mixins. See [Testing](docs/testing.md). (#44, #50)
26
+ - `Squishling.verify`/`verify!` check every squished class without a request (output schema and typed result, resolved
27
+ escalation, params and squawk, harness and judge, the keywords `include_when:`/`squish_when` take, and the adapter),
28
+ reporting every problem; `verify!` raises `VerificationError`. Squished classes are tracked in a weak registry
29
+ (`Squishling.squished_classes`). A `squishling:verify` rake task (with a Rails Railtie) and a `be_squishling_valid`
30
+ RSpec matcher (`require "squishling/rspec"`) wrap it. See [Verifying squished classes](docs/verifying.md). (#48, #67)
31
+ - Per-call token budgets: `max_total_output_tokens:` and `max_total_input_tokens:` (on `squishling`, `squish`,
32
+ `squish!`, and `default_max_total_*` in `Squishling.configure`; layered limit by limit, `false` lifts one) cap what one
33
+ call spends across its escalation attempts, squishsum sample rounds, and judge. A request in flight is never cut short,
34
+ so a call can end above the limit by one request. Once a limit is reached the call raises `BudgetExhaustedError`, an
35
+ `InvalidOutputError`, so `squish_fallback` still receives it. (#55, #68)
36
+ - `squawk_on:` (`:all`, `:rejections`, `:retries`, `:final`, or a list) filters which attempts reach the `squawk` hook,
37
+ settable with `config.squawk_on`, `squishling squawk_on:`, and `squish :method, squawk_on:`. The default `:all` keeps
38
+ the old behavior. (#58, #69)
39
+ - `Adapter::Response#finish_reason` (a Symbol such as `:stop` or `:max_tokens`), filled by both built-in adapters.
40
+ `InvalidOutputError` gains `finish_reason` and `truncated?`, with a fixed note in its message and log line when the
41
+ reply was cut off, and `squawk` metadata gains `finish_reason`. (#54, #60)
42
+ - `InvalidOutputError#error_details`: `InvalidOutputError::Detail` entries (`message`, `source` of `:schema`,
43
+ `:validator`, `:parse` or `:squishling`) with a `model_derived?` predicate that is true only for schema errors, which
44
+ can name a key the model chose. Escape just those when forwarding messages to a chat webhook, HTML, or markdown.
45
+ `#errors` is still an Array of Strings (now frozen). (#57, #62)
46
+ - `RequestRejectedError` (a `ConfigurationError`) for a provider's 400, with `provider_status` and `provider_code` read
47
+ as plain tokens from the error body (`INVALID_ARGUMENT`, `invalid_request_error`). The provider's message is still
48
+ never repeated. (#52, #53)
49
+ - `retry_temperature:` on an escalation step sets the temperature for same-chat retries only. (#56, #61)
50
+
51
+ ### Changed
52
+
53
+ - **Breaking change:** `ruby_llm` is no longer a runtime dependency of the gem. Apps that use the RubyLLM adapter (the
54
+ default when no adapter is configured and `ruby_llm` is loadable) must now list `gem "ruby_llm"` in their own
55
+ Gemfile, and `require "ruby_llm"` before `RubyLLM.configure` if they don't use `Bundler.require`; or switch to
56
+ `config.adapter = :openai_compatible`. (#47)
57
+ - **Behavior change:** a schema property named like a method the typed result needs (`class`, `hash`, `to_h`,
58
+ `members`, `inspect`, `is_a?`, ...) now raises `ConfigurationError` when the schema is built, before any request,
59
+ instead of replacing that method on the result class and failing late. Rename such fields (`class` to
60
+ `ticket_class`); the reserved names are listed in [Output schemas](docs/schemas.md). (#43, #49)
61
+ - **Behavior change:** when a same-chat retry returns the exact reply it was just rejected for, Squishling skips the
62
+ step's remaining attempts and moves to the next step (or raises `InvalidOutputError` right away on the last step);
63
+ samples end early the same way. `InvalidOutputError#attempts` counts requests actually made, `squawk` metadata gains
64
+ `repeated:`, and `final:` now means no attempt follows a rejection. Size a step's `attempts:` for replies that can
65
+ differ. (#56, #61)
66
+ - A provider 400's "Check params" hint now names a param only when the provider's error body names one you configured,
67
+ and otherwise says the params, output schema, or input may be invalid. An array-wrapped overlong-context 400 from the
68
+ OpenAI-compatible adapter is now an `LLMError`. (#53)
69
+
70
+ ### Fixed
71
+
72
+ - A raw Hash `output_schema` that omits `type: "object"` no longer lets a non-object reply (a String, Array, or number)
73
+ reach the typed result as a `NoMethodError`; it is now invalid output, retried, escalated, and sent to
74
+ `squish_fallback` like any other. Schemas built with Schematist always had `type: "object"`.
75
+ - `RequestRejectedError`'s `provider_status` and `provider_code` no longer accept a hyphenated token, so a key-shaped
76
+ string a gateway echoes into an error body is never repeated in our message. The RubyLLM adapter's "invalid params"
77
+ `ConfigurationError` now names the param keys, not their values.
78
+
79
+ ### Internal
80
+
81
+ - Development dependencies: `ruby_llm` 2.1.1, `coolhand` 0.10.0, and `rubocop` 1.92.1; CI pins `ruby/setup-ruby` v1.335.0
82
+ so the matrix resolves Ruby 3.3.13, 3.4.12, and 4.0.9. A `finish-release` skill now tags and publishes a reviewed
83
+ release-prep PR. (#45, #51, #59, #63, #64, #65)
84
+
85
+ ## [0.4.0] - 2026-10-18
86
+
87
+ ### Added
88
+
89
+ - Custom harnesses: `Squishling.register_harness` (with `Squishling.harnesses` and `reset_harnesses!`) and
90
+ `Squishling::Harness::Base` let you write your own harness and use it anywhere a built-in works (`harness:`,
91
+ `squish!`, `default_harness`). A harness gets a `Squishling::Run` with `escalate`, `samples`, `judge`, `agree?`,
92
+ `stage`, and `log`, so it keeps model, provider, and params resolution, `squish_validate`, concurrent samples, and the
93
+ normal failure handling; it can only return typed results its `Run` produced, and its options are validated when it
94
+ is declared. The built-in `:escalation`, `:squishsum`, and `:judged_squishsum` now run on the same interface.
95
+ `squawk` metadata gains `role:` and `sample:`. See [Custom harnesses](docs/harnesses.md#custom-harnesses). (#31)
96
+ - Round-robin models: escalation steps that share a `priority:` form a tier. Each call rotates which model in the tier
97
+ goes first (counted per class and method, shared across threads), and the tier's other models are tried before the
98
+ next priority. `rotate: false` on a step pins its tier to the order written. (#38)
99
+ - `config.observer`: an around-style hook (`around(call) { ... }`) wrapped around every provider request, on the thread
100
+ that makes it, so a `:squishsum` sample's observer runs on its worker thread. It is told the request's `Observer::Call`
101
+ and gets back only an `Observer::Usage` (token counts and model), never the response. An observer that yields twice
102
+ or never raises `ConfigurationError`, and one that rescues the request's error has it raised again. See
103
+ [Observers](docs/observers.md). (#25, #40)
104
+ - A bundled Coolhand observer (`observer = :coolhand`) that declares each call's workload (`Class#method`) and template
105
+ (`squish:L<n>`, `sample:a`, or `judge`) through `Coolhand.with_context`. The workload's description is
106
+ `Class#method`, or `Class#method (rescue)` when a `squish!` call handed the call to the LLM (`Observer::Call#handed_off`),
107
+ never the purpose text. It needs coolhand 0.9.0 or newer. See
108
+ [Coolhand Labs](docs/coolhand.md). (#40)
109
+ - A docs page and landing-page section on the architecture, an explainer site in `site/` deployed to GitHub Pages, and
110
+ per-field `description:` documented as the best practice over `purpose` guidance. (#30, #33, #34, #36)
111
+
112
+ ### Changed
113
+
114
+ - **Behavior change:** when the `coolhand` gem is loaded and configured (an API key, not disabled), Squishling now
115
+ declares each request's workload and template to it automatically. Nothing changes for apps without Coolhand. To
116
+ opt out, set `config.observer = false`.
117
+ - **Behavior change:** two escalation steps may now share a `priority:` (they used to raise `duplicate order:`), and
118
+ they rotate per call as described above. If you relied on a fixed order between tied steps, give them distinct
119
+ priorities or set `rotate: false`.
120
+ - `order:` on an escalation step is deprecated in favor of `priority:`. It still works, with a deprecation warning, and
121
+ giving both on one step raises `ConfigurationError`. Judge and harness steps take no `priority:` or `rotate:`. (#38)
122
+ - **Behavior change:** `LLMError` and the `ConfigurationError` for a rejected request (400), bad credentials, or a
123
+ forbidden key no longer repeat the provider's own message, which can echo request content or part of an API key into
124
+ logs and error trackers. They carry the error class and HTTP status (`RubyLLM::RateLimitError, HTTP 429`); the
125
+ provider's message is still on `cause`. If you matched on provider text in `e.message`, read `e.cause.message`
126
+ instead.
127
+ - `docs/measuring-tokens.md` now points to observers for bringing your own metrics instead of walking through a
128
+ hand-rolled one. (#40)
129
+
130
+ ### Removed
131
+
132
+ - **Breaking change:** the `:ensemble` and `:judged_ensemble` harnesses added in 0.3.0 are gone. `harness: :ensemble`
133
+ or `:judged_ensemble` (and `default_harness`) now raise `ConfigurationError` (unknown harness). Use `:squishsum` or
134
+ `:judged_squishsum`, which sample the first escalation step twice with the judge defaulting to the next step, or
135
+ write a custom harness. (#22, #24)
136
+
137
+ ### Internal
138
+
139
+ - CI gains an exhale duplication and complexity gate (`bin/exhale`), and dependabot bumps `rubocop` (1.92), `ruby/setup-ruby`,
140
+ and `actions/upload-artifact`. These are development-only changes. (#27, #28, #29, #41)
141
+
142
+ ## [0.3.0] - 2026-10-10
11
143
 
12
144
  ### Added
13
145
 
data/README.md CHANGED
@@ -2,6 +2,10 @@
2
2
  <img src="assets/squishling-logo.png" alt="Squishling" width="420">
3
3
  </h1>
4
4
 
5
+ <p align="center">
6
+ <strong><a href="https://coolhand-labs.github.io/squishling/">📖 Read the Squishling explainer →</a></strong>
7
+ </p>
8
+
5
9
  [![CI](https://github.com/Coolhand-Labs/squishling/actions/workflows/ci.yml/badge.svg)](https://github.com/Coolhand-Labs/squishling/actions/workflows/ci.yml)
6
10
  [![Gem Version](https://badge.fury.io/rb/squishling.svg)](https://badge.fury.io/rb/squishling)
7
11
 
@@ -14,7 +18,17 @@ Either way it returns the same strict-schema-validated, typed result, so callers
14
18
  Think of code as a cache for AI judgment. Every line you write is something to maintain, so the LLM is the
15
19
  default and code has to earn its place: you build it for the inputs that carry the volume, and the LLM keeps
16
20
  handling the rest. New to the idea? Read
17
- [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai).
21
+ [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai) or browse the
22
+ [talk slides](https://michael.carroll.io/talks/2026/elastic-software/).
23
+
24
+ Or try it on the [explainer site](https://coolhand-labs.github.io/squishling/):
25
+
26
+ - [Spectrum slider](https://coolhand-labs.github.io/squishling/#spectrum): drag from traditional software to fully
27
+ LLM and see the tradeoff at each point
28
+ - [Token-cost calculator](https://coolhand-labs.github.io/squishling/#number): price your tech debt in tokens
29
+ against code upkeep
30
+ - [Walkthrough](https://coolhand-labs.github.io/squishling/#squishling): describe it, tune it, pay for accuracy,
31
+ harden the common shapes
18
32
 
19
33
  ## Use cases
20
34
 
@@ -81,6 +95,12 @@ Both calls return the same result class. If the LLM can't deliver either, `squis
81
95
  return, or the error is raised with the original `ParseError` as its cause. See
82
96
  [Handing off to the LLM](docs/routing.md#handing-off-to-the-llm-with-squish).
83
97
 
98
+ ### Retire code you no longer need
99
+
100
+ Go the other way too. Find the parsers and handlers that serve a thin tail of your traffic and squish them. When a
101
+ path's monthly token cost is lower than what its code costs to maintain, delete the code and relish the feeling of a
102
+ slimmer code base (or let your AI coding agent relish it).
103
+
84
104
  ### Measure your tech debt in tokens
85
105
 
86
106
  Code you haven't written is debt you haven't taken on. A squished path has a running cost you can read off a
@@ -110,12 +130,17 @@ end
110
130
  gem "squishling"
111
131
  ```
112
132
 
113
- Requires Ruby 3.3+ and RubyLLM 2.1+. Configure your provider API keys in RubyLLM as usual, then optionally set a
114
- universal model, or an escalation of models to try in order:
133
+ Requires Ruby 3.3+ and an [adapter](docs/adapters.md) to reach a model: the `ruby_llm` gem (2.1+; Squishling uses it
134
+ automatically when it is in your bundle), or any OpenAI-compatible endpoint, which needs no extra gem. Squishling no
135
+ longer installs or requires `ruby_llm` for you, so add `gem "ruby_llm"` next to it if you use RubyLLM (and
136
+ `require "ruby_llm"` before `RubyLLM.configure` if your app doesn't load its Gemfile with `Bundler.require`). With RubyLLM,
137
+ configure your provider API keys there as usual. Then choose an adapter, and optionally set a universal model, or an
138
+ escalation of models to try in order:
115
139
 
116
140
  ```ruby
117
141
  Squishling.configure do |config|
118
- config.default_model = "claude-sonnet-5-5" # falls back to RubyLLM's default when nil
142
+ config.adapter = :ruby_llm # or { type: :openai_compatible, base_url: "http://localhost:11434/v1" }
143
+ config.default_model = "claude-sonnet-5-5" # falls back to the adapter's default when nil
119
144
  # or: config.default_escalation = [{ model: "claude-haiku-4-5", attempts: 2 }, "claude-sonnet-5-5", "claude-opus-5-5"]
120
145
  end
121
146
  ```
@@ -131,17 +156,25 @@ validated on both paths). A squished call goes to the LLM when:
131
156
 
132
157
  Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM output.
133
158
 
159
+ **Best practice: describe fields in the schema, not the purpose.** Put per-field guidance in a `description:` on
160
+ the field (`string :status, description: "..."`, or `optional(:eta, description: "...") { string }`). The model
161
+ sees it right where the value goes, and the purpose stays short and about the goal. See
162
+ [Describing fields](docs/schemas.md#describing-fields).
163
+
134
164
  ## Features
135
165
 
136
166
  - **One contract, two paths**: Ruby returns and LLM output are validated against the same strict schema and
137
167
  returned as the same typed `Data` objects. `squished?` tells you which path served a call.
138
168
  - **Your code as context**: `append_to_purpose` adds sections to the prompt, including a class's or method's
139
169
  own Ruby source.
140
- - **Any RubyLLM provider and model**: OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, OpenRouter, and more.
141
- Set a universal, per-class, per-method, or per-call model, plus layered generation params (temperature,
170
+ - **[Bring your own adapter](docs/adapters.md)**: Squishling doesn't depend on any one LLM client. Use the
171
+ [RubyLLM](https://rubyllm.com) adapter (OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, OpenRouter, and more),
172
+ the OpenAI-compatible adapter (OpenAI, Ollama, vLLM, LM Studio, Together, Groq, … over plain HTTP), or write your
173
+ own in one method.
174
+ - **Any provider and model**: set a universal, per-class, per-method, or per-call model, plus layered generation params (temperature,
142
175
  reasoning effort, top_p, …).
143
- - **Model escalation**: declare an `escalation:` instead of a `model:`, with per-step `attempts:`, `order:`, provider,
144
- params, and `forward_rejected:`. Invalid output (with its errors fed back) or a provider failure moves on to the next attempt. Steps can
176
+ - **Model escalation**: declare an `escalation:` instead of a `model:`, with per-step `attempts:`, `priority:` (models tied on priority take turns going first), provider,
177
+ params, `forward_rejected:`, `retry_temperature:`, and `include_when:` (leave a model out of calls where a Proc is falsy). Invalid output (with its errors fed back) or a provider failure moves on to the next attempt. Steps can
145
178
  cross providers, e.g. a local Qwen model on Ollama, then Anthropic Claude Haiku on AWS Bedrock, then Claude Opus:
146
179
 
147
180
  ```ruby
@@ -168,13 +201,17 @@ Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM
168
201
  - [`:escalation`](docs/configuration.md#models-and-escalation) (default): try each attempt until one passes
169
202
  - [`:squishsum`](docs/harnesses.md#samples): two concurrent samples, accepted only if they agree
170
203
  - [`:judged_squishsum`](docs/harnesses.md#the-judge): a chat or Jev judge picks between disagreeing samples
171
- - [`:ensemble` and `:judged_ensemble`](docs/harnesses.md#ensembles): the same checks across two different models
204
+ - [Your own](docs/harnesses.md#custom-harnesses): `Squishling.register_harness` plugs a custom harness into the same
205
+ machinery (samples, judges, `squawk`, `squish_fallback`)
206
+ - **Cost caps**: `max_total_output_tokens:` and `max_total_input_tokens:` stop a call that keeps failing from
207
+ spending without limit (see [Capping a call's cost](docs/failures.md#capping-a-calls-cost-token-budgets)).
172
208
  - **Output contracts**: beyond the strict schema, conditional rules (`given`) and Ruby checks (`squish_validate`)
173
209
  reject bad output and trigger the next attempt.
174
210
  - **Defined failure behavior**: provider errors become `Squishling::LLMError`, and `squish_fallback` lets you decide
175
211
  what to return when every attempt fails.
176
- - **Observability**: the opt-in [`squawk`](docs/failures.md#observing-every-attempt-squawk) hook sends every attempt's
177
- raw output to your error tracker. Model output stays out of error messages and logs otherwise.
212
+ - **Observability**: the opt-in [`squawk`](docs/failures.md#observing-every-attempt-squawk) hook sends each attempt's
213
+ raw output (or only the rejected and retried ones) to your error tracker. Model output stays out of
214
+ error messages and logs otherwise.
178
215
  - **Opt-in context**: only method arguments, the instance state you name with `squish_context`, the `context:` you
179
216
  pass to `squish!`, and the source you choose to append are sent to the provider. The one addition: when
180
217
  [escalation](docs/configuration.md#models-and-escalation) moves to the next step, that step also sees the previous
@@ -182,29 +219,40 @@ Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM
182
219
 
183
220
  ## Documentation
184
221
 
222
+ - [Explainer site](https://coolhand-labs.github.io/squishling/): interactive walkthrough, spectrum slider, and
223
+ token-cost calculator
185
224
  - [Configuration](docs/configuration.md): options, models and escalation, providers, generation params, inheritance
225
+ - [Adapters](docs/adapters.md): the RubyLLM and OpenAI-compatible adapters, and writing your own
186
226
  - [Routing](docs/routing.md): when a call goes to the LLM, `squish!`, `append_to_purpose`, hardening a path,
187
227
  what the LLM sees
188
228
  - [Output schemas](docs/schemas.md): schema forms, strict mode, typed results, optional vs. empty, contracts
189
229
  - [Failure handling](docs/failures.md): escalation, `squish_validate`, error classes, fallbacks
190
- - [Harnesses](docs/harnesses.md): escalation, squishsum, ensemble, and judges (chat or Jev)
230
+ - [Harnesses](docs/harnesses.md): escalation, squishsum, judges (chat or Jev), and custom harnesses
231
+ - [Verifying squished classes](docs/verifying.md): `Squishling.verify!`, a rake task, and an RSpec matcher that check
232
+ every class ahead of runtime, without a request
191
233
  - [Naming and collisions](docs/naming.md): the methods `include Squishling` adds and what happens when a name is taken
192
234
  - [Measuring token spend](docs/measuring-tokens.md): see what each squished path costs, with Coolhand Labs,
193
- the `squawk` hook, OpenTelemetry, LangSmith, or RubyLLM's instrumenter
235
+ the `squawk` hook, your own observer, or OpenTelemetry
236
+ - [Observers](docs/observers.md): wrap every provider request, and the built-in [Coolhand Labs](docs/coolhand.md) observer
237
+ that files each call under its workload and template
238
+ - [Testing](docs/testing.md): `require "squishling/testing"`: a scripted adapter, call assertions, result and error
239
+ stubs, and schema-driven fixtures, for Minitest and RSpec
194
240
  - [Live examples](examples/README.md): end-to-end tests against Anthropic Claude Haiku and OpenAI GPT-6 Luna
195
241
 
196
242
  ## Development
197
243
 
198
244
  ```sh
199
245
  bin/setup
200
- bundle exec rake # RSpec (offline; RubyLLM is stubbed) + RuboCop
246
+ bundle exec rake # RSpec (offline; adapters are faked or stubbed) + RuboCop
201
247
  ```
202
248
 
203
249
  See [AGENTS.md](AGENTS.md) for repo conventions, and [SECURITY.md](SECURITY.md) for reporting vulnerabilities.
250
+ Have a suggestion to improve Squishling? [Open an issue](https://github.com/Coolhand-Labs/squishling/issues).
204
251
 
205
252
  ## Credits
206
253
 
207
- Inspired by [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai).
254
+ Inspired by [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai); see also
255
+ the [talk slides](https://michael.carroll.io/talks/2026/elastic-software/) from XO Ruby NYC 2026.
208
256
 
209
257
  ## License
210
258
 
data/docs/adapters.md ADDED
@@ -0,0 +1,189 @@
1
+ # Adapters
2
+
3
+ An **adapter** is how Squishling reaches a model. Squishling builds the request (system prompt, conversation so far,
4
+ strict output schema, generation params), the adapter makes one completion and returns the model's output, and
5
+ Squishling does the rest: validation, `squish_validate`, retries and escalation, harnesses, `squawk`, and fallbacks.
6
+ Two adapters ship, and you can write your own.
7
+
8
+ | Adapter | Reaches | Needs |
9
+ |---|---|---|
10
+ | [`:ruby_llm`](#ruby_llm) | OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, OpenRouter, and every provider [RubyLLM](https://rubyllm.com) supports | `gem "ruby_llm"` |
11
+ | [`:openai_compatible`](#openai-compatible-endpoints) | Any server that speaks OpenAI's Chat Completions API: OpenAI, Ollama, vLLM, LM Studio, llama.cpp, Together, Groq, … | nothing (Ruby's `Net::HTTP`) |
12
+ | [Your own](#writing-your-own) | Anything | a class with one method |
13
+
14
+ Squishling itself depends on neither RubyLLM nor any provider SDK.
15
+
16
+ ## Choosing one
17
+
18
+ ```ruby
19
+ Squishling.configure do |config|
20
+ config.adapter = :ruby_llm # a built-in or registered adapter, by name
21
+ # config.adapter = { type: :openai_compatible, base_url: "http://localhost:11434/v1", default_model: "llama3.2" }
22
+ # config.adapter = MyAdapter.new # or any adapter instance
23
+ end
24
+ ```
25
+
26
+ An adapter is built when you assign it, so a missing gem or a bad option fails at boot, not on the first request.
27
+ There is one adapter per process; every squishling class, method, and escalation step goes through it.
28
+
29
+ **When you set none**, Squishling uses the RubyLLM adapter if the `ruby_llm` gem is in your bundle, so apps that
30
+ configured RubyLLM before adapters existed keep working. Squishling no longer requires `ruby_llm` when it loads, so an
31
+ app that doesn't load its Gemfile with `Bundler.require` should `require "ruby_llm"` before `RubyLLM.configure`. Without the gem, the first squished call
32
+ raises `Squishling::ConfigurationError` saying what to add.
33
+
34
+ ## `:ruby_llm`
35
+
36
+ Everything works as it did before adapters: configure provider API keys with `RubyLLM.configure`, name models, and
37
+ use `provider:` for models missing from RubyLLM's registry. See [Configuration](configuration.md). Transport-level
38
+ retries are RubyLLM's (`RubyLLM.config.max_retries`, `request_timeout`).
39
+
40
+ This is the only built-in adapter that supports `type: :judgment` judges ([System One decision models](harnesses.md#the-judge)).
41
+
42
+ ## OpenAI-compatible endpoints
43
+
44
+ ```ruby
45
+ Squishling.configure do |config|
46
+ config.adapter = {
47
+ type: :openai_compatible,
48
+ base_url: "http://localhost:11434/v1", # where /chat/completions lives
49
+ api_key: ENV["LLM_API_KEY"], # optional for local servers
50
+ default_model: "llama3.2" # for steps that name no model
51
+ }
52
+ config.default_model = "llama3.2"
53
+ end
54
+ ```
55
+
56
+ | Option | Default | Description |
57
+ |---|---|---|
58
+ | `base_url` | `https://api.openai.com/v1` | The API root. Requests go to `{base_url}/chat/completions`. Must be `http(s)` and carry no credentials, query, or fragment. |
59
+ | `api_key` | `ENV["OPENAI_API_KEY"]` for OpenAI's own endpoint, else none | A String, or a callable returning one (called once per completion, outside the adapter's retry handling, so rotating keys work and its exceptions reach you unchanged). Sent as a bearer token; omitted when blank. The key is sent as-is, so use `https://` for anything off your machine or trusted network. `OPENAI_API_KEY` is read only for OpenAI's own `https://api.openai.com`, so it can never go to another host: pass `api_key:` for any other endpoint that needs one. |
60
+ | `headers` | `{}` | Extra request headers, e.g. `{ "OpenAI-Organization" => "org-…" }` or a gateway's own. |
61
+ | `default_model` | `nil` | The model for steps that don't name one. Without it, a step with no model raises `ConfigurationError`. |
62
+ | `open_timeout` / `read_timeout` | `10` / `120` | Seconds. |
63
+ | `max_retries` | `2` | Timeouts, connection and protocol failures, HTTP 408, 429, and 5xx are retried this many times with exponential backoff (honoring `Retry-After`, up to 30 seconds) before they surface as an `LLMError`. A timed-out request may still have been processed by the server, so a retry can be billed twice; set `max_retries: 0` if that matters more than resilience. |
64
+ | `max_tokens_param` | `:max_completion_tokens` | The body key for `max_output_tokens`. Use `:max_tokens` for servers that don't know the newer name. |
65
+ | `structured_output` | `:json_schema` | `:json_schema` asks for strict structured output. For servers without it use `:json_object`, or `:none`: the schema then goes into the system prompt in words. Squishling validates every answer itself either way. |
66
+
67
+ **How params map.** `temperature` → `temperature`; `max_output_tokens` → `max_tokens_param`; `thinking: { effort: }` →
68
+ `reasoning_effort` (`true`/`false` leave the model's default, and a `budget:` or `display:` raises
69
+ `ConfigurationError`, since Chat Completions has no field for them). Any other param (`top_p`, `seed`, …) goes into
70
+ the request body as given. Squishling's own keys (`model`, `messages`, `response_format`, `n`, `stream`) always win.
71
+ `provider:` has no meaning here and is ignored.
72
+
73
+ **Failures.** Every HTTP status of 300 or above except 408, 429, and 5xx (401, 403, 404, 400, 422, a redirect, …) raises `ConfigurationError`, a `RequestRejectedError` for a 400 or other rejected request (never retried or escalated; a 400 that
74
+ reports an overlong context, and a 413, are `LLMError`s, since they depend on the input; a 400 counts as an overlong
75
+ context when its error code is `context_length_exceeded` or its message uses the usual wording ("context length", "too many
76
+ tokens", …), which the message is only matched against, never repeated). 408, 429, 5xx, timeouts, and
77
+ connection errors raise `LLMError` once the adapter's own retries are used up. Messages carry the status or error
78
+ class only; a rejected request (400) also carries the server's error status and code when they are plain tokens (as
79
+ `provider_status` and `provider_code`). The server's message, the response body, and the API key never appear in an error or log. Redirects
80
+ aren't followed, so the key can't be sent to another host.
81
+
82
+ ## Writing your own
83
+
84
+ Subclass `Squishling::Adapter::Base`, implement `complete`, and return a `Squishling::Adapter::Response`:
85
+
86
+ ```ruby
87
+ class AcmeAdapter < Squishling::Adapter::Base
88
+ def initialize(client: Acme::Client.new)
89
+ super()
90
+ @client = client
91
+ end
92
+
93
+ def complete(request)
94
+ reply = @client.chat(
95
+ model: request.model,
96
+ system: request.instructions,
97
+ messages: request.messages, # [{ role: :user | :assistant, content: "…" }, …]
98
+ json_schema: request.schema["schema"], # strict; Squishling validates the answer regardless
99
+ **request.params
100
+ )
101
+ Squishling::Adapter::Response.new(
102
+ content: reply.text, # the raw String (or a Hash you already parsed)
103
+ model: reply.model,
104
+ tokens: { input_tokens: reply.usage.in, output_tokens: reply.usage.out }
105
+ )
106
+ rescue Acme::AuthError
107
+ raise configuration_error(request, "Acme rejected the credentials")
108
+ rescue Acme::BadRequest => e
109
+ # a 400: repeats only the body's status and code, and points at a param only if the provider named one
110
+ raise rejected_request_error(request, via: "Acme, HTTP 400", body: e.response_body, http_status: 400)
111
+ rescue Acme::Error => e
112
+ raise llm_error(request, "Acme failed (#{e.class})")
113
+ end
114
+ end
115
+
116
+ Squishling.configure { |config| config.adapter = AcmeAdapter.new }
117
+ # or register it by name, to use config.adapter = :acme (or { type: :acme, client: … }):
118
+ Squishling.register_adapter(:acme, AcmeAdapter)
119
+ ```
120
+
121
+ ### The request
122
+
123
+ `Squishling::Adapter::Request` carries everything for one completion:
124
+
125
+ | Field | |
126
+ |---|---|
127
+ | `label` | `"Class#method"`, to prefix your error messages (the `configuration_error` / `llm_error` helpers do; `rejected_request_error(request, via:, body:, http_status:)` builds the error for a rejected request, from a JSON String or String-keyed Hash body). |
128
+ | `model`, `provider` | The escalation step's, or `nil` when it names none (use your own default). What `provider:` means is up to you. |
129
+ | `instructions` | The system prompt. |
130
+ | `messages` | The conversation so far, ending with the user message to answer. After invalid output, Squishling appends your previous answer and a retry message; you never keep history. |
131
+ | `schema` | `{ "name", "description", "schema", "strict" => true }`: the strict structured-output payload, without the conditional keywords providers can't take. |
132
+ | `params` | The resolved [generation params](configuration.md#generation-params): the portable `:temperature`, `:max_output_tokens`, `:thinking`, plus provider-specific keys. Squishling has already rejected the keys it owns. Translate the portable ones for your provider; pass the rest through if it takes them. |
133
+
134
+ ### The response
135
+
136
+ `Squishling::Adapter::Response.new(content:, model:, tokens:, finish_reason:)`. `content` is the model's output as a
137
+ String, or a Hash if you parsed it; `nil` or empty counts as invalid output and is retried. `model` is the id the
138
+ provider reports. `tokens` is a Hash (`:input_tokens`, `:output_tokens`, and anything else you have); `{}` when
139
+ unknown. These feed [`squawk`'s `usage`](failures.md#observing-every-attempt-squawk), the [observer](observers.md)'s `Usage`, and the
140
+ [token budget](failures.md#capping-a-calls-cost-token-budgets), which counts `:input_tokens` and `:output_tokens`
141
+ (a response with none never exhausts a budget).
142
+ `finish_reason` is why the provider stopped, as a Symbol: `:stop`, `:max_tokens` (the output cap cut the reply off),
143
+ `:content_filter`, `:tool_calls`, or the provider's own spelling; leave it out (`nil`) when unknown. Map your
144
+ provider's "length" reason to `:max_tokens`, so Squishling can [tell a cutoff from a wrong
145
+ answer](failures.md#a-cutoff-or-a-wrong-answer).
146
+
147
+ ### The error contract
148
+
149
+ | Raise | When | What Squishling does |
150
+ |---|---|---|
151
+ | `Squishling::ConfigurationError` | Bad credentials, a request the provider rejects, an unknown model: anything a retry can't fix | Raised at once. Never retried, escalated, or sent to `squish_fallback`. |
152
+ | `Squishling::LLMError` | Rate limit, 5xx, timeout, connection failure, an overlong context: anything worth another go | Moves to the next attempt; raised from the last one (and goes to `squish_fallback`). |
153
+
154
+ Keep the provider's own message out of both: it can echo request content or part of a key, and ours reach logs and
155
+ error trackers. Put the original on `cause` (raising inside a `rescue` does that for you). Any other exception
156
+ propagates unwrapped and is never retried, so map your transport's errors. Return anything but an
157
+ `Adapter::Response` and Squishling raises `ConfigurationError`.
158
+
159
+ ### Optional hooks
160
+
161
+ - **`judgments?` and `judge(input, questions:, step:, label:)`**: System One decision models for `type: :judgment` judges.
162
+ `judge` returns an object answering `[:winner]` with something that responds to `choice` and `probabilities`.
163
+ Without `judgments?`, such a judge raises `ConfigurationError` before any request. If the object has `tokens` (with
164
+ `input` and `output` readers), they count toward the call's token budget.
165
+ - **`capture_context` and `with_context(context) { … }`**: samples (the `:squishsum` harnesses) send their requests from
166
+ worker threads. `capture_context` runs on the caller's thread and returns whatever your tracing needs; `with_context`
167
+ re-establishes it around each worker's request. The RubyLLM adapter uses this for its instrumentation workflow.
168
+
169
+ ### Thread safety
170
+
171
+ Adapters are called concurrently from worker threads when a harness takes samples. Keep no unsynchronized mutable
172
+ state (a per-request HTTP connection, as the OpenAI-compatible adapter does, is the simple way).
173
+
174
+ ### Testing against your adapter
175
+
176
+ To test your *application* (not an adapter), use [`squishling/testing`](testing.md): a scripted adapter with request
177
+ inspection, call assertions, and stubs. To unit-test an adapter you wrote, a canned one is enough:
178
+
179
+ ```ruby
180
+ class CannedAdapter < Squishling::Adapter::Base
181
+ def initialize(*replies) = (@replies = replies)
182
+
183
+ def complete(request)
184
+ Squishling::Adapter::Response.new(content: @replies.shift, model: request.model, tokens: {})
185
+ end
186
+ end
187
+
188
+ Squishling.configure { |config| config.adapter = CannedAdapter.new({ "label" => "bug" }) }
189
+ ```