squishling 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +133 -1
- data/README.md +63 -15
- data/docs/adapters.md +189 -0
- data/docs/configuration.md +89 -24
- data/docs/coolhand.md +66 -0
- data/docs/failures.md +129 -20
- data/docs/harnesses.md +186 -48
- data/docs/measuring-tokens.md +36 -42
- data/docs/observers.md +80 -0
- data/docs/routing.md +7 -2
- data/docs/schemas.md +33 -6
- data/docs/testing.md +183 -0
- data/docs/verifying.md +84 -0
- data/lib/squishling/adapter/error_body.rb +47 -0
- data/lib/squishling/adapter.rb +275 -0
- data/lib/squishling/adapters/openai_compatible.rb +304 -0
- data/lib/squishling/adapters/openai_options.rb +72 -0
- data/lib/squishling/adapters/ruby_llm.rb +193 -0
- data/lib/squishling/budget.rb +71 -0
- data/lib/squishling/class_methods.rb +43 -7
- data/lib/squishling/class_registry.rb +63 -0
- data/lib/squishling/configuration.rb +72 -8
- data/lib/squishling/definition.rb +95 -17
- data/lib/squishling/errors.rb +97 -12
- data/lib/squishling/harness/base.rb +136 -0
- data/lib/squishling/harness/escalation.rb +22 -0
- data/lib/squishling/harness/judged_squishsum.rb +83 -0
- data/lib/squishling/harness/option_types.rb +94 -0
- data/lib/squishling/harness/registry.rb +41 -0
- data/lib/squishling/harness/squishsum.rb +22 -0
- data/lib/squishling/harness.rb +25 -99
- data/lib/squishling/invoker.rb +28 -233
- data/lib/squishling/judge.rb +78 -21
- data/lib/squishling/llm_client.rb +73 -109
- data/lib/squishling/model_path.rb +117 -27
- data/lib/squishling/named_registry.rb +64 -0
- data/lib/squishling/observer.rb +109 -0
- data/lib/squishling/observers/coolhand.rb +86 -0
- data/lib/squishling/output_check.rb +13 -9
- data/lib/squishling/params.rb +15 -27
- data/lib/squishling/railtie.rb +10 -0
- data/lib/squishling/result.rb +37 -16
- data/lib/squishling/rotation.rb +31 -0
- data/lib/squishling/rspec.rb +18 -0
- data/lib/squishling/run/messages.rb +79 -0
- data/lib/squishling/run/rejections.rb +43 -0
- data/lib/squishling/run.rb +320 -0
- data/lib/squishling/schema.rb +10 -9
- data/lib/squishling/squawk.rb +33 -3
- data/lib/squishling/tasks/squishling.rake +18 -0
- data/lib/squishling/testing/adapter.rb +194 -0
- data/lib/squishling/testing/checks.rb +39 -0
- data/lib/squishling/testing/direct.rb +41 -0
- data/lib/squishling/testing/errors.rb +18 -0
- data/lib/squishling/testing/fixtures.rb +206 -0
- data/lib/squishling/testing/helpers.rb +71 -0
- data/lib/squishling/testing/minitest.rb +32 -0
- data/lib/squishling/testing/observer.rb +31 -0
- data/lib/squishling/testing/reply.rb +60 -0
- data/lib/squishling/testing/request.rb +75 -0
- data/lib/squishling/testing/rspec.rb +30 -0
- data/lib/squishling/testing/script.rb +189 -0
- data/lib/squishling/testing/state.rb +73 -0
- data/lib/squishling/testing/stubs.rb +112 -0
- data/lib/squishling/testing.rb +126 -0
- data/lib/squishling/verification.rb +39 -0
- data/lib/squishling/verifier.rb +179 -0
- data/lib/squishling/version.rb +1 -1
- data/lib/squishling.rb +89 -4
- metadata +53 -21
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2c50560e08f93a644589fb8ab529a41eca9ae201b81b9dd9233ecc9ea86873c5
|
|
4
|
+
data.tar.gz: f8e2e5bca400137917567449ce299b59a05b40af787631154144f6b986ba8dbb
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: fa16bccb992c1a7b040254e6c909bb1a36588c4e9fa068ce8eb7d6b5bcf4f2e0394429c15b5d4ab7e90ac39f2cf286367eb44475b7abfce9b9a2f177e1388b3f
|
|
7
|
+
data.tar.gz: 97998785808aae5c1182fa6e0780990f512e2ea653856b853e89fbb330e09621f653355aabc6be2e294d2a13c031df5f94143e3bad85fa3f2315609d445bda0d
|
data/CHANGELOG.md
CHANGED
|
@@ -7,7 +7,139 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
-
## [0.
|
|
10
|
+
## [0.5.0] - 2026-10-28
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- Pluggable adapters: provider requests go through an adapter (`Squishling::Adapter::Base`, a stateless
|
|
15
|
+
`complete(request)`), set with `config.adapter` or registered with `Squishling.register_adapter`. Two ship:
|
|
16
|
+
`:ruby_llm` (the previous behavior) and `:openai_compatible`, plain `Net::HTTP` against any `/chat/completions`
|
|
17
|
+
endpoint with retries and strict `json_schema` output, which keeps keys and provider text out of errors, logs, and
|
|
18
|
+
`inspect`. See [Adapters](docs/adapters.md). (#47)
|
|
19
|
+
- `include_when:` on an escalation step: a Proc called like `squish_when` (the method's inputs as keywords, on the
|
|
20
|
+
instance) that leaves the step out of a call when falsy, for example a model without a BAA. Steps are filtered before
|
|
21
|
+
rotation, and a call left with no steps raises `ConfigurationError`. (#46)
|
|
22
|
+
- `squishling/testing` (opt-in, `require "squishling/testing"`): a scripted adapter that records requests and
|
|
23
|
+
conversations, `assert_no_llm_call`/`forbid_provider_calls!`, schema-validated `stub_result`/`stub_error`,
|
|
24
|
+
`validate`/`fallback_for`/`preview`, schema-driven `valid_reply_for`/`invalid_reply_for`, squawk capture, and
|
|
25
|
+
Minitest and RSpec mixins. See [Testing](docs/testing.md). (#44, #50)
|
|
26
|
+
- `Squishling.verify`/`verify!` check every squished class without a request (output schema and typed result, resolved
|
|
27
|
+
escalation, params and squawk, harness and judge, the keywords `include_when:`/`squish_when` take, and the adapter),
|
|
28
|
+
reporting every problem; `verify!` raises `VerificationError`. Squished classes are tracked in a weak registry
|
|
29
|
+
(`Squishling.squished_classes`). A `squishling:verify` rake task (with a Rails Railtie) and a `be_squishling_valid`
|
|
30
|
+
RSpec matcher (`require "squishling/rspec"`) wrap it. See [Verifying squished classes](docs/verifying.md). (#48, #67)
|
|
31
|
+
- Per-call token budgets: `max_total_output_tokens:` and `max_total_input_tokens:` (on `squishling`, `squish`,
|
|
32
|
+
`squish!`, and `default_max_total_*` in `Squishling.configure`; layered limit by limit, `false` lifts one) cap what one
|
|
33
|
+
call spends across its escalation attempts, squishsum sample rounds, and judge. A request in flight is never cut short,
|
|
34
|
+
so a call can end above the limit by one request. Once a limit is reached the call raises `BudgetExhaustedError`, an
|
|
35
|
+
`InvalidOutputError`, so `squish_fallback` still receives it. (#55, #68)
|
|
36
|
+
- `squawk_on:` (`:all`, `:rejections`, `:retries`, `:final`, or a list) filters which attempts reach the `squawk` hook,
|
|
37
|
+
settable with `config.squawk_on`, `squishling squawk_on:`, and `squish :method, squawk_on:`. The default `:all` keeps
|
|
38
|
+
the old behavior. (#58, #69)
|
|
39
|
+
- `Adapter::Response#finish_reason` (a Symbol such as `:stop` or `:max_tokens`), filled by both built-in adapters.
|
|
40
|
+
`InvalidOutputError` gains `finish_reason` and `truncated?`, with a fixed note in its message and log line when the
|
|
41
|
+
reply was cut off, and `squawk` metadata gains `finish_reason`. (#54, #60)
|
|
42
|
+
- `InvalidOutputError#error_details`: `InvalidOutputError::Detail` entries (`message`, `source` of `:schema`,
|
|
43
|
+
`:validator`, `:parse` or `:squishling`) with a `model_derived?` predicate that is true only for schema errors, which
|
|
44
|
+
can name a key the model chose. Escape just those when forwarding messages to a chat webhook, HTML, or markdown.
|
|
45
|
+
`#errors` is still an Array of Strings (now frozen). (#57, #62)
|
|
46
|
+
- `RequestRejectedError` (a `ConfigurationError`) for a provider's 400, with `provider_status` and `provider_code` read
|
|
47
|
+
as plain tokens from the error body (`INVALID_ARGUMENT`, `invalid_request_error`). The provider's message is still
|
|
48
|
+
never repeated. (#52, #53)
|
|
49
|
+
- `retry_temperature:` on an escalation step sets the temperature for same-chat retries only. (#56, #61)
|
|
50
|
+
|
|
51
|
+
### Changed
|
|
52
|
+
|
|
53
|
+
- **Breaking change:** `ruby_llm` is no longer a runtime dependency of the gem. Apps that use the RubyLLM adapter (the
|
|
54
|
+
default when no adapter is configured and `ruby_llm` is loadable) must now list `gem "ruby_llm"` in their own
|
|
55
|
+
Gemfile, and `require "ruby_llm"` before `RubyLLM.configure` if they don't use `Bundler.require`; or switch to
|
|
56
|
+
`config.adapter = :openai_compatible`. (#47)
|
|
57
|
+
- **Behavior change:** a schema property named like a method the typed result needs (`class`, `hash`, `to_h`,
|
|
58
|
+
`members`, `inspect`, `is_a?`, ...) now raises `ConfigurationError` when the schema is built, before any request,
|
|
59
|
+
instead of replacing that method on the result class and failing late. Rename such fields (`class` to
|
|
60
|
+
`ticket_class`); the reserved names are listed in [Output schemas](docs/schemas.md). (#43, #49)
|
|
61
|
+
- **Behavior change:** when a same-chat retry returns the exact reply it was just rejected for, Squishling skips the
|
|
62
|
+
step's remaining attempts and moves to the next step (or raises `InvalidOutputError` right away on the last step);
|
|
63
|
+
samples end early the same way. `InvalidOutputError#attempts` counts requests actually made, `squawk` metadata gains
|
|
64
|
+
`repeated:`, and `final:` now means no attempt follows a rejection. Size a step's `attempts:` for replies that can
|
|
65
|
+
differ. (#56, #61)
|
|
66
|
+
- A provider 400's "Check params" hint now names a param only when the provider's error body names one you configured,
|
|
67
|
+
and otherwise says the params, output schema, or input may be invalid. An array-wrapped overlong-context 400 from the
|
|
68
|
+
OpenAI-compatible adapter is now an `LLMError`. (#53)
|
|
69
|
+
|
|
70
|
+
### Fixed
|
|
71
|
+
|
|
72
|
+
- A raw Hash `output_schema` that omits `type: "object"` no longer lets a non-object reply (a String, Array, or number)
|
|
73
|
+
reach the typed result as a `NoMethodError`; it is now invalid output, retried, escalated, and sent to
|
|
74
|
+
`squish_fallback` like any other. Schemas built with Schematist always had `type: "object"`.
|
|
75
|
+
- `RequestRejectedError`'s `provider_status` and `provider_code` no longer accept a hyphenated token, so a key-shaped
|
|
76
|
+
string a gateway echoes into an error body is never repeated in our message. The RubyLLM adapter's "invalid params"
|
|
77
|
+
`ConfigurationError` now names the param keys, not their values.
|
|
78
|
+
|
|
79
|
+
### Internal
|
|
80
|
+
|
|
81
|
+
- Development dependencies: `ruby_llm` 2.1.1, `coolhand` 0.10.0, and `rubocop` 1.92.1; CI pins `ruby/setup-ruby` v1.335.0
|
|
82
|
+
so the matrix resolves Ruby 3.3.13, 3.4.12, and 4.0.9. A `finish-release` skill now tags and publishes a reviewed
|
|
83
|
+
release-prep PR. (#45, #51, #59, #63, #64, #65)
|
|
84
|
+
|
|
85
|
+
## [0.4.0] - 2026-10-18
|
|
86
|
+
|
|
87
|
+
### Added
|
|
88
|
+
|
|
89
|
+
- Custom harnesses: `Squishling.register_harness` (with `Squishling.harnesses` and `reset_harnesses!`) and
|
|
90
|
+
`Squishling::Harness::Base` let you write your own harness and use it anywhere a built-in works (`harness:`,
|
|
91
|
+
`squish!`, `default_harness`). A harness gets a `Squishling::Run` with `escalate`, `samples`, `judge`, `agree?`,
|
|
92
|
+
`stage`, and `log`, so it keeps model, provider, and params resolution, `squish_validate`, concurrent samples, and the
|
|
93
|
+
normal failure handling; it can only return typed results its `Run` produced, and its options are validated when it
|
|
94
|
+
is declared. The built-in `:escalation`, `:squishsum`, and `:judged_squishsum` now run on the same interface.
|
|
95
|
+
`squawk` metadata gains `role:` and `sample:`. See [Custom harnesses](docs/harnesses.md#custom-harnesses). (#31)
|
|
96
|
+
- Round-robin models: escalation steps that share a `priority:` form a tier. Each call rotates which model in the tier
|
|
97
|
+
goes first (counted per class and method, shared across threads), and the tier's other models are tried before the
|
|
98
|
+
next priority. `rotate: false` on a step pins its tier to the order written. (#38)
|
|
99
|
+
- `config.observer`: an around-style hook (`around(call) { ... }`) wrapped around every provider request, on the thread
|
|
100
|
+
that makes it, so a `:squishsum` sample's observer runs on its worker thread. It is told the request's `Observer::Call`
|
|
101
|
+
and gets back only an `Observer::Usage` (token counts and model), never the response. An observer that yields twice
|
|
102
|
+
or never raises `ConfigurationError`, and one that rescues the request's error has it raised again. See
|
|
103
|
+
[Observers](docs/observers.md). (#25, #40)
|
|
104
|
+
- A bundled Coolhand observer (`observer = :coolhand`) that declares each call's workload (`Class#method`) and template
|
|
105
|
+
(`squish:L<n>`, `sample:a`, or `judge`) through `Coolhand.with_context`. The workload's description is
|
|
106
|
+
`Class#method`, or `Class#method (rescue)` when a `squish!` call handed the call to the LLM (`Observer::Call#handed_off`),
|
|
107
|
+
never the purpose text. It needs coolhand 0.9.0 or newer. See
|
|
108
|
+
[Coolhand Labs](docs/coolhand.md). (#40)
|
|
109
|
+
- A docs page and landing-page section on the architecture, an explainer site in `site/` deployed to GitHub Pages, and
|
|
110
|
+
per-field `description:` documented as the best practice over `purpose` guidance. (#30, #33, #34, #36)
|
|
111
|
+
|
|
112
|
+
### Changed
|
|
113
|
+
|
|
114
|
+
- **Behavior change:** when the `coolhand` gem is loaded and configured (an API key, not disabled), Squishling now
|
|
115
|
+
declares each request's workload and template to it automatically. Nothing changes for apps without Coolhand. To
|
|
116
|
+
opt out, set `config.observer = false`.
|
|
117
|
+
- **Behavior change:** two escalation steps may now share a `priority:` (they used to raise `duplicate order:`), and
|
|
118
|
+
they rotate per call as described above. If you relied on a fixed order between tied steps, give them distinct
|
|
119
|
+
priorities or set `rotate: false`.
|
|
120
|
+
- `order:` on an escalation step is deprecated in favor of `priority:`. It still works, with a deprecation warning, and
|
|
121
|
+
giving both on one step raises `ConfigurationError`. Judge and harness steps take no `priority:` or `rotate:`. (#38)
|
|
122
|
+
- **Behavior change:** `LLMError` and the `ConfigurationError` for a rejected request (400), bad credentials, or a
|
|
123
|
+
forbidden key no longer repeat the provider's own message, which can echo request content or part of an API key into
|
|
124
|
+
logs and error trackers. They carry the error class and HTTP status (`RubyLLM::RateLimitError, HTTP 429`); the
|
|
125
|
+
provider's message is still on `cause`. If you matched on provider text in `e.message`, read `e.cause.message`
|
|
126
|
+
instead.
|
|
127
|
+
- `docs/measuring-tokens.md` now points to observers for bringing your own metrics instead of walking through a
|
|
128
|
+
hand-rolled one. (#40)
|
|
129
|
+
|
|
130
|
+
### Removed
|
|
131
|
+
|
|
132
|
+
- **Breaking change:** the `:ensemble` and `:judged_ensemble` harnesses added in 0.3.0 are gone. `harness: :ensemble`
|
|
133
|
+
or `:judged_ensemble` (and `default_harness`) now raise `ConfigurationError` (unknown harness). Use `:squishsum` or
|
|
134
|
+
`:judged_squishsum`, which sample the first escalation step twice with the judge defaulting to the next step, or
|
|
135
|
+
write a custom harness. (#22, #24)
|
|
136
|
+
|
|
137
|
+
### Internal
|
|
138
|
+
|
|
139
|
+
- CI gains an exhale duplication and complexity gate (`bin/exhale`), and dependabot bumps `rubocop` (1.92), `ruby/setup-ruby`,
|
|
140
|
+
and `actions/upload-artifact`. These are development-only changes. (#27, #28, #29, #41)
|
|
141
|
+
|
|
142
|
+
## [0.3.0] - 2026-10-10
|
|
11
143
|
|
|
12
144
|
### Added
|
|
13
145
|
|
data/README.md
CHANGED
|
@@ -2,6 +2,10 @@
|
|
|
2
2
|
<img src="assets/squishling-logo.png" alt="Squishling" width="420">
|
|
3
3
|
</h1>
|
|
4
4
|
|
|
5
|
+
<p align="center">
|
|
6
|
+
<strong><a href="https://coolhand-labs.github.io/squishling/">📖 Read the Squishling explainer →</a></strong>
|
|
7
|
+
</p>
|
|
8
|
+
|
|
5
9
|
[](https://github.com/Coolhand-Labs/squishling/actions/workflows/ci.yml)
|
|
6
10
|
[](https://badge.fury.io/rb/squishling)
|
|
7
11
|
|
|
@@ -14,7 +18,17 @@ Either way it returns the same strict-schema-validated, typed result, so callers
|
|
|
14
18
|
Think of code as a cache for AI judgment. Every line you write is something to maintain, so the LLM is the
|
|
15
19
|
default and code has to earn its place: you build it for the inputs that carry the volume, and the LLM keeps
|
|
16
20
|
handling the rest. New to the idea? Read
|
|
17
|
-
[Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai)
|
|
21
|
+
[Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai) or browse the
|
|
22
|
+
[talk slides](https://michael.carroll.io/talks/2026/elastic-software/).
|
|
23
|
+
|
|
24
|
+
Or try it on the [explainer site](https://coolhand-labs.github.io/squishling/):
|
|
25
|
+
|
|
26
|
+
- [Spectrum slider](https://coolhand-labs.github.io/squishling/#spectrum): drag from traditional software to fully
|
|
27
|
+
LLM and see the tradeoff at each point
|
|
28
|
+
- [Token-cost calculator](https://coolhand-labs.github.io/squishling/#number): price your tech debt in tokens
|
|
29
|
+
against code upkeep
|
|
30
|
+
- [Walkthrough](https://coolhand-labs.github.io/squishling/#squishling): describe it, tune it, pay for accuracy,
|
|
31
|
+
harden the common shapes
|
|
18
32
|
|
|
19
33
|
## Use cases
|
|
20
34
|
|
|
@@ -81,6 +95,12 @@ Both calls return the same result class. If the LLM can't deliver either, `squis
|
|
|
81
95
|
return, or the error is raised with the original `ParseError` as its cause. See
|
|
82
96
|
[Handing off to the LLM](docs/routing.md#handing-off-to-the-llm-with-squish).
|
|
83
97
|
|
|
98
|
+
### Retire code you no longer need
|
|
99
|
+
|
|
100
|
+
Go the other way too. Find the parsers and handlers that serve a thin tail of your traffic and squish them. When a
|
|
101
|
+
path's monthly token cost is lower than what its code costs to maintain, delete the code and relish the feeling of a
|
|
102
|
+
slimmer code base (or let your AI coding agent relish it).
|
|
103
|
+
|
|
84
104
|
### Measure your tech debt in tokens
|
|
85
105
|
|
|
86
106
|
Code you haven't written is debt you haven't taken on. A squished path has a running cost you can read off a
|
|
@@ -110,12 +130,17 @@ end
|
|
|
110
130
|
gem "squishling"
|
|
111
131
|
```
|
|
112
132
|
|
|
113
|
-
Requires Ruby 3.3+ and
|
|
114
|
-
|
|
133
|
+
Requires Ruby 3.3+ and an [adapter](docs/adapters.md) to reach a model: the `ruby_llm` gem (2.1+; Squishling uses it
|
|
134
|
+
automatically when it is in your bundle), or any OpenAI-compatible endpoint, which needs no extra gem. Squishling no
|
|
135
|
+
longer installs or requires `ruby_llm` for you, so add `gem "ruby_llm"` next to it if you use RubyLLM (and
|
|
136
|
+
`require "ruby_llm"` before `RubyLLM.configure` if your app doesn't load its Gemfile with `Bundler.require`). With RubyLLM,
|
|
137
|
+
configure your provider API keys there as usual. Then choose an adapter, and optionally set a universal model, or an
|
|
138
|
+
escalation of models to try in order:
|
|
115
139
|
|
|
116
140
|
```ruby
|
|
117
141
|
Squishling.configure do |config|
|
|
118
|
-
config.
|
|
142
|
+
config.adapter = :ruby_llm # or { type: :openai_compatible, base_url: "http://localhost:11434/v1" }
|
|
143
|
+
config.default_model = "claude-sonnet-5-5" # falls back to the adapter's default when nil
|
|
119
144
|
# or: config.default_escalation = [{ model: "claude-haiku-4-5", attempts: 2 }, "claude-sonnet-5-5", "claude-opus-5-5"]
|
|
120
145
|
end
|
|
121
146
|
```
|
|
@@ -131,17 +156,25 @@ validated on both paths). A squished call goes to the LLM when:
|
|
|
131
156
|
|
|
132
157
|
Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM output.
|
|
133
158
|
|
|
159
|
+
**Best practice: describe fields in the schema, not the purpose.** Put per-field guidance in a `description:` on
|
|
160
|
+
the field (`string :status, description: "..."`, or `optional(:eta, description: "...") { string }`). The model
|
|
161
|
+
sees it right where the value goes, and the purpose stays short and about the goal. See
|
|
162
|
+
[Describing fields](docs/schemas.md#describing-fields).
|
|
163
|
+
|
|
134
164
|
## Features
|
|
135
165
|
|
|
136
166
|
- **One contract, two paths**: Ruby returns and LLM output are validated against the same strict schema and
|
|
137
167
|
returned as the same typed `Data` objects. `squished?` tells you which path served a call.
|
|
138
168
|
- **Your code as context**: `append_to_purpose` adds sections to the prompt, including a class's or method's
|
|
139
169
|
own Ruby source.
|
|
140
|
-
- **
|
|
141
|
-
|
|
170
|
+
- **[Bring your own adapter](docs/adapters.md)**: Squishling doesn't depend on any one LLM client. Use the
|
|
171
|
+
[RubyLLM](https://rubyllm.com) adapter (OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, OpenRouter, and more),
|
|
172
|
+
the OpenAI-compatible adapter (OpenAI, Ollama, vLLM, LM Studio, Together, Groq, … over plain HTTP), or write your
|
|
173
|
+
own in one method.
|
|
174
|
+
- **Any provider and model**: set a universal, per-class, per-method, or per-call model, plus layered generation params (temperature,
|
|
142
175
|
reasoning effort, top_p, …).
|
|
143
|
-
- **Model escalation**: declare an `escalation:` instead of a `model:`, with per-step `attempts:`, `
|
|
144
|
-
params, and `
|
|
176
|
+
- **Model escalation**: declare an `escalation:` instead of a `model:`, with per-step `attempts:`, `priority:` (models tied on priority take turns going first), provider,
|
|
177
|
+
params, `forward_rejected:`, `retry_temperature:`, and `include_when:` (leave a model out of calls where a Proc is falsy). Invalid output (with its errors fed back) or a provider failure moves on to the next attempt. Steps can
|
|
145
178
|
cross providers, e.g. a local Qwen model on Ollama, then Anthropic Claude Haiku on AWS Bedrock, then Claude Opus:
|
|
146
179
|
|
|
147
180
|
```ruby
|
|
@@ -168,13 +201,17 @@ Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM
|
|
|
168
201
|
- [`:escalation`](docs/configuration.md#models-and-escalation) (default): try each attempt until one passes
|
|
169
202
|
- [`:squishsum`](docs/harnesses.md#samples): two concurrent samples, accepted only if they agree
|
|
170
203
|
- [`:judged_squishsum`](docs/harnesses.md#the-judge): a chat or Jev judge picks between disagreeing samples
|
|
171
|
-
- [
|
|
204
|
+
- [Your own](docs/harnesses.md#custom-harnesses): `Squishling.register_harness` plugs a custom harness into the same
|
|
205
|
+
machinery (samples, judges, `squawk`, `squish_fallback`)
|
|
206
|
+
- **Cost caps**: `max_total_output_tokens:` and `max_total_input_tokens:` stop a call that keeps failing from
|
|
207
|
+
spending without limit (see [Capping a call's cost](docs/failures.md#capping-a-calls-cost-token-budgets)).
|
|
172
208
|
- **Output contracts**: beyond the strict schema, conditional rules (`given`) and Ruby checks (`squish_validate`)
|
|
173
209
|
reject bad output and trigger the next attempt.
|
|
174
210
|
- **Defined failure behavior**: provider errors become `Squishling::LLMError`, and `squish_fallback` lets you decide
|
|
175
211
|
what to return when every attempt fails.
|
|
176
|
-
- **Observability**: the opt-in [`squawk`](docs/failures.md#observing-every-attempt-squawk) hook sends
|
|
177
|
-
raw output to your error tracker. Model output stays out of
|
|
212
|
+
- **Observability**: the opt-in [`squawk`](docs/failures.md#observing-every-attempt-squawk) hook sends each attempt's
|
|
213
|
+
raw output (or only the rejected and retried ones) to your error tracker. Model output stays out of
|
|
214
|
+
error messages and logs otherwise.
|
|
178
215
|
- **Opt-in context**: only method arguments, the instance state you name with `squish_context`, the `context:` you
|
|
179
216
|
pass to `squish!`, and the source you choose to append are sent to the provider. The one addition: when
|
|
180
217
|
[escalation](docs/configuration.md#models-and-escalation) moves to the next step, that step also sees the previous
|
|
@@ -182,29 +219,40 @@ Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM
|
|
|
182
219
|
|
|
183
220
|
## Documentation
|
|
184
221
|
|
|
222
|
+
- [Explainer site](https://coolhand-labs.github.io/squishling/): interactive walkthrough, spectrum slider, and
|
|
223
|
+
token-cost calculator
|
|
185
224
|
- [Configuration](docs/configuration.md): options, models and escalation, providers, generation params, inheritance
|
|
225
|
+
- [Adapters](docs/adapters.md): the RubyLLM and OpenAI-compatible adapters, and writing your own
|
|
186
226
|
- [Routing](docs/routing.md): when a call goes to the LLM, `squish!`, `append_to_purpose`, hardening a path,
|
|
187
227
|
what the LLM sees
|
|
188
228
|
- [Output schemas](docs/schemas.md): schema forms, strict mode, typed results, optional vs. empty, contracts
|
|
189
229
|
- [Failure handling](docs/failures.md): escalation, `squish_validate`, error classes, fallbacks
|
|
190
|
-
- [Harnesses](docs/harnesses.md): escalation, squishsum,
|
|
230
|
+
- [Harnesses](docs/harnesses.md): escalation, squishsum, judges (chat or Jev), and custom harnesses
|
|
231
|
+
- [Verifying squished classes](docs/verifying.md): `Squishling.verify!`, a rake task, and an RSpec matcher that check
|
|
232
|
+
every class ahead of runtime, without a request
|
|
191
233
|
- [Naming and collisions](docs/naming.md): the methods `include Squishling` adds and what happens when a name is taken
|
|
192
234
|
- [Measuring token spend](docs/measuring-tokens.md): see what each squished path costs, with Coolhand Labs,
|
|
193
|
-
the `squawk` hook,
|
|
235
|
+
the `squawk` hook, your own observer, or OpenTelemetry
|
|
236
|
+
- [Observers](docs/observers.md): wrap every provider request, and the built-in [Coolhand Labs](docs/coolhand.md) observer
|
|
237
|
+
that files each call under its workload and template
|
|
238
|
+
- [Testing](docs/testing.md): `require "squishling/testing"`: a scripted adapter, call assertions, result and error
|
|
239
|
+
stubs, and schema-driven fixtures, for Minitest and RSpec
|
|
194
240
|
- [Live examples](examples/README.md): end-to-end tests against Anthropic Claude Haiku and OpenAI GPT-6 Luna
|
|
195
241
|
|
|
196
242
|
## Development
|
|
197
243
|
|
|
198
244
|
```sh
|
|
199
245
|
bin/setup
|
|
200
|
-
bundle exec rake # RSpec (offline;
|
|
246
|
+
bundle exec rake # RSpec (offline; adapters are faked or stubbed) + RuboCop
|
|
201
247
|
```
|
|
202
248
|
|
|
203
249
|
See [AGENTS.md](AGENTS.md) for repo conventions, and [SECURITY.md](SECURITY.md) for reporting vulnerabilities.
|
|
250
|
+
Have a suggestion to improve Squishling? [Open an issue](https://github.com/Coolhand-Labs/squishling/issues).
|
|
204
251
|
|
|
205
252
|
## Credits
|
|
206
253
|
|
|
207
|
-
Inspired by [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai)
|
|
254
|
+
Inspired by [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai); see also
|
|
255
|
+
the [talk slides](https://michael.carroll.io/talks/2026/elastic-software/) from XO Ruby NYC 2026.
|
|
208
256
|
|
|
209
257
|
## License
|
|
210
258
|
|
data/docs/adapters.md
ADDED
|
@@ -0,0 +1,189 @@
|
|
|
1
|
+
# Adapters
|
|
2
|
+
|
|
3
|
+
An **adapter** is how Squishling reaches a model. Squishling builds the request (system prompt, conversation so far,
|
|
4
|
+
strict output schema, generation params), the adapter makes one completion and returns the model's output, and
|
|
5
|
+
Squishling does the rest: validation, `squish_validate`, retries and escalation, harnesses, `squawk`, and fallbacks.
|
|
6
|
+
Two adapters ship, and you can write your own.
|
|
7
|
+
|
|
8
|
+
| Adapter | Reaches | Needs |
|
|
9
|
+
|---|---|---|
|
|
10
|
+
| [`:ruby_llm`](#ruby_llm) | OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, OpenRouter, and every provider [RubyLLM](https://rubyllm.com) supports | `gem "ruby_llm"` |
|
|
11
|
+
| [`:openai_compatible`](#openai-compatible-endpoints) | Any server that speaks OpenAI's Chat Completions API: OpenAI, Ollama, vLLM, LM Studio, llama.cpp, Together, Groq, … | nothing (Ruby's `Net::HTTP`) |
|
|
12
|
+
| [Your own](#writing-your-own) | Anything | a class with one method |
|
|
13
|
+
|
|
14
|
+
Squishling itself depends on neither RubyLLM nor any provider SDK.
|
|
15
|
+
|
|
16
|
+
## Choosing one
|
|
17
|
+
|
|
18
|
+
```ruby
|
|
19
|
+
Squishling.configure do |config|
|
|
20
|
+
config.adapter = :ruby_llm # a built-in or registered adapter, by name
|
|
21
|
+
# config.adapter = { type: :openai_compatible, base_url: "http://localhost:11434/v1", default_model: "llama3.2" }
|
|
22
|
+
# config.adapter = MyAdapter.new # or any adapter instance
|
|
23
|
+
end
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
An adapter is built when you assign it, so a missing gem or a bad option fails at boot, not on the first request.
|
|
27
|
+
There is one adapter per process; every squishling class, method, and escalation step goes through it.
|
|
28
|
+
|
|
29
|
+
**When you set none**, Squishling uses the RubyLLM adapter if the `ruby_llm` gem is in your bundle, so apps that
|
|
30
|
+
configured RubyLLM before adapters existed keep working. Squishling no longer requires `ruby_llm` when it loads, so an
|
|
31
|
+
app that doesn't load its Gemfile with `Bundler.require` should `require "ruby_llm"` before `RubyLLM.configure`. Without the gem, the first squished call
|
|
32
|
+
raises `Squishling::ConfigurationError` saying what to add.
|
|
33
|
+
|
|
34
|
+
## `:ruby_llm`
|
|
35
|
+
|
|
36
|
+
Everything works as it did before adapters: configure provider API keys with `RubyLLM.configure`, name models, and
|
|
37
|
+
use `provider:` for models missing from RubyLLM's registry. See [Configuration](configuration.md). Transport-level
|
|
38
|
+
retries are RubyLLM's (`RubyLLM.config.max_retries`, `request_timeout`).
|
|
39
|
+
|
|
40
|
+
This is the only built-in adapter that supports `type: :judgment` judges ([System One decision models](harnesses.md#the-judge)).
|
|
41
|
+
|
|
42
|
+
## OpenAI-compatible endpoints
|
|
43
|
+
|
|
44
|
+
```ruby
|
|
45
|
+
Squishling.configure do |config|
|
|
46
|
+
config.adapter = {
|
|
47
|
+
type: :openai_compatible,
|
|
48
|
+
base_url: "http://localhost:11434/v1", # where /chat/completions lives
|
|
49
|
+
api_key: ENV["LLM_API_KEY"], # optional for local servers
|
|
50
|
+
default_model: "llama3.2" # for steps that name no model
|
|
51
|
+
}
|
|
52
|
+
config.default_model = "llama3.2"
|
|
53
|
+
end
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
| Option | Default | Description |
|
|
57
|
+
|---|---|---|
|
|
58
|
+
| `base_url` | `https://api.openai.com/v1` | The API root. Requests go to `{base_url}/chat/completions`. Must be `http(s)` and carry no credentials, query, or fragment. |
|
|
59
|
+
| `api_key` | `ENV["OPENAI_API_KEY"]` for OpenAI's own endpoint, else none | A String, or a callable returning one (called once per completion, outside the adapter's retry handling, so rotating keys work and its exceptions reach you unchanged). Sent as a bearer token; omitted when blank. The key is sent as-is, so use `https://` for anything off your machine or trusted network. `OPENAI_API_KEY` is read only for OpenAI's own `https://api.openai.com`, so it can never go to another host: pass `api_key:` for any other endpoint that needs one. |
|
|
60
|
+
| `headers` | `{}` | Extra request headers, e.g. `{ "OpenAI-Organization" => "org-…" }` or a gateway's own. |
|
|
61
|
+
| `default_model` | `nil` | The model for steps that don't name one. Without it, a step with no model raises `ConfigurationError`. |
|
|
62
|
+
| `open_timeout` / `read_timeout` | `10` / `120` | Seconds. |
|
|
63
|
+
| `max_retries` | `2` | Timeouts, connection and protocol failures, HTTP 408, 429, and 5xx are retried this many times with exponential backoff (honoring `Retry-After`, up to 30 seconds) before they surface as an `LLMError`. A timed-out request may still have been processed by the server, so a retry can be billed twice; set `max_retries: 0` if that matters more than resilience. |
|
|
64
|
+
| `max_tokens_param` | `:max_completion_tokens` | The body key for `max_output_tokens`. Use `:max_tokens` for servers that don't know the newer name. |
|
|
65
|
+
| `structured_output` | `:json_schema` | `:json_schema` asks for strict structured output. For servers without it use `:json_object`, or `:none`: the schema then goes into the system prompt in words. Squishling validates every answer itself either way. |
|
|
66
|
+
|
|
67
|
+
**How params map.** `temperature` → `temperature`; `max_output_tokens` → `max_tokens_param`; `thinking: { effort: }` →
|
|
68
|
+
`reasoning_effort` (`true`/`false` leave the model's default, and a `budget:` or `display:` raises
|
|
69
|
+
`ConfigurationError`, since Chat Completions has no field for them). Any other param (`top_p`, `seed`, …) goes into
|
|
70
|
+
the request body as given. Squishling's own keys (`model`, `messages`, `response_format`, `n`, `stream`) always win.
|
|
71
|
+
`provider:` has no meaning here and is ignored.
|
|
72
|
+
|
|
73
|
+
**Failures.** Every HTTP status of 300 or above except 408, 429, and 5xx (401, 403, 404, 400, 422, a redirect, …) raises `ConfigurationError`, a `RequestRejectedError` for a 400 or other rejected request (never retried or escalated; a 400 that
|
|
74
|
+
reports an overlong context, and a 413, are `LLMError`s, since they depend on the input; a 400 counts as an overlong
|
|
75
|
+
context when its error code is `context_length_exceeded` or its message uses the usual wording ("context length", "too many
|
|
76
|
+
tokens", …), which the message is only matched against, never repeated). 408, 429, 5xx, timeouts, and
|
|
77
|
+
connection errors raise `LLMError` once the adapter's own retries are used up. Messages carry the status or error
|
|
78
|
+
class only; a rejected request (400) also carries the server's error status and code when they are plain tokens (as
|
|
79
|
+
`provider_status` and `provider_code`). The server's message, the response body, and the API key never appear in an error or log. Redirects
|
|
80
|
+
aren't followed, so the key can't be sent to another host.
|
|
81
|
+
|
|
82
|
+
## Writing your own
|
|
83
|
+
|
|
84
|
+
Subclass `Squishling::Adapter::Base`, implement `complete`, and return a `Squishling::Adapter::Response`:
|
|
85
|
+
|
|
86
|
+
```ruby
|
|
87
|
+
class AcmeAdapter < Squishling::Adapter::Base
|
|
88
|
+
def initialize(client: Acme::Client.new)
|
|
89
|
+
super()
|
|
90
|
+
@client = client
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
def complete(request)
|
|
94
|
+
reply = @client.chat(
|
|
95
|
+
model: request.model,
|
|
96
|
+
system: request.instructions,
|
|
97
|
+
messages: request.messages, # [{ role: :user | :assistant, content: "…" }, …]
|
|
98
|
+
json_schema: request.schema["schema"], # strict; Squishling validates the answer regardless
|
|
99
|
+
**request.params
|
|
100
|
+
)
|
|
101
|
+
Squishling::Adapter::Response.new(
|
|
102
|
+
content: reply.text, # the raw String (or a Hash you already parsed)
|
|
103
|
+
model: reply.model,
|
|
104
|
+
tokens: { input_tokens: reply.usage.in, output_tokens: reply.usage.out }
|
|
105
|
+
)
|
|
106
|
+
rescue Acme::AuthError
|
|
107
|
+
raise configuration_error(request, "Acme rejected the credentials")
|
|
108
|
+
rescue Acme::BadRequest => e
|
|
109
|
+
# a 400: repeats only the body's status and code, and points at a param only if the provider named one
|
|
110
|
+
raise rejected_request_error(request, via: "Acme, HTTP 400", body: e.response_body, http_status: 400)
|
|
111
|
+
rescue Acme::Error => e
|
|
112
|
+
raise llm_error(request, "Acme failed (#{e.class})")
|
|
113
|
+
end
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
Squishling.configure { |config| config.adapter = AcmeAdapter.new }
|
|
117
|
+
# or register it by name, to use config.adapter = :acme (or { type: :acme, client: … }):
|
|
118
|
+
Squishling.register_adapter(:acme, AcmeAdapter)
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
### The request
|
|
122
|
+
|
|
123
|
+
`Squishling::Adapter::Request` carries everything for one completion:
|
|
124
|
+
|
|
125
|
+
| Field | |
|
|
126
|
+
|---|---|
|
|
127
|
+
| `label` | `"Class#method"`, to prefix your error messages (the `configuration_error` / `llm_error` helpers do; `rejected_request_error(request, via:, body:, http_status:)` builds the error for a rejected request, from a JSON String or String-keyed Hash body). |
|
|
128
|
+
| `model`, `provider` | The escalation step's, or `nil` when it names none (use your own default). What `provider:` means is up to you. |
|
|
129
|
+
| `instructions` | The system prompt. |
|
|
130
|
+
| `messages` | The conversation so far, ending with the user message to answer. After invalid output, Squishling appends your previous answer and a retry message; you never keep history. |
|
|
131
|
+
| `schema` | `{ "name", "description", "schema", "strict" => true }`: the strict structured-output payload, without the conditional keywords providers can't take. |
|
|
132
|
+
| `params` | The resolved [generation params](configuration.md#generation-params): the portable `:temperature`, `:max_output_tokens`, `:thinking`, plus provider-specific keys. Squishling has already rejected the keys it owns. Translate the portable ones for your provider; pass the rest through if it takes them. |
|
|
133
|
+
|
|
134
|
+
### The response
|
|
135
|
+
|
|
136
|
+
`Squishling::Adapter::Response.new(content:, model:, tokens:, finish_reason:)`. `content` is the model's output as a
|
|
137
|
+
String, or a Hash if you parsed it; `nil` or empty counts as invalid output and is retried. `model` is the id the
|
|
138
|
+
provider reports. `tokens` is a Hash (`:input_tokens`, `:output_tokens`, and anything else you have); `{}` when
|
|
139
|
+
unknown. These feed [`squawk`'s `usage`](failures.md#observing-every-attempt-squawk), the [observer](observers.md)'s `Usage`, and the
|
|
140
|
+
[token budget](failures.md#capping-a-calls-cost-token-budgets), which counts `:input_tokens` and `:output_tokens`
|
|
141
|
+
(a response with none never exhausts a budget).
|
|
142
|
+
`finish_reason` is why the provider stopped, as a Symbol: `:stop`, `:max_tokens` (the output cap cut the reply off),
|
|
143
|
+
`:content_filter`, `:tool_calls`, or the provider's own spelling; leave it out (`nil`) when unknown. Map your
|
|
144
|
+
provider's "length" reason to `:max_tokens`, so Squishling can [tell a cutoff from a wrong
|
|
145
|
+
answer](failures.md#a-cutoff-or-a-wrong-answer).
|
|
146
|
+
|
|
147
|
+
### The error contract
|
|
148
|
+
|
|
149
|
+
| Raise | When | What Squishling does |
|
|
150
|
+
|---|---|---|
|
|
151
|
+
| `Squishling::ConfigurationError` | Bad credentials, a request the provider rejects, an unknown model: anything a retry can't fix | Raised at once. Never retried, escalated, or sent to `squish_fallback`. |
|
|
152
|
+
| `Squishling::LLMError` | Rate limit, 5xx, timeout, connection failure, an overlong context: anything worth another go | Moves to the next attempt; raised from the last one (and goes to `squish_fallback`). |
|
|
153
|
+
|
|
154
|
+
Keep the provider's own message out of both: it can echo request content or part of a key, and ours reach logs and
|
|
155
|
+
error trackers. Put the original on `cause` (raising inside a `rescue` does that for you). Any other exception
|
|
156
|
+
propagates unwrapped and is never retried, so map your transport's errors. Return anything but an
|
|
157
|
+
`Adapter::Response` and Squishling raises `ConfigurationError`.
|
|
158
|
+
|
|
159
|
+
### Optional hooks
|
|
160
|
+
|
|
161
|
+
- **`judgments?` and `judge(input, questions:, step:, label:)`**: System One decision models for `type: :judgment` judges.
|
|
162
|
+
`judge` returns an object answering `[:winner]` with something that responds to `choice` and `probabilities`.
|
|
163
|
+
Without `judgments?`, such a judge raises `ConfigurationError` before any request. If the object has `tokens` (with
|
|
164
|
+
`input` and `output` readers), they count toward the call's token budget.
|
|
165
|
+
- **`capture_context` and `with_context(context) { … }`**: samples (the `:squishsum` harnesses) send their requests from
|
|
166
|
+
worker threads. `capture_context` runs on the caller's thread and returns whatever your tracing needs; `with_context`
|
|
167
|
+
re-establishes it around each worker's request. The RubyLLM adapter uses this for its instrumentation workflow.
|
|
168
|
+
|
|
169
|
+
### Thread safety
|
|
170
|
+
|
|
171
|
+
Adapters are called concurrently from worker threads when a harness takes samples. Keep no unsynchronized mutable
|
|
172
|
+
state (a per-request HTTP connection, as the OpenAI-compatible adapter does, is the simple way).
|
|
173
|
+
|
|
174
|
+
### Testing against your adapter
|
|
175
|
+
|
|
176
|
+
To test your *application* (not an adapter), use [`squishling/testing`](testing.md): a scripted adapter with request
|
|
177
|
+
inspection, call assertions, and stubs. To unit-test an adapter you wrote, a canned one is enough:
|
|
178
|
+
|
|
179
|
+
```ruby
|
|
180
|
+
class CannedAdapter < Squishling::Adapter::Base
|
|
181
|
+
def initialize(*replies) = (@replies = replies)
|
|
182
|
+
|
|
183
|
+
def complete(request)
|
|
184
|
+
Squishling::Adapter::Response.new(content: @replies.shift, model: request.model, tokens: {})
|
|
185
|
+
end
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
Squishling.configure { |config| config.adapter = CannedAdapter.new({ "label" => "bug" }) }
|
|
189
|
+
```
|