squishling 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: '0748f552c985521890d365ebc0818ec59c4336f93fb183d1769f49ed8cd32349'
4
- data.tar.gz: '05128ecf8d7156e0c8ca084bf8a1860b754596017df2cdfdcc35e2242138fa7b'
3
+ metadata.gz: 0fb55da97a0a032802afcd90b8e56d497a42194206ad90f07d71bc46c70a9e49
4
+ data.tar.gz: 79430c869cda0726357c42c1ad1fdc7cbf9108a00ebcd56814df4e45acf99db6
5
5
  SHA512:
6
- metadata.gz: 2be443b51d2dc1460a410fbf0850bed09e33fff5f7ad894d4bd92e3f47d428d22df7bfc0371036c090a6b806f466b1d930de3a9caf8d1b044185fd4000b765bf
7
- data.tar.gz: 3a18d6e041f5807c966677fea5499de8af5d0f28146cb98bbe205141598ee6a01a2d598b1326df3b78227455dcc124c69cf95afacabf9a1689a1cc382a652414
6
+ metadata.gz: 95f83f1f99d86b5b2fddcc0d5e7d845aff73dbb7cc85620eafb93f999edd69181e8a72c726409cdb6ec11fa1286264947c48ac5f3467c1fb368755ef8ad95b62
7
+ data.tar.gz: 4ad6fbec6185bb9e4369c972b9f561bce840db75a96ba02c619fe82d1726e15230330b8f457c740017b08e412b3da2c364fd40f657db3128b28cab9c01ba993b
data/CHANGELOG.md CHANGED
@@ -7,7 +7,64 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
- ## [0.3.0] - 2026-10-09
10
+ ## [0.4.0] - 2026-10-18
11
+
12
+ ### Added
13
+
14
+ - Custom harnesses: `Squishling.register_harness` (with `Squishling.harnesses` and `reset_harnesses!`) and
15
+ `Squishling::Harness::Base` let you write your own harness and use it anywhere a built-in works (`harness:`,
16
+ `squish!`, `default_harness`). A harness gets a `Squishling::Run` with `escalate`, `samples`, `judge`, `agree?`,
17
+ `stage`, and `log`, so it keeps model, provider, and params resolution, `squish_validate`, concurrent samples, and the
18
+ normal failure handling; it can only return typed results its `Run` produced, and its options are validated when it
19
+ is declared. The built-in `:escalation`, `:squishsum`, and `:judged_squishsum` now run on the same interface.
20
+ `squawk` metadata gains `role:` and `sample:`. See [Custom harnesses](docs/harnesses.md#custom-harnesses). (#31)
21
+ - Round-robin models: escalation steps that share a `priority:` form a tier. Each call rotates which model in the tier
22
+ goes first (counted per class and method, shared across threads), and the tier's other models are tried before the
23
+ next priority. `rotate: false` on a step pins its tier to the order written. (#38)
24
+ - `config.observer`: an around-style hook (`around(call) { ... }`) wrapped around every provider request, on the thread
25
+ that makes it, so a `:squishsum` sample's observer runs on its worker thread. It is told the request's `Observer::Call`
26
+ and gets back only an `Observer::Usage` (token counts and model), never the response. An observer that yields twice
27
+ or never raises `ConfigurationError`, and one that rescues the request's error has it raised again. See
28
+ [Observers](docs/observers.md). (#25, #40)
29
+ - A bundled Coolhand observer (`observer = :coolhand`) that declares each call's workload (`Class#method`) and template
30
+ (`squish:L<n>`, `sample:a`, or `judge`) through `Coolhand.with_context`. The workload's description is
31
+ `Class#method`, or `Class#method (rescue)` when a `squish!` call handed the call to the LLM (`Observer::Call#handed_off`),
32
+ never the purpose text. It needs coolhand 0.9.0 or newer. See
33
+ [Coolhand Labs](docs/coolhand.md). (#40)
34
+ - A docs page and landing-page section on the architecture, an explainer site in `site/` deployed to GitHub Pages, and
35
+ per-field `description:` documented as the best practice over `purpose` guidance. (#30, #33, #34, #36)
36
+
37
+ ### Changed
38
+
39
+ - **Behavior change:** when the `coolhand` gem is loaded and configured (an API key, not disabled), Squishling now
40
+ declares each request's workload and template to it automatically. Nothing changes for apps without Coolhand. To
41
+ opt out, set `config.observer = false`.
42
+ - **Behavior change:** two escalation steps may now share a `priority:` (they used to raise `duplicate order:`), and
43
+ they rotate per call as described above. If you relied on a fixed order between tied steps, give them distinct
44
+ priorities or set `rotate: false`.
45
+ - `order:` on an escalation step is deprecated in favor of `priority:`. It still works, with a deprecation warning, and
46
+ giving both on one step raises `ConfigurationError`. Judge and harness steps take no `priority:` or `rotate:`. (#38)
47
+ - **Behavior change:** `LLMError` and the `ConfigurationError` for a rejected request (400), bad credentials, or a
48
+ forbidden key no longer repeat the provider's own message, which can echo request content or part of an API key into
49
+ logs and error trackers. They carry the error class and HTTP status (`RubyLLM::RateLimitError, HTTP 429`); the
50
+ provider's message is still on `cause`. If you matched on provider text in `e.message`, read `e.cause.message`
51
+ instead.
52
+ - `docs/measuring-tokens.md` now points to observers for bringing your own metrics instead of walking through a
53
+ hand-rolled one. (#40)
54
+
55
+ ### Removed
56
+
57
+ - **Breaking change:** the `:ensemble` and `:judged_ensemble` harnesses added in 0.3.0 are gone. `harness: :ensemble`
58
+ or `:judged_ensemble` (and `default_harness`) now raise `ConfigurationError` (unknown harness). Use `:squishsum` or
59
+ `:judged_squishsum`, which sample the first escalation step twice with the judge defaulting to the next step, or
60
+ write a custom harness. (#22, #24)
61
+
62
+ ### Internal
63
+
64
+ - CI gains an exhale duplication and complexity gate (`bin/exhale`), and dependabot bumps `rubocop` (1.92), `ruby/setup-ruby`,
65
+ and `actions/upload-artifact`. These are development-only changes. (#27, #28, #29, #41)
66
+
67
+ ## [0.3.0] - 2026-10-10
11
68
 
12
69
  ### Added
13
70
 
data/README.md CHANGED
@@ -2,6 +2,10 @@
2
2
  <img src="assets/squishling-logo.png" alt="Squishling" width="420">
3
3
  </h1>
4
4
 
5
+ <p align="center">
6
+ <strong><a href="https://coolhand-labs.github.io/squishling/">📖 Read the Squishling explainer →</a></strong>
7
+ </p>
8
+
5
9
  [![CI](https://github.com/Coolhand-Labs/squishling/actions/workflows/ci.yml/badge.svg)](https://github.com/Coolhand-Labs/squishling/actions/workflows/ci.yml)
6
10
  [![Gem Version](https://badge.fury.io/rb/squishling.svg)](https://badge.fury.io/rb/squishling)
7
11
 
@@ -14,7 +18,17 @@ Either way it returns the same strict-schema-validated, typed result, so callers
14
18
  Think of code as a cache for AI judgment. Every line you write is something to maintain, so the LLM is the
15
19
  default and code has to earn its place: you build it for the inputs that carry the volume, and the LLM keeps
16
20
  handling the rest. New to the idea? Read
17
- [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai).
21
+ [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai) or browse the
22
+ [talk slides](https://michael.carroll.io/talks/2026/elastic-software/).
23
+
24
+ Or try it on the [explainer site](https://coolhand-labs.github.io/squishling/):
25
+
26
+ - [Spectrum slider](https://coolhand-labs.github.io/squishling/#spectrum): drag from traditional software to fully
27
+ LLM and see the tradeoff at each point
28
+ - [Token-cost calculator](https://coolhand-labs.github.io/squishling/#number): price your tech debt in tokens
29
+ against code upkeep
30
+ - [Walkthrough](https://coolhand-labs.github.io/squishling/#squishling): describe it, tune it, pay for accuracy,
31
+ harden the common shapes
18
32
 
19
33
  ## Use cases
20
34
 
@@ -81,6 +95,12 @@ Both calls return the same result class. If the LLM can't deliver either, `squis
81
95
  return, or the error is raised with the original `ParseError` as its cause. See
82
96
  [Handing off to the LLM](docs/routing.md#handing-off-to-the-llm-with-squish).
83
97
 
98
+ ### Retire code you no longer need
99
+
100
+ Go the other way too. Find the parsers and handlers that serve a thin tail of your traffic and squish them. When a
101
+ path's monthly token cost is lower than what its code costs to maintain, delete the code and relish the feeling of a
102
+ slimmer code base (or let your AI coding agent relish it).
103
+
84
104
  ### Measure your tech debt in tokens
85
105
 
86
106
  Code you haven't written is debt you haven't taken on. A squished path has a running cost you can read off a
@@ -131,6 +151,11 @@ validated on both paths). A squished call goes to the LLM when:
131
151
 
132
152
  Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM output.
133
153
 
154
+ **Best practice: describe fields in the schema, not the purpose.** Put per-field guidance in a `description:` on
155
+ the field (`string :status, description: "..."`, or `optional(:eta, description: "...") { string }`). The model
156
+ sees it right where the value goes, and the purpose stays short and about the goal. See
157
+ [Describing fields](docs/schemas.md#describing-fields).
158
+
134
159
  ## Features
135
160
 
136
161
  - **One contract, two paths**: Ruby returns and LLM output are validated against the same strict schema and
@@ -140,7 +165,7 @@ Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM
140
165
  - **Any RubyLLM provider and model**: OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, OpenRouter, and more.
141
166
  Set a universal, per-class, per-method, or per-call model, plus layered generation params (temperature,
142
167
  reasoning effort, top_p, …).
143
- - **Model escalation**: declare an `escalation:` instead of a `model:`, with per-step `attempts:`, `order:`, provider,
168
+ - **Model escalation**: declare an `escalation:` instead of a `model:`, with per-step `attempts:`, `priority:` (models tied on priority take turns going first), provider,
144
169
  params, and `forward_rejected:`. Invalid output (with its errors fed back) or a provider failure moves on to the next attempt. Steps can
145
170
  cross providers, e.g. a local Qwen model on Ollama, then Anthropic Claude Haiku on AWS Bedrock, then Claude Opus:
146
171
 
@@ -168,7 +193,8 @@ Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM
168
193
  - [`:escalation`](docs/configuration.md#models-and-escalation) (default): try each attempt until one passes
169
194
  - [`:squishsum`](docs/harnesses.md#samples): two concurrent samples, accepted only if they agree
170
195
  - [`:judged_squishsum`](docs/harnesses.md#the-judge): a chat or Jev judge picks between disagreeing samples
171
- - [`:ensemble` and `:judged_ensemble`](docs/harnesses.md#ensembles): the same checks across two different models
196
+ - [Your own](docs/harnesses.md#custom-harnesses): `Squishling.register_harness` plugs a custom harness into the same
197
+ machinery (samples, judges, `squawk`, `squish_fallback`)
172
198
  - **Output contracts**: beyond the strict schema, conditional rules (`given`) and Ruby checks (`squish_validate`)
173
199
  reject bad output and trigger the next attempt.
174
200
  - **Defined failure behavior**: provider errors become `Squishling::LLMError`, and `squish_fallback` lets you decide
@@ -182,15 +208,19 @@ Otherwise the Ruby runs, and whatever it returns is validated and typed like LLM
182
208
 
183
209
  ## Documentation
184
210
 
211
+ - [Explainer site](https://coolhand-labs.github.io/squishling/): interactive walkthrough, spectrum slider, and
212
+ token-cost calculator
185
213
  - [Configuration](docs/configuration.md): options, models and escalation, providers, generation params, inheritance
186
214
  - [Routing](docs/routing.md): when a call goes to the LLM, `squish!`, `append_to_purpose`, hardening a path,
187
215
  what the LLM sees
188
216
  - [Output schemas](docs/schemas.md): schema forms, strict mode, typed results, optional vs. empty, contracts
189
217
  - [Failure handling](docs/failures.md): escalation, `squish_validate`, error classes, fallbacks
190
- - [Harnesses](docs/harnesses.md): escalation, squishsum, ensemble, and judges (chat or Jev)
218
+ - [Harnesses](docs/harnesses.md): escalation, squishsum, judges (chat or Jev), and custom harnesses
191
219
  - [Naming and collisions](docs/naming.md): the methods `include Squishling` adds and what happens when a name is taken
192
220
  - [Measuring token spend](docs/measuring-tokens.md): see what each squished path costs, with Coolhand Labs,
193
- the `squawk` hook, OpenTelemetry, LangSmith, or RubyLLM's instrumenter
221
+ the `squawk` hook, your own observer, or OpenTelemetry
222
+ - [Observers](docs/observers.md): wrap every provider request, and the built-in [Coolhand Labs](docs/coolhand.md) observer
223
+ that files each call under its workload and template
194
224
  - [Live examples](examples/README.md): end-to-end tests against Anthropic Claude Haiku and OpenAI GPT-6 Luna
195
225
 
196
226
  ## Development
@@ -201,10 +231,12 @@ bundle exec rake # RSpec (offline; RubyLLM is stubbed) + RuboCop
201
231
  ```
202
232
 
203
233
  See [AGENTS.md](AGENTS.md) for repo conventions, and [SECURITY.md](SECURITY.md) for reporting vulnerabilities.
234
+ Have a suggestion to improve Squishling? [Open an issue](https://github.com/Coolhand-Labs/squishling/issues).
204
235
 
205
236
  ## Credits
206
237
 
207
- Inspired by [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai).
238
+ Inspired by [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai); see also
239
+ the [talk slides](https://michael.carroll.io/talks/2026/elastic-software/) from XO Ruby NYC 2026.
208
240
 
209
241
  ## License
210
242
 
@@ -29,7 +29,8 @@ end
29
29
  | `default_provider` | `nil` | The provider for `default_model`/`default_escalation` steps that don't name one. Only needed for models missing from RubyLLM's registry (see below). |
30
30
  | `default_params` | `{}` | Generation params for every call (temperature, reasoning effort, top_p, …), overridable per class, per method, and per escalation step. See [Generation params](#generation-params). |
31
31
  | `squawk` | `nil` | A callable run after every LLM attempt with the raw output, for sending it to an error tracker or tracing tool. Also settable per class and per method. See [Observing every attempt](failures.md#observing-every-attempt-squawk). |
32
- | `default_harness` | `nil` (`:escalation`) | How every squishling class that doesn't declare its own uses its escalation: `:escalation`, `:squishsum`, `:judged_squishsum`, `:ensemble`, `:judged_ensemble`, or a Hash with `type:` and options. See [Harnesses](harnesses.md). |
32
+ | `observer` | `nil` | An around-style hook for every provider request: `:coolhand`, any object responding to `around(call)`, or `false` for none. `nil` uses the [Coolhand observer](coolhand.md) when the `coolhand` gem is configured. See [Observers](observers.md). |
33
+ | `default_harness` | `nil` (`:escalation`) | How every squishling class that doesn't declare its own uses its escalation: `:escalation`, `:squishsum`, `:judged_squishsum`, a [registered custom harness](harnesses.md#custom-harnesses), or a Hash with `type:` and options. See [Harnesses](harnesses.md). |
33
34
  | `logger` | `nil` | Any `Logger`. Debug lines when a call routes to the LLM, warnings on each escalation and when a fallback is used. Never includes the raw model response; see [what ends up in errors and logs](failures.md#what-ends-up-in-errors-and-logs). |
34
35
 
35
36
  Transport-level retries (rate limits, 5xx, timeouts) are configured on RubyLLM itself
@@ -62,21 +63,50 @@ Each step is a model name, or a Hash with:
62
63
  |---|---|---|
63
64
  | `model:` | required | The model id. |
64
65
  | `attempts:` | `1` | How many attempts this step gets before escalating to the next one. |
65
- | `order:` | position | An Integer; steps run lowest first. Give every step an `order:` or none (duplicates are rejected). Gaps are fine. |
66
+ | `priority:` | position | An Integer; steps run lowest first. Give every step a `priority:` or none. Steps with the same priority are one tier (see [Rotating models](#rotating-models-within-a-priority)). Gaps are fine. `order:` is the deprecated name for it. |
67
+ | `rotate:` | `true` | Set `false` on a step to keep its tier in the order written instead of rotating it. |
66
68
  | `provider:` | the level's `provider:` | See [Providers](#providers-and-newly-released-models). |
67
69
  | `forward_rejected:` | `true` | Set `false` to start this step from the original input alone, without the previous step's rejected output and its errors (see below). |
68
70
  | `params:` | `{}` | [Generation params](#generation-params) for this step, merged over the class and method params key by key. |
69
71
 
70
- Without `order:`, steps run in the order written. With it, the order is explicit and doesn't depend on position:
72
+ Without `priority:`, steps run in the order written. With it, the order is explicit and doesn't depend on position:
71
73
 
72
74
  ```ruby
73
75
  squishling escalation: [
74
- { model: "claude-opus-5-5", order: 20, params: { thinking: { effort: :high } } },
75
- { model: "claude-haiku-4-5", order: 0, attempts: 2 },
76
- { model: "claude-sonnet-5-5", order: 10 }
76
+ { model: "claude-opus-5-5", priority: 30, params: { thinking: { effort: :high } } },
77
+ { model: "claude-haiku-4-5", priority: 10, attempts: 2 },
78
+ { model: "gemma-4b", priority: 10 },
79
+ { model: "claude-sonnet-5-5", priority: 20 }
77
80
  ]
78
81
  ```
79
82
 
83
+ **Best practice:** number priorities 10, 20, 30, and so on, so a model can be slotted in between later (a `15`)
84
+ without renumbering the rest. Haiku and Gemma share the lowest priority here, so they take turns going first (see below).
85
+
86
+ ### Rotating models within a priority
87
+
88
+ Give several models the same `priority:` and they share a tier. Each call rotates which of them goes first, so
89
+ every model gets an even share of the real work over many calls, a fair test of how well each handles it:
90
+
91
+ ```ruby
92
+ squishling escalation: [
93
+ { model: "claude-haiku-4-5", priority: 10 },
94
+ { model: "gemma-4b", priority: 10 },
95
+ { model: "claude-sonnet-5-5", priority: 20 }
96
+ ]
97
+ # call 1: haiku, gemma-4b, sonnet call 2: gemma-4b, haiku, sonnet call 3: haiku, gemma-4b, sonnet
98
+ ```
99
+
100
+ - If the model that goes first fails (invalid output or a provider failure, after its own `attempts:`), the other
101
+ models in the tier are tried, in rotated order, before the call escalates to the next priority.
102
+ - Each class and method counts its own calls (`AClass#a`, `AClass#b` and `BClass#a` rotate independently), shared across
103
+ threads, but not across processes: each process (a Puma worker, say) starts its own count at the first model, and it
104
+ restarts with the process. A `squish!` whose own escalation has a rotating tier counts against its method too.
105
+ - A tier rotates by its declared order, so ties keep the order they were written in. A tier with a single model
106
+ never rotates, and neither does an escalation without `priority:`. `rotate: false` on any step of a tier keeps the
107
+ whole tier in the order written, whatever its other steps say.
108
+ - Rotation also moves which stages the [`:squishsum` and `:judged_squishsum`](harnesses.md) harnesses sample first.
109
+
80
110
  `params:` on a step let one step switch to a reasoning model that rejects sampling params:
81
111
 
82
112
  ```ruby
data/docs/coolhand.md ADDED
@@ -0,0 +1,66 @@
1
+ # Coolhand Labs
2
+
3
+ Squishling ships an [observer](observers.md) for [Coolhand Labs](https://coolhandlabs.com/). With it, every LLM call a
4
+ squished method makes is filed under a named **workload** and **template** in Coolhand. Squishling knows exactly
5
+ where each call sits (which method, which escalation level, which sample, which judge), so it declares that for you.
6
+
7
+ ## Setup
8
+
9
+ Add and configure the [`coolhand`](https://github.com/Coolhand-Labs/coolhand-ruby) gem as described in
10
+ [Measuring token spend](measuring-tokens.md#coolhand-labs). **That's all.** Squishling defaults to the Coolhand
11
+ observer whenever Coolhand is already available in your codebase: loaded, with an `api_key`, and not disabled
12
+ (`config.enabled = false`). Nothing is added to Squishling's dependencies, and nothing happens when the gem isn't
13
+ there. It needs a version of the `coolhand` gem that has `Coolhand.with_context` (declared workloads); with an older
14
+ one, the observer stays off.
15
+
16
+ To turn it off, set `observer` to `false`:
17
+
18
+ ```ruby
19
+ Squishling.configure do |config|
20
+ config.observer = false # never declare, even though Coolhand is configured
21
+ end
22
+ ```
23
+
24
+ The other settings:
25
+
26
+ ```ruby
27
+ Squishling.configure do |config|
28
+ config.observer = :coolhand # always declare; raises ConfigurationError now if the gem is missing or too old
29
+ config.observer = nil # the default: declare when Coolhand is loaded and configured
30
+ end
31
+ ```
32
+
33
+ To report somewhere else, or as well, use any [other observer](observers.md). `config.observer` holds one.
34
+
35
+ ## What is declared
36
+
37
+ Each request runs inside `Coolhand.with_context(reporter: "squishling", workload: ..., template: ...)`, so Coolhand's
38
+ Net::HTTP interception tags it with the declaration.
39
+
40
+ | | Key | Name |
41
+ |---|---|---|
42
+ | Workload | `Class#method` | `Class#method` (its `description` is `Class#method`, or `Class#method (rescue)` when a [`squish!`](routing.md#handing-off-to-the-llm-with-squish) call handed it to the LLM) |
43
+ | Template | `Class#method/<role>` | `Class#method · <role>` |
44
+
45
+ `<role>` is where the request sits:
46
+
47
+ | Role | Request |
48
+ |---|---|
49
+ | `squish:L<n>` | Attempt *n* (1-based) of the [escalation](configuration.md#models-and-escalation), counting each attempt of each step (a [custom harness](harnesses.md#custom-harnesses) that escalates stage by stage restarts at L1 for each) |
50
+ | `sample:a`, `sample:b`, ... | A [squishsum](harnesses.md) sample (which runs on a worker thread; the declaration is made on that thread) |
51
+ | `judge` | A harness's judge, chat or System One (all of a judge's attempts share this one template) |
52
+
53
+ So `Parser#call/squish:L2` is the second attempt of `Parser#call`, e.g. `Parser#call · squish L2` in Coolhand.
54
+
55
+ - The model is deliberately not in the key. It is on the log, and the position is the stable identity.
56
+ - Repeated attempts on the same step count as separate levels, because the level is the attempt's position in the
57
+ expanded escalation.
58
+ - A class name long enough to push a key past Coolhand's 255-character limit is left undeclared rather than
59
+ breaking the call.
60
+ - Squishling's declaration is made inside any `Coolhand.with_context` you wrapped around the call, so for the fields it
61
+ sets, Squishling's wins.
62
+ - The description never contains the purpose text, which can be a proc that differs per call. The workload's key
63
+ and name stay `Class#method` either way, so calls that went through `squish!` and calls that didn't share one
64
+ workload.
65
+ - `observer = :coolhand` loads the gem when it is assigned, so assign it after `coolhand` is available (the default
66
+ needs no ordering).
data/docs/failures.md CHANGED
@@ -10,12 +10,12 @@ A failed attempt moves on to the next one; once the last fails, the error is rai
10
10
  |---|---|
11
11
  | Rate limit, 5xx, overload, timeout, connection error | RubyLLM retries at the HTTP level first (`RubyLLM.config.max_retries`, default 3). If it still fails, moves to the next attempt in a fresh chat; on the last attempt, raises `Squishling::LLMError` with the original exception as its `cause`. |
12
12
  | Bad API key, unknown model, missing provider config | Raises `Squishling::ConfigurationError`. Never escalated, never sent to the fallback. |
13
- | Provider rejects the request (400 Bad Request), e.g. an unsupported `temperature` or a schema it won't accept | Raises `ConfigurationError` with the provider's message and the params in use. Never escalated, never sent to the fallback, so a setup mistake can't be silently covered up on every call. |
13
+ | Provider rejects the request (400 Bad Request), e.g. an unsupported `temperature` or a schema it won't accept | Raises `ConfigurationError` naming the error class and HTTP status, and the params in use; the provider's own message is on `cause`. Never escalated, never sent to the fallback, so a setup mistake can't be silently covered up on every call. |
14
14
  | Empty or `nil` response (e.g. a refusal or a max-tokens cutoff) | Moves to the next attempt, then raises `Squishling::InvalidOutputError` |
15
15
  | Malformed or truncated JSON | Moves to the next attempt, then raises `InvalidOutputError`. JSON wrapped in a markdown code fence is accepted. |
16
16
  | JSON that doesn't match the schema (wrong types, missing or extra keys, `null` in a non-`optional` field, a broken [conditional rule](schemas.md#contracts-beyond-the-schema), root not an object) | Moves to the next attempt with the validation errors, then raises `InvalidOutputError` |
17
17
  | Schema-valid output that [`squish_validate`](#output-checks-squish_validate) rejects | Moves to the next attempt with your messages, then raises `InvalidOutputError` |
18
- | With a [squishsum or ensemble harness](harnesses.md): two valid samples that differ, with no judge or a judge that rejects both | Raises `Squishling::DisagreementError` (an `InvalidOutputError`), carrying both typed `candidates` |
18
+ | With a [squishsum harness](harnesses.md): two valid samples that differ, with no judge or a judge that rejects both | Raises `Squishling::DisagreementError` (an `InvalidOutputError`), carrying both typed `candidates` |
19
19
 
20
20
  Squishling validates output itself with [json_schemer](https://github.com/davishmcclurg/json_schemer), because
21
21
  RubyLLM doesn't, and some providers don't enforce strict mode. When the next attempt is on the same step (same
@@ -42,6 +42,9 @@ Model output can echo sensitive input, so the raw response is kept in one place:
42
42
  the message, the logger, and the next attempt's prompt.
43
43
  - A fixed message when a response contains a value JSON can't represent (`1e400` parses to `Infinity`, or a string
44
44
  with invalid UTF-8).
45
+ - For a failed provider request (`LLMError`, and the `ConfigurationError` for a 400, bad credentials, or a forbidden
46
+ key), only the error class and HTTP status. The provider's own message can echo request content or part of an API
47
+ key, so it stays on `cause` (`error.cause.message`) for you to read or log deliberately.
45
48
  - At most the first 20 of these errors, plus a count of the rest, so a very large malformed response can't flood
46
49
  the retry message or the log.
47
50
  - For a [`DisagreementError`](harnesses.md#when-the-harness-fails), only what Squishling wrote: a `:judgment` judge's
@@ -69,7 +72,7 @@ end
69
72
  |---|---|
70
73
  | `output` | The raw response: a Hash or String, or `nil` when the provider call itself failed |
71
74
  | `error` | `nil` for an accepted attempt. Otherwise the `InvalidOutputError` (with `errors` and `raw`) or `LLMError` that ended the attempt |
72
- | `metadata` | `label` (`"Class#method"`), `attempt`, `attempts` (the escalation's length; under a harness, the sample's or judge's own attempts), `final` (the last attempt), `model`, `provider`, `params`, `input` (the JSON sent: arguments and named context only), `usage` (token counts, when RubyLLM reports them) |
75
+ | `metadata` | `label` (`"Class#method"`), `attempt`, `attempts` (the escalation's length; under a harness, the sample's or judge's own attempts), `final` (the last attempt), `model`, `provider`, `params`, `input` (the JSON sent: arguments and named context only), `usage` (token counts, when RubyLLM reports them), `role` (`nil` for an escalation attempt, `:sample`, or `:judge`), `sample` (`"a"`, `"b"`, ... for a sample's attempt, else `nil`) |
73
76
 
74
77
  - Set it globally with `config.squawk`, per class with `squishling squawk: ...`, or per method with
75
78
  `squish :triage, squawk: ...`. The method's hook wins over the class's, which wins over the configured one;
@@ -79,7 +82,8 @@ end
79
82
  - It runs inline, so keep it quick. Exceptions it raises propagate unchanged and fail the call, like any of your own
80
83
  code, so rescue inside the hook if an outage in your tracing tool shouldn't.
81
84
  - `output` is the same object the result is built from, so treat it as read-only.
82
- - Under a [harness](harnesses.md), it runs for every sample and judge attempt too, each with its own `input`.
85
+ - Under a [harness](harnesses.md), it runs for every sample and judge attempt too, each with its own `input`, and
86
+ `role`/`sample` say which one it was. A [custom harness](harnesses.md#reporting-to-squawk) reports the same way.
83
87
  - It isn't called for a `ConfigurationError`, for deterministic and fallback returns, or for an attempt where your own
84
88
  `squish_validate` raises (that exception propagates first).
85
89
 
@@ -118,7 +122,7 @@ end
118
122
  | `Squishling::Error` | Base class for everything below. Raised directly for misuse at call time, e.g. `result` or `squish!` called outside a squished method |
119
123
  | `Squishling::ConfigurationError` | Missing purpose or schema, a non-strict schema, invalid or reserved params, an invalid `model:`/`escalation:` declaration, an invalid `append_to_purpose` item or unavailable source, bad credentials, an unknown model, a request the provider rejects (400) |
120
124
  | `Squishling::InvalidOutputError` | LLM output still invalid (schema or `squish_validate`) after every attempt in the escalation, or a deterministic/fallback return that doesn't match the schema |
121
- | `Squishling::DisagreementError` | A subclass of `InvalidOutputError`: a [squishsum or ensemble harness](harnesses.md#when-the-harness-fails)'s samples disagreed and no judge accepted either. `candidates`, `verdict`, and `reason` describe what happened |
125
+ | `Squishling::DisagreementError` | A subclass of `InvalidOutputError`: a [squishsum harness](harnesses.md#when-the-harness-fails)'s samples disagreed and no judge accepted either. `candidates`, `verdict`, and `reason` describe what happened |
122
126
  | `Squishling::LLMError` | The provider call failed on the last attempt in the escalation, after RubyLLM's own retries, including context-length errors (`cause` holds the original) |
123
127
 
124
128
  Errors raised by your own Ruby code are not wrapped.