squishling 0.1.0 → 0.3.0

data/docs/failures.md

@@ -2,34 +2,131 @@
 
 ## When the LLM fails
 
+Every squished call works through its [escalation](configuration.md#models-and-escalation), one attempt at a time.
+A failed attempt moves on to the next one; once the last fails, the error is raised (or handed to the
+[fallback](#fallbacks)). A plain `model:` makes a single attempt.
+
 | Failure | What Squishling does |
 |---|---|
-| Rate limit, 5xx, overload, timeout, connection error | RubyLLM retries at the HTTP level (`RubyLLM.config.max_retries`, default 3). If it still fails, raises `Squishling::LLMError`; the original exception is its `cause`. |
-| Bad API key, unknown model, missing provider config | Raises `Squishling::ConfigurationError`. Never retried, never sent to the fallback. |
-| Provider rejects the request (400 Bad Request), e.g. an unsupported `temperature` or a schema it won't accept | Raises `ConfigurationError` with the provider's message and the params in use. Never retried, never sent to the fallback, so a setup mistake can't be silently covered up on every call. |
-| Empty or `nil` response (e.g. a refusal or a max-tokens cutoff) | Re-asks, then raises `Squishling::InvalidOutputError` |
-| Malformed or truncated JSON | Re-asks, then raises `InvalidOutputError`. JSON wrapped in a markdown code fence is accepted. |
-| JSON that doesn't match the schema (wrong types, missing or extra keys, root not an object) | Re-asks with the validation errors, then raises `InvalidOutputError` |
+| Rate limit, 5xx, overload, timeout, connection error | RubyLLM retries at the HTTP level first (`RubyLLM.config.max_retries`, default 3). If it still fails, moves to the next attempt in a fresh chat; on the last attempt, raises `Squishling::LLMError` with the original exception as its `cause`. |
+| Bad API key, unknown model, missing provider config | Raises `Squishling::ConfigurationError`. Never escalated, never sent to the fallback. |
+| Provider rejects the request (400 Bad Request), e.g. an unsupported `temperature` or a schema it won't accept | Raises `ConfigurationError` with the provider's message and the params in use. Never escalated, never sent to the fallback, so a setup mistake can't be silently covered up on every call. |
+| Empty or `nil` response (e.g. a refusal or a max-tokens cutoff) | Moves to the next attempt, then raises `Squishling::InvalidOutputError` |
+| Malformed or truncated JSON | Moves to the next attempt, then raises `InvalidOutputError`. JSON wrapped in a markdown code fence is accepted. |
+| JSON that doesn't match the schema (wrong types, missing or extra keys, `null` in a non-`optional` field, a broken [conditional rule](schemas.md#contracts-beyond-the-schema), root not an object) | Moves to the next attempt with the validation errors, then raises `InvalidOutputError` |
+| Schema-valid output that [`squish_validate`](#output-checks-squish_validate) rejects | Moves to the next attempt with your messages, then raises `InvalidOutputError` |
+| With a [squishsum or ensemble harness](harnesses.md): two valid samples that differ, with no judge or a judge that rejects both | Raises `Squishling::DisagreementError` (an `InvalidOutputError`), carrying both typed `candidates` |
 
 Squishling validates output itself with [json_schemer](https://github.com/davishmcclurg/json_schemer), because
-RubyLLM doesn't, and some providers don't enforce strict mode. Re-asks happen in the same conversation, so the model
-sees what it got wrong. `Squishling.config.max_retries` (default 1) sets how many follow-ups are allowed.
-`InvalidOutputError` exposes `errors`, `raw` (the last response), and `attempts`.
+RubyLLM doesn't, and some providers don't enforce strict mode. When the next attempt is on the same step (same
+model, provider, params, and `forward_rejected:`), the re-ask happens in the same conversation, so the model sees
+what it got wrong. A different step gets a fresh chat with the original input plus the rejected output (truncated to
+4,000 characters) and its errors, so that output is sent to the next step's provider, which may not be the one that
+produced it. Set `forward_rejected: false` on a step to leave both out. `InvalidOutputError` exposes `errors`, `raw`
+(the last response), `attempts`, and `models` (the model tried on each attempt); for a
+[`DisagreementError`](harnesses.md#when-the-harness-fails), `raw` holds both candidates, `attempts` is `nil`, and
+`models` has one entry per role (sample a, sample b, then the judge). With a `logger` configured, every
+escalation is logged as a warning.
+
+### What ends up in errors and logs
+
+Model output can echo sensitive input, so the raw response is kept in one place: `InvalidOutputError#raw` (plus the opt-in `squawk` hook below).
+`InvalidOutputError#message`, `#errors`, and `config.logger` warnings are built from these:
+
+- For unparseable JSON, only the position (`response was not valid JSON (at line 1 column 7)`), never the parser's
+  snippet of the response.
+- JSON Schema errors, which name the schema path and the rule that failed. They never contain the rejected value, but
+  they do name an extra key the model added (`object property at /foo is a disallowed additional property`), and
+  that key name is chosen by the model.
+- Your own `squish_validate` messages, verbatim. If you interpolate result values into them, those values reach
+  the message, the logger, and the next attempt's prompt.
+- A fixed message when a response contains a value JSON can't represent (`1e400` parses to `Infinity`, or a string
+  with invalid UTF-8).
+- At most the first 20 of these errors, plus a count of the rest, so a very large malformed response can't flood
+  the retry message or the log.
+- For a [`DisagreementError`](harnesses.md#when-the-harness-fails), only what Squishling wrote: a `:judgment` judge's
+  choice and probability. A chat judge's `reason` is model-written, so it is available as `#reason` but never in the
+  message or logs.
+
+The same applies to anything you log yourself, such as `error.message` in a [fallback](#fallbacks). Use `error.raw`
+only where you are willing to store model output. To send the output of every attempt somewhere on purpose, use
+[`squawk`](#observing-every-attempt-squawk).
+
+## Observing every attempt (`squawk`)
+
+`squawk` is an opt-in hook for sending what the model returned to an error tracker or tracing tool. It runs after
+every LLM attempt, accepted or rejected, and does nothing unless you set it:
+
+```ruby
+Squishling.configure do |config|
+  config.squawk = lambda do |output:, metadata:, error:|
+    ObservabilitySolution.record(output, metadata, error) if error
+  end
+end
+```
+
+| Keyword | Value |
+|---|---|
+| `output` | The raw response: a Hash or String, or `nil` when the provider call itself failed |
+| `error` | `nil` for an accepted attempt. Otherwise the `InvalidOutputError` (with `errors` and `raw`) or `LLMError` that ended the attempt |
+| `metadata` | `label` (`"Class#method"`), `attempt`, `attempts` (the escalation's length; under a harness, the sample's or judge's own attempts), `final` (the last attempt), `model`, `provider`, `params`, `input` (the JSON sent: arguments and named context only), `usage` (token counts, when RubyLLM reports them) |
+
+- Set it globally with `config.squawk`, per class with `squishling squawk: ...`, or per method with
+  `squish :triage, squawk: ...`. The method's hook wins over the class's, which wins over the configured one;
+  `squawk: false` silences an inherited hook. Subclasses and `squish!` calls inherit it.
+- Any object that responds to `call` works. The hook receives only the keywords it declares, unless it takes `**`,
+  so a lambda that wants just `error:` is fine and new metadata fields won't break it.
+- It runs inline, so keep it quick. Exceptions it raises propagate unchanged and fail the call, like any of your own
+  code, so rescue inside the hook if an outage in your tracing tool shouldn't.
+- `output` is the same object the result is built from, so treat it as read-only.
+- Under a [harness](harnesses.md), it runs for every sample and judge attempt too, each with its own `input`.
+- It isn't called for a `ConfigurationError`, for deterministic and fallback returns, or for an attempt where your own
+  `squish_validate` raises (that exception propagates first).
+
+## Output checks (`squish_validate`)
+
+The schema covers shape and types. For rules it can't express, such as cross-field arithmetic or a lookup against
+your data, add a Ruby check. It runs only on LLM output that already matches the schema, receives the typed result
+plus the method's inputs as keywords, and runs against the instance:
+
+```ruby
+class InvoiceParser
+  include Squishling
+  # ...
+  squish_validate do |result, **|
+    errors = []
+    errors << "total must equal the sum of line_items" unless result.total == result.line_items.sum(&:amount)
+    errors << "unknown currency" unless Currency.supported?(result.currency)
+    errors
+  end
+end
+```
+
+- Return `nil`, `true`, `""`, or `[]` to accept the output.
+- Return a message or an array of messages to reject it. The messages are sent to the next attempt and end up in
+  `InvalidOutputError#errors` if every attempt fails. `false` rejects with a generic message.
+- A [dry-validation](https://dry-rb.org/gems/dry-validation/) result works too, if your app already uses it:
+  `squish_validate { |result, **| InvoiceContract.new.call(result.to_h) }`. Squishling doesn't depend on dry-rb.
+- Per method: `squish :triage, validate: ->(result, **inputs) { ... }`. Subclasses inherit the class-level check.
+- It doesn't run on deterministic or fallback returns: those are your own code.
+- Exceptions raised inside it propagate unwrapped, like any of your own code.
 
 ## Errors
 
 | Error | Raised when |
 |---|---|
 | `Squishling::Error` | Base class for everything below. Raised directly for misuse at call time, e.g. `result` or `squish!` called outside a squished method |
-| `Squishling::ConfigurationError` | Missing instructions or schema, a non-strict schema, invalid or reserved params, an invalid `append_instructions` item or unavailable source, bad credentials, an unknown model, a request the provider rejects (400) |
-| `Squishling::InvalidOutputError` | LLM output still invalid after `max_retries`, or a deterministic/fallback return that doesn't match the schema |
-| `Squishling::LLMError` | The provider call failed after RubyLLM's own retries, including context-length errors (`cause` holds the original) |
+| `Squishling::ConfigurationError` | Missing purpose or schema, a non-strict schema, invalid or reserved params, an invalid `model:`/`escalation:` declaration, an invalid `append_to_purpose` item or unavailable source, bad credentials, an unknown model, a request the provider rejects (400) |
+| `Squishling::InvalidOutputError` | LLM output still invalid (schema or `squish_validate`) after every attempt in the escalation, or a deterministic/fallback return that doesn't match the schema |
+| `Squishling::DisagreementError` | A subclass of `InvalidOutputError`: a [squishsum or ensemble harness](harnesses.md#when-the-harness-fails)'s samples disagreed and no judge accepted either. `candidates`, `verdict`, and `reason` describe what happened |
+| `Squishling::LLMError` | The provider call failed on the last attempt in the escalation, after RubyLLM's own retries, including context-length errors (`cause` holds the original) |
 
 Errors raised by your own Ruby code are not wrapped.
 
 ## Fallbacks
 
-Use `squish_fallback` to decide what happens when the LLM path fails with `InvalidOutputError` or `LLMError`.
+Use `squish_fallback` to decide what happens when the LLM path fails with `InvalidOutputError` (including a
+`DisagreementError`) or `LLMError`. It's the one fallback for every [harness](harnesses.md).
 It receives the error plus the method's inputs as keywords, and runs against the instance:
 
 ```ruby
@@ -49,7 +146,7 @@ end
 - Per method: `squish :triage, fallback: ->(error, **inputs) { ... }`.
 - Subclasses inherit the class-level fallback.
 - Calls handed to the LLM with `squish!` use the fallback too. A fallback can't call `squish!` itself; that
-  raises `Squishling::Error` rather than looping. See [Escalating from Ruby](routing.md#escalating-from-ruby-with-squish).
+  raises `Squishling::Error` rather than looping. See [Handing off to the LLM](routing.md#handing-off-to-the-llm-with-squish).
 - Without a fallback, the error propagates.
 
 Routing to Ruby code isn't automatic on failure. The predicate already chose the LLM for this input, so the

data/docs/harnesses.md

@@ -0,0 +1,198 @@
+# Harnesses: escalation, squishsum, and ensemble
+
+A squished call's **harness** decides how it uses its [escalation](configuration.md#models-and-escalation):
+
+| Harness | What it does | Requests per call |
+|---|---|---|
+| `:escalation` (default) | Tries each attempt in order until an output passes the schema and `squish_validate`. | 1 or more |
+| `:squishsum` | Asks the escalation's **first step** twice, concurrently. Identical outputs are accepted; different ones raise `Squishling::DisagreementError`. | 2 or more |
+| `:judged_squishsum` | Like `:squishsum`, but when the two outputs differ, a **judge** picks one or rejects both. | 2 or more, plus 1 or more judge requests when they differ |
+| `:ensemble` | Asks the escalation's **first and second steps** once each, concurrently, so two different models are compared. Identical outputs are accepted; different ones raise `DisagreementError`. | 2 or more |
+| `:judged_ensemble` | Like `:ensemble`, but when the two outputs differ, a **judge** picks one or rejects both. | 2 or more, plus 1 or more judge requests when they differ |
+
+Use a squishsum harness when a confidently wrong answer costs more than a second request: one sample can make a
+mistake, but two independent samples rarely make the same one. Use an [ensemble](#ensembles) when the two samples
+should come from different models: a model rarely disagrees with itself at low temperature, but a cheap and a
+stronger model often disagree on exactly the cases the cheap one gets confidently wrong.
+
+```ruby
+class TicketTriager
+  include Squishling
+
+  squishling escalation: [{ model: "claude-haiku-4-5", attempts: 2 }, "claude-sonnet-5-5"],
+             harness: :judged_squishsum   # two Haiku samples; Sonnet judges when they differ
+  purpose "Assign a priority and a team."
+  output_schema do
+    string :priority, enum: %w[low medium high]
+    string :team
+  end
+end
+```
+
+## Declaring a harness
+
+`harness:` takes a type, or a Hash with `type:` and its options:
+
+```ruby
+Squishling.configure { |c| c.default_harness = :squishsum }      # every class without its own
+
+squishling harness: :judged_squishsum                            # this class (and subclasses)
+squish :triage, harness: { type: :squishsum,                     # one method
+                           compare: ->(a, b, **) { a.priority == b.priority } }
+squish!(harness: :escalation)                                    # one call (see routing.md)
+```
+
+The first level that declares one wins: `squish!`, then the method, then the class, then
+`Squishling.config.default_harness`, then `:escalation`. Options aren't merged across levels.
+
+| Option | Harnesses | Description |
+|---|---|---|
+| `type:` | all | `:escalation`, `:squishsum`, `:judged_squishsum`, `:ensemble`, or `:judged_ensemble` (required in the Hash form). |
+| `compare:` | squishsum and ensemble ones | `->(a, b, **inputs) { ... }`, evaluated against the instance with the two typed results and the method's inputs. Truthy means the samples agree. Default: the two outputs are exactly equal. |
+| `judge:` | judged ones | The judge step (see [The judge](#the-judge)). Default: the escalation's next step after the sampled ones. |
+| `judge_instructions:` | judged ones | A String, or a Proc evaluated against the instance, that replaces the [default judge prompt](#the-default-judge-prompt). |
+
+## Samples
+
+Under `:squishsum`, both samples run on the escalation's first step, with its model, provider, and params
+(an [ensemble](#ensembles) runs them on its first and second steps):
+
+- **They're independent.** Each sample has its own chat, and the two requests are sent concurrently on separate
+  threads. Only the provider requests run on those threads. Parsing, schema validation, `squish_validate`,
+  `compare:`, and the judge all run on the calling thread, so your Squishling callbacks never run concurrently.
+  RubyLLM instrumentation subscribers (e.g. `ActiveSupport::Notifications` listeners for `chat.ruby_llm`) do see
+  each sample's request on its worker thread, outside the Rails executor, so keep them thread-safe.
+- **Each one is retried like an escalation step.** A sample with invalid output is re-asked in its own chat, up to
+  their step's `attempts:`, while a sample that already passed keeps its result. A failed request gets a fresh
+  chat. A sample still failing after its attempts fails the call with `InvalidOutputError` or `LLMError`, as
+  usual. Under `:squishsum`, later escalation steps are never used for samples (see [Ensembles](#ensembles)).
+- **Agreement is exact by default.** Free-text fields seldom match word for word, so pass `compare:` to decide
+  which fields must match:
+
+  ```ruby
+  squishling harness: { type: :squishsum, compare: ->(a, b, **) { a.priority == b.priority && a.team == b.team } }
+  ```
+
+  When the samples agree, the first one is returned.
+
+## Ensembles
+
+`:ensemble` and `:judged_ensemble` sample two *different* steps instead of one step twice. Sample `a` is the
+escalation's first step and sample `b` its second, each with its own model, provider, and params:
+
+```ruby
+class TicketTriager
+  include Squishling
+
+  squishling escalation: [{ model: "claude-haiku-4-5", attempts: 2 }, "claude-sonnet-5-5", "claude-opus-5-5"],
+             harness: :judged_ensemble   # a = Haiku, b = Sonnet, Opus judges when they differ
+  # purpose, output_schema, ...
+end
+```
+
+- A step's `attempts:` are that sample's retries (here Haiku gets two attempts, Sonnet one), exactly as in
+  [Samples](#samples). Steps after the second are never sampled.
+- Everything else matches squishsum: independence and concurrency, `compare:`, `DisagreementError`, and the
+  judge. `DisagreementError#models` names the model behind each sample.
+- The judge defaults to the escalation's **third** step. With only two steps, a `:judged_ensemble` needs `judge:`.
+  An ensemble with fewer than two steps, or a judged one with neither a third step nor `judge:`, raises
+  `ConfigurationError` before any request.
+- A global `default_harness = :ensemble` therefore needs every class to declare at least two steps, and a
+  `squish!(model: ...)` override (one step) can't be combined with an ensemble harness.
+- Steps are told apart by value, so two adjacent steps with the same model, provider, and params count as one step.
+
+## The judge
+
+With a judged harness, two valid samples that disagree go to a judge. The judge picks candidate `a` or `b`,
+whose typed result is returned (`squished?` is `true`), or it picks `neither`, which raises
+`DisagreementError`.
+
+The judge is:
+
+1. the `judge:` step, if declared: a model name, or a Hash with `model:`, `provider:`, `params:`, and `attempts:`
+   (as in an escalation step, without `order:`), plus `type:`; or
+2. the escalation's next step after the sampled ones (the second under `:judged_squishsum`, the third under
+   `:judged_ensemble`), with its attempts.
+
+A judged harness with neither raises `ConfigurationError` before any sample is requested. A `judge:`
+step uses only its own `provider:`; it doesn't inherit the class or method `provider:`, which belongs to their models
+(a chat judge's params, though, do merge over the method's).
+
+### Chat judges
+
+By default the judge is a chat model, asked for a strict `{ "verdict": "a" | "b" | "neither", "reason": "..." }`.
+It gets the same context the samples had, plus the samples themselves:
+
+- **System prompt:** the judge prompt, then the operation's own purpose (including any `append_to_purpose`
+  sections).
+- **User message:** JSON with the samples' `input` (`arguments` and `context`), the `output_schema`, and the
+  `candidates` `a` and `b`.
+
+A verdict that doesn't match its schema is retried per the judge step's `attempts:`; a judge that never returns a
+valid verdict raises `InvalidOutputError`. A chat judge's params are the method's
+[generation params](configuration.md#generation-params) with its own `params:` merged over them.
+
+### System One judgment models (Jev)
+
+A judge can also be a System One decision model, such as TypeSafe's Jev, via
+[RubyLLM judgments](https://rubyllm.com/judgments/) (RubyLLM 2.1+). Decision models return calibrated
+probabilities rather than text, so they are much faster and cheaper than a chat judge.
+
+```ruby
+squishling harness: {
+  type: :judged_squishsum,
+  judge: { model: "jev-latest", type: :judgment, min_confidence: 0.8 }
+}
+```
+
+Squishling asks one `choice` question (`a`, `b`, or `neither`), with the judge prompt as its instructions and the
+operation's purpose, input, output schema, and candidates as the judgment input. A pick of `a` or `b` is
+accepted only when its probability is at least `min_confidence:` (default `0.8`). Otherwise, as with `neither`,
+`DisagreementError` is raised with the choice and its probability in its `reason`. A judgment judge sends only its own
+`params:` (as RubyLLM `provider_options:`). `attempts:` retries a failed request. Configure the provider's
+credentials in RubyLLM as usual.
+
+### The default judge prompt
+
+```text
+You are an impartial judge. Two independent attempts at the same operation returned different outputs,
+candidate "a" and candidate "b". Both already match the required output format, so judge only their
+content. You are given the operation's purpose and its input. Decide which candidate correctly and
+faithfully carries out the purpose for this input. If both do (for example, they differ only in
+wording), choose either one, preferring "a". Choose "neither" only when both are wrong or you can't tell
+whether either is correct. Never combine them or invent a third answer.
+```
+
+It is available as `Squishling::Harness::DEFAULT_JUDGE_INSTRUCTIONS`. Replace it with `judge_instructions:`:
+
+```ruby
+squishling harness: { type: :judged_squishsum,
+                      judge_instructions: "Pick the candidate whose priority follows our SLA rules: ..." }
+```
+
+## When the harness fails
+
+A disagreement raises `Squishling::DisagreementError`, a subclass of `InvalidOutputError`. It's raised when the
+samples differ with no judge, or when the judge rejects both. Like any failure on the LLM path, it goes to
+[`squish_fallback`](failures.md#fallbacks), which can still return one of the candidates:
+
+```ruby
+squish_fallback do |error, **|
+  raise error unless error.is_a?(Squishling::DisagreementError)
+
+  error.candidates.first   # already typed and validated; squished? is true
+end
+```
+
+| Attribute | Value |
+|---|---|
+| `candidates` | The two typed results, `[a, b]` |
+| `verdict` | `nil` when there was no judge; `:neither` when the judge rejected both |
+| `reason` | The judge's reason (or, for a judgment model, the choice and its probability). A chat judge's reason is model-written, so it is not part of `message` or the logs; a judgment model's choice and probability are |
+| `raw` | Both candidates as Hashes, `[a.to_h, b.to_h]` |
+| `attempts` | `nil` |
+| `models` | The model behind each role: sample `a`, sample `b`, then the judge if one ran (one entry each, not one per attempt) |
+
+Other failures are unchanged. A sample or judge that never produces valid output raises `InvalidOutputError`, a
+failed request on the last attempt raises `LLMError`, and setup mistakes (including a provider 400 on either
+sample) raise `ConfigurationError`, which is never sent to the fallback.

data/docs/measuring-tokens.md

@@ -0,0 +1,148 @@
+# Measuring token spend
+
+A squished path costs tokens on every call. A Ruby path costs engineer time once, plus upkeep. Measuring the
+first tells you when the second is worth paying.
+
+## What Squishling gives you
+
+- `result.squished?` says which path served a call, so you can count LLM calls per class.
+- The [`squawk` hook](failures.md#observing-every-attempt-squawk) runs after every LLM attempt and receives the
+  method (`label`, e.g. `"ShipmentUpdate#call"`), the model and provider, which attempt it was, and the token usage
+  RubyLLM reported. That is enough to attribute spend to a squishling method without any other tooling.
+- Cost in dollars comes from RubyLLM's model pricing. Multiply the reported tokens by your model's price, or read it
+  from RubyLLM directly (see [the instrumenter](#lower-level-rubyllms-instrumenter)).
+
+## Deciding when to write the code
+
+For each squished path, over a month:
+
+```
+token cost  =  calls × (input tokens × input price + output tokens × output price)
+code cost   =  time to write it  +  upkeep  (new cases, format changes, on-call)
+```
+
+Write the Ruby when the token cost keeps exceeding the code cost, and narrow `squish_when` so only the inputs
+Ruby can't handle still go to the LLM. See [Hardening a path](routing.md#hardening-a-path). If the token cost
+stays small, the debt you haven't taken on is cheaper than the code.
+
+## Tools
+
+| Option | Good for | Setup |
+|---|---|---|
+| [Coolhand Labs](#coolhand-labs) | Picking up your squishling calls automatically and measuring their cost and accuracy | The `coolhand` gem and an initializer |
+| [Custom: the `squawk` hook](#custom-the-squawk-hook) | Logging or metrics in your own stack, with no new dependency | A lambda and one config line |
+| [OpenTelemetry](#opentelemetry) | Sending traces to Datadog, Langfuse, Arize, Braintrust, LangSmith, or any OTel backend | The OpenTelemetry SDK, an exporter, and one line |
+| [LangSmith](#langsmith) | Browsing traces and token usage in LangSmith | Through OpenTelemetry |
+
+### Coolhand Labs
+
+Add and initialize the [`coolhand`](https://github.com/Coolhand-Labs/coolhand-ruby) gem. It intercepts the LLM
+requests your app makes, so it picks up your squishling calls automatically and measures their cost, and their
+accuracy through Coolhand's feedback tools. No changes to your squishling classes.
+
+```ruby
+# Gemfile
+gem "coolhand"
+
+# config/initializers/coolhand.rb
+Coolhand.configure do |config|
+  config.api_key = ENV.fetch("COOLHAND_API_KEY")
+end
+```
+
+Get an API key at [coolhandlabs.com](https://coolhandlabs.com/). To keep the data in your own infrastructure, point
+`config.base_url` at a Coolhand-compatible endpoint of your own.
+
+### Custom: the `squawk` hook
+
+`config.squawk` runs after every LLM attempt, accepted or rejected, and is handed the method's label, the model,
+and the token usage RubyLLM reported. A lambda can take just the keywords it needs:
+
+```ruby
+Squishling.configure do |config|
+  config.squawk = lambda do |metadata:, **|
+    usage = metadata[:usage] || {}   # nil when the provider call failed; counts it didn't report are absent
+    Rails.logger.info("squishling #{metadata[:label]} model=#{metadata[:model]} " \
+                      "attempt=#{metadata[:attempt]}/#{metadata[:attempts]} " \
+                      "in=#{usage[:input_tokens]} out=#{usage[:output_tokens]}")
+  end
+end
+```
+
+Each line carries the `"Class#method"` that made the call, so grouping by label gives you tokens per squishling. A call
+that stays in Ruby makes no LLM request and never reaches the hook, so a label that stops appearing is a path that has
+been hardened. You can also set the hook per class (`squishling squawk: ...`) or per method. See
+[Observing every attempt](failures.md#observing-every-attempt-squawk) for every keyword and the rules for exceptions.
+
+### Lower level: RubyLLM's instrumenter
+
+To meter every RubyLLM request in your app, not only Squishling's, use RubyLLM's own instrumentation. It reports a
+`usage.ruby_llm` event for each provider request, with `model`, `provider`, `status`, `tokens` (`input`, `output`,
+and more) and `cost`. Give it an object that responds to `instrument(name, payload)` and optionally takes a block. In
+Rails, `ActiveSupport::Notifications` is used automatically.
+
+```ruby
+class TokenMeter
+  def instrument(name, payload = {})
+    if name == "usage.ruby_llm"
+      Rails.logger.info("llm model=#{payload[:model]} workflow=#{payload[:workflow_name]} " \
+                        "in=#{payload[:tokens].input} out=#{payload[:tokens].output}")
+    end
+    block_given? ? yield : nil
+  end
+end
+
+RubyLLM.configure { |config| config.instrumenter = TokenMeter.new }
+```
+
+These events don't know which squishling made the request. To attribute the spend, wrap the call in a RubyLLM
+workflow, and every event emitted inside the block carries its name, id and any `metadata:` you pass. (The
+`squawk` hook above needs no wrapping.)
+
+```ruby
+RubyLLM.workflow("ShipmentUpdate", metadata: { carrier: "acme-freight" }) do
+  ShipmentUpdate.call(carrier: "acme-freight", payload: body)
+end
+```
+
+### OpenTelemetry
+
+RubyLLM 2.1+ ships its own OpenTelemetry tracer. Each request becomes a span with the model, request settings and
+token usage (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`), following the OpenTelemetry GenAI
+conventions. It does not record message text. Your app owns the SDK and exporters.
+
+```ruby
+# Gemfile
+gem "opentelemetry-sdk"
+gem "opentelemetry-exporter-otlp"   # or your backend's exporter
+
+# config/initializers/opentelemetry.rb
+OpenTelemetry::SDK.configure        # reads the standard OTEL_* environment variables
+RubyLLM::OpenTelemetry.enable       # starts tracing RubyLLM requests
+```
+
+Spans don't say which squishling made the request, so pair this with a `RubyLLM.workflow` (see
+[above](#lower-level-rubyllms-instrumenter)) or group by model and prompt in your backend.
+
+### LangSmith
+
+LangSmith accepts OpenTelemetry traces, so the OpenTelemetry setup above covers it: point the OTLP exporter at
+LangSmith. See LangSmith's
+[OpenTelemetry docs](https://docs.smith.langchain.com/observability/how_to_guides/tracing/trace_with_opentelemetry)
+for the endpoint and headers. A community Ruby gem, [`langsmith-sdk`](https://github.com/felipekb/langsmith-ruby-sdk),
+also exists; it isn't published by LangChain.
+
+## Privacy
+
+Squishling only sends the model what you name: the method arguments, any `squish_context` values, and any source you
+append. Tools that sit on the request path are a separate route out of your process, and they see both sides:
+
+- **Request interceptors, such as `coolhand`,** record the prompts (your purpose text and those inputs) and the
+  model's responses. Coolhand sends them to coolhandlabs.com unless you point `config.base_url` at an endpoint of your
+  own.
+- **A `squawk` hook that forwards `output:` or `metadata[:input]`** sends model output and your inputs wherever you
+  send them. That is the sanctioned route for model output; see
+  [Observing every attempt](failures.md#observing-every-attempt-squawk). The examples above log only metadata.
+- **OpenTelemetry** with RubyLLM's tracer records usage and request settings, not message text.
+
+Check what your tool stores before pointing it at inputs that contain personal data.

data/docs/naming.md

@@ -0,0 +1,43 @@
+# Naming and collisions
+
+`include Squishling` adds a small set of methods to your class. This page lists them and explains what happens
+when one of those names is already taken.
+
+## What Squishling adds
+
+| Where | Names |
+|---|---|
+| Class-level DSL | `squishling`, `purpose`, `append_to_purpose`, `output_schema`, `squish`, `squish_when`, `squish_fallback`, `squish_validate`, `squish_context`, `squished_methods`, `call`, and the `squishling_*` readers |
+| Instance | `squishling_result`, `squish!`, and `result` (an alias for `squishling_result`) |
+
+Methods your class defines itself always win over these, whether you define them before or after the `include`.
+Only methods your class *inherits* can be shadowed.
+
+## Inherited collisions raise
+
+If a parent class or an earlier-included module already defines a class-level DSL method, `squish!` or
+`squishling_result`, `include Squishling` raises `Squishling::ConfigurationError` listing every clash and where
+it comes from:
+
+```
+ConfigurationError: MyApp inherits methods that Squishling would override: MyApp.call (from Sinatra::Base's
+class methods). Include Squishling in a plain Ruby class instead ...
+```
+
+This is deliberate. Squishling's modules sit ahead of inherited ones, so without the check
+`Sinatra::Base.call(env)` (the Rack entry point), or any `call` or `purpose` your parent class defines, would be
+replaced silently. Put Squishling in a plain Ruby class and have your framework object call it instead.
+
+## `result` is skipped when taken
+
+`result` is a convenience for `squishling_result`. If the class already has a `result` (its own or inherited), it is
+left alone and Squishling does not add the alias. Use `squishling_result(...)` instead; it is always available.
+
+**ActiveRecord caveat.** Column readers are generated lazily, so Squishling can't see a `result` column when you
+include it. Its `result` would then shadow the column reader. On a model with a `result` column, call
+`squishling_result` (or keep Squishling out of the model and use a plain class).
+
+## Not a collision
+
+ActiveSupport adds `String#squish` and `String#squish!`. Those live on `String`; Squishling's `squish!` lives on
+your class, so they never meet. They only look alike when reading code in a Rails app.