squishling 0.5.1 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 14b40817f3f49682f1ed311aa5c05007dfc7f55957c9cf89683fd26fea962aa7
-  data.tar.gz: 32b93a239eaa8f5994675a24fca6598e5e0a9edeb8d77efa97ebcf9783db6aa4
+  metadata.gz: 1d16d5c74c4eb8fee390ba827002dee27418f0cc66694d1fc2762893b1c027fb
+  data.tar.gz: 385bad5ba43f729d35260e7362955c35d5ffe7b4db731d1d6c0abe285e62d569
 SHA512:
-  metadata.gz: 184955d83ece437fbf86d7f93c2103a1c61302d8df9b70f7430b49b2896c7aab42e04d1f7503203979ff1087ed8e05b67baeb767c9283c43f4bafa82c720abd8
-  data.tar.gz: 351cdbe7b06b587e2c236ff75012fae0bf2fa526613a2d79e6b6c37aac643547bc22cfc74f73cb2e965dc31dd5f0234a7addf034fce8b1de5ea8dd74dab529d3
+  metadata.gz: 2a06a78caf781e12ea7dca4208bca5cc5bb008578cb3a26095b790477d82d2e0ebe5f574599b13acd36ffdfe83e0e269059e89b9f9c8285b61f55376f37e6002
+  data.tar.gz: 940b86dfd9a8d612360fdc76e05532320dc65ac5f02bbfcf1f5600a863b399f569de650f373921fdb2ac8665edf1b039ec329b19e4ef6b5b662c135341bbca7b

data/CHANGELOG.md

@@ -7,6 +7,47 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.6.0] - 2026-11-01
+
+Prompt caching, on by default. Read **Changed** before upgrading: requests now ask the provider to cache the stable
+prompt, which can bill a cache write on providers that charge for one.
+
+### Added
+
+- Prompt caching: a `cache` setting that asks the provider to cache the stable prompt (the purpose and the input note in
+  the system prompt), so a repeated call can read it back instead of paying for it again. It is `true`, `false`, or a
+  Hash of options such as `{ ttl: "1h" }`, set with `config.default_cache`, `squishling cache:`, `squish :method,
+  cache:`, `squish!(cache:)`, or an escalation step's `cache:`; the more specific level wins. There is no class-level
+  `cache false` macro, since it would collide with `ActiveRecord::Base.cache`: use `squishling cache: false`. The RubyLLM
+  adapter asks for it with `Chat#with_caching` and sends no cache controls when it is off; a System One judgment is never
+  cached. See [Prompt caching](docs/configuration.md#prompt-caching). (#79, #80)
+- `Observer::Usage` gains `cache_read_tokens` and `cache_write_tokens`, and `Adapter::Request`, `ModelPath::Step`, and
+  `Testing::Request` gain a `cache` member (defaulting to `false`, so existing constructors keep working). Adapters that
+  report cache usage put `:cache_read_tokens` and `:cache_write_tokens` in `Adapter::Response#tokens`. (#79, #80)
+- `config.budget_counts_cache_reads` (default `false`, `true` or `false` only): whether the tokens a request reads from the
+  provider's prompt cache count toward `max_total_input_tokens`. (#97)
+
+### Changed
+
+- **Behavior change:** prompt caching is on by default (`config.default_cache` is `true`). With the RubyLLM adapter, a
+  provider that bills for cache writes (such as Anthropic Claude) may charge a write premium on the first call. Set
+  `config.default_cache = false`, or `cache: false` on a class, method, or step, to opt out. A model that can't take
+  cache controls may reject the request, which raises `ConfigurationError`: set `cache: false` for that model. The
+  OpenAI-compatible adapter ignores the setting. (#79, #80)
+- **Behavior change:** `max_total_input_tokens` now counts the tokens written to the prompt cache, which RubyLLM
+  reports apart from `input_tokens`, so a call that writes a large cached prompt can reach the budget sooner than before.
+  Tokens read from the cache are left out unless `budget_counts_cache_reads` is `true`. Output tokens are unchanged.
+  (#79, #80, #97)
+- **Behavior change:** the OpenAI-compatible adapter reports `input_tokens` net of cached tokens, as every adapter
+  does (the cached reads go to `cache_read_tokens`, and a server's cache writes to `cache_write_tokens`). A custom
+  adapter's `input_tokens` should likewise exclude cached tokens. Code that read `input_tokens` from `squawk` usage or an
+  observer for this adapter will see a smaller number on cache hits. (#79, #80)
+
+### Documented
+
+- The adapter docs list the separate [`squishling-apple_fm`](https://github.com/Coolhand-Labs/squishling-apple-fm-adapter)
+  and [`squishling-claude_code`](https://github.com/Coolhand-Labs/squishling-claude-code-adapter) adapter gems. (#94)
+
 ## [0.5.1] - 2026-10-31
 
 The v0.5.0 red-team follow-ups (#70): data-egress, secret-leakage, and misclassification fixes. Several of them change

data/README.md

@@ -172,7 +172,7 @@ sees it right where the value goes, and the purpose stays short and about the go
   the OpenAI-compatible adapter (OpenAI, Ollama, vLLM, LM Studio, Together, Groq, … over plain HTTP), or write your
   own in one method.
 - **Any provider and model**: set a universal, per-class, per-method, or per-call model, plus layered generation params (temperature,
-  reasoning effort, top_p, …).
+  reasoning effort, top_p, …) and [prompt caching](docs/configuration.md#prompt-caching) (on by default, opt out per class, method, or call).
 - **Model escalation**: declare an `escalation:` instead of a `model:`, with per-step `attempts:`, `priority:` (models tied on priority take turns going first), provider,
   params, `forward_rejected:`, `retry_temperature:`, and `include_when:` (leave a model out of calls where a Proc is falsy). Invalid output (with its errors fed back) or a provider failure moves on to the next attempt. Steps can
   cross providers, e.g. a local Qwen model on Ollama, then Anthropic Claude Haiku on AWS Bedrock, then Claude Opus:

data/docs/adapters.md

@@ -3,12 +3,14 @@
 An **adapter** is how Squishling reaches a model. Squishling builds the request (system prompt, conversation so far,
 strict output schema, generation params), the adapter makes one completion and returns the model's output, and
 Squishling does the rest: validation, `squish_validate`, retries and escalation, harnesses, `squawk`, and fallbacks.
-Two adapters ship, and you can write your own.
+Two adapters ship, separate gems add more (below), and you can write your own.
 
 | Adapter | Reaches | Needs |
 |---|---|---|
 | [`:ruby_llm`](#ruby_llm) | OpenAI, Anthropic Claude, Google Gemini, AWS Bedrock, OpenRouter, and every provider [RubyLLM](https://rubyllm.com) supports | `gem "ruby_llm"` |
 | [`:openai_compatible`](#openai-compatible-endpoints) | Any server that speaks OpenAI's Chat Completions API: OpenAI, Ollama, vLLM, LM Studio, llama.cpp, Together, Groq, … | nothing (Ruby's `Net::HTTP`) |
+| [`squishling-apple_fm`](https://github.com/Coolhand-Labs/squishling-apple-fm-adapter) (separate gem) | The on-device model built into macOS (Apple Intelligence): free and offline, for development | `gem "squishling-apple_fm"`, macOS 26+ on Apple Silicon |
+| [`squishling-claude_code`](https://github.com/Coolhand-Labs/squishling-claude-code-adapter) (separate gem) | Your local Claude Code CLI (`claude -p`), on your own Claude subscription instead of API billing, for development | `gem "squishling-claude_code"`, Claude Code installed and logged in |
 | [Your own](#writing-your-own) | Anything | a class with one method |
 
 Squishling itself depends on neither RubyLLM nor any provider SDK.
@@ -176,15 +178,17 @@ Squishling.register_adapter(:acme, AcmeAdapter)
 | `messages` | The conversation so far, ending with the user message to answer. After invalid output, Squishling appends your previous answer and a retry message; you never keep history. |
 | `schema` | `{ "name", "description", "schema", "strict" => true }`: the strict structured-output payload, without the conditional keywords providers can't take. |
 | `params` | The resolved [generation params](configuration.md#generation-params): the portable `:temperature`, `:max_output_tokens`, `:thinking`, plus provider-specific keys. Squishling has already rejected the keys it owns. Translate the portable ones for your provider; pass the rest through if it takes them. |
+| `cache` | The [prompt caching](configuration.md#prompt-caching) to ask for: `false` (none), or a Hash of options (`{}` for the provider's default, `{ ttl: "1h" }`). Ignore it if your provider has no prompt caching or caches on its own. |
 
 ### The response
 
 `Squishling::Adapter::Response.new(content:, model:, tokens:, finish_reason:)`. `content` is the model's output as a
 String, or a Hash if you parsed it; `nil` or empty counts as invalid output and is retried. `model` is the id the
-provider reports. `tokens` is a Hash (`:input_tokens`, `:output_tokens`, and anything else you have); `{}` when
+provider reports. `tokens` is a Hash (`:input_tokens`, which leaves out the tokens read from or written to the prompt cache, `:output_tokens`, `:cache_read_tokens`, `:cache_write_tokens`, and anything else you have); `{}` when
 unknown. These feed [`squawk`'s `usage`](failures.md#observing-every-attempt-squawk), the [observer](observers.md)'s `Usage`, and the
-[token budget](failures.md#capping-a-calls-cost-token-budgets), which counts `:input_tokens` and `:output_tokens`
-(a response with none never exhausts a budget).
+[token budget](failures.md#capping-a-calls-cost-token-budgets), which counts `:input_tokens`, `:cache_write_tokens` and
+`:output_tokens`, and `:cache_read_tokens` only with `budget_counts_cache_reads` (a response with none never exhausts a
+budget).
 `finish_reason` is why the provider stopped, as a Symbol: `:stop`, `:max_tokens` (the output cap cut the reply off),
 `:content_filter`, `:tool_calls`, or the provider's own spelling; leave it out (`nil`) when unknown. Map your
 provider's "length" reason to `:max_tokens`, so Squishling can [tell a cutoff from a wrong

data/docs/configuration.md

@@ -32,7 +32,9 @@ end
 | `default_escalation` | `nil` | The universal [escalation](#models-and-escalation): models tried in order. Assigning it clears `default_model`, and vice versa. |
 | `default_provider` | `nil` | The provider for `default_model`/`default_escalation` steps that don't name one. What it means depends on the adapter: RubyLLM needs it only for models missing from its registry (see below); the OpenAI-compatible adapter ignores it. |
 | `default_params` | `{}` | Generation params for every call (temperature, reasoning effort, top_p, …), overridable per class, per method, and per escalation step. See [Generation params](#generation-params). |
+| `default_cache` | `true` | Asks the provider to cache the stable prompt on every request: `true`, `false`, or a Hash of options such as `{ ttl: "1h" }`. Overridable per class, per method, per escalation step, and per `squish!` call. See [Prompt caching](#prompt-caching). |
 | `default_max_total_output_tokens`, `default_max_total_input_tokens` | `nil` (no limit) | The most output (or input) tokens one call may use across all its requests; once reached, the call raises `BudgetExhaustedError` instead of making another attempt. Overridable per class (`squishling max_total_output_tokens: ...`), per method, and per `squish!` call. See [Capping a call's cost](failures.md#capping-a-calls-cost-token-budgets). |
+| `budget_counts_cache_reads` | `false` | Whether the input tokens a request reads from the provider's prompt cache count toward `max_total_input_tokens`. Off by default, since reads are cheap; tokens written to the cache always count. Global only. See [Capping a call's cost](failures.md#capping-a-calls-cost-token-budgets). |
 | `squawk` | `nil` | A callable run after every LLM attempt (see `squawk_on`), and once more when a token budget or a disagreement stops the call, with the raw output, for sending it to an error tracker or tracing tool. Also settable per class and per method. See [Observing every attempt](failures.md#observing-every-attempt-squawk). |
 | `squawk_on` | `:all` | Which attempts reach the `squawk` hook: `:all`, `:rejections` (errors only), `:retries` (errors plus any accepted attempt after the first), `:final` (the last attempt the escalation allows), or a list of them (any one holding runs the hook). A call [stopped](failures.md#a-call-that-is-stopped) by a token budget or a disagreement is always reported. Also settable per class and per method. |
 | `observer` | `nil` | An around-style hook for every provider request: `:coolhand`, any object responding to `around(call)`, or `false` for none. `nil` uses the [Coolhand observer](coolhand.md) when the `coolhand` gem is configured. See [Observers](observers.md). |
@@ -76,6 +78,7 @@ Each step is a model name, or a Hash with:
 | `retry_temperature:` | none | A Numeric `temperature` for this step's retries in the same chat (not its first attempt), to nudge a model that keeps sending the same rejected reply. Don't use it on a step that sets `thinking`, since reasoning models reject `temperature`. |
 | `forward_rejected:` | `true` | Set `false` to start this step from the original input alone, without the previous step's rejected output and its errors (see below). |
 | `params:` | `{}` | [Generation params](#generation-params) for this step, merged over the class and method params key by key. |
+| `cache:` | the level's `cache:` | [Prompt caching](#prompt-caching) for this step: `false`, `true`, or a Hash of options. It replaces the level's value, since caching behaves differently per model and provider. |
 
 Without `priority:`, steps run in the order written. With it, the order is explicit and doesn't depend on position:
 
@@ -166,7 +169,7 @@ end
 Passing both `model:` and `escalation:` in one declaration raises `ConfigurationError`, and so does a list passed as
 `model:`. Across separate declarations (a reopened class, or two config assignments), the latest one wins.
 
-Consecutive attempts of the same step (same model, provider, params, `forward_rejected:`, and `retry_temperature:`, e.g. `attempts: 2`)
+Consecutive attempts of the same step (same model, provider, params, `cache:`, `forward_rejected:`, and `retry_temperature:`, e.g. `attempts: 2`)
 continue one conversation. Moving to a different step starts a fresh chat with the original input plus the previous
 output and why it was rejected, so no provider-specific history crosses providers. That rejected output is
 model-generated and is sent to the next step's provider, even when it is a different one, truncated to 4,000
@@ -257,9 +260,53 @@ end
 - If a provider rejects a param, the call raises `Squishling::ConfigurationError` naming the params. It isn't
   retried or sent to `squish_fallback`. See [Failure handling](failures.md).
 
+## Prompt caching
+
+Squishling puts the stable text (the purpose and the note on the inputs) in the system prompt and the
+per-call data in the user message, which is the order provider prompt caches want. With the RubyLLM adapter, caching is
+on by default: each request goes out with RubyLLM's `with_caching`, so Anthropic Claude (whose caching is opt-in) caches
+the prompt, and a call's retries within one escalation step reuse it. The cache point RubyLLM sets covers the request up
+to its last message, so the per-call data is written to the cache too, and only a later request that shares that prefix
+reads it back. The OpenAI-compatible adapter ignores the setting, and providers that cache on their own (OpenAI, Google
+Gemini) still do.
+
+`cache` is `true`, `false`, or a Hash of options for the provider (`{ ttl: "1h" }`). Each level overrides the one above
+it, and `nil` inherits; a Hash replaces the level above's rather than merging into it:
+
+```ruby
+Squishling.configure { |c| c.default_cache = false }              # every call (default: true)
+
+class Triager
+  include Squishling
+  squishling cache: false                                         # this class (and subclasses)
+
+  squish :triage, cache: { ttl: "1h" } do                         # one method
+    string :priority
+  end
+end
+
+squish!(cache: false, escalation: ["claude-opus-5-5"])            # one hand-off
+# escalation: [{ model: "claude-haiku-4-5", cache: false }, "claude-sonnet-5-5"]   # one step
+```
+
+- **Writing a cache entry costs extra.** Anthropic charges a premium for the write (about 1.25x the input price for the
+  5-minute lifetime, more for `ttl: "1h"`) and pays it back only when a later request reads the entry. A class with a
+  long prompt that is called once and never again costs slightly more with caching on; set `cache: false` for it.
+- **Not every model takes cache controls.** A model that can't cache may reject them (a provider 400, which is a
+  `ConfigurationError`, never retried); set `cache: false` for it, per class, method, or escalation step.
+- **A prefix below the model's minimum is not cached.** Providers ignore caching under a model-specific minimum, so a
+  short prompt is a harmless no-op, not an error.
+- **An option the provider doesn't take raises `ConfigurationError`** when the request is made (Anthropic accepts `ttl`,
+  for instance). Explicit Gemini content caches (`id:`) are not supported.
+- **Judgments** (`type: :judgment` judges) don't use a chat, so they are never cached.
+- **Token budgets** count the tokens written to the cache but not those read back, so a hit doesn't spend
+  `max_total_input_tokens`; `budget_counts_cache_reads = true` counts reads too.
+- **See whether it hits** in `cache_read_tokens` and `cache_write_tokens`: they are in `squawk`'s `usage` and on the
+  [observer](observers.md#what-yield-returns)'s `Usage` (nil when the provider reports none).
+
 ## Inheritance
 
-Subclasses inherit the model or escalation and its provider (as a pair), harness, generation params (merged key by key), token budgets (limit by limit), purpose, output schema,
+Subclasses inherit the model or escalation and its provider (as a pair), harness, generation params (merged key by key), prompt caching, token budgets (limit by limit), purpose, output schema,
 `squish_when` predicate, `squish_context` names, `squish_validate`, `squish_fallback`, and every `squish` declaration. Overrides in a subclass, including
 overridden methods, are routed the same way.
 
@@ -269,6 +316,6 @@ overridden methods, are routed the same way.
 ## Per-call overrides
 
 Inside a squished method, `squish!` sends the call to the LLM with its own `purpose:`,
-`append_to_purpose:`, `context:`, `model:` or `escalation:` (with `provider:`), `params:`, `harness:`, and
+`append_to_purpose:`, `context:`, `model:` or `escalation:` (with `provider:`), `params:`, `cache:`, `harness:`, and
 `max_total_output_tokens:`/`max_total_input_tokens:`. Each layers over the method and class
 settings the same way they layer over each other. See [Handing off to the LLM](routing.md#handing-off-to-the-llm-with-squish).

data/docs/failures.md

@@ -155,7 +155,10 @@ Things to know:
   limit by one request (for the squishsum harnesses, one round of samples).
 - Tokens are the counts the [adapter](adapters.md) reports (`output_tokens` includes a reasoning model's thinking
   tokens for providers that count them that way). A request that failed reports none, and a provider that reports no
-  usage never exhausts a budget.
+  usage never exhausts a budget. The input total includes the tokens written to the prompt cache
+  (`cache_write_tokens`) but not those read from it (`cache_read_tokens`), which adapters report apart from
+  `input_tokens`: with [prompt caching](configuration.md#prompt-caching) on, a hit on a long cached prompt doesn't spend
+  the budget. Set `config.budget_counts_cache_reads = true` in `Squishling.configure` to count reads too.
 - The limits are positive Integers. Set them with `Squishling.configure` (`default_max_total_output_tokens`,
   `default_max_total_input_tokens`), `squishling`, `squish :method, ...`, or a [`squish!`](routing.md#handing-off-to-the-llm-with-squish) call.
   The more specific level wins, limit by limit, and `false` lifts a limit set at a wider level.

data/docs/measuring-tokens.md

@@ -93,7 +93,8 @@ can meter spend knowing which method, escalation level, sample, or judge made th
 ```ruby
 class SpendObserver
   def around(call)
-    usage = yield # input_tokens, output_tokens (nil if unreported), and model; never the content
+    usage = yield # input_tokens (not counting cached ones), output_tokens, cache_read_tokens, cache_write_tokens
+                  # (each nil if unreported), and model; never the content
     tags = ["label:#{call.label}", "role:#{call.role || 'squish'}", "model:#{usage.model}"]
     Metrics.increment("squishling.tokens.in", usage.input_tokens.to_i, tags:)
     Metrics.increment("squishling.tokens.out", usage.output_tokens.to_i, tags:)

data/docs/observers.md

@@ -26,7 +26,8 @@ raises `ConfigurationError`.
 
 ## What `yield` returns
 
-The block returns a `Squishling::Observer::Usage`: the request's `input_tokens` and `output_tokens` (nil when the
+The block returns a `Squishling::Observer::Usage`: the request's `input_tokens` and `output_tokens`, the
+`cache_read_tokens` and `cache_write_tokens` of [prompt caching](configuration.md#prompt-caching) (each nil when the
 provider didn't report them) and the `model` that answered. That is enough to meter spend:
 
 ```ruby

data/docs/routing.md

@@ -98,7 +98,7 @@ end
 - `purpose:` (replaces the class's)
 - `append_to_purpose:` (added to the class's, see [Appending to the purpose](#appending-to-the-purpose))
 - `output_schema:` (or a schema block)
-- `model:` or `escalation:`, `provider:`, and `params:` (generation params; see [Configuration](configuration.md))
+- `model:` or `escalation:`, `provider:`, `params:` (generation params), and `cache:` (prompt caching); see [Configuration](configuration.md)
 - `harness:` (see [Harnesses](harnesses.md), including your own)
 - `max_total_output_tokens:` and `max_total_input_tokens:` (see [Capping a call's cost](failures.md#capping-a-calls-cost-token-budgets))
 - `when:`, a predicate proc
@@ -143,6 +143,7 @@ end
 | `append_to_purpose:` | Added to the declared sections; `false` (alone or first in an Array) drops them for this call |
 | `purpose:` | Replaces the purpose |
 | `model:` or `escalation:`, `provider:`, `params:` | E.g. send this call to a stronger model, or a whole [escalation](configuration.md#models-and-escalation), when Ruby fails. A `provider:` needs a `model:` or `escalation:`; `params:` merge key by key over the declared ones. |
+| `cache:` | Replaces the declared [prompt caching](configuration.md#prompt-caching) setting; `false` turns it off for this call |
 | `harness:` | Replaces the declared [harness](harnesses.md), e.g. `harness: :judged_squishsum` to double-check a hand-off |
 | `max_total_output_tokens:`, `max_total_input_tokens:` | Cap what this call may spend, e.g. a tighter budget for a hand-off; `false` lifts a declared cap. See [Capping a call's cost](failures.md#capping-a-calls-cost-token-budgets). |
 

data/docs/testing.md

@@ -92,7 +92,7 @@ and `#verdict` are the real ones; rescue them as you would in production.
 |---|---|
 | `arguments`, `context` | the method's arguments and the `squish_context` / `squish!` context the model was sent |
 | `system_prompt`, `output_schema` | the purpose (with `append_to_purpose` sections) and the strict schema |
-| `model`, `provider`, `params` | the step it went to |
+| `model`, `provider`, `params`, `cache` | the step it went to (`cache` is `false` when [prompt caching](configuration.md#prompt-caching) was off for it) |
 | `role`, `sample`, `attempt`, `attempts` | `nil`/`:sample`/`:judge`, `"a"`, and where it is in the escalation |
 | `conversation` | the chat it belongs to |
 | `errors` | the validation messages a retry or escalation carried back to the model |

data/lib/squishling/adapter.rb

@@ -12,15 +12,22 @@ module Squishling
     # structured-output payload ({ "name", "description", "schema", "strict" => true }). params is the resolved
     # generation params: the portable :temperature, :max_output_tokens, and :thinking, plus any provider-specific keys
     # (see Params). label ("Class#method") prefixes the adapter's error messages. model and provider are the step's
-    # (nil when it names none).
-    Request = Data.define(:label, :model, :provider, :instructions, :messages, :schema, :params)
+    # (nil when it names none). cache is the prompt caching to ask the provider for: false (none), or a Hash of options
+    # ({} for the provider's default, { ttl: "1h" }), see Cache. An adapter whose provider has no prompt caching, or
+    # caches on its own, ignores it.
+    Request = Data.define(:label, :model, :provider, :instructions, :messages, :schema, :params, :cache) do
+      def initialize(label:, model:, provider:, instructions:, messages:, schema:, params:, cache: false)
+        super
+      end
+    end
 
     # Stands in for an empty reply when the conversation is replayed: some servers reject an empty assistant message.
     EMPTY_REPLY = "(no output)"
 
     # What a completion returns. content is the model's output: the raw String, or a Hash the adapter already parsed.
-    # model is the model id the provider reports. tokens is a Hash of counts (:input_tokens, :output_tokens, and
-    # whatever else the provider reports, such as :cache_read_tokens and :thinking_tokens), empty when unknown.
+    # model is the model id the provider reports. tokens is a Hash of counts (:input_tokens, the input not read from or
+    # written to the prompt cache, :output_tokens, and whatever else the provider reports, such as :cache_read_tokens,
+    # :cache_write_tokens and :thinking_tokens), empty when unknown.
     # finish_reason is why the provider stopped, as a Symbol: :stop, :max_tokens (the output cap cut the reply off),
     # :content_filter, :tool_calls, or the provider's own spelling; nil when it reported none.
     Response = Data.define(:content, :model, :tokens, :finish_reason) do

data/lib/squishling/adapters/openai_compatible.rb

@@ -292,14 +292,25 @@ module Squishling
       def tokens_of(usage)
         return {} unless usage.is_a?(Hash)
 
+        cached = detail(usage, "prompt_tokens_details", "cached_tokens")
+        written = detail(usage, "prompt_tokens_details", "cache_write_tokens")
         {
-          input_tokens: usage["prompt_tokens"],
+          input_tokens: uncached(usage["prompt_tokens"], cached, written),
           output_tokens: usage["completion_tokens"],
-          cache_read_tokens: detail(usage, "prompt_tokens_details", "cached_tokens"),
+          cache_read_tokens: cached,
+          cache_write_tokens: written,
           thinking_tokens: detail(usage, "completion_tokens_details", "reasoning_tokens")
         }.select { |_, count| count.is_a?(Integer) }
       end
 
+      # prompt_tokens counts the tokens read from and written to the cache; input_tokens is the rest, as every adapter
+      # reports it (see Adapter::Response), so they aren't counted twice.
+      def uncached(prompt_tokens, *cached)
+        return prompt_tokens unless prompt_tokens.is_a?(Integer)
+
+        [prompt_tokens - cached.grep(Integer).sum, 0].max
+      end
+
       # A count inside a usage sub-object, which a server might send in any shape.
       def detail(usage, group, key)
         usage[group][key] if usage[group].is_a?(Hash)

data/lib/squishling/adapters/ruby_llm.rb

@@ -38,7 +38,7 @@ module Squishling
         chat = build_chat(request)
         *history, last = request.messages
         history.each { |message| chat.add_message(role: message[:role], content: message[:content]) }
-        response = guarded(request) { chat.ask(last[:content]) }
+        response = guarded(request) { ask(chat, request, last[:content]) }
         Adapter::Response.new(content: response.content, model: response.model, tokens: tokens_of(response),
           finish_reason: response.finish_reason)
       end
@@ -82,6 +82,17 @@ module Squishling
 
       private
 
+      # RubyLLM checks a caching option against the provider when it renders the request, and raises ArgumentError for
+      # one it can't use. That is the developer's setting, so it is a setup mistake rather than a failed request.
+      def ask(chat, request, content)
+        chat.ask(content)
+      rescue ArgumentError => e
+        raise unless request.cache && e.message.match?(/caching/i) && raised_by_ruby_llm?(e)
+
+        raise configuration_error(request,
+          "RubyLLM rejected the request, cache #{request.cache.inspect} (#{e.message})")
+      end
+
       def tokens_of(response)
         response.tokens ? response.tokens.to_h : {}
       end
@@ -106,6 +117,7 @@ module Squishling
         chat.with_instructions(request.instructions)
         chat.with_schema(request.schema)
         apply_params(chat, request)
+        apply_cache(chat, request)
         chat
       rescue ::RubyLLM::ModelNotFoundError, ::RubyLLM::ConfigurationError => e
         raise configuration_error(request, e.message)
@@ -124,6 +136,12 @@ module Squishling
         raise configuration_error(request, "invalid params #{params.keys.inspect} (#{e.message})")
       end
 
+      # Prompt caching is only asked for when the request carries it: off sends no cache controls. RubyLLM checks the
+      # options when it renders the request, so an unusable one is reported by #ask.
+      def apply_cache(chat, request)
+        chat.with_caching(request.cache) if request.cache
+      end
+
       # true: the model's default thinking; false: off; a Hash: { effort:, budget:, display: }.
       def apply_thinking(chat, thinking)
         thinking.is_a?(Hash) ? chat.with_thinking(**thinking) : chat.with_thinking(thinking)

data/lib/squishling/budget.rb

@@ -33,17 +33,20 @@ module Squishling
     # threads, so the totals are taken under a lock. A response without token counts adds nothing, so a provider that
     # reports no usage never exhausts a budget.
     class Tally
-      def initialize(budget)
+      def initialize(budget, cache_reads: false)
         @budget = budget
+        @cache_reads = cache_reads
         @used = { input_tokens: 0, output_tokens: 0 }
         @lock = Mutex.new
       end
 
-      # Counts a response's tokens: an Adapter::Response, or a judgment (see Observer.usage).
+      # Counts a response's tokens: an Adapter::Response, or a judgment (see Observer.usage). The input total includes
+      # the tokens written to the prompt cache, and those read from it only when cache_reads is set; adapters report
+      # both apart from input_tokens.
       def add(response)
         usage = Observer.usage(response)
         @lock.synchronize do
-          @used[:input_tokens] += count(usage.input_tokens)
+          @used[:input_tokens] += input_count(usage)
           @used[:output_tokens] += count(usage.output_tokens)
         end
         response
@@ -63,6 +66,11 @@ module Squishling
 
       private
 
+      def input_count(usage)
+        total = count(usage.input_tokens) + count(usage.cache_write_tokens)
+        @cache_reads ? total + count(usage.cache_read_tokens) : total
+      end
+
       def count(tokens)
         tokens.is_a?(Integer) && tokens.positive? ? tokens : 0
       end

data/lib/squishling/cache.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+module Squishling
+  # Prompt caching: whether a request asks the provider to cache the stable prompt (the purpose and the input note in
+  # the system prompt), layered config -> class -> method -> squish! like generation params, and per escalation step.
+  # A level declares true, false, or a Hash of options (ttl: "1h"), and nil leaves it to the level above. What a
+  # request carries is false (no cache controls) or a Hash of options ({} for the provider's default). See
+  # docs/configuration.md.
+  module Cache
+    module_function
+
+    # One level's value, validated: nil (inherit), true, false, or a frozen Hash with Symbol keys. Explicit provider
+    # caches (id:) are not supported: an adapter holds no state between requests.
+    def normalize(value, label)
+      return value if value.nil? || [true, false].include?(value)
+      unless value.is_a?(Hash)
+        raise ConfigurationError, "#{label} must be true, false, or a Hash of options, got #{value.inspect}"
+      end
+
+      options = value.transform_keys(&:to_sym)
+      raise ConfigurationError, "#{label}: id: (an explicit provider cache) isn't supported" if options.key?(:id)
+
+      options.freeze
+    end
+
+    # What the request carries: the last level that declares a value wins (a Hash replaces, it doesn't merge), and no
+    # level at all means off. true is the provider's default options.
+    def resolve(*layers)
+      value = layers.reverse.find { |layer| !layer.nil? }
+      value == true ? {}.freeze : value || false
+    end
+  end
+end

data/lib/squishling/class_methods.rb

@@ -15,6 +15,9 @@ module Squishling
     # parent's provider.
     # params: are generation params merged over the configured defaults (see Configuration#default_params):
     #   squishling params: { temperature: 0.1, top_p: 0.9 }
+    # cache: asks the provider to cache the stable prompt (true by default, see Cache); false opts the class out, and
+    # a Hash such as { ttl: "1h" } passes options through:
+    #   squishling cache: false
     # append_to_purpose: adds sections after the purpose (see #append_to_purpose):
     #   squishling append_to_purpose: ["The Ruby that handles well-formed input:", self]
     # harness: chooses how the escalation is used (see Harness):
@@ -29,7 +32,7 @@ module Squishling
     # squawk_on: limits which attempts reach the hook: :all (the default), :rejections, :retries, :final, or a
     # list of them (any one holding runs the hook):
     #   squishling squawk: hook, squawk_on: %i[rejections final]
-    def squishling(model: nil, escalation: nil, provider: nil, params: nil, harness: nil, purpose: nil,
+    def squishling(model: nil, escalation: nil, provider: nil, params: nil, cache: nil, harness: nil, purpose: nil,
       append_to_purpose: nil, output_schema: nil, squawk: nil, squawk_on: nil, max_total_output_tokens: nil,
       max_total_input_tokens: nil)
       path = ModelPath.declare(model, escalation, to_s)
@@ -44,8 +47,7 @@ module Squishling
       # provider meant for its parent's.
       @squishling_model_path = path if path
       @squishling_provider = provider if provider || path
-      @squishling_params = Params.normalize(params, "#{self} params") if params
-      @squishling_harness = Harness.normalize(harness, to_s) unless harness.nil?
+      declare_generation(params, cache, harness)
       self.purpose(purpose) if purpose
       self.append_to_purpose(append_to_purpose) unless append_to_purpose.nil?
       self.output_schema(output_schema) if output_schema
@@ -76,6 +78,11 @@ module Squishling
       squishling_lookup(:@squishling_provider)
     end
 
+    # The class's (or nearest ancestor's) cache setting (see Cache), nil when none declared it.
+    def squishling_cache
+      squishling_lookup(:@squishling_cache)
+    end
+
     def squishling_harness
       squishling_lookup(:@squishling_harness)
     end
@@ -171,13 +178,14 @@ module Squishling
     #     string :priority
     #   end
     def squish(*names, purpose: nil, append_to_purpose: nil, output_schema: nil, model: nil, escalation: nil,
-      provider: nil, params: nil, harness: nil, when: nil, fallback: nil, validate: nil, squawk: nil,
-      squawk_on: nil, max_total_output_tokens: nil, max_total_input_tokens: nil, &schema_block)
+      provider: nil, params: nil, cache: nil, harness: nil, when: nil, fallback: nil, validate: nil,
+      squawk: nil, squawk_on: nil, max_total_output_tokens: nil, max_total_input_tokens: nil, &schema_block)
       schema = schema_block ? Schematist::Schema.create(&schema_block) : output_schema
       model = ModelPath.declare(model, escalation, "#{self} squish")
       raise ConfigurationError, "#{self} squish: provider: needs a model: or escalation:" if provider && !model
 
       params &&= Params.normalize(params, "#{self} squish params")
+      cache = Cache.normalize(cache, "#{self} squish cache")
       harness = Harness.normalize(harness, "#{self} squish") unless harness.nil?
       unless append_to_purpose.nil?
         append_to_purpose = Appendices.normalize(append_to_purpose, "#{self} squish append_to_purpose")
@@ -185,7 +193,7 @@ module Squishling
       squawk = Squawk.validate(squawk, "#{self} squish")
       squawk_on = Squawk.validate_on(squawk_on, "#{self} squish")
       budget = Budget.build("#{self} squish", output: max_total_output_tokens, input: max_total_input_tokens)
-      options = { purpose:, append_to_purpose:, output_schema: schema, model:, provider:, params:, harness:,
+      options = { purpose:, append_to_purpose:, output_schema: schema, model:, provider:, params:, cache:, harness:,
                   predicate: binding.local_variable_get(:when), fallback:, validator: validate, squawk:,
                   squawk_on:, budget: }.compact
 
@@ -216,6 +224,13 @@ module Squishling
 
     private
 
+    # nil leaves a level unset (so the parent's applies); false for cache: is stored, to opt the class out.
+    def declare_generation(params, cache, harness)
+      @squishling_params = Params.normalize(params, "#{self} params") if params
+      @squishling_cache = Cache.normalize(cache, "#{self} cache") unless cache.nil?
+      @squishling_harness = Harness.normalize(harness, to_s) unless harness.nil?
+    end
+
     # nil leaves a level unset (so the parent's applies); false for squawk: is stored, to silence it.
     def declare_squawk(squawk, squawk_on)
       @squishling_squawk = Squawk.validate(squawk, to_s) unless squawk.nil?

data/lib/squishling/configuration.rb

@@ -24,6 +24,14 @@ module Squishling
     # adapter for its provider; any other key (top_p, seed, ...) is passed to the provider as-is.
     attr_reader :default_params
 
+    # Whether requests ask the provider to cache the stable prompt: true (the default), false, or a Hash of options
+    # such as { ttl: "1h" } (see Cache). Overridable per class, per method, per escalation step, and per squish! call.
+    attr_reader :default_cache
+
+    # Whether the tokens a request reads from the provider's prompt cache count toward max_total_input_tokens: false
+    # (the default) leaves them out, true counts them. Tokens written to the cache always count (see Budget).
+    attr_reader :budget_counts_cache_reads
+
     # How every squishling class that doesn't declare its own harness uses its escalation (see Harness):
     # :escalation (the default), :squishsum, :judged_squishsum, a harness you registered, or a Hash with type: and
     # its options.
@@ -67,6 +75,8 @@ module Squishling
       @default_model_path = nil
       @default_provider = nil
       @default_params = {}
+      @default_cache = true
+      @budget_counts_cache_reads = false
       @default_harness = nil
       @logger = nil
       @squawk = nil
@@ -131,6 +141,21 @@ module Squishling
       @default_params = Params.normalize(params, "default_params")
     end
 
+    # nil restores the default (true).
+    def default_cache=(cache)
+      @default_cache = cache.nil? || Cache.normalize(cache, "default_cache")
+    end
+
+    # true or false; unlike default_cache=, nil is not accepted.
+    def budget_counts_cache_reads=(counts)
+      unless [true, false].include?(counts)
+        raise ConfigurationError,
+          "Squishling.configure: budget_counts_cache_reads must be true or false, got #{counts.inspect}"
+      end
+
+      @budget_counts_cache_reads = counts
+    end
+
     def default_harness=(harness)
       @default_harness = harness.nil? ? nil : Harness.normalize(harness, "default_harness")
     end

data/lib/squishling/definition.rb

@@ -13,8 +13,8 @@ module Squishling
     attr_reader :klass, :name, :call_context, :handed_off
 
     def initialize(klass:, name:, purpose: nil, append_to_purpose: nil, output_schema: nil, model: nil,
-      provider: nil, params: nil, harness: nil, predicate: nil, fallback: nil, validator: nil, squawk: nil,
-      squawk_on: nil, budget: nil, call_context: {}, handed_off: false)
+      provider: nil, params: nil, cache: nil, harness: nil, predicate: nil, fallback: nil, validator: nil,
+      squawk: nil, squawk_on: nil, budget: nil, call_context: {}, handed_off: false)
       @klass = klass
       @name = name
       @purpose = purpose
@@ -23,6 +23,7 @@ module Squishling
       @model = model
       @provider = provider
       @params = params
+      @cache = cache
       @harness = harness
       @predicate = predicate
       @fallback = fallback
@@ -37,15 +38,16 @@ module Squishling
     # This definition with one call's overrides on top. The output schema, predicate, fallback, and validator
     # can't be overridden: the call must still return the method's result type.
     def for_call(purpose: nil, append_to_purpose: nil, context: nil, model: nil, escalation: nil,
-      provider: nil, params: nil, harness: nil, max_total_output_tokens: nil, max_total_input_tokens: nil)
+      provider: nil, params: nil, cache: nil, harness: nil, max_total_output_tokens: nil,
+      max_total_input_tokens: nil)
       call_path = ModelPath.declare(model, escalation, "#{label} squish!")
       raise ConfigurationError, "#{label}: squish! provider: needs a model: or escalation:" if provider && !call_path
-      unless context.nil? || (context.is_a?(Hash) && context.each_key.all?(NAME_KEY))
-        raise ConfigurationError, "#{label}: squish! context: must be a Hash with String or Symbol keys"
-      end
+
+      check_context!(context)
 
       appended = Appendices.normalize(append_to_purpose, "#{label} squish!") unless append_to_purpose.nil?
       call_params = params && Params.normalize(params, "#{label} squish! params")
+      call_cache = Cache.normalize(cache, "#{label} squish! cache")
       call_harness = Harness.normalize(harness, "#{label} squish!") unless harness.nil?
       call_budget = Budget.build("#{label} squish!", output: max_total_output_tokens, input: max_total_input_tokens)
       self.class.new(
@@ -56,6 +58,7 @@ module Squishling
         model: call_path || @model, provider: call_path ? provider : @provider,
         # merge, not Params.resolve: a nil at the method level must still unset the class's key.
         params: call_params ? (@params || {}).merge(call_params) : @params,
+        cache: [call_cache, @cache].find { |level| !level.nil? },
         harness: call_harness || @harness, budget: @budget.merge(call_budget),
         call_context: @call_context.merge((context || {}).transform_keys(&:to_sym)),
         handed_off: true # only squish! builds a definition this way; it is not part of a declared one
@@ -101,7 +104,7 @@ module Squishling
       else
         0
       end
-      ModelPath.steps(eligible, provider:, params:, turn:)
+      ModelPath.steps(eligible, provider:, params:, cache:, turn:)
     end
 
     # The steps and provider of the first level (method, class, or config) that declares a model or escalation, or
@@ -139,6 +142,12 @@ module Squishling
       Params.resolve(Squishling.config.default_params, klass.squishling_params, @params)
     end
 
+    # Prompt caching: false, or the options to ask the provider for (see Cache). The config default, overridden by the
+    # class, then the method (and a squish! call).
+    def cache
+      Cache.resolve(Squishling.config.default_cache, klass.squishling_cache, @cache)
+    end
+
     # The token limits of one call (see Budget): config defaults, overridden limit by limit by the class, then by the
     # method (and a squish! call).
     def budget
@@ -247,6 +256,12 @@ module Squishling
 
     private
 
+    def check_context!(context)
+      return if context.nil? || (context.is_a?(Hash) && context.each_key.all?(NAME_KEY))
+
+      raise ConfigurationError, "#{label}: squish! context: must be a Hash with String or Symbol keys"
+    end
+
     # Values JSON can't represent (NaN, Infinity, cycles) are invalid output, not a raw JSON error.
     def jsonify_return(attrs)
       [Schema.jsonify(attrs), []]

data/lib/squishling/judge.rb

@@ -111,6 +111,11 @@ module Squishling
       value.is_a?(String) && !value.strip.empty?
     end
 
+    # A judgment is not a chat, so it has nothing to cache whatever its step says.
+    def judgment_steps(judge)
+      ModelPath.steps([judge], provider: nil, params: {}).map { |step| step.with(cache: false) }
+    end
+
     def judgment?
       @judge&.fetch(:type) == :judgment
     end
@@ -119,8 +124,9 @@ module Squishling
     # under its own, like any escalation step; a judgment judge sends only its own params, as provider options.
     def resolve_steps(path)
       if (judge = @judge)
-        params = judge[:type] == :judgment ? {} : @definition.params
-        return ModelPath.steps([judge], provider: nil, params:)
+        return judgment_steps(judge) if judge[:type] == :judgment
+
+        return ModelPath.steps([judge], provider: nil, params: @definition.params, cache: @definition.cache)
       end
 
       rest = path.drop_while { |step| step == path.first }

data/lib/squishling/llm_client.rb

@@ -16,7 +16,7 @@ module Squishling
       @harness = harness
       @handed_off = handed_off
       @adapter = Squishling.config.resolved_adapter
-      @tally = Budget::Tally.new(budget)
+      @tally = Budget::Tally.new(budget, cache_reads: Squishling.config.budget_counts_cache_reads)
     end
 
     # The budget limit this client's requests have reached ({ limit:, max:, used: }), or nil. Checked before each
@@ -89,7 +89,8 @@ module Squishling
     def complete(chat, message, step, reask)
       messages = chat.messages + [{ role: :user, content: message }]
       request = Adapter::Request.new(label: @label, model: step.model, provider: step.provider,
-        instructions: chat.instructions, messages:, schema: chat.schema, params: step.request_params(reask))
+        instructions: chat.instructions, messages:, schema: chat.schema, params: step.request_params(reask),
+        cache: step.cache)
       response = @adapter.complete(request)
       unless response.is_a?(Adapter::Response)
         raise ConfigurationError, "#{@label}: the #{@adapter.class} adapter must return a " \

data/lib/squishling/model_path.rb

@@ -17,9 +17,10 @@ module Squishling
   # retry_temperature: (a Numeric) is the temperature for the step's same-chat retries after the first attempt.
   module ModelPath
     # One attempt: the model, its provider, the fully resolved generation params, whether the previous
-    # step's rejected output is shown to it, and the temperature its retries in the same chat use, if any.
-    Step = Data.define(:model, :provider, :params, :forward_rejected, :retry_temperature) do
-      def initialize(model:, provider:, params:, forward_rejected:, retry_temperature: nil)
+    # step's rejected output is shown to it, the temperature its retries in the same chat use, if any, and the
+    # prompt caching options to request (false for none, see Cache).
+    Step = Data.define(:model, :provider, :params, :forward_rejected, :retry_temperature, :cache) do
+      def initialize(model:, provider:, params:, forward_rejected:, retry_temperature: nil, cache: false)
         super
       end
 
@@ -35,7 +36,7 @@ module Squishling
       end
     end
 
-    STEP_KEYS = %i[model provider params attempts priority order rotate forward_rejected include_when
+    STEP_KEYS = %i[model provider params cache attempts priority order rotate forward_rejected include_when
                    retry_temperature].freeze
 
     module_function
@@ -72,13 +73,14 @@ module Squishling
     end
 
     # One Step per attempt. The level's provider applies to steps that don't name their own, and each
-    # step's params are merged over the level-resolved params. turn is this call's place in the method's rotation
+    # step's params are merged over the level-resolved params, and its cache: replaces the level-resolved one (a Hash
+    # or false, see Cache). turn is this call's place in the method's rotation
     # (see Rotation): each tier of tied priorities starts turn steps further along.
-    def steps(path, provider:, params:, turn: 0)
+    def steps(path, provider:, params:, cache: false, turn: 0)
       rotated(path, turn).flat_map do |step|
         attempt = Step.new(model: step[:model], provider: step[:provider] || provider,
           params: Params.resolve(params, step[:params]), forward_rejected: step[:forward_rejected],
-          retry_temperature: step[:retry_temperature])
+          retry_temperature: step[:retry_temperature], cache: Cache.resolve(cache, step[:cache]))
         [attempt] * step[:attempts]
       end
     end
@@ -131,7 +133,7 @@ module Squishling
     def normalize_step(step, label, extras: [])
       case step
       when String, Symbol
-        { model: model_name(step, label), provider: nil, params: nil, attempts: 1, forward_rejected: true }
+        { model: model_name(step, label), provider: nil, params: nil, cache: nil, attempts: 1, forward_rejected: true }
       when Hash then normalize_hash(step, label, extras)
       else
         raise ConfigurationError, "#{label}: each step must be a model name or a Hash with model:, got #{step.inspect}"
@@ -169,7 +171,8 @@ module Squishling
       end
 
       params = step[:params] && Params.normalize(step[:params], "#{label} (#{model}) params")
-      normalized = { model:, provider: step[:provider], params:, attempts:, forward_rejected: }
+      cache = Cache.normalize(step[:cache], "#{label} (#{model}) cache")
+      normalized = { model:, provider: step[:provider], params:, cache:, attempts:, forward_rejected: }
       normalized.merge(step.slice(:priority, :rotate, :include_when, :retry_temperature, *extras))
     end
 

data/lib/squishling/observer.rb

@@ -17,8 +17,13 @@ module Squishling
       :request)
 
     # What an observer's block returns: the request's token usage and the model that answered, never the content.
-    # A count the provider didn't report is nil.
-    Usage = Data.define(:input_tokens, :output_tokens, :model)
+    # A count the provider didn't report is nil. cache_read_tokens and cache_write_tokens are the input tokens served
+    # from, and written to, the provider's prompt cache (see Cache), so a hit is visible.
+    Usage = Data.define(:input_tokens, :output_tokens, :model, :cache_read_tokens, :cache_write_tokens) do
+      def initialize(input_tokens:, output_tokens:, model:, cache_read_tokens: nil, cache_write_tokens: nil)
+        super
+      end
+    end
 
     BUILT_IN = %i[coolhand].freeze
 
@@ -96,6 +101,8 @@ module Squishling
       model = response.respond_to?(:model) ? response.model : nil
       Usage.new(input_tokens: token_count(tokens, :input_tokens, :input),
         output_tokens: token_count(tokens, :output_tokens, :output),
+        cache_read_tokens: token_count(tokens, :cache_read_tokens, :cache_read),
+        cache_write_tokens: token_count(tokens, :cache_write_tokens, :cache_write),
         model: model.respond_to?(:id) ? model.id : model)
     end
 

data/lib/squishling/run.rb

@@ -73,7 +73,7 @@ module Squishling
       end
 
       step = Harness::OptionTypes.check(:step, declaration, label, :"harness step")
-      ModelPath.steps([step], provider: nil, params: @definition.params).freeze
+      ModelPath.steps([step], provider: nil, params: @definition.params, cache: @definition.cache).freeze
     end
 
     # Makes each attempt of the stages in order (the escalation's steps when none are given) until an output

data/lib/squishling/testing/adapter.rb

@@ -82,7 +82,8 @@ module Squishling
         payload = Schema.jsonify(input)
         @requests << Request.new(label:, type: :judgment, role: :judge, sample: nil, attempt: call[:attempt],
           attempts: call[:attempts], model: step.model, provider: step.provider, params: step.params,
-          system_prompt: nil, output_schema: nil, messages: [], conversation: nil, payload:, reply: outcome)
+          cache: step.cache, system_prompt: nil, output_schema: nil, messages: [], conversation: nil, payload:,
+          reply: outcome)
         outcome
       end
 
@@ -150,8 +151,9 @@ module Squishling
       def record(request, call, outcome, conversation)
         @requests << Request.new(label: request.label, type: :chat, role: call[:role], sample: call[:sample]&.to_s,
           attempt: call[:attempt], attempts: call[:attempts], model: request.model, provider: request.provider,
-          params: request.params, system_prompt: request.instructions, output_schema: request.schema["schema"],
-          messages: request.messages, conversation:, payload: Request.payload_of(request.messages), reply: outcome)
+          params: request.params, cache: request.cache, system_prompt: request.instructions,
+          output_schema: request.schema["schema"], messages: request.messages, conversation:,
+          payload: Request.payload_of(request.messages), reply: outcome)
       end
     end
 

data/lib/squishling/testing/request.rb

@@ -12,9 +12,14 @@ module Squishling
     # sent. For a judge they are those of the call being judged, and candidates holds the two samples. messages is
     # the conversation as sent (empty for a System One judgment, whose payload is the input). conversation is the
     # index of the chat in Adapter#conversations (nil for a judgment): a retry is the same conversation, an
-    # escalation a new one. reply is the Adapter::Response, or the error raised instead.
+    # escalation a new one. reply is the Adapter::Response, or the error raised instead. cache is the prompt caching
+    # asked for: false, or a Hash of options.
     Request = Data.define(:label, :type, :role, :sample, :attempt, :attempts, :model, :provider, :params,
-      :system_prompt, :output_schema, :messages, :conversation, :payload, :reply) do
+      :system_prompt, :output_schema, :messages, :conversation, :payload, :reply, :cache) do
+      def initialize(reply:, cache: false, **attributes)
+        super
+      end
+
       def arguments
         input["arguments"]
       end

data/lib/squishling/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Squishling
-  VERSION = "0.5.1"
+  VERSION = "0.6.0"
 end

data/lib/squishling.rb

@@ -8,6 +8,7 @@ require_relative "squishling/version"
 require_relative "squishling/errors"
 require_relative "squishling/configuration"
 require_relative "squishling/params"
+require_relative "squishling/cache"
 require_relative "squishling/named_registry"
 require_relative "squishling/secret"
 require_relative "squishling/adapter/error_body"
@@ -163,11 +164,12 @@ module Squishling
   # it); return it from the method. The overrides apply to this call only: append_to_purpose adds to (or,
   # with false, replaces) the declared sections, context is sent alongside the declared squish_context,
   # model/escalation (one or the other)/provider/purpose/harness replace the declared ones, and params merge
-  # key by key over them. max_total_output_tokens/max_total_input_tokens cap what the call may spend across all its
+  # key by key over them. cache: replaces the declared prompt caching setting (false turns it off, see Cache).
+  # max_total_output_tokens/max_total_input_tokens cap what the call may spend across all its
   # requests (false lifts a declared cap).
   def squish!(append_to_purpose: nil, context: nil, purpose: nil, model: nil, escalation: nil, provider: nil,
-    params: nil, harness: nil, max_total_output_tokens: nil, max_total_input_tokens: nil)
+    params: nil, cache: nil, harness: nil, max_total_output_tokens: nil, max_total_input_tokens: nil)
     Router.hand_off(self, { append_to_purpose:, context:, purpose:, model:, escalation:, provider:, params:,
-                            harness:, max_total_output_tokens:, max_total_input_tokens: })
+                            cache:, harness:, max_total_output_tokens:, max_total_input_tokens: })
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: squishling
 version: !ruby/object:Gem::Version
-  version: 0.5.1
+  version: 0.6.0
 platform: ruby
 authors:
 - Michael Carroll
@@ -78,6 +78,7 @@ files:
 - lib/squishling/adapters/ruby_llm.rb
 - lib/squishling/appendices.rb
 - lib/squishling/budget.rb
+- lib/squishling/cache.rb
 - lib/squishling/class_methods.rb
 - lib/squishling/class_registry.rb
 - lib/squishling/collisions.rb