squishling 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +123 -0
- data/README.md +117 -35
- data/docs/configuration.md +113 -32
- data/docs/failures.md +111 -14
- data/docs/harnesses.md +198 -0
- data/docs/measuring-tokens.md +148 -0
- data/docs/naming.md +43 -0
- data/docs/routing.md +46 -31
- data/docs/schemas.md +32 -2
- data/lib/squishling/appendices.rb +3 -3
- data/lib/squishling/class_methods.rb +81 -31
- data/lib/squishling/collisions.rb +54 -0
- data/lib/squishling/configuration.rb +58 -9
- data/lib/squishling/definition.rb +78 -39
- data/lib/squishling/errors.rb +29 -5
- data/lib/squishling/harness.rb +153 -0
- data/lib/squishling/invoker.rb +223 -74
- data/lib/squishling/judge.rb +142 -0
- data/lib/squishling/llm_client.rb +150 -0
- data/lib/squishling/model_path.rb +129 -0
- data/lib/squishling/output_check.rb +109 -0
- data/lib/squishling/params.rb +39 -2
- data/lib/squishling/router.rb +13 -7
- data/lib/squishling/schema.rb +45 -3
- data/lib/squishling/source.rb +5 -5
- data/lib/squishling/squawk.rb +52 -0
- data/lib/squishling/version.rb +1 -1
- data/lib/squishling.rb +25 -5
- metadata +13 -3
data/docs/routing.md
CHANGED
|
@@ -18,7 +18,7 @@ A squished call runs its Ruby implementation unless one of these sends it to the
|
|
|
18
18
|
|---|---|---|
|
|
19
19
|
| `squish_when` (or `when:`) predicate is truthy | class, or per method | before Ruby runs |
|
|
20
20
|
| The method has no implementation: it isn't defined, or it raises `NotImplementedError` | the method body | when Ruby gives up |
|
|
21
|
-
| `squish!` | inside the method, e.g. in a `rescue` | after Ruby has partly run, see [
|
|
21
|
+
| `squish!` | inside the method, e.g. in a `rescue` | after Ruby has partly run, see [Handing off to the LLM](#handing-off-to-the-llm-with-squish) |
|
|
22
22
|
|
|
23
23
|
With no predicate and a working implementation, every call runs Ruby.
|
|
24
24
|
|
|
@@ -35,24 +35,27 @@ end
|
|
|
35
35
|
def summarize(text) = raise NotImplementedError # elastic until someone writes it
|
|
36
36
|
```
|
|
37
37
|
|
|
38
|
-
When the Ruby implementation runs,
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
38
|
+
When the Ruby implementation runs, whatever it returns (a `Hash`, `nil`, a string, another schema's result, …)
|
|
39
|
+
is validated against the schema and turned into the same typed result the LLM path produces.
|
|
40
|
+
`squishling_result(...)` does the same explicitly; `result(...)` is a convenience alias, skipped when the class
|
|
41
|
+
already has a `result` (see [Naming and collisions](naming.md)). Invalid deterministic output raises
|
|
42
|
+
`Squishling::InvalidOutputError` too, so a hardened path can't silently drift from the contract.
|
|
42
43
|
|
|
43
44
|
A `NotImplementedError` raised anywhere inside the method, including from code it calls, also routes to the
|
|
44
45
|
LLM.
|
|
45
46
|
|
|
46
|
-
Each call is routed on its own, including a squished method that calls itself on smaller inputs
|
|
47
|
+
Each call is routed on its own, including a squished method that calls itself on smaller inputs (those
|
|
48
|
+
recursive calls must return schema-valid values too). A subclass
|
|
47
49
|
override that calls `super` is one call: it's routed once, at the subclass. A call to the method from its own
|
|
48
|
-
`squish_when` or `squish_fallback` (or
|
|
49
|
-
implementation, so a fallback can hand the input back to Ruby with `call(**inputs)`.
|
|
50
|
+
`squish_when` or `squish_fallback` (or a purpose proc) isn't routed again: it runs the Ruby
|
|
51
|
+
implementation, so a fallback can hand the input back to Ruby with `call(**inputs)`. These inner calls
|
|
52
|
+
return a `Hash` as the typed result and any other value unchanged; only the outermost return is validated in full.
|
|
50
53
|
|
|
51
54
|
## Hardening a path
|
|
52
55
|
|
|
53
56
|
This is the workflow from [Elastic Software](https://everythingengineer.substack.com/p/beginners-write-software-with-ai):
|
|
54
57
|
|
|
55
|
-
1. Ship the class with
|
|
58
|
+
1. Ship the class with purpose and a schema but no implementation. Every call goes to the LLM.
|
|
56
59
|
2. Watch which inputs carry the volume. `squished?` on each result tells you which path served it.
|
|
57
60
|
3. Write Ruby for the high-volume cases and narrow `squish_when` so only the rest go to the LLM.
|
|
58
61
|
|
|
@@ -68,7 +71,7 @@ class TicketTriager
|
|
|
68
71
|
|
|
69
72
|
squish_context :customer_tier, :product # instance state sent alongside the arguments
|
|
70
73
|
|
|
71
|
-
squish :triage,
|
|
74
|
+
squish :triage, purpose: "Assign a priority and team.", model: "claude-haiku-4-5" do
|
|
72
75
|
string :priority, enum: %w[low med high]
|
|
73
76
|
string :team
|
|
74
77
|
end
|
|
@@ -83,16 +86,17 @@ end
|
|
|
83
86
|
|
|
84
87
|
`squish` accepts:
|
|
85
88
|
|
|
86
|
-
- `
|
|
87
|
-
- `
|
|
89
|
+
- `purpose:` (replaces the class's)
|
|
90
|
+
- `append_to_purpose:` (added to the class's, see [Appending to the purpose](#appending-to-the-purpose))
|
|
88
91
|
- `output_schema:` (or a schema block)
|
|
89
|
-
- `model:`, `provider:`, and `params:` (generation params; see [Configuration](configuration.md))
|
|
92
|
+
- `model:` or `escalation:`, `provider:`, and `params:` (generation params; see [Configuration](configuration.md))
|
|
93
|
+
- `harness:` (see [Harnesses](harnesses.md))
|
|
90
94
|
- `when:`, a predicate proc
|
|
91
|
-
- `fallback:` (see [Failure handling](failures.md))
|
|
95
|
+
- `validate:` and `fallback:` (see [Failure handling](failures.md))
|
|
92
96
|
|
|
93
97
|
`squish` can come before or after the method's `def`.
|
|
94
98
|
|
|
95
|
-
##
|
|
99
|
+
## Handing off to the LLM with `squish!`
|
|
96
100
|
|
|
97
101
|
Call `squish!` inside a squished method to hand *this call* to the LLM: for example, when the Ruby parser
|
|
98
102
|
fails on an input it wasn't written for. It sends the call's arguments, as any LLM call would, and returns the
|
|
@@ -103,8 +107,8 @@ from the method.
|
|
|
103
107
|
class InvoiceParser
|
|
104
108
|
include Squishling
|
|
105
109
|
|
|
106
|
-
|
|
107
|
-
|
|
110
|
+
purpose "Extract invoice fields from the client's raw data."
|
|
111
|
+
append_to_purpose "Here is the Ruby that parses well-formed invoices, for context on the logic and goals:",
|
|
108
112
|
self
|
|
109
113
|
output_schema do
|
|
110
114
|
string :invoice_number
|
|
@@ -115,7 +119,7 @@ class InvoiceParser
|
|
|
115
119
|
parsed = AcmeParser.parse(data)
|
|
116
120
|
result(invoice_number: parsed.id, total: parsed.sum)
|
|
117
121
|
rescue AcmeParser::ParseError => e
|
|
118
|
-
squish!(
|
|
122
|
+
squish!(append_to_purpose: "The Ruby parser above failed on this input; the error is in the context.",
|
|
119
123
|
context: { parse_error: e })
|
|
120
124
|
end
|
|
121
125
|
end
|
|
@@ -126,9 +130,10 @@ end
|
|
|
126
130
|
| Option | Effect |
|
|
127
131
|
|---|---|
|
|
128
132
|
| `context:` | A Hash sent under `"context"` with any `squish_context` values (a same-named key wins). Exceptions are sent as `{ "class", "message" }`, never their backtrace. |
|
|
129
|
-
| `
|
|
130
|
-
| `
|
|
131
|
-
| `model:`, `provider:`, `params:` | E.g.
|
|
133
|
+
| `append_to_purpose:` | Added to the declared sections; `false` (alone or first in an Array) drops them for this call |
|
|
134
|
+
| `purpose:` | Replaces the purpose |
|
|
135
|
+
| `model:` or `escalation:`, `provider:`, `params:` | E.g. send this call to a stronger model, or a whole [escalation](configuration.md#models-and-escalation), when Ruby fails. A `provider:` needs a `model:` or `escalation:`; `params:` merge key by key over the declared ones. |
|
|
136
|
+
| `harness:` | Replaces the declared [harness](harnesses.md), e.g. `harness: :judged_squishsum` to double-check a hand-off |
|
|
132
137
|
|
|
133
138
|
- **The output schema can't be overridden.** The call still returns the method's result type.
|
|
134
139
|
- **Failures** go through the normal LLM path: a declared `squish_fallback` is used, otherwise
|
|
@@ -148,9 +153,9 @@ end
|
|
|
148
153
|
or in a parent implementation reached through `super`. Calling it anywhere else, or from a `squish_fallback`
|
|
149
154
|
(which would loop), raises `Squishling::Error`.
|
|
150
155
|
|
|
151
|
-
## Appending to the
|
|
156
|
+
## Appending to the purpose
|
|
152
157
|
|
|
153
|
-
`
|
|
158
|
+
`append_to_purpose` adds sections to the system prompt after the purpose. It takes items, an Array of
|
|
154
159
|
them, or a block (treated as a Proc item). Each item is one of:
|
|
155
160
|
|
|
156
161
|
| Item | Sent as |
|
|
@@ -164,19 +169,19 @@ them, or a block (treated as a Proc item). Each item is one of:
|
|
|
164
169
|
class InvoiceParser
|
|
165
170
|
include Squishling
|
|
166
171
|
|
|
167
|
-
|
|
168
|
-
|
|
172
|
+
append_to_purpose "The Ruby that parses well-formed invoices:", self, AcmeParser
|
|
173
|
+
append_to_purpose -> { "This client's invoices are in #{currency}." }
|
|
169
174
|
|
|
170
|
-
squish :summarize,
|
|
175
|
+
squish :summarize, append_to_purpose: false do # no appendices for this method
|
|
171
176
|
string :summary
|
|
172
177
|
end
|
|
173
178
|
end
|
|
174
179
|
```
|
|
175
180
|
|
|
176
|
-
The same option goes in the class-wide call: `squishling
|
|
181
|
+
The same option goes in the class-wide call: `squishling append_to_purpose: ["...", self]`.
|
|
177
182
|
|
|
178
|
-
Sections are added down the chain: class, then subclass, then `squish :name,
|
|
179
|
-
`squish!(
|
|
183
|
+
Sections are added down the chain: class, then subclass, then `squish :name, append_to_purpose:`, then
|
|
184
|
+
`squish!(append_to_purpose:)`. `false` drops everything declared above it, so `[false, "Only this."]`
|
|
180
185
|
replaces it.
|
|
181
186
|
|
|
182
187
|
Source is read with Ruby's own parser (Prism) the first time it's needed and cached. Some limits:
|
|
@@ -195,8 +200,8 @@ Source is read with Ruby's own parser (Prism) the first time it's needed and cac
|
|
|
195
200
|
|
|
196
201
|
## What the LLM sees
|
|
197
202
|
|
|
198
|
-
- **System prompt:** your
|
|
199
|
-
`
|
|
203
|
+
- **System prompt:** your purpose (a String, or a Proc evaluated against the instance), then any
|
|
204
|
+
`append_to_purpose` sections, then a short note describing the input format.
|
|
200
205
|
- **User message:** JSON with the method's arguments, mapped to their parameter names. Any
|
|
201
206
|
`squish_context` values, and a `squish!` call's `context:`, go under `"context"`:
|
|
202
207
|
|
|
@@ -205,6 +210,16 @@ Source is read with Ruby's own parser (Prism) the first time it's needed and cac
|
|
|
205
210
|
"context": { "customer_tier": "enterprise", "product": "API" } }
|
|
206
211
|
```
|
|
207
212
|
|
|
213
|
+
- **Squishsum and ensemble harnesses:** both samples get exactly this input, and a [judge](harnesses.md#the-judge) gets
|
|
214
|
+
the same purpose and input plus both samples' outputs.
|
|
215
|
+
- **Retries and escalation:** another attempt of the same step gets the validation errors in the same conversation. A
|
|
216
|
+
later [escalation](configuration.md#models-and-escalation) step, which may be a different provider (say, local
|
|
217
|
+
Ollama, then hosted Anthropic Claude), gets the same JSON plus the previous model's rejected output and the
|
|
218
|
+
errors, including any messages your [`squish_validate`](failures.md#output-checks-squish_validate) check
|
|
219
|
+
returned. The rejected output is model-generated and is truncated to 4,000 characters; set
|
|
220
|
+
`forward_rejected: false` on a step to start it from the original input only, without the rejected output or
|
|
221
|
+
its errors. Don't put data in those messages that you wouldn't send as an argument.
|
|
222
|
+
|
|
208
223
|
`squish_context` names are read from a method of that name if there is one, otherwise from the instance
|
|
209
224
|
variable. Only context you name is sent. Instance variables are never dumped wholesale, so API clients,
|
|
210
225
|
database connections, and secrets stay out of the prompt.
|
data/docs/schemas.md
CHANGED
|
@@ -59,7 +59,8 @@ r.squished? # => true when it came from the LLM
|
|
|
59
59
|
```
|
|
60
60
|
|
|
61
61
|
On the deterministic path, return a `Hash` or build the result with `result(...)`. Either way it's validated
|
|
62
|
-
against the schema
|
|
62
|
+
against the schema, and so is anything else you return (`nil`, a string, another schema's result), which raises
|
|
63
|
+
`Squishling::InvalidOutputError` unless it matches.
|
|
63
64
|
|
|
64
65
|
## Optional vs. empty
|
|
65
66
|
|
|
@@ -84,4 +85,33 @@ end
|
|
|
84
85
|
branch as usual: `vitals` is a `Data` object or `nil`, and a list of objects is a list of `Data` objects. In a raw
|
|
85
86
|
JSON Schema, `type: ["array", "null"]` works too. A union with more than one non-null branch is ambiguous, so its
|
|
86
87
|
values come back as plain hashes. If you need the model to tell "none" apart from "not mentioned", say so in your
|
|
87
|
-
|
|
88
|
+
purpose.
|
|
89
|
+
|
|
90
|
+
## Contracts beyond the schema
|
|
91
|
+
|
|
92
|
+
Every response is validated against the full schema, so some contracts are already enforced:
|
|
93
|
+
|
|
94
|
+
- **A non-`optional` field can't be `null`.** `string :reason` rejects `"reason": null`. Make a field `optional` only
|
|
95
|
+
when "not provided" is a real answer.
|
|
96
|
+
- **Constraints** such as `enum`, `min_length`, `pattern`, `minimum`, and `min_items` are checked locally, even where
|
|
97
|
+
a provider's strict mode ignores them.
|
|
98
|
+
|
|
99
|
+
For rules that depend on another field, use Schematist's conditionals. A field can be nullable in general but
|
|
100
|
+
required when a condition holds:
|
|
101
|
+
|
|
102
|
+
```ruby
|
|
103
|
+
output_schema do
|
|
104
|
+
string :status, enum: %w[approved rejected]
|
|
105
|
+
optional(:reason) { string } # nil is fine when approved...
|
|
106
|
+
given(status: "rejected") { string :reason, min_length: 1 } # ...but not when rejected
|
|
107
|
+
end
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Providers' strict modes don't support conditional keywords (`if`/`then`/`else` from `given`, and `dependentRequired`/
|
|
111
|
+
`dependentSchemas` from `dependent`), so Squishling keeps them out of the schema it sends and enforces them itself.
|
|
112
|
+
Output that breaks a rule is rejected like any other invalid output: the call moves on to the next attempt in its
|
|
113
|
+
[escalation](configuration.md#models-and-escalation) with the errors. The model doesn't see these rules
|
|
114
|
+
up front, so state them in your purpose too.
|
|
115
|
+
|
|
116
|
+
For anything a schema can't express (sums, lookups against your data), use
|
|
117
|
+
[`squish_validate`](failures.md#output-checks-squish_validate).
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
module Squishling
|
|
4
|
-
#
|
|
4
|
+
# append_to_purpose: extra system-prompt sections placed after the purpose, layered
|
|
5
5
|
# class -> subclass -> method -> call. Each level adds to the levels above it; `false` drops them.
|
|
6
6
|
module Appendices
|
|
7
7
|
ITEM_TYPES = [String, Module, Method, UnboundMethod, Proc].freeze
|
|
@@ -29,7 +29,7 @@ module Squishling
|
|
|
29
29
|
|
|
30
30
|
values = receiver.instance_exec(&item)
|
|
31
31
|
(values.is_a?(Array) ? values : [values]).select(&:itself).map do |value|
|
|
32
|
-
check!(value, "#{label}
|
|
32
|
+
check!(value, "#{label} append_to_purpose proc", procs: false)
|
|
33
33
|
render_item(value)
|
|
34
34
|
end
|
|
35
35
|
end
|
|
@@ -43,7 +43,7 @@ module Squishling
|
|
|
43
43
|
allowed = procs ? ITEM_TYPES : ITEM_TYPES - [Proc]
|
|
44
44
|
return if allowed.any? { |type| item.is_a?(type) }
|
|
45
45
|
|
|
46
|
-
raise ConfigurationError, "#{label}:
|
|
46
|
+
raise ConfigurationError, "#{label}: append_to_purpose items must be Strings, classes or modules, " \
|
|
47
47
|
"#{procs ? 'methods, or procs' : 'or methods'} (got #{describe(item)})"
|
|
48
48
|
end
|
|
49
49
|
|
|
@@ -6,59 +6,89 @@ module Squishling
|
|
|
6
6
|
DEFAULT_METHOD = :call
|
|
7
7
|
|
|
8
8
|
# Set class-wide options in one call:
|
|
9
|
-
# squishling model: "claude-sonnet-5-5",
|
|
9
|
+
# squishling model: "claude-sonnet-5-5", purpose: "...", output_schema: MySchema
|
|
10
|
+
# model: is a single attempt; escalation: is a list of models tried in order (see ModelPath):
|
|
11
|
+
# squishling escalation: [{ model: "claude-haiku-4-5", attempts: 2 }, "claude-sonnet-5-5", "claude-opus-5-5"]
|
|
10
12
|
# Pass provider: alongside model: for models missing from RubyLLM's registry, e.g.
|
|
11
|
-
# squishling model: "gpt-
|
|
13
|
+
# squishling model: "gpt-7-preview", provider: :openai
|
|
14
|
+
# provider: needs a model: or escalation: (declared on the class), and a subclass's model never inherits its
|
|
15
|
+
# parent's provider.
|
|
12
16
|
# params: are generation params merged over the configured defaults (see Configuration#default_params):
|
|
13
17
|
# squishling params: { temperature: 0.1, top_p: 0.9 }
|
|
14
|
-
#
|
|
15
|
-
# squishling
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
18
|
+
# append_to_purpose: adds sections after the purpose (see #append_to_purpose):
|
|
19
|
+
# squishling append_to_purpose: ["The Ruby that handles well-formed input:", self]
|
|
20
|
+
# harness: chooses how the escalation is used (see Harness):
|
|
21
|
+
# squishling harness: :judged_squishsum
|
|
22
|
+
# squawk: is called after every LLM attempt with the raw output (see Configuration#squawk); false silences
|
|
23
|
+
# an inherited one:
|
|
24
|
+
# squishling squawk: ->(output:, metadata:, error:) { Tracer.record(output, metadata, error) }
|
|
25
|
+
def squishling(model: nil, escalation: nil, provider: nil, params: nil, harness: nil, purpose: nil,
|
|
26
|
+
append_to_purpose: nil, output_schema: nil, squawk: nil)
|
|
27
|
+
path = ModelPath.declare(model, escalation, to_s)
|
|
28
|
+
# A provider belongs to the model declared at the same level. Without a model here it would apply to
|
|
29
|
+
# nothing (the model comes from the config or a parent class), so it is an error unless this class already
|
|
30
|
+
# declared its own.
|
|
31
|
+
if provider && !path && !instance_variable_defined?(:@squishling_model_path)
|
|
32
|
+
raise ConfigurationError, "#{self}: provider: needs a model: or escalation: declared on this class"
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# Stored (even as nil) whenever a model is, so a subclass that declares its own model never inherits a
|
|
36
|
+
# provider meant for its parent's.
|
|
37
|
+
@squishling_model_path = path if path
|
|
38
|
+
@squishling_provider = provider if provider || path
|
|
20
39
|
@squishling_params = Params.normalize(params, "#{self} params") if params
|
|
21
|
-
|
|
22
|
-
self.
|
|
40
|
+
@squishling_harness = Harness.normalize(harness, to_s) unless harness.nil?
|
|
41
|
+
self.purpose(purpose) if purpose
|
|
42
|
+
self.append_to_purpose(append_to_purpose) unless append_to_purpose.nil?
|
|
23
43
|
self.output_schema(output_schema) if output_schema
|
|
44
|
+
@squishling_squawk = Squawk.validate(squawk, to_s) unless squawk.nil?
|
|
24
45
|
self
|
|
25
46
|
end
|
|
26
47
|
|
|
27
|
-
def
|
|
28
|
-
squishling_lookup(:@
|
|
48
|
+
def squishling_squawk
|
|
49
|
+
squishling_lookup(:@squishling_squawk)
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# The class's (or nearest ancestor's) model or escalation, as normalized steps.
|
|
53
|
+
def squishling_model_path
|
|
54
|
+
squishling_lookup(:@squishling_model_path)
|
|
29
55
|
end
|
|
30
56
|
|
|
31
57
|
def squishling_provider
|
|
32
58
|
squishling_lookup(:@squishling_provider)
|
|
33
59
|
end
|
|
34
60
|
|
|
61
|
+
def squishling_harness
|
|
62
|
+
squishling_lookup(:@squishling_harness)
|
|
63
|
+
end
|
|
64
|
+
|
|
35
65
|
# Generation params merged down the inheritance chain, so a subclass overrides individual keys.
|
|
36
66
|
def squishling_params
|
|
37
67
|
squishling_inherited(:squishling_params, {}).merge(@squishling_params || {})
|
|
38
68
|
end
|
|
39
69
|
|
|
40
70
|
# The system prompt. A String, or a Proc evaluated against the instance.
|
|
41
|
-
def
|
|
42
|
-
return squishling_lookup(:@
|
|
71
|
+
def purpose(text = nil, &block)
|
|
72
|
+
return squishling_lookup(:@squishling_purpose) if text.nil? && block.nil?
|
|
43
73
|
|
|
44
|
-
@
|
|
74
|
+
@squishling_purpose = block || text
|
|
45
75
|
end
|
|
46
76
|
|
|
47
|
-
# Sections appended to the system prompt after the
|
|
77
|
+
# Sections appended to the system prompt after the purpose, added to by subclasses, `squish`, and
|
|
48
78
|
# `squish!`. Items: Strings; a class or module (`self` for this class) or a method (`instance_method(:call)`),
|
|
49
79
|
# sent as its Ruby source; or a Proc evaluated against the instance. `false` drops inherited items.
|
|
50
|
-
#
|
|
51
|
-
#
|
|
52
|
-
def
|
|
80
|
+
# append_to_purpose "Here is the Ruby that parses well-formed invoices:", self
|
|
81
|
+
# append_to_purpose { "This client's invoices are in #{currency}." }
|
|
82
|
+
def append_to_purpose(*items, &block)
|
|
53
83
|
items = items.first if items.size == 1 && items.first.is_a?(Array)
|
|
54
84
|
items += [block] if block
|
|
55
|
-
(@
|
|
85
|
+
(@squishling_append_to_purpose ||= []).concat(Appendices.normalize(items, "#{self} append_to_purpose"))
|
|
56
86
|
self
|
|
57
87
|
end
|
|
58
88
|
|
|
59
89
|
# Every level's items in declaration order, `false` markers included (see Appendices.resolve).
|
|
60
|
-
def
|
|
61
|
-
squishling_inherited(:
|
|
90
|
+
def squishling_append_to_purpose
|
|
91
|
+
squishling_inherited(:squishling_append_to_purpose, []) + (@squishling_append_to_purpose || [])
|
|
62
92
|
end
|
|
63
93
|
|
|
64
94
|
# The output format: a Schematist::Schema subclass (RubyLLM::Schema with the ruby_llm-schema shim),
|
|
@@ -81,7 +111,7 @@ module Squishling
|
|
|
81
111
|
|
|
82
112
|
# Called with the error and the method's inputs (as keywords) when the LLM path fails with an
|
|
83
113
|
# InvalidOutputError or LLMError, evaluated against the instance. Its return value is used as
|
|
84
|
-
# the result (
|
|
114
|
+
# the result (validated against the schema and typed like a deterministic return); re-raise to propagate.
|
|
85
115
|
# squish_fallback { |error, **inputs| { priority: "medium", team: "support" } }
|
|
86
116
|
def squish_fallback(&block)
|
|
87
117
|
@squishling_fallback = block
|
|
@@ -91,6 +121,19 @@ module Squishling
|
|
|
91
121
|
squishling_lookup(:@squishling_fallback)
|
|
92
122
|
end
|
|
93
123
|
|
|
124
|
+
# Extra checks on LLM output that already matches the schema, called with the typed result and the
|
|
125
|
+
# method's inputs (as keywords), evaluated against the instance. Return nil (or true, "", []) to accept,
|
|
126
|
+
# or error message(s) to reject the output and move on to the next attempt, with the messages fed back:
|
|
127
|
+
# squish_validate { |result, **| "total must equal the line items" if result.total != result.line_items.sum }
|
|
128
|
+
# A dry-validation style result (responding to success? and errors) is accepted too.
|
|
129
|
+
def squish_validate(&block)
|
|
130
|
+
@squishling_validator = block
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
def squishling_validator
|
|
134
|
+
squishling_lookup(:@squishling_validator)
|
|
135
|
+
end
|
|
136
|
+
|
|
94
137
|
# Instance state (attributes or instance variables) to send to the LLM alongside the arguments.
|
|
95
138
|
def squish_context(*names)
|
|
96
139
|
(@squishling_context_names ||= []).concat(names.map(&:to_sym))
|
|
@@ -101,19 +144,26 @@ module Squishling
|
|
|
101
144
|
end
|
|
102
145
|
|
|
103
146
|
# Make methods elastic. Each may override the class-level settings:
|
|
104
|
-
# squish :triage,
|
|
105
|
-
# fallback: ->(error, **) { { priority: "medium" } },
|
|
147
|
+
# squish :triage, purpose: "...", escalation: %w[claude-haiku-4-5 claude-sonnet-5-5], when: ->(**) { true },
|
|
148
|
+
# fallback: ->(error, **) { { priority: "medium" } }, append_to_purpose: [...],
|
|
149
|
+
# validate: ->(result, **) { "team is required" if result.team.empty? },
|
|
150
|
+
# harness: :squishsum, squawk: ->(output:, error:, **) { Tracer.record(output, error) } do
|
|
106
151
|
# string :priority
|
|
107
152
|
# end
|
|
108
|
-
def squish(*names,
|
|
109
|
-
params: nil, when: nil, fallback: nil, &schema_block)
|
|
153
|
+
def squish(*names, purpose: nil, append_to_purpose: nil, output_schema: nil, model: nil, escalation: nil,
|
|
154
|
+
provider: nil, params: nil, harness: nil, when: nil, fallback: nil, validate: nil, squawk: nil, &schema_block)
|
|
110
155
|
schema = schema_block ? Schematist::Schema.create(&schema_block) : output_schema
|
|
156
|
+
model = ModelPath.declare(model, escalation, "#{self} squish")
|
|
157
|
+
raise ConfigurationError, "#{self} squish: provider: needs a model: or escalation:" if provider && !model
|
|
158
|
+
|
|
111
159
|
params &&= Params.normalize(params, "#{self} squish params")
|
|
112
|
-
unless
|
|
113
|
-
|
|
160
|
+
harness = Harness.normalize(harness, "#{self} squish") unless harness.nil?
|
|
161
|
+
unless append_to_purpose.nil?
|
|
162
|
+
append_to_purpose = Appendices.normalize(append_to_purpose, "#{self} squish append_to_purpose")
|
|
114
163
|
end
|
|
115
|
-
|
|
116
|
-
|
|
164
|
+
squawk = Squawk.validate(squawk, "#{self} squish")
|
|
165
|
+
options = { purpose:, append_to_purpose:, output_schema: schema, model:, provider:, params:, harness:,
|
|
166
|
+
predicate: binding.local_variable_get(:when), fallback:, validator: validate, squawk: }.compact
|
|
117
167
|
|
|
118
168
|
names.map(&:to_sym).each do |name|
|
|
119
169
|
(@squishling_methods ||= {})[name] = options
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Squishling
|
|
4
|
+
# `include Squishling` puts its methods ahead of inherited ones, so a class that inherits a method of the same
|
|
5
|
+
# name (Sinatra::Base.call, a parent's own `purpose`, ...) would be silently shadowed. Methods the class
|
|
6
|
+
# defines itself always win, so they are never reported.
|
|
7
|
+
module Collisions
|
|
8
|
+
INSTANCE_METHODS = %i[squish! squishling_result].freeze
|
|
9
|
+
|
|
10
|
+
module_function
|
|
11
|
+
|
|
12
|
+
# Raises ConfigurationError naming every inherited method Squishling would override. Call it before
|
|
13
|
+
# extending the class with ClassMethods, and after the module is in the class's ancestors.
|
|
14
|
+
def check!(base)
|
|
15
|
+
found = class_collisions(base) + instance_collisions(base)
|
|
16
|
+
return if found.empty?
|
|
17
|
+
|
|
18
|
+
raise ConfigurationError,
|
|
19
|
+
"#{base} inherits methods that Squishling would override: #{found.join(', ')}. " \
|
|
20
|
+
"Include Squishling in a plain Ruby class instead (compose the framework object rather than inheriting it)"
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def class_collisions(base)
|
|
24
|
+
ClassMethods.public_instance_methods(false).filter_map do |name|
|
|
25
|
+
next if name == :inherited || !base.respond_to?(name, true)
|
|
26
|
+
|
|
27
|
+
owner = base.method(name).owner
|
|
28
|
+
next if owner == base.singleton_class || owner == ClassMethods
|
|
29
|
+
|
|
30
|
+
"#{base}.#{name} (from #{describe(owner)})"
|
|
31
|
+
end
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def instance_collisions(base)
|
|
35
|
+
ancestors = base.ancestors
|
|
36
|
+
inherited = ancestors.drop(ancestors.index(Squishling) + 1)
|
|
37
|
+
|
|
38
|
+
INSTANCE_METHODS.filter_map do |name|
|
|
39
|
+
next if defined_in?(base, name)
|
|
40
|
+
|
|
41
|
+
owner = inherited.find { |mod| defined_in?(mod, name) }
|
|
42
|
+
"#{base}##{name} (from #{describe(owner)})" if owner
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
def defined_in?(mod, name)
|
|
47
|
+
mod.method_defined?(name, false) || mod.private_method_defined?(name, false)
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
def describe(owner)
|
|
51
|
+
owner.singleton_class? ? "#{owner.attached_object}'s class methods" : owner.to_s
|
|
52
|
+
end
|
|
53
|
+
end
|
|
54
|
+
end
|
|
@@ -2,12 +2,20 @@
|
|
|
2
2
|
|
|
3
3
|
module Squishling
|
|
4
4
|
class Configuration
|
|
5
|
-
# Model used by every squishling class that doesn't declare its own.
|
|
6
|
-
# When
|
|
7
|
-
|
|
5
|
+
# Model used by every squishling class that doesn't declare its own (a single attempt).
|
|
6
|
+
# When neither this nor default_escalation is set, RubyLLM's own default model is used.
|
|
7
|
+
attr_reader :default_model
|
|
8
8
|
|
|
9
|
-
#
|
|
10
|
-
#
|
|
9
|
+
# Models tried in order by every squishling class that doesn't declare its own, e.g.
|
|
10
|
+
# [{ model: "claude-haiku-4-5", attempts: 2 }, "claude-sonnet-5-5", "claude-opus-5-5"]
|
|
11
|
+
# See ModelPath. default_model and default_escalation set the same thing: assigning one clears the other.
|
|
12
|
+
attr_reader :default_escalation
|
|
13
|
+
|
|
14
|
+
# The validated default_model or default_escalation steps (nil when neither is set).
|
|
15
|
+
attr_reader :default_model_path
|
|
16
|
+
|
|
17
|
+
# Provider for default_model/default_escalation steps that don't name their own (e.g. :openai). Only
|
|
18
|
+
# needed for models missing from RubyLLM's registry.
|
|
11
19
|
attr_accessor :default_provider
|
|
12
20
|
|
|
13
21
|
# Generation params applied to every call, overridable per class and per method.
|
|
@@ -15,22 +23,63 @@ module Squishling
|
|
|
15
23
|
# with_thinking; any other key (top_p, max_tokens, seed, ...) is passed to the provider as-is.
|
|
16
24
|
attr_reader :default_params
|
|
17
25
|
|
|
18
|
-
# How
|
|
19
|
-
|
|
26
|
+
# How every squishling class that doesn't declare its own harness uses its escalation (see Harness):
|
|
27
|
+
# :escalation (the default), :squishsum, :judged_squishsum, :ensemble, :judged_ensemble, or a Hash with
|
|
28
|
+
# type: and its options.
|
|
29
|
+
attr_reader :default_harness
|
|
20
30
|
|
|
21
|
-
# Optional Logger for routing decisions.
|
|
31
|
+
# Optional Logger for routing and escalation decisions.
|
|
22
32
|
attr_accessor :logger
|
|
23
33
|
|
|
34
|
+
# Optional callable run after every LLM attempt with the raw output (see Squawk), e.g.
|
|
35
|
+
# ->(output:, metadata:, error:) { Tracer.record(output, metadata, error) }
|
|
36
|
+
attr_reader :squawk
|
|
37
|
+
|
|
24
38
|
def initialize
|
|
25
39
|
@default_model = nil
|
|
40
|
+
@default_escalation = nil
|
|
41
|
+
@default_model_path = nil
|
|
26
42
|
@default_provider = nil
|
|
27
43
|
@default_params = {}
|
|
28
|
-
@
|
|
44
|
+
@default_harness = nil
|
|
29
45
|
@logger = nil
|
|
46
|
+
@squawk = nil
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def default_model=(model)
|
|
50
|
+
@default_model_path = ModelPath.from_model(model, "default_model")
|
|
51
|
+
@default_escalation = nil
|
|
52
|
+
@default_model = model
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def default_escalation=(escalation)
|
|
56
|
+
@default_model_path = ModelPath.from_escalation(escalation, "default_escalation")
|
|
57
|
+
@default_model = nil
|
|
58
|
+
@default_escalation = escalation
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def squawk=(hook)
|
|
62
|
+
@squawk = Squawk.validate(hook, "Squishling.configure")
|
|
30
63
|
end
|
|
31
64
|
|
|
32
65
|
def default_params=(params)
|
|
33
66
|
@default_params = Params.normalize(params, "default_params")
|
|
34
67
|
end
|
|
68
|
+
|
|
69
|
+
def default_harness=(harness)
|
|
70
|
+
@default_harness = harness.nil? ? nil : Harness.normalize(harness, "default_harness")
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
MAX_RETRIES_REMOVED = "max_retries was removed; use default_escalation (or a class/method escalation:) " \
|
|
74
|
+
"with attempts:, e.g. [{ model: \"claude-haiku-4-5\", attempts: 2 }]"
|
|
75
|
+
|
|
76
|
+
# Removed: the escalation decides how many attempts run.
|
|
77
|
+
def max_retries
|
|
78
|
+
raise ConfigurationError, MAX_RETRIES_REMOVED
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def max_retries=(_value)
|
|
82
|
+
raise ConfigurationError, MAX_RETRIES_REMOVED
|
|
83
|
+
end
|
|
35
84
|
end
|
|
36
85
|
end
|