crystil 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +35 -0
- data/LICENSE +1 -1
- data/README.md +12 -1
- data/lib/crystil/version.rb +1 -1
- data/lib/crystil/wrappers/anthropic.rb +138 -2
- data/lib/crystil/wrappers/base.rb +4 -0
- data/lib/crystil/wrappers/event_stream.rb +205 -0
- data/lib/crystil/wrappers/geminiai.rb +43 -7
- data/lib/crystil/wrappers/groq.rb +7 -7
- data/sig/crystil/wrappers/anthropic.rbs +12 -0
- data/sig/crystil/wrappers/base.rbs +4 -1
- data/sig/crystil/wrappers/event_stream.rbs +24 -0
- data/sig/crystil/wrappers/geminiai.rbs +9 -0
- metadata +4 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 90d050805fee5db26efd12b183842cb70ff4a971bac3d8cc9a8e9a5717f0174e
|
|
4
|
+
data.tar.gz: a205e8dd09052a6f686efc427b609b8b6ceebde6c147fd54708d66350f374954
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 4ed5f37da282a51c97d8262452927ece7cdaaecf02139eeec6c1c8b84d854b435483d776e8d8c49436e3f4ced7e1ca5700b1d42ca1675e24848894531e73c17d
|
|
7
|
+
data.tar.gz: 58b439278a99afe07d4c89b63824a0b08a08e6d52b5a2555b8629b6c52afc32251b8251b8e9c3885f0738778d4473c975fa9ace9ec1451d38cdc86a3de42f63c
|
data/CHANGELOG.md
CHANGED
|
@@ -75,3 +75,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
75
75
|
- Default endpoints now resolve to `api.crystil.com` and `collector.crystil.com`.
|
|
76
76
|
|
|
77
77
|
No API surface, method signature, or request payload changed — this release is a rename only.
|
|
78
|
+
|
|
79
|
+
## [0.6.0] - 2026-09-18
|
|
80
|
+
|
|
81
|
+
### Added
|
|
82
|
+
- Capture Anthropic streaming calls. `messages.stream` and `messages.stream_raw` are now
|
|
83
|
+
wrapped, and their telemetry is submitted once the caller has read the stream, carrying
|
|
84
|
+
the events as the unmerged list the backend's streamed extractor reads — so a streamed
|
|
85
|
+
call reports its content and token usage instead of nothing at all. Sentinel's relevance
|
|
86
|
+
check now runs on these calls too. `messages.create` is unchanged.
|
|
87
|
+
|
|
88
|
+
A read that ends early — `break`, or any of `first` / `take` / `find` / `any?` — is
|
|
89
|
+
recorded too, carrying the events received so far. So is a read cut short by an error,
|
|
90
|
+
which reports as failed but keeps the partial content and token counts rather than
|
|
91
|
+
discarding them.
|
|
92
|
+
|
|
93
|
+
### Fixed
|
|
94
|
+
- Record which model a `gemini-ai` call used. That gem fixes the model on the client and puts
|
|
95
|
+
it in the request path rather than the request body, so a failed call — a local raise, a 4xx,
|
|
96
|
+
a timeout — carried no model anywhere: not in the query, and not in the response, which is
|
|
97
|
+
the error. The backend could not tell which model the call was for and quarantined the record
|
|
98
|
+
instead of storing it as the zero-cost failed call it is. The wrapper now reads the model off
|
|
99
|
+
the client and records it on the query, so failed `gemini-ai` calls report like every other
|
|
100
|
+
provider's. Successful calls are unaffected — their response already named the model.
|
|
101
|
+
|
|
102
|
+
The model is added only to the copy Crystil records, for both the analytics payload and the
|
|
103
|
+
sentinel relevance check. Your parameters are not modified and the outbound request to Google
|
|
104
|
+
is unchanged. A `model` you pass yourself is kept as-is.
|
|
105
|
+
|
|
106
|
+
### Notes
|
|
107
|
+
- `messages.create(stream: true)` raises `ArgumentError` in the `anthropic` gem, which is
|
|
108
|
+
why the wrapper patches the two streaming methods rather than the create path the
|
|
109
|
+
JavaScript and Python SDKs use.
|
|
110
|
+
- `google-genai` (v0.1.1) still has no streaming wrapper because the gem exposes no
|
|
111
|
+
streaming method. Streamed Gemini calls through the `gemini-ai` gem were already captured.
|
|
112
|
+
|
data/LICENSE
CHANGED
data/README.md
CHANGED
|
@@ -37,6 +37,11 @@ no API key stored in GitHub, and **you should never run `gem push` or
|
|
|
37
37
|
`rake release` by hand** — the release's safety checks only exist in CI, and a
|
|
38
38
|
version number can never be reused on RubyGems once taken.
|
|
39
39
|
|
|
40
|
+
The supported path is the **`/run-release` skill**
|
|
41
|
+
([.claude/skills/run-release/SKILL.md](.claude/skills/run-release/SKILL.md)), which walks these
|
|
42
|
+
steps, runs the same checks CI will run before anything irreversible happens, and stops for
|
|
43
|
+
confirmation before merging and before publishing. The manual steps below are the fallback.
|
|
44
|
+
|
|
40
45
|
To release version `X.Y.Z`:
|
|
41
46
|
|
|
42
47
|
1. **Update the version number** in [lib/crystil/version.rb](lib/crystil/version.rb):
|
|
@@ -60,13 +65,19 @@ To release version `X.Y.Z`:
|
|
|
60
65
|
`--target`, and pushing a tag on its own triggers nothing.
|
|
61
66
|
```bash
|
|
62
67
|
gh release create vX.Y.Z --target main --title "vX.Y.Z" \
|
|
63
|
-
--notes "$(
|
|
68
|
+
--notes "$(awk '/^## \[X\.Y\.Z\]/{f=1;next} f&&/^## \[/{exit} f' CHANGELOG.md)"
|
|
64
69
|
```
|
|
65
70
|
Pass the notes explicitly as above (or `--generate-notes` for an
|
|
66
71
|
auto-generated commit summary). Don't use `--notes-from-tag` here — when gh
|
|
67
72
|
creates the tag it is lightweight, so that flag falls back to the commit
|
|
68
73
|
message and your notes end up reading "Merge pull request #NN".
|
|
69
74
|
|
|
75
|
+
Use the `awk` form, not `sed -n '/^## \[X.Y.Z\]/,/^## \[/p' | sed '$d'`.
|
|
76
|
+
CHANGELOG.md is oldest-first, so the version you are releasing is the last
|
|
77
|
+
section in the file; with no following `## [` to stop at, the `sed` form runs
|
|
78
|
+
to EOF and `sed '$d'` then silently eats its final line. `awk` stops at the
|
|
79
|
+
next header or EOF and drops nothing.
|
|
80
|
+
|
|
70
81
|
Or use the GitHub UI: *Releases → Draft a new release*, create the tag
|
|
71
82
|
`vX.Y.Z` against `main`, then **Publish release**. Saving a draft does not
|
|
72
83
|
trigger anything; only publishing does.
|
data/lib/crystil/version.rb
CHANGED
|
@@ -6,6 +6,8 @@ module Crystil
|
|
|
6
6
|
module Wrappers
|
|
7
7
|
# Wrapper for Anthropic Ruby client
|
|
8
8
|
class Anthropic
|
|
9
|
+
REQUEST_OPTIONS_KEYS = [:request_options, "request_options"].freeze
|
|
10
|
+
|
|
9
11
|
def initialize(config, collector, sentinel = nil)
|
|
10
12
|
@config = config
|
|
11
13
|
@collector = collector
|
|
@@ -30,6 +32,57 @@ module Crystil
|
|
|
30
32
|
client
|
|
31
33
|
end
|
|
32
34
|
|
|
35
|
+
# A copy of the params without `request_options`, whose `extra_headers` can hold credentials.
|
|
36
|
+
# Used for telemetry and sentinel only; the gem still gets the original.
|
|
37
|
+
def self.recorded_params(params)
|
|
38
|
+
return params unless params.is_a?(Hash)
|
|
39
|
+
|
|
40
|
+
params.reject { |key, _| REQUEST_OPTIONS_KEYS.include?(key) }
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# The query as it went over the wire.
|
|
44
|
+
#
|
|
45
|
+
# The caller never passes `stream:` to `messages.stream` / `stream_raw` —
|
|
46
|
+
# the gem stores it on the parsed params before building the request — so
|
|
47
|
+
# recording it here keeps the payload honest about what was actually sent.
|
|
48
|
+
# Returns a new hash; the caller's is never mutated.
|
|
49
|
+
def self.streamed_query(params)
|
|
50
|
+
return params unless params.is_a?(Hash)
|
|
51
|
+
|
|
52
|
+
params.merge(stream: true)
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
# The enumerable whose events are the ones the API actually sent.
|
|
56
|
+
#
|
|
57
|
+
# `stream_raw` hands back an `Anthropic::Internal::Stream` whose own events
|
|
58
|
+
# are the raw ones, so it is watched directly.
|
|
59
|
+
#
|
|
60
|
+
# `stream` hands back a `MessageStream`, and watching *that* would record
|
|
61
|
+
# the wrong list. Its `each` yields a mix: it adds synthesized `text` /
|
|
62
|
+
# `input_json` / `thinking` events and replaces `content_block_stop` and
|
|
63
|
+
# `message_stop` with richer versions of its own. The backend needs the
|
|
64
|
+
# real `content_block_stop` to parse a tool call's accumulated JSON. Its
|
|
65
|
+
# `@raw_stream` ivar holds the untouched API stream and is read lazily (the
|
|
66
|
+
# iterator is a `chain_fused` Enumerator whose body runs on first
|
|
67
|
+
# iteration), so patching `each` on the object already sitting there is
|
|
68
|
+
# picked up when the caller starts reading.
|
|
69
|
+
#
|
|
70
|
+
# `@raw_stream` is private to the gem, and the gemspec floats across the
|
|
71
|
+
# 1.x line, so a future release could move or rename it. When that inner
|
|
72
|
+
# stream cannot be found, the `stream` path records nothing rather than
|
|
73
|
+
# falling back to the outer object: a streamed payload missing
|
|
74
|
+
# `message_start` makes the backend's extractor raise, so emitting nothing
|
|
75
|
+
# beats emitting a list it cannot read.
|
|
76
|
+
def self.raw_event_stream(stream, method_name)
|
|
77
|
+
return nil unless stream.respond_to?(:each)
|
|
78
|
+
return stream if method_name == :stream_raw
|
|
79
|
+
return nil unless stream.respond_to?(:instance_variable_defined?)
|
|
80
|
+
return nil unless stream.instance_variable_defined?(:@raw_stream)
|
|
81
|
+
|
|
82
|
+
inner = stream.instance_variable_get(:@raw_stream)
|
|
83
|
+
inner if inner.respond_to?(:each)
|
|
84
|
+
end
|
|
85
|
+
|
|
33
86
|
private
|
|
34
87
|
|
|
35
88
|
def validate_client!(client)
|
|
@@ -47,6 +100,17 @@ module Crystil
|
|
|
47
100
|
messages_resource.instance_variable_set(:@crystil_collector, client.instance_variable_get(:@crystil_collector))
|
|
48
101
|
messages_resource.instance_variable_set(:@crystil_sentinel, client.instance_variable_get(:@crystil_sentinel))
|
|
49
102
|
|
|
103
|
+
wrap_create_method(messages_resource)
|
|
104
|
+
|
|
105
|
+
# `messages.create(stream: true)` raises ArgumentError in this gem — it
|
|
106
|
+
# routes streaming through its own methods, which bypass `create`
|
|
107
|
+
# entirely. Wrap them only when present, so mocked clients that define
|
|
108
|
+
# just `create` still register.
|
|
109
|
+
wrap_streaming_method(messages_resource, :stream) if messages_resource.respond_to?(:stream)
|
|
110
|
+
wrap_streaming_method(messages_resource, :stream_raw) if messages_resource.respond_to?(:stream_raw)
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
def wrap_create_method(messages_resource)
|
|
50
114
|
# Store the original create method
|
|
51
115
|
original_create = messages_resource.method(:create)
|
|
52
116
|
|
|
@@ -59,7 +123,7 @@ module Crystil
|
|
|
59
123
|
version = defined?(::Anthropic::VERSION) ? ::Anthropic::VERSION : nil
|
|
60
124
|
|
|
61
125
|
# Extract parameters for sentinel and analytics
|
|
62
|
-
params = kwargs.any? ? kwargs : (args.first || {})
|
|
126
|
+
params = Crystil::Wrappers::Anthropic.recorded_params(kwargs.any? ? kwargs : (args.first || {}))
|
|
63
127
|
sentinel = instance_variable_get(:@crystil_sentinel)
|
|
64
128
|
sentinel&.raise_if_irrelevant!(
|
|
65
129
|
title: ANTHROPIC_CLIENT_TITLE,
|
|
@@ -91,7 +155,7 @@ module Crystil
|
|
|
91
155
|
# We don't want to send intercepts to collector
|
|
92
156
|
raise e
|
|
93
157
|
rescue StandardError => e
|
|
94
|
-
params = kwargs.any? ? kwargs : (args.first || {})
|
|
158
|
+
params = Crystil::Wrappers::Anthropic.recorded_params(kwargs.any? ? kwargs : (args.first || {}))
|
|
95
159
|
|
|
96
160
|
crystil_submit_error_analytics(
|
|
97
161
|
method: :create,
|
|
@@ -107,6 +171,78 @@ module Crystil
|
|
|
107
171
|
raise e
|
|
108
172
|
end
|
|
109
173
|
end
|
|
174
|
+
|
|
175
|
+
# Wraps `messages.stream` / `messages.stream_raw`.
|
|
176
|
+
#
|
|
177
|
+
# Both return a lazy stream, so there is nothing to record when the method
|
|
178
|
+
# returns — the payload is the events, and none have arrived yet. The
|
|
179
|
+
# response is instead recorded once the caller has read the stream, as the
|
|
180
|
+
# unmerged event list the backend's streamed Anthropic extractor expects.
|
|
181
|
+
# See `Base#crystil_record_event_stream`.
|
|
182
|
+
def wrap_streaming_method(messages_resource, method_name)
|
|
183
|
+
original_method = messages_resource.method(method_name)
|
|
184
|
+
|
|
185
|
+
messages_resource.define_singleton_method(method_name) do |*args, **kwargs, &block|
|
|
186
|
+
# Include Base module methods
|
|
187
|
+
extend Base unless singleton_class.include?(Base)
|
|
188
|
+
|
|
189
|
+
start_time = Time.now
|
|
190
|
+
version = defined?(::Anthropic::VERSION) ? ::Anthropic::VERSION : nil
|
|
191
|
+
|
|
192
|
+
# Extract parameters for sentinel and analytics
|
|
193
|
+
params = Crystil::Wrappers::Anthropic.recorded_params(kwargs.any? ? kwargs : (args.first || {}))
|
|
194
|
+
sentinel = instance_variable_get(:@crystil_sentinel)
|
|
195
|
+
sentinel&.raise_if_irrelevant!(
|
|
196
|
+
title: ANTHROPIC_CLIENT_TITLE,
|
|
197
|
+
request: params,
|
|
198
|
+
version: version
|
|
199
|
+
)
|
|
200
|
+
|
|
201
|
+
# Call original method
|
|
202
|
+
stream = if kwargs.any?
|
|
203
|
+
original_method.call(**kwargs, &block)
|
|
204
|
+
else
|
|
205
|
+
original_method.call(*args, &block)
|
|
206
|
+
end
|
|
207
|
+
|
|
208
|
+
events = Crystil::Wrappers::Anthropic.raw_event_stream(stream, method_name)
|
|
209
|
+
|
|
210
|
+
if events
|
|
211
|
+
crystil_record_event_stream(
|
|
212
|
+
events,
|
|
213
|
+
closer: stream,
|
|
214
|
+
method: method_name,
|
|
215
|
+
# Copied now, not at report time, so changes the caller makes while reading don't leak in.
|
|
216
|
+
kwargs: deep_copy(Crystil::Wrappers::Anthropic.streamed_query(params)),
|
|
217
|
+
start_time: start_time,
|
|
218
|
+
title: ANTHROPIC_CLIENT_TITLE,
|
|
219
|
+
version: version
|
|
220
|
+
)
|
|
221
|
+
end
|
|
222
|
+
|
|
223
|
+
# The caller's own object, so `accumulated_message`, `text`,
|
|
224
|
+
# `until_done` and the rest of its surface keep working.
|
|
225
|
+
stream
|
|
226
|
+
rescue CrystilRequestInterceptedError => e
|
|
227
|
+
# We don't want to send intercepts to collector
|
|
228
|
+
raise e
|
|
229
|
+
rescue StandardError => e
|
|
230
|
+
params = Crystil::Wrappers::Anthropic.recorded_params(kwargs.any? ? kwargs : (args.first || {}))
|
|
231
|
+
|
|
232
|
+
crystil_submit_error_analytics(
|
|
233
|
+
method: method_name,
|
|
234
|
+
args: [],
|
|
235
|
+
kwargs: Crystil::Wrappers::Anthropic.streamed_query(params),
|
|
236
|
+
error: e,
|
|
237
|
+
start_time: start_time,
|
|
238
|
+
end_time: Time.now,
|
|
239
|
+
title: ANTHROPIC_CLIENT_TITLE,
|
|
240
|
+
version: version
|
|
241
|
+
)
|
|
242
|
+
|
|
243
|
+
raise e
|
|
244
|
+
end
|
|
245
|
+
end
|
|
110
246
|
end
|
|
111
247
|
end
|
|
112
248
|
end
|
|
@@ -2,10 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
require "time"
|
|
4
4
|
|
|
5
|
+
require_relative "event_stream"
|
|
6
|
+
|
|
5
7
|
module Crystil
|
|
6
8
|
module Wrappers
|
|
7
9
|
# Base functionality for all provider wrappers
|
|
8
10
|
module Base
|
|
11
|
+
include EventStream
|
|
12
|
+
|
|
9
13
|
def crystil_wrap_method(method_name, _provider_name)
|
|
10
14
|
return if method(method_name).source_location&.first&.include?("crystil")
|
|
11
15
|
|
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "fiber"
|
|
4
|
+
require "json"
|
|
5
|
+
|
|
6
|
+
module Crystil
|
|
7
|
+
module Wrappers
|
|
8
|
+
# Recording a lazy event stream as the list of events the provider sent.
|
|
9
|
+
#
|
|
10
|
+
# Mixed into `Base`, so a wrapper reaches it the same way it reaches
|
|
11
|
+
# `crystil_merge_streaming_chunk`. The two are alternatives, picked by what
|
|
12
|
+
# the backend extractor for that provider reads: merge the chunks into one
|
|
13
|
+
# object for OpenAI and Groq, record the events as a list for Anthropic.
|
|
14
|
+
module EventStream
|
|
15
|
+
STREAM_CUT_SHORT_MESSAGE = "Stream read ended before the provider finished sending events"
|
|
16
|
+
STREAM_CLOSED_UNREAD_MESSAGE = "Stream closed before any events were read"
|
|
17
|
+
|
|
18
|
+
# Record a lazy event stream as the unmerged list of events the provider
|
|
19
|
+
# sent, once the caller has finished reading it.
|
|
20
|
+
#
|
|
21
|
+
# This is the counterpart to `crystil_merge_streaming_chunk` for providers
|
|
22
|
+
# whose backend extractor reads a *list*. Anthropic is one: its streamed
|
|
23
|
+
# extractor keys off `conversation.response` being a list and takes the
|
|
24
|
+
# model off `message_start`. Merging the events first would flatten them
|
|
25
|
+
# into an object with no top-level model, which the backend cannot read.
|
|
26
|
+
#
|
|
27
|
+
# The stream object is returned as-is with only its `each` replaced, so
|
|
28
|
+
# everything else the provider hung on it keeps working. Enumerable
|
|
29
|
+
# methods (`map`, `to_a`, `first`, ...) route through `each` and are
|
|
30
|
+
# recorded too; `to_enum` reads past it, so a caller using that gets an
|
|
31
|
+
# untraced call rather than a broken one.
|
|
32
|
+
#
|
|
33
|
+
# Analytics fire once per stream however many times it is iterated, and
|
|
34
|
+
# only after the read ends — never per event.
|
|
35
|
+
# `closer` is the caller's object (Anthropic's `MessageStream`, not `@raw_stream`); its `close` reports too,
|
|
36
|
+
# since the call is billed before any read.
|
|
37
|
+
def crystil_record_event_stream(stream, method:, kwargs:, start_time:, title: nil, version: nil, closer: stream)
|
|
38
|
+
return stream unless stream.respond_to?(:each)
|
|
39
|
+
|
|
40
|
+
original_each = stream.method(:each)
|
|
41
|
+
# Per stream, not per read, so a `close` after a partial read that never
|
|
42
|
+
# reached its `ensure` (external iteration via `next`) still has them.
|
|
43
|
+
events = []
|
|
44
|
+
reading_fibers = []
|
|
45
|
+
reported = false
|
|
46
|
+
claim_report = -> { reported ? false : (reported = true) }
|
|
47
|
+
recorder = self
|
|
48
|
+
report = lambda do |error|
|
|
49
|
+
recorder.crystil_report_event_stream(
|
|
50
|
+
events: events.dup, error: error, method: method, kwargs: kwargs,
|
|
51
|
+
start_time: start_time, title: title, version: version
|
|
52
|
+
)
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
stream.define_singleton_method(:each) do |*args, &block|
|
|
56
|
+
next original_each.call(*args, &block) if block.nil?
|
|
57
|
+
|
|
58
|
+
failure = nil
|
|
59
|
+
finished = false
|
|
60
|
+
in_caller_block = false
|
|
61
|
+
reading_fibers << Fiber.current
|
|
62
|
+
|
|
63
|
+
begin
|
|
64
|
+
result = original_each.call(*args) do |event|
|
|
65
|
+
events << Crystil::Wrappers::EventStream.crystil_event_to_data(event)
|
|
66
|
+
in_caller_block = true
|
|
67
|
+
block.call(event)
|
|
68
|
+
in_caller_block = false
|
|
69
|
+
end
|
|
70
|
+
finished = true
|
|
71
|
+
result
|
|
72
|
+
rescue Exception => e # rubocop:disable Lint/RescueException -- re-raised; Interrupt must count as failed
|
|
73
|
+
# An error from the caller's own code is theirs, not the provider's, and its message may hold their data.
|
|
74
|
+
failure = e unless in_caller_block
|
|
75
|
+
raise e
|
|
76
|
+
ensure
|
|
77
|
+
reading_fibers.delete_at(reading_fibers.rindex(Fiber.current) || reading_fibers.size)
|
|
78
|
+
# `ensure`, not the end of the `begin` body: `break` and every
|
|
79
|
+
# Enumerable method built on it (`first`, `take`, `find`, `any?`)
|
|
80
|
+
# unwind past both the success path and the `rescue`. Reporting
|
|
81
|
+
# there would leave a caller who stops early — a cancel button, a
|
|
82
|
+
# timeout guard — with a billed call and no record of it, which is
|
|
83
|
+
# the exact failure this recorder exists to prevent.
|
|
84
|
+
# A `break`/`throw` from the caller's block is a deliberate stop; the same exit while waiting
|
|
85
|
+
# on the provider is a cut-off, e.g. `Timeout.timeout` (throws on Ruby 3.2; raises on 3.3+).
|
|
86
|
+
failure ||= Crystil::StreamError.new(STREAM_CUT_SHORT_MESSAGE) unless finished || in_caller_block
|
|
87
|
+
|
|
88
|
+
report.call(failure) if claim_report.call
|
|
89
|
+
end
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
crystil_route_enumerators(stream)
|
|
93
|
+
crystil_report_on_close(closer, reading_fibers, claim_report) do
|
|
94
|
+
# Closed with nothing read is a call we saw start but not finish; after a partial read it is a deliberate
|
|
95
|
+
# stop, reported like a `break`.
|
|
96
|
+
report.call(events.empty? ? Crystil::StreamError.new(STREAM_CLOSED_UNREAD_MESSAGE) : nil)
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
stream
|
|
100
|
+
rescue StandardError
|
|
101
|
+
# Instrumentation must never be the reason a call fails. A stream that
|
|
102
|
+
# cannot take a singleton method (frozen, or with `each` defined
|
|
103
|
+
# somewhere we cannot reach) simply goes unrecorded, the same fail-open
|
|
104
|
+
# stance sentinel and `extract_response` take.
|
|
105
|
+
stream
|
|
106
|
+
end
|
|
107
|
+
|
|
108
|
+
# Anthropic's `Internal::Stream#to_enum` hands out its inner iterator, skipping the patched `each`.
|
|
109
|
+
# Route it back through so `next`-driven reads are recorded too.
|
|
110
|
+
def crystil_route_enumerators(stream)
|
|
111
|
+
%i[to_enum enum_for].each do |name|
|
|
112
|
+
stream.define_singleton_method(name) do |meth = :each, *args, &blk|
|
|
113
|
+
::Kernel.instance_method(:to_enum).bind(self).call(meth, *args, &blk)
|
|
114
|
+
end
|
|
115
|
+
end
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
# Wrap `close` so it reports when no read has. A `close` from inside the
|
|
119
|
+
# caller's own read loop is left to that read's `ensure`, which sees the whole read.
|
|
120
|
+
def crystil_report_on_close(closer, reading_fibers, claim_report, &report)
|
|
121
|
+
return unless closer.respond_to?(:close)
|
|
122
|
+
|
|
123
|
+
original_close = closer.method(:close)
|
|
124
|
+
closer.define_singleton_method(:close) do |*args, &block|
|
|
125
|
+
original_close.call(*args, &block)
|
|
126
|
+
ensure
|
|
127
|
+
report.call if !reading_fibers.include?(Fiber.current) && claim_report.call
|
|
128
|
+
end
|
|
129
|
+
end
|
|
130
|
+
|
|
131
|
+
# Submit the one payload for a finished read.
|
|
132
|
+
#
|
|
133
|
+
# The events captured are what the provider actually sent, and what it
|
|
134
|
+
# billed for, so they are kept even when the read ended early or badly —
|
|
135
|
+
# a stream cut off by a network drop still tells us the input tokens. An
|
|
136
|
+
# Anthropic event list always opens with `message_start`, so any non-empty
|
|
137
|
+
# capture is one the backend extractor can read; an empty one is not, and
|
|
138
|
+
# falls back to the plain error shape, or to sending nothing at all.
|
|
139
|
+
def crystil_report_event_stream(events:, error:, method:, kwargs:, start_time:, title: nil, version: nil)
|
|
140
|
+
response = events.empty? ? nil : events
|
|
141
|
+
|
|
142
|
+
if error
|
|
143
|
+
crystil_submit_error_analytics(
|
|
144
|
+
method: method, args: [], kwargs: kwargs, error: error,
|
|
145
|
+
start_time: start_time, end_time: Time.now,
|
|
146
|
+
title: title, version: version, response: response
|
|
147
|
+
)
|
|
148
|
+
elsif response
|
|
149
|
+
crystil_submit_analytics(
|
|
150
|
+
method: method, args: [], kwargs: kwargs, response: response,
|
|
151
|
+
start_time: start_time, end_time: Time.now, title: title, version: version
|
|
152
|
+
)
|
|
153
|
+
end
|
|
154
|
+
rescue StandardError => e
|
|
155
|
+
# This runs from the recorder's `ensure`, so anything raised here would
|
|
156
|
+
# escape the caller's `each` — failing a call that succeeded, or
|
|
157
|
+
# replacing the provider's own error with ours. Reporting is best-effort.
|
|
158
|
+
warn "Crystil: Failed to record stream analytics: #{e.message}"
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
# A streamed event as plain data, captured the moment it goes past.
|
|
162
|
+
#
|
|
163
|
+
# Converting here rather than at submission time is deliberate. Anthropic's
|
|
164
|
+
# `MessageStream` accumulates into the events' own nested objects as later
|
|
165
|
+
# ones arrive — `message_start`'s `message`, and each `content_block_start`'s
|
|
166
|
+
# `content_block` — so a reference held until the stream ends would report
|
|
167
|
+
# the final snapshot as though the opening events had carried it.
|
|
168
|
+
# `extract_response` also deep-copies an Array as-is, so provider objects
|
|
169
|
+
# left in the list would reach the collector unserialized.
|
|
170
|
+
#
|
|
171
|
+
# Provider event models have a shallow `to_h`; their `deep_to_h` is a real copy. Objects without
|
|
172
|
+
# one are copied through a JSON round trip, and plain Hash/Array events through Marshal.
|
|
173
|
+
def self.crystil_event_to_data(event)
|
|
174
|
+
return event if event.nil?
|
|
175
|
+
return crystil_deep_copy(event) if event.is_a?(Hash) || event.is_a?(Array)
|
|
176
|
+
return event.deep_to_h if event.respond_to?(:deep_to_h)
|
|
177
|
+
|
|
178
|
+
data = JSON.parse(JSON.generate(event))
|
|
179
|
+
|
|
180
|
+
# `JSON.generate` does not raise on an object with no JSON-aware
|
|
181
|
+
# `to_json`: `Object#to_json` emits the object's `to_s`, which parses
|
|
182
|
+
# back as a String. Anything but a Hash or Array means the round trip
|
|
183
|
+
# produced an inspect string rather than the event's data.
|
|
184
|
+
return data if data.is_a?(Hash) || data.is_a?(Array)
|
|
185
|
+
|
|
186
|
+
crystil_event_to_hash(event)
|
|
187
|
+
rescue StandardError
|
|
188
|
+
crystil_event_to_hash(event)
|
|
189
|
+
end
|
|
190
|
+
|
|
191
|
+
def self.crystil_event_to_hash(event)
|
|
192
|
+
event.respond_to?(:to_h) ? event.to_h : event
|
|
193
|
+
rescue StandardError
|
|
194
|
+
event
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
# Plain data that cannot be marshaled (a Proc inside, say) is kept as-is rather than dropped.
|
|
198
|
+
def self.crystil_deep_copy(data)
|
|
199
|
+
Marshal.load(Marshal.dump(data))
|
|
200
|
+
rescue StandardError
|
|
201
|
+
data
|
|
202
|
+
end
|
|
203
|
+
end
|
|
204
|
+
end
|
|
205
|
+
end
|
|
@@ -4,8 +4,14 @@ require_relative "constants"
|
|
|
4
4
|
|
|
5
5
|
module Crystil
|
|
6
6
|
module Wrappers
|
|
7
|
-
# Wrapper for Google GenerativeAI Ruby client (gemini-ai gem
|
|
7
|
+
# Wrapper for the Google GenerativeAI Ruby client (gemini-ai gem v4.3.0).
|
|
8
|
+
# Alone among the wrappers, the recorded query carries a field the caller
|
|
9
|
+
# never sent — see `extract_model`.
|
|
8
10
|
class GeminiAI
|
|
11
|
+
# Shape of the gem's `@model_address`. Anchored at the end so a path a
|
|
12
|
+
# future version builds differently reads as unknown, not as a wrong model.
|
|
13
|
+
MODEL_ADDRESS_PATTERN = %r{(?:\A|/)models/(?<model>[^/]+)\z}.freeze
|
|
14
|
+
|
|
9
15
|
def initialize(config, collector, sentinel = nil)
|
|
10
16
|
@config = config
|
|
11
17
|
@collector = collector
|
|
@@ -31,6 +37,24 @@ module Crystil
|
|
|
31
37
|
client
|
|
32
38
|
end
|
|
33
39
|
|
|
40
|
+
# The model this client is pinned to, or nil if unreadable. The gem keeps
|
|
41
|
+
# it on the client, so a failed call names the model nowhere else.
|
|
42
|
+
def self.extract_model(client)
|
|
43
|
+
address = client.instance_variable_get(:@model_address)
|
|
44
|
+
return nil unless address.is_a?(String)
|
|
45
|
+
|
|
46
|
+
address[MODEL_ADDRESS_PATTERN, :model]
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# A copy of the caller's parameters with the client's model filled in, for
|
|
50
|
+
# recording only. Their own model wins; their hash is never mutated.
|
|
51
|
+
def self.parameters_with_model(parameters, model)
|
|
52
|
+
return parameters unless parameters.is_a?(Hash) && model
|
|
53
|
+
return parameters if parameters.key?(:model) || parameters.key?("model")
|
|
54
|
+
|
|
55
|
+
parameters.merge(model: model)
|
|
56
|
+
end
|
|
57
|
+
|
|
34
58
|
private
|
|
35
59
|
|
|
36
60
|
def validate_client!(client)
|
|
@@ -60,10 +84,16 @@ module Crystil
|
|
|
60
84
|
start_time = Time.now
|
|
61
85
|
version = defined?(::Gemini::GEM) && ::Gemini::GEM.is_a?(Hash) ? ::Gemini::GEM[:version] : nil
|
|
62
86
|
|
|
87
|
+
# The gem pins the model to the client rather than the request, so
|
|
88
|
+
# record it alongside the caller's parameters — a failed call has no
|
|
89
|
+
# response to read it from. See GeminiAI.extract_model.
|
|
90
|
+
model = Crystil::Wrappers::GeminiAI.extract_model(self)
|
|
91
|
+
recorded_query = Crystil::Wrappers::GeminiAI.parameters_with_model(parameters, model)
|
|
92
|
+
|
|
63
93
|
sentinel = instance_variable_get(:@crystil_sentinel)
|
|
64
94
|
sentinel&.raise_if_irrelevant!(
|
|
65
95
|
title: GOOGLE_CLIENT_TITLE,
|
|
66
|
-
request:
|
|
96
|
+
request: recorded_query,
|
|
67
97
|
version: version
|
|
68
98
|
)
|
|
69
99
|
|
|
@@ -74,7 +104,7 @@ module Crystil
|
|
|
74
104
|
crystil_submit_analytics(
|
|
75
105
|
method: :generate_content,
|
|
76
106
|
args: [],
|
|
77
|
-
kwargs:
|
|
107
|
+
kwargs: recorded_query,
|
|
78
108
|
response: response,
|
|
79
109
|
start_time: start_time,
|
|
80
110
|
end_time: Time.now,
|
|
@@ -90,7 +120,7 @@ module Crystil
|
|
|
90
120
|
crystil_submit_error_analytics(
|
|
91
121
|
method: :generate_content,
|
|
92
122
|
args: [],
|
|
93
|
-
kwargs:
|
|
123
|
+
kwargs: recorded_query,
|
|
94
124
|
error: e,
|
|
95
125
|
start_time: start_time,
|
|
96
126
|
end_time: Time.now,
|
|
@@ -126,10 +156,16 @@ module Crystil
|
|
|
126
156
|
start_time = Time.now
|
|
127
157
|
version = defined?(::Gemini::GEM) && ::Gemini::GEM.is_a?(Hash) ? ::Gemini::GEM[:version] : nil
|
|
128
158
|
|
|
159
|
+
# The gem pins the model to the client rather than the request, so
|
|
160
|
+
# record it alongside the caller's parameters — a failed call has no
|
|
161
|
+
# response to read it from. See GeminiAI.extract_model.
|
|
162
|
+
model = Crystil::Wrappers::GeminiAI.extract_model(self)
|
|
163
|
+
recorded_query = Crystil::Wrappers::GeminiAI.parameters_with_model(parameters, model)
|
|
164
|
+
|
|
129
165
|
sentinel = instance_variable_get(:@crystil_sentinel)
|
|
130
166
|
sentinel&.raise_if_irrelevant!(
|
|
131
167
|
title: GOOGLE_CLIENT_TITLE,
|
|
132
|
-
request:
|
|
168
|
+
request: recorded_query,
|
|
133
169
|
version: version
|
|
134
170
|
)
|
|
135
171
|
|
|
@@ -140,7 +176,7 @@ module Crystil
|
|
|
140
176
|
crystil_submit_analytics(
|
|
141
177
|
method: :stream_generate_content,
|
|
142
178
|
args: [],
|
|
143
|
-
kwargs:
|
|
179
|
+
kwargs: recorded_query,
|
|
144
180
|
response: response,
|
|
145
181
|
start_time: start_time,
|
|
146
182
|
end_time: Time.now,
|
|
@@ -156,7 +192,7 @@ module Crystil
|
|
|
156
192
|
crystil_submit_error_analytics(
|
|
157
193
|
method: :stream_generate_content,
|
|
158
194
|
args: [],
|
|
159
|
-
kwargs:
|
|
195
|
+
kwargs: recorded_query,
|
|
160
196
|
error: e,
|
|
161
197
|
start_time: start_time,
|
|
162
198
|
end_time: Time.now,
|
|
@@ -10,10 +10,10 @@ module Crystil
|
|
|
10
10
|
# OpenAI gpt-oss, Qwen, compound systems), so it sits in
|
|
11
11
|
# `conversation.client.provider = "groq"` rather than `title`. `title` is
|
|
12
12
|
# derived per-call from the model-ID prefix — e.g.
|
|
13
|
-
# `meta-llama/llama-
|
|
14
|
-
# `openai/gpt-oss-20b` → `"openai"`.
|
|
15
|
-
# (`
|
|
16
|
-
# alphanumeric run (`"
|
|
13
|
+
# `meta-llama/llama-prompt-guard-2-86m` → `"meta-llama"`,
|
|
14
|
+
# `openai/gpt-oss-20b` → `"openai"`. Un-prefixed IDs
|
|
15
|
+
# (`allam-2-7b`, `whisper-large-v3`) fall back to the first
|
|
16
|
+
# alphanumeric run (`"allam"`, `"whisper"`); anything unparseable falls
|
|
17
17
|
# back to `GROQ_PROVIDER` (`"groq"`). `title` is never nil.
|
|
18
18
|
#
|
|
19
19
|
# Unlike the JS / Python groq-sdk, the Ruby `groq` gem's `Client#chat`
|
|
@@ -65,10 +65,10 @@ module Crystil
|
|
|
65
65
|
#
|
|
66
66
|
# Rules:
|
|
67
67
|
# - String with `/`: return everything before the first `/`
|
|
68
|
-
# (`meta-llama/llama-
|
|
69
|
-
# → `"openai"`).
|
|
68
|
+
# (`meta-llama/llama-prompt-guard-2-86m` → `"meta-llama"`,
|
|
69
|
+
# `openai/gpt-oss-20b` → `"openai"`).
|
|
70
70
|
# - String without `/`: return the first run of alphanumeric characters
|
|
71
|
-
# (`
|
|
71
|
+
# (`allam-2-7b` → `"allam"`, `whisper-large-v3` → `"whisper"`,
|
|
72
72
|
# `gpt-4o` → `"gpt"`).
|
|
73
73
|
# - Anything else (non-string, empty, unrecognized shape): return
|
|
74
74
|
# `fallback`.
|
|
@@ -7,12 +7,24 @@ module Crystil
|
|
|
7
7
|
@collector: Crystil::Collector
|
|
8
8
|
@sentinel: Crystil::Sentinel?
|
|
9
9
|
|
|
10
|
+
REQUEST_OPTIONS_KEYS: Array[Symbol | String]
|
|
11
|
+
|
|
10
12
|
def initialize: (Crystil::Config config, Crystil::Collector collector, ?Crystil::Sentinel? sentinel) -> void
|
|
11
13
|
def register: (untyped client) -> untyped
|
|
12
14
|
|
|
15
|
+
# A copy without `request_options`; a non-Hash is handed back unchanged.
|
|
16
|
+
def self.recorded_params: (untyped params) -> untyped
|
|
17
|
+
# `untyped` in, `untyped` out: the caller's params are whatever the gem
|
|
18
|
+
# accepts, and a non-Hash is handed back unchanged.
|
|
19
|
+
def self.streamed_query: (untyped params) -> untyped
|
|
20
|
+
# nil when there is no readable event stream, so nothing is recorded.
|
|
21
|
+
def self.raw_event_stream: (untyped stream, Symbol method_name) -> untyped?
|
|
22
|
+
|
|
13
23
|
private
|
|
14
24
|
def validate_client!: (untyped client) -> void
|
|
15
25
|
def wrap_messages_method: (untyped client) -> void
|
|
26
|
+
def wrap_create_method: (untyped messages_resource) -> void
|
|
27
|
+
def wrap_streaming_method: (untyped messages_resource, Symbol method_name) -> void
|
|
16
28
|
end
|
|
17
29
|
end
|
|
18
30
|
end
|
|
@@ -5,9 +5,12 @@ module Crystil
|
|
|
5
5
|
module Base
|
|
6
6
|
def crystil_wrap_method: (Symbol method_name, String _provider_name) -> void
|
|
7
7
|
def crystil_submit_analytics: (method: Symbol, args: Array[untyped], kwargs: Hash[Symbol, untyped], response: untyped, start_time: Time, end_time: Time, ?provider: String?, ?title: String?, ?version: String?, ?status: String, ?exception: String?) -> void
|
|
8
|
-
def crystil_submit_error_analytics: (method: Symbol, args: Array[untyped], kwargs: Hash[Symbol, untyped], error:
|
|
8
|
+
def crystil_submit_error_analytics: (method: Symbol, args: Array[untyped], kwargs: Hash[Symbol, untyped], error: Exception, start_time: Time, end_time: Time, ?provider: String?, ?title: String?, ?version: String?, ?response: untyped) -> void
|
|
9
9
|
def crystil_merge_streaming_chunk: (Hash[String | Symbol, untyped] accumulated, Hash[String | Symbol, untyped] chunk) -> Hash[String | Symbol, untyped]
|
|
10
10
|
def crystil_normalize_openai_chunk: (untyped chunk) -> untyped
|
|
11
|
+
# `crystil_record_event_stream` / `crystil_report_event_stream` arrive via
|
|
12
|
+
# `include EventStream`; they are declared in event_stream.rbs, not here.
|
|
13
|
+
include EventStream
|
|
11
14
|
|
|
12
15
|
private
|
|
13
16
|
def extract_query: (Symbol _method, Array[untyped] _args, Hash[Symbol, untyped] kwargs) -> Hash[Symbol, untyped]
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# Signatures for the list-shaped streaming recorder. `untyped` throughout on
|
|
2
|
+
# the stream and event surface: the objects come from whichever provider gem
|
|
3
|
+
# the caller registered, and the module deliberately knows nothing about them
|
|
4
|
+
# beyond `each`.
|
|
5
|
+
module Crystil
|
|
6
|
+
module Wrappers
|
|
7
|
+
module EventStream
|
|
8
|
+
STREAM_CUT_SHORT_MESSAGE: String
|
|
9
|
+
STREAM_CLOSED_UNREAD_MESSAGE: String
|
|
10
|
+
|
|
11
|
+
# Returns the stream it was given, with only `each` replaced — or
|
|
12
|
+
# untouched when it cannot be patched.
|
|
13
|
+
def crystil_record_event_stream: (untyped stream, method: Symbol, kwargs: untyped, start_time: Time, ?title: String?, ?version: String?, ?closer: untyped) -> untyped
|
|
14
|
+
# Overrides `to_enum` / `enum_for` so external iteration goes through the recorded `each`.
|
|
15
|
+
def crystil_route_enumerators: (untyped stream) -> void
|
|
16
|
+
# Leaves `close` alone when the object has none.
|
|
17
|
+
def crystil_report_on_close: (untyped closer, Array[Fiber] reading_fibers, ^() -> bool claim_report) { () -> void } -> void
|
|
18
|
+
def crystil_report_event_stream: (events: Array[untyped], error: Exception?, method: Symbol, kwargs: untyped, start_time: Time, ?title: String?, ?version: String?) -> void
|
|
19
|
+
def self.crystil_event_to_data: (untyped event) -> untyped
|
|
20
|
+
def self.crystil_event_to_hash: (untyped event) -> untyped
|
|
21
|
+
def self.crystil_deep_copy: (untyped data) -> untyped
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
@@ -3,6 +3,8 @@
|
|
|
3
3
|
module Crystil
|
|
4
4
|
module Wrappers
|
|
5
5
|
class GeminiAI
|
|
6
|
+
MODEL_ADDRESS_PATTERN: Regexp
|
|
7
|
+
|
|
6
8
|
@config: Crystil::Config
|
|
7
9
|
@collector: Crystil::Collector
|
|
8
10
|
@sentinel: Crystil::Sentinel?
|
|
@@ -10,6 +12,13 @@ module Crystil
|
|
|
10
12
|
def initialize: (Crystil::Config config, Crystil::Collector collector, ?Crystil::Sentinel? sentinel) -> void
|
|
11
13
|
def register: (untyped client) -> untyped
|
|
12
14
|
|
|
15
|
+
# Returns nil when the client's @model_address is absent or unreadable.
|
|
16
|
+
def self.extract_model: (untyped client) -> String?
|
|
17
|
+
|
|
18
|
+
# Returns `parameters` unchanged when the model is unknown or already
|
|
19
|
+
# present, so the return type follows the input rather than being a Hash.
|
|
20
|
+
def self.parameters_with_model: (untyped parameters, String? model) -> untyped
|
|
21
|
+
|
|
13
22
|
private
|
|
14
23
|
def validate_client!: (untyped client) -> void
|
|
15
24
|
def wrap_generate_content_method: (untyped client) -> void
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: crystil
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.6.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Crystil
|
|
@@ -48,6 +48,7 @@ files:
|
|
|
48
48
|
- lib/crystil/wrappers/anthropic.rb
|
|
49
49
|
- lib/crystil/wrappers/base.rb
|
|
50
50
|
- lib/crystil/wrappers/constants.rb
|
|
51
|
+
- lib/crystil/wrappers/event_stream.rb
|
|
51
52
|
- lib/crystil/wrappers/geminiai.rb
|
|
52
53
|
- lib/crystil/wrappers/google.rb
|
|
53
54
|
- lib/crystil/wrappers/groq.rb
|
|
@@ -68,6 +69,7 @@ files:
|
|
|
68
69
|
- sig/crystil/wrappers/anthropic.rbs
|
|
69
70
|
- sig/crystil/wrappers/base.rbs
|
|
70
71
|
- sig/crystil/wrappers/constants.rbs
|
|
72
|
+
- sig/crystil/wrappers/event_stream.rbs
|
|
71
73
|
- sig/crystil/wrappers/geminiai.rbs
|
|
72
74
|
- sig/crystil/wrappers/google.rbs
|
|
73
75
|
- sig/crystil/wrappers/groq.rbs
|
|
@@ -94,7 +96,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
94
96
|
- !ruby/object:Gem::Version
|
|
95
97
|
version: '0'
|
|
96
98
|
requirements: []
|
|
97
|
-
rubygems_version: 4.0.
|
|
99
|
+
rubygems_version: 4.0.21
|
|
98
100
|
specification_version: 4
|
|
99
101
|
summary: Cost visibility for AI agents
|
|
100
102
|
test_files: []
|