coolhand 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,218 @@
1
+ # Reading Templates (Search + Get)
2
+
3
+ `Coolhand::TemplateService` reads back the LLM request templates your logs are matched against.
4
+ It wraps two read-only endpoints:
5
+
6
+ | method | endpoint |
7
+ |---|---|
8
+ | `search_templates` | `GET /api/v2/llm_request_templates` |
9
+ | `get_template` | `GET /api/v2/llm_request_templates/{id}` |
10
+
11
+ Both require your **private** API key. The public key is write-only on this API and is rejected
12
+ exactly like an invalid one.
13
+
14
+ ```ruby
15
+ require "coolhand"
16
+
17
+ Coolhand.configure do |config|
18
+ config.api_key = ENV.fetch("COOLHAND_PRIVATE_API_KEY")
19
+ end
20
+
21
+ templates = Coolhand.template_service
22
+
23
+ result = templates.search_templates(search: "summar", status: "published")
24
+ result.templates.each { |template| puts "#{template[:name]} (#{template[:log_count]} logs)" }
25
+
26
+ detail = templates.get_template(result.templates.first[:id])
27
+ puts detail[:user_prompt_pattern]
28
+ ```
29
+
30
+ ## What this is not
31
+
32
+ This is **not** a port of the `search_templates` MCP tool, and the two do not agree:
33
+
34
+ - `log_count` here counts only directly-collected client logs — the same records
35
+ `GET /api/v2/llm_request_logs?template_id=…` returns. Evals, bakeoff comparisons and synthetic
36
+ logs are excluded, so this number is often lower than the MCP tool's.
37
+ - Templates whose workload has been archived are **returned** here, not hidden, so the list agrees
38
+ with `get_template`, which can always fetch such a template by id. Narrow with `workload_id`
39
+ instead.
40
+
41
+ Template *creation, update and deprecation* stay on the MCP surface. This REST surface is
42
+ read-only, and there is no version-history sub-resource.
43
+
44
+ ## `search_templates(search: nil, workload_id: nil, status: nil, include_deprecated: nil, include_system: nil, page: nil, per: nil)`
45
+
46
+ Search is a *parameter* on the list endpoint rather than a route of its own, so this is one method
47
+ rather than a list/search pair.
48
+
49
+ All filters are optional keyword arguments, and their names are the wire names — Ruby's convention
50
+ and this API's already agree, so there is no second spelling to translate through.
51
+
52
+ | keyword | type | notes |
53
+ |---|---|---|
54
+ | `search` | String | Case-insensitive **literal** substring match on the template name. `%` and `_` are escaped server-side and match themselves — do not escape them again. |
55
+ | `workload_id` | String | Workload hashid. One that does not decode, or that belongs to another client, is a `422` rather than an empty list. |
56
+ | `status` | String | `"draft"`, `"published"` or `"failure"`. Any other non-empty value is a `422` from the server; empty is treated as no filter. |
57
+ | `include_deprecated` | Boolean | Include templates with a non-null `deprecated_at`. Defaults to `false` server-side. |
58
+ | `include_system` | Boolean | Include the `Unmatched` / `Ignored API Calls` buckets. Defaults to `false` server-side. |
59
+ | `page` | Integer | 1-based. |
60
+ | `per` | Integer | Default 25, max 100, both enforced server-side. |
61
+
62
+ Two things it deliberately does **not** do:
63
+
64
+ - **There is no `client_id` keyword.** The client is always derived from the authenticating API key
65
+ and cannot be supplied, so passing one raises `ArgumentError` rather than reaching the wire.
66
+ - **`status` is not checked against a client-side allowlist.** The API definition enumerates the
67
+ values on the *query parameter*, but leaves the `status` field on the *response* an unconstrained
68
+ string. Sending an unrecognised value gets you the server's `422`; a status the server gains
69
+ later works without a gem release.
70
+
71
+ `per_page` is accepted on the wire as an alias for `per`. This gem only ever sends `per` — one knob
72
+ is enough.
73
+
74
+ ### Return value
75
+
76
+ A `Coolhand::TemplateSearchResult` — a `Struct`, so `#templates`, `#pagination`, `#to_h` and
77
+ pattern matching all work:
78
+
79
+ ```ruby
80
+ result = Coolhand.template_service.search_templates(include_system: true)
81
+
82
+ result.templates # => [{ id: "kp9npvc8qq2q", name: "Unmatched", ... }, ...]
83
+ result.pagination # => #<struct Coolhand::Pagination current_page=1, ...>
84
+ ```
85
+
86
+ **Rows are plain Hashes with Symbol keys**, the same shape `create_feedback` and `create_log`
87
+ already return. They are handed back exactly as the API rendered them rather than being copied into
88
+ a value object, so a field the server adds later reaches you instead of being silently dropped on
89
+ the way through. That is also why there is no `system_template?` predicate: read the wire field,
90
+ `template[:system_template]`.
91
+
92
+ Rows are ordered newest first (`created_at DESC`, with a primary-key tiebreaker so paging is stable
93
+ across requests). Each carries:
94
+
95
+ | field | type | notes |
96
+ |---|---|---|
97
+ | `:id` | String | Hashid, never the integer primary key. |
98
+ | `:name` | String | Never null, but may be blank on a draft. |
99
+ | `:status` | String or nil | `draft` / `published` / `failure`. |
100
+ | `:version` | String or nil | |
101
+ | `:group` | String or nil | `chat`, `user_prompt`, `user_prompt_with_system_prompt`, `embedding`, `other`. |
102
+ | `:workload_id` | String | Workload hashid; never null. |
103
+ | `:workload_name` | String | Never null. |
104
+ | `:system_template` | Boolean | True for the `Unmatched` / `Ignored API Calls` buckets. |
105
+ | `:deprecated_at` | String or nil | ISO-8601 UTC. Non-null means the template has been superseded. |
106
+ | `:log_count` | Integer | Directly-collected client logs only — see [What this is not](#what-this-is-not). |
107
+ | `:created_at` | String | ISO-8601 UTC. |
108
+ | `:updated_at` | String | ISO-8601 UTC. |
109
+
110
+ **Prompt patterns are not in the list.** They come from `get_template` only.
111
+
112
+ `pagination` is a `Coolhand::Pagination` built from the endpoint's `X-Page`, `X-Per-Page`,
113
+ `X-Total-Count` and `X-Total-Pages` response headers — never from the size of the array, which only
114
+ ever describes the page in hand. Unlike `/llm_request_logs` there is no `include_total` opt-out
115
+ here; the headers are always sent.
116
+
117
+ | field | type |
118
+ |---|---|
119
+ | `current_page` | Integer |
120
+ | `per_page` | Integer |
121
+ | `total_count` | Integer |
122
+ | `total_pages` | Integer |
123
+ | `has_next_page` | Boolean |
124
+ | `has_prev_page` | Boolean |
125
+
126
+ ### System templates and the empty default list
127
+
128
+ Every Coolhand client is created with two system buckets, `Unmatched` and `Ignored API Calls`. They
129
+ are hidden unless you pass `include_system: true`, so a client with no templates of its own gets an
130
+ empty array rather than those two rows. `Unmatched` is what you inspect when logs are misrouting:
131
+
132
+ ```ruby
133
+ unmatched = Coolhand.template_service
134
+ .search_templates(include_system: true, search: "unmatched")
135
+ .templates
136
+ .first
137
+
138
+ puts unmatched[:log_count]
139
+ ```
140
+
141
+ ## `get_template(id)`
142
+
143
+ `id` is the template hashid — the `:id` field from `search_templates`.
144
+
145
+ Returns a Hash with every list field above, **plus** the full untruncated regexes the list omits:
146
+
147
+ | field | type |
148
+ |---|---|
149
+ | `:user_prompt_pattern` | String or nil |
150
+ | `:system_prompt_pattern` | String or nil |
151
+
152
+ Unlike the list, this filters on nothing but client ownership: a deprecated or system template is
153
+ reachable by id with **no opt-in flag**, because inspecting one of those is the usual reason to
154
+ fetch a template directly.
155
+
156
+ ## Errors
157
+
158
+ The write methods in this gem (`create_feedback`, `create_log`, `send_llm_request_log`) log a
159
+ failure and return `nil` — instrumentation must never be the reason a host app falls over. **The
160
+ read methods are the opposite and raise**, because a caller that asked for data has to be able to
161
+ tell a `404` from a timeout from a genuinely empty result.
162
+
163
+ | raised | when |
164
+ |---|---|
165
+ | `Coolhand::HttpError` | The server answered with a non-2xx status. `#status` is the HTTP status code and `#body` the response body. |
166
+ | `Coolhand::Error` | No API key configured, a transport failure, a body that is not valid JSON, or a blank `id` passed to `get_template`. |
167
+
168
+ `Coolhand::HttpError` is a `Coolhand::Error`, so `rescue Coolhand::Error` catches both.
169
+
170
+ **Branch on `#status`, never on the message:**
171
+
172
+ ```ruby
173
+ begin
174
+ result = Coolhand.template_service.search_templates
175
+ rescue Coolhand::HttpError => e
176
+ case e.status
177
+ when 401 then warn "Private API key required — the public key cannot read"
178
+ when 422 then warn "Bad filter: #{e.body}"
179
+ when 504 then warn "Timed out aggregating log_count — narrow the query and retry"
180
+ else raise
181
+ end
182
+ end
183
+ ```
184
+
185
+ | status | meaning |
186
+ |---|---|
187
+ | `401` | Missing, invalid, or public API key. |
188
+ | `404` | `get_template` only. Unknown id, **or** one belonging to another client — existence is not disclosed, so this is never a `403`. |
189
+ | `422` | Unrecognised `status`, or a `workload_id` that does not decode or belongs to another client. |
190
+ | `504` | See below. |
191
+
192
+ ### `504` is expected, and retryable
193
+
194
+ `log_count` aggregates over `llm_request_logs`, so its cost scales with how many logs the matched
195
+ templates hold — the `Unmatched` bucket can hold every log that never matched a template. Every
196
+ query behind these responses is bounded by a 10-second statement timeout, and exceeding it returns
197
+ `504` rather than hanging. It reaches you as an `HttpError` with `status == 504`, not folded into a
198
+ generic server error, precisely so you can narrow and retry:
199
+
200
+ ```ruby
201
+ templates = Coolhand.template_service
202
+
203
+ begin
204
+ templates.search_templates(include_system: true)
205
+ rescue Coolhand::HttpError => e
206
+ raise unless e.status == 504
207
+
208
+ # Narrow the aggregate: one workload at a time, smaller pages.
209
+ templates.search_templates(include_system: true, workload_id: workload_id, per: 10)
210
+ end
211
+ ```
212
+
213
+ This is also why the read path waits far longer than the write path before giving up. A write
214
+ times out after 5 seconds because it runs inline in your app's own request. A read allows 60: the
215
+ server bounds each *statement* behind a response at 10 seconds, but one response runs several, so
216
+ a slow-but-working `include_system=true` call measured 7-15 seconds against a development database
217
+ while returning `200` every time. Giving up sooner would report a working endpoint as a transport
218
+ failure, and would pre-empt the `504` you are meant to see and retry.
data/docs/vertex.md ADDED
@@ -0,0 +1,54 @@
1
+ # Google Vertex AI Batch Result Logging
2
+
3
+ Log completed Google Vertex AI batch prediction job results the same way regular synchronous calls are logged. Unlike the [OpenAI batch handler](openai.md), this is not webhook-driven — you call `Coolhand::Vertex::BatchResultProcessor` directly from your own batch-job callback/polling code.
4
+
5
+ For monitoring regular (non-batch) Vertex AI calls, no extra setup is required beyond `Coolhand.configure` — `aiplatform.googleapis.com` is intercepted by default.
6
+
7
+ Requires Rails — `Coolhand::Vertex::BatchResultProcessor` logs via `Rails.logger` internally. `config.capture = false` and `Coolhand.without_capture` do not suppress these logs: unlike the passive Net::HTTP interceptor, calling this processor is an explicit, deliberate act, so it always sends.
8
+
9
+ ## Usage
10
+
11
+ ```ruby
12
+ require "coolhand/vertex/batch_result_processor"
13
+ ```
14
+
15
+ - Call the `Coolhand::Vertex::BatchResultProcessor` service with `batch_info` and the downloaded batch results.
16
+ - Optionally pass `model:` to populate the logged `model` field. It falls back to a `"model"` key in `batch_info` if present, and is omitted from the payload entirely when neither is available. If the value (from either source) looks like a Vertex model resource path (`publishers/<publisher>/models/<id>` or `projects/<project>/locations/<location>/models/<id>`), it's normalized down to just the trailing id/version before being logged; any other string is sent as-is.
17
+ - `batch_info["name"]` must be the exact Vertex job resource name (`projects/<project>/locations/<location>/batchPredictionJobs/<job-id>`), and `batch_info["startTime"]`/`["endTime"]` must be valid ISO 8601 timestamps — a batch with a missing or malformed resource name or timestamps is skipped entirely (logged as an error) rather than sent with a broken URL.
18
+ - If `endTime` precedes `startTime` (e.g. clock skew), the batch is still logged — `duration_ms` is clamped to `0` and a warning is logged — rather than dropping the results.
19
+ - Each element of the `batch_results` array passed to `.call` must be a `Hash` with a `"request"` and/or `"response"` key (the request/response body to log for that item — either key alone is accepted). Any other shape is treated as malformed and skipped, with the skip counted in a `Rails.logger.warn` summary rather than raised.
20
+
21
+ ## Minimal example
22
+
23
+ Only the key lines are shown — wire this into your own batch callback service.
24
+
25
+ ```ruby
26
+ class Vertex::BatchCallbackProcessor < BaseService
27
+ option :batch_request, model: BatchApiRequest
28
+ option :batch_info
29
+
30
+ def call
31
+ case batch_info["state"]
32
+ when "JOB_STATE_PENDING"
33
+ nil
34
+ when "JOB_STATE_RUNNING", "JOB_STATE_QUEUED"
35
+ batch_request.update!(status: "processing")
36
+
37
+ Coolhand::Vertex::BatchResultProcessor.new(batch_info:).call
38
+ when "JOB_STATE_SUCCEEDED"
39
+ output_file_id = batch_info["outputInfo"]["gcsOutputDirectory"]
40
+ results = download_batch_results(output_file_id)
41
+ results.each { |batch_item| process_batch_result(batch_item) }
42
+
43
+ batch_request.update!(status: "completed", completed_at: Time.current, output_file_id:)
44
+
45
+ Coolhand::Vertex::BatchResultProcessor.new(batch_info:, model: batch_request.llm_model).call(results)
46
+
47
+ # Clean up GCS files after successful processing
48
+ cleanup_gcs_files(output_file_id)
49
+ when "JOB_STATE_FAILED"
50
+ handle_failed_batch(batch_info["error"]["message"])
51
+ end
52
+ end
53
+ end
54
+ ```
@@ -4,9 +4,19 @@ require "net/http"
4
4
  require "uri"
5
5
  require "json"
6
6
  require_relative "collector"
7
+ require_relative "errors"
8
+ require_relative "read_requests"
7
9
 
8
10
  module Coolhand
9
11
  class ApiService
12
+ # The GET half of this class. See Coolhand::ReadRequests for why reads raise where writes
13
+ # log and return nil.
14
+ include ReadRequests
15
+
16
+ # Caps how much of a failed response body is interpolated into a log line or an exception
17
+ # message. Without it an oversized body from a proxy or gateway becomes the message.
18
+ ERROR_BODY_LIMIT = 2000
19
+
10
20
  attr_reader :api_endpoint
11
21
 
12
22
  def initialize(endpoint = "v2/llm_request_logs")
@@ -77,12 +87,7 @@ module Coolhand
77
87
  http.read_timeout = 5
78
88
 
79
89
  request = Net::HTTP::Post.new(uri.request_uri)
80
- headers = create_request_options(payload)
81
- headers.each do |key, value|
82
- # Ensure header values are UTF-8 encoded
83
- encoded_value = value.is_a?(String) ? value.dup.force_encoding("UTF-8") : value
84
- request[key] = encoded_value
85
- end
90
+ apply_headers(request, create_request_options(payload))
86
91
 
87
92
  # Clean payload and ensure UTF-8 encoding before JSON generation
88
93
  cleaned_payload = sanitize_payload_for_json(payload)
@@ -92,21 +97,20 @@ module Coolhand
92
97
  request.body = json_body.force_encoding("UTF-8")
93
98
 
94
99
  begin
95
- response = http.request(request)
100
+ # This request goes through the same patched Net::HTTP as the host
101
+ # app's real LLM calls. Without this, a base_url/intercept_addresses
102
+ # configuration that also matches Coolhand's own API host would
103
+ # re-intercept this log-shipping call, generating a second log
104
+ # request that itself gets intercepted, and so on — unbounded
105
+ # request amplification.
106
+ response = Coolhand.without_capture { http.request(request) }
96
107
 
97
108
  if response.is_a?(Net::HTTPSuccess)
98
109
  result = JSON.parse(response.body, symbolize_names: true)
99
110
  log success_message
100
111
  result
101
112
  else
102
- body = response.body.force_encoding("UTF-8") if response.body
103
- # Only show first part of HTML error pages
104
- error_msg = if body&.include?("<!DOCTYPE html>")
105
- "#{body[0..200]}... [HTML error page truncated]"
106
- else
107
- body
108
- end
109
- log "❌ Request failed: #{response.code} - #{error_msg}"
113
+ log "❌ Request failed: #{response.code} - #{format_error_body(response.body)}"
110
114
  nil
111
115
  end
112
116
  rescue StandardError => e
@@ -204,6 +208,26 @@ module Coolhand
204
208
 
205
209
  private
206
210
 
211
+ # Shared by the write path's failure log and the read path's raised message, so a large
212
+ # response body never dumps a whole document into either. An HTML error page (a proxy's 502,
213
+ # say) is cut short hard; anything else keeps enough to diagnose from and no more.
214
+ def format_error_body(body)
215
+ return nil if body.nil?
216
+
217
+ text = body.dup.force_encoding("UTF-8")
218
+ return "#{text[0..200]}... [HTML error page truncated]" if text.include?("<!DOCTYPE html>")
219
+ return text if text.length <= ERROR_BODY_LIMIT
220
+
221
+ "#{text[0, ERROR_BODY_LIMIT]}... [truncated]"
222
+ end
223
+
224
+ def apply_headers(request, headers)
225
+ headers.each do |key, value|
226
+ # Net::HTTP rejects header values that are not UTF-8.
227
+ request[key] = value.is_a?(String) ? value.dup.force_encoding("UTF-8") : value
228
+ end
229
+ end
230
+
207
231
  def missing_api_key?
208
232
  return false if Coolhand.required_field?(api_key)
209
233
 
@@ -275,8 +299,24 @@ module Coolhand
275
299
  return if silent
276
300
 
277
301
  puts "\n🎉 LOGGING OpenAI API Call #{@api_endpoint}"
278
- puts captured_data
302
+
303
+ if debug_mode?
304
+ puts captured_data
305
+ else
306
+ puts request_body_summary(captured_data)
307
+ end
308
+
279
309
  puts "📤 Sending to: #{@api_endpoint}"
280
310
  end
311
+
312
+ def request_body_summary(captured_data)
313
+ return "captured_data: (unavailable)" unless captured_data.is_a?(Hash)
314
+
315
+ id = captured_data[:id] || captured_data["id"] || "N/A"
316
+ body = captured_data[:request_body] || captured_data["request_body"]
317
+ "id: #{id}, request_body: #{body.to_json.bytesize} bytes"
318
+ rescue StandardError
319
+ "captured_data: (unavailable)"
320
+ end
281
321
  end
282
322
  end
@@ -7,10 +7,11 @@ module Coolhand
7
7
 
8
8
  # Matches any header whose *name* signals sensitive content, regardless of
9
9
  # provider — covers known keys (x-api-key, x-goog-api-key, openai-api-key),
10
- # AWS SigV4 session tokens (x-amz-security-token), and future/unknown
11
- # providers using a similarly-named header. Shared with LoggerService so
12
- # the two logging paths (interceptor + webhook forwarding) stay consistent.
13
- SENSITIVE_HEADER_PATTERN = /key|token|secret|signature|authorization/i
10
+ # AWS SigV4 session tokens (x-amz-security-token), session/CSRF cookies
11
+ # (cookie, set-cookie), and future/unknown providers using a
12
+ # similarly-named header. Shared with LoggerService so the two logging
13
+ # paths (interceptor + webhook forwarding) stay consistent.
14
+ SENSITIVE_HEADER_PATTERN = /key|token|secret|signature|authorization|cookie/i
14
15
 
15
16
  def sanitize_headers(headers)
16
17
  return {} if headers.nil?
@@ -22,7 +23,9 @@ module Coolhand
22
23
  begin
23
24
  headers.to_hash.transform_keys(&:to_s).transform_values { |v| normalize_header_value(v) }
24
25
  rescue StandardError
25
- # fall through to other enumeration strategies
26
+ # Deliberately fails closed to an empty hash rather than trying each_header/each on
27
+ # this object next — an object whose own to_hash raises is untrustworthy enough that
28
+ # guessing at another enumeration strategy isn't worth the risk of a second failure.
26
29
  nil
27
30
  end
28
31
  elsif headers.respond_to?(:each_header)
@@ -69,50 +72,67 @@ module Coolhand
69
72
  end
70
73
  end
71
74
 
75
+ # Matches query param *names* that signal sensitive content, the same way
76
+ # SENSITIVE_HEADER_PATTERN does for headers — covers exact params this
77
+ # gem already knew about (key/token/secret) as well as presigned-URL
78
+ # credential params used by AWS (X-Amz-Signature, X-Amz-Credential,
79
+ # X-Amz-Security-Token) and Google Cloud (X-Goog-Signature,
80
+ # X-Goog-Credential) storage APIs.
81
+ SENSITIVE_QUERY_PARAM_PATTERN = /key|token|secret|sig|credential|password|auth/i
82
+
72
83
  def sanitize_url(url)
73
84
  uri = URI.parse(url)
74
- return url unless uri.query
75
-
76
- sensitive = %w[key api_key apikey token access_token secret]
77
- params = URI.decode_www_form(uri.query)
78
- redacted = false
79
- params.map! do |n, v|
80
- if sensitive.include?(n.downcase)
81
- redacted = true
82
- [n, "[REDACTED]"]
83
- else
84
- [n, v]
85
- end
85
+ modified = false
86
+
87
+ if uri.userinfo
88
+ # URI userinfo syntax disallows "[" / "]", so this can't reuse the [REDACTED]
89
+ # placeholder used elsewhere.
90
+ uri.userinfo = "REDACTED"
91
+ modified = true
86
92
  end
87
93
 
88
- if redacted
89
- uri.query = URI.encode_www_form(params)
90
- uri.to_s
91
- else
92
- url
94
+ if uri.query
95
+ params = URI.decode_www_form(uri.query)
96
+ redacted_query = false
97
+ params.map! do |n, v|
98
+ if n.match?(SENSITIVE_QUERY_PARAM_PATTERN)
99
+ redacted_query = true
100
+ [n, "[REDACTED]"]
101
+ else
102
+ [n, v]
103
+ end
104
+ end
105
+ if redacted_query
106
+ uri.query = URI.encode_www_form(params)
107
+ modified = true
108
+ end
93
109
  end
110
+
111
+ modified ? uri.to_s : url
94
112
  rescue URI::InvalidURIError
95
113
  url
96
114
  end
97
115
 
98
116
  def send_complete_request_log(request_id:, method:, url:, request_headers:, request_body:, response_headers:,
99
- response_body:, status_code:, start_time:, end_time:, duration_ms:, is_streaming:)
100
- request_data = {
101
- raw_request: {
102
- id: request_id,
103
- timestamp: start_time.iso8601,
104
- method: method.to_s.downcase,
105
- url: sanitize_url(url),
106
- headers: request_headers,
107
- request_body: request_body,
108
- response_headers: response_headers,
109
- response_body: response_body,
110
- status_code: status_code,
111
- duration_ms: duration_ms,
112
- completed_at: end_time.iso8601,
113
- is_streaming: is_streaming
114
- }
117
+ response_body:, status_code:, start_time:, end_time:, duration_ms:, is_streaming:, source_api: nil, model: nil)
118
+ raw_request = {
119
+ id: request_id,
120
+ timestamp: start_time.iso8601,
121
+ method: method.to_s.downcase,
122
+ url: sanitize_url(url),
123
+ headers: sanitize_headers(request_headers),
124
+ request_body: request_body,
125
+ response_headers: sanitize_headers(response_headers),
126
+ response_body: response_body,
127
+ status_code: status_code,
128
+ duration_ms: duration_ms,
129
+ completed_at: end_time.iso8601,
130
+ is_streaming: is_streaming
115
131
  }
132
+ raw_request[:source_api] = source_api if Coolhand.required_field?(source_api)
133
+ raw_request[:model] = model if Coolhand.required_field?(model)
134
+
135
+ request_data = { raw_request: raw_request }
116
136
 
117
137
  api_service = Coolhand::ApiService.new
118
138
  api_service.send_llm_request_log(request_data)
@@ -3,22 +3,34 @@
3
3
  require "yaml"
4
4
  require "uri"
5
5
 
6
+ require_relative "open_ai/webhook_id_store"
7
+
6
8
  module Coolhand
7
9
  # Handles all configuration settings for the gem.
8
10
  class Configuration
9
- DEFAULT_EXCLUDE_API_PATTERNS = YAML.load_file(
11
+ DEFAULT_EXCLUDE_API_PATTERNS = YAML.safe_load_file(
10
12
  File.join(__dir__, "default_exclude_api_patterns.yml")
11
13
  ).freeze
12
14
 
13
- DEFAULT_INTERCEPT_ADDRESSES = YAML.load_file(
15
+ DEFAULT_INTERCEPT_ADDRESSES = YAML.safe_load_file(
14
16
  File.join(__dir__, "default_intercept_addresses.yml")
15
17
  ).freeze
16
18
 
19
+ DEFAULT_INTERCEPT_PATH_PATTERNS = YAML.safe_load_file(
20
+ File.join(__dir__, "default_intercept_path_patterns.yml")
21
+ ).freeze
22
+
17
23
  BASE_URL_ERROR_MSG = "base_url must use https:// (or http://localhost / http://127.0.0.1 for local dev)"
18
24
  LOOPBACK_HOSTS = %w[localhost 127.0.0.1 ::1].freeze
19
25
 
20
- attr_accessor :api_key, :environment, :silent, :debug_mode, :capture, :exclude_api_patterns, :enabled
21
- attr_reader :intercept_addresses, :base_url
26
+ # 1 MB generous for real chat/completion payloads, small enough to stop
27
+ # a multi-MB file upload from being logged in full when its content-type
28
+ # happens to look JSON-ish (or is unset).
29
+ DEFAULT_MAX_CAPTURED_BODY_BYTES = 1_000_000
30
+
31
+ attr_accessor :api_key, :environment, :silent, :debug_mode, :capture, :exclude_api_patterns, :enabled,
32
+ :max_captured_body_bytes, :webhook_replay_tolerance_seconds, :webhook_id_store
33
+ attr_reader :intercept_addresses, :intercept_path_patterns, :base_url
22
34
 
23
35
  def initialize
24
36
  # Set defaults
@@ -26,20 +38,41 @@ module Coolhand
26
38
  @api_key = nil
27
39
  @silent = false
28
40
  @intercept_addresses = DEFAULT_INTERCEPT_ADDRESSES.dup
41
+ @intercept_path_patterns = DEFAULT_INTERCEPT_PATH_PATTERNS.dup
29
42
  self.base_url = "https://coolhandlabs.com/api"
30
43
  @debug_mode = false
31
44
  @capture = true
32
45
  @exclude_api_patterns = DEFAULT_EXCLUDE_API_PATTERNS.dup
33
46
  @enabled = true
47
+ @max_captured_body_bytes = DEFAULT_MAX_CAPTURED_BODY_BYTES
48
+ @webhook_replay_tolerance_seconds = 300
49
+ @webhook_id_store = Coolhand::OpenAi::WebhookIdStore.new
34
50
  end
35
51
 
36
- # Custom setter that preserves defaults when nil/empty array is provided
52
+ # intercept_addresses is a required allow-list: NetHttpInterceptor#intercept? only
53
+ # captures a request whose URL matches an entry here, so an empty list would mean
54
+ # "never capture anything" (validate! deliberately rejects that). Unlike
55
+ # exclude_api_patterns, `= []` is not a supported way to disable it — use
56
+ # config.enabled = false or config.capture = false to disable capture entirely.
57
+ # nil/empty here is treated as "leave the current value alone" rather than cleared.
37
58
  def intercept_addresses=(value)
38
- return if value.nil? || (value.is_a?(Array) && value.empty?)
59
+ if value.nil? || (value.is_a?(Array) && value.empty?)
60
+ Coolhand.log "⚠️ Coolhand: intercept_addresses = #{value.inspect} is ignored " \
61
+ "(would disable capture entirely) — keeping the current value. " \
62
+ "Use config.enabled = false or config.capture = false to disable capture."
63
+ return
64
+ end
39
65
 
40
66
  @intercept_addresses = value.is_a?(Array) ? value : [value]
41
67
  end
42
68
 
69
+ # Custom setter that preserves defaults when nil/empty array is provided
70
+ def intercept_path_patterns=(value)
71
+ return if value.nil? || (value.is_a?(Array) && value.empty?)
72
+
73
+ @intercept_path_patterns = value.is_a?(Array) ? value : [value]
74
+ end
75
+
43
76
  def base_url=(value)
44
77
  stripped = value&.sub(%r{/+\z}, "")
45
78
  raise Error, BASE_URL_ERROR_MSG unless stripped.nil? || valid_base_url?(stripped)
@@ -1,6 +1,7 @@
1
1
  # Coolhand default exclude API patterns
2
- # These substrings are matched against request URLs after the intercept_addresses
3
- # allow-list passes. Matching URLs are skipped and not forwarded as llm_request_logs.
2
+ # These substrings are matched against the request's path (not the full URL or
3
+ # query string). A request whose path matches is skipped and not forwarded as
4
+ # an llm_request_log, regardless of whether it would otherwise match intercept_addresses.
4
5
  #
5
6
  # Users can extend defaults: c.exclude_api_patterns << "/myOperationalPath/"
6
7
  # Users can override entirely: c.exclude_api_patterns = ["/only_this/"]
@@ -1,9 +1,15 @@
1
1
  # Coolhand default intercept addresses
2
- # These substrings are matched against request URLs to decide whether a request
3
- # should be captured and forwarded as an llm_request_log.
2
+ # These are matched against the request's parsed host (exact match, or a
3
+ # dot-boundary suffix match, case-insensitive) to decide whether a request
4
+ # should be captured and forwarded as an llm_request_log. A single "*" in an
5
+ # entry matches exactly one host label — e.g. "bedrock-runtime.*.amazonaws.com"
6
+ # matches "bedrock-runtime.us-east-1.amazonaws.com" but nothing else.
4
7
  #
5
8
  # Users can extend defaults: c.intercept_addresses << "my.custom.api.com"
6
9
  # Users can override entirely: c.intercept_addresses = ["only.this.com"]
10
+ # This list cannot be emptied — it's required (unlike exclude_api_patterns, which
11
+ # can be disabled with `= []`). To disable capture entirely, use
12
+ # c.enabled = false or c.capture = false instead.
7
13
 
8
14
  - "api.openai.com"
9
15
  - "api.anthropic.com"
@@ -11,9 +17,8 @@
11
17
  - "generativelanguage.googleapis.com"
12
18
  - "models.github.ai"
13
19
  - "models.inference.ai.azure.com"
14
- - ":generateContent"
15
- - ":streamGenerateContent"
16
20
  - "aiplatform.googleapis.com"
17
21
  - "gateway.ai.cloudflare.com"
18
- - "bedrock-runtime"
22
+ - "bedrock-runtime.*.amazonaws.com"
19
23
  - "openrouter.ai"
24
+ - "opencode.ai"
@@ -0,0 +1,12 @@
1
+ # Colon-action path suffixes from Google's API Discovery convention
2
+ # (e.g. ".../v1beta/models/gemini-pro:generateContent"). These have no host
3
+ # component of their own, so — unlike intercept_addresses — they are only
4
+ # ever matched against the *path* of requests whose host is already
5
+ # googleapis.com or a googleapis.com subdomain. They can never match on
6
+ # their own against an unrelated/attacker-controlled host.
7
+ #
8
+ # Users can extend defaults: c.intercept_path_patterns << ":myAction"
9
+ # Users can override entirely: c.intercept_path_patterns = [":onlyThis"]
10
+
11
+ - ":generateContent"
12
+ - ":streamGenerateContent"