coolhand 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: fc0fc356a6cf69d4a0c67a906000cab9590d9fc2a452efec8037a7f541efa561
4
- data.tar.gz: ec6f7a136ae1d8e139bf638175b81a2dda063b9bf19d0199ab7f7135cf6d833d
3
+ metadata.gz: 3f66ea8b94e39769d66543c75987c2108397f89c93dc53748753b396ca76d921
4
+ data.tar.gz: 560b9ec89c71b31095b5f579bb270b295a79b87a7edbec495179f6f1b764f77e
5
5
  SHA512:
6
- metadata.gz: 131465dcaf94ae8cc4b5dce47252fe92effc9693ba4de6e61247714c682a717934fa3144cf3228bf754fe89a53803051f32a895c3a42fd3c07990d4cf74286a5
7
- data.tar.gz: 26e03d0f27b9acbeb36ff6a0e0ee11cca55b8142cd17d494972013534ba4d8b3753b29fb3565dedb8c0f8daf1e26d7af443353999f5956f5b1b7b8d7ea139585
6
+ metadata.gz: ebd806bb3744d3fd16673a98f7746b144b468a158ec855b32a4ba142cf110ff4a78b03ea1b95106b8f29d011425bbe0ec44b8f853acb87b9e4037cf87843991f
7
+ data.tar.gz: 3a8078f5dbe7f3a5c7a0892b391a7085d54b5a6aa1db386075e34470c89802ca0bd6bb5dc5654dffa7fcb2595da3df7b2210a52568595c5c77929e435d1a13b8
data/.rubocop.yml CHANGED
@@ -125,6 +125,12 @@ Layout/EndAlignment:
125
125
  Naming/VariableNumber:
126
126
  Enabled: false
127
127
 
128
+ Naming/PredicateMethod:
129
+ Enabled: false
130
+
131
+ Naming/MethodParameterName:
132
+ Enabled: false
133
+
128
134
  # Lint rules
129
135
  Lint/EmptyClass:
130
136
  Enabled: false
@@ -139,6 +145,12 @@ RSpec/HookArgument:
139
145
  RSpec/MultipleExpectations:
140
146
  Enabled: false
141
147
 
148
+ RSpec/VerifiedDoubleReference:
149
+ Enabled: false
150
+
151
+ RSpec/MessageChain:
152
+ Enabled: false
153
+
142
154
  RSpec/ExampleLength:
143
155
  Enabled: false
144
156
 
data/CHANGELOG.md CHANGED
@@ -5,6 +5,73 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [Unreleased]
9
+
10
+ ## [0.4.0] - 2026-06-22
11
+
12
+ ### Added
13
+ - **More default intercept addresses** — Vertex AI (`aiplatform.googleapis.com`), Cloudflare AI Gateway (`gateway.ai.cloudflare.com`), AWS Bedrock OpenAI-compatible endpoint (`bedrock-runtime`), and OpenRouter (`openrouter.ai`) are now monitored out of the box with no configuration required (#66).
14
+ - **`Configuration#enabled` flag** — Set `config.enabled = false` (e.g. `config.enabled = Rails.env.production?`) to skip all patching and validation globally without restructuring your configure block (#68).
15
+ - **Feedback `creator_type` field** — Pass `creator_type: 'human'`, `'agent'`, or `'unknown'` when submitting feedback to identify who originated the feedback; matches the Coolhand API field (#67).
16
+
17
+ ### Changed
18
+ - **Deferred `api_key` validation** — A missing `api_key` no longer raises at `Coolhand.configure` time. Intercepted requests are silently skipped (with a warning log) when the key is absent, so apps that boot without a key in CI or non-production environments no longer crash (#68).
19
+
20
+ ### Dependencies
21
+ - Bumped `faraday` from 2.14.1 to 2.14.2 (#63).
22
+
23
+ ## [0.3.0] - 2026-05-14
24
+
25
+ ### 🚀 Major Changes
26
+ - **Unified Net::HTTP Interceptor** - Replaced dual interceptor architecture (Faraday + Anthropic) with a single `NetHttpInterceptor` that captures all HTTP traffic via `Module#prepend`
27
+ - **Simplified Namespace** - Removed `Coolhand::Ruby` namespace; all classes now under `Coolhand` directly (e.g., `Coolhand::FeedbackService` instead of `Coolhand::Ruby::FeedbackService`)
28
+ - **Ruby 4.0 Compatibility** - Full support for Ruby 4.0 with conditional debugger dependencies
29
+ - **Google Gemini API Support** - `generativelanguage.googleapis.com` and `:streamGenerateContent` added to default `intercept_addresses`; both `generateContent` and `streamGenerateContent` endpoints are intercepted out of the box
30
+ - **Anthropic API Support Restored** - `api.anthropic.com` added to default `intercept_addresses`; accidentally dropped during the v0.3.0 refactor that replaced `AnthropicInterceptor` with the unified `NetHttpInterceptor`
31
+ - **URL Query Parameter Sanitization** - New `sanitize_url` helper redacts sensitive query parameters (`key`, `api_key`, `apikey`, `token`, `access_token`, `secret`) before logging; protects API keys passed as URL params (common with Gemini's `?key=` pattern)
32
+
33
+ ### ✨ New Features
34
+ - **GitHub Models API** - `models.github.ai` (current endpoint) and `models.inference.ai.azure.com` (deprecated endpoint) added to default `intercept_addresses`; calls routed through GitHub Copilot credentials are now captured automatically without manual configuration. Default intercept addresses are loaded from `default_intercept_addresses.yml` to make future additions a single-line YAML change.
35
+ - **`config.base_url`** - Configurable API destination for self-hosted deployments. Defaults to `https://coolhandlabs.com/api`; set to any `https://` URL to redirect logs and feedback POSTs to your own backend. `http://localhost` and `http://127.0.0.1` are also accepted for local development. Trailing slashes are normalized automatically.
36
+ - **Feedback `sentiment` field** - New string field for feedback: `'like'`, `'dislike'`, or `'neutral'`. Preferred over the boolean `like` field for richer signal.
37
+ - **Feedback `workload_hashid` field** - New string field to associate feedback with a specific workload.
38
+ - **Batch Processing Support** - New `Coolhand::OpenAi::BatchResultProcessor` and `Coolhand::Vertex::BatchResultProcessor` for logging completed async batch jobs as individual `llm_request_log` entries
39
+ - **OpenAI Webhook Validation** - New `Coolhand::OpenAi::WebhookValidator` verifies webhook signatures using HMAC-SHA256 with timing-safe comparison; lenient in development, strict in production/staging
40
+ - **WebhookInterceptor Rails Module** - `Coolhand::WebhookInterceptor` mixin for Rails controllers to validate and dispatch OpenAI batch completion webhooks automatically
41
+ - **Capture Control** - New `config.capture` global toggle (default: `true`) and `config.debug_mode` (captures locally, skips API forwarding) for fine-grained interception control
42
+ - **Thread-Safe Block Control** - `Coolhand.with_capture { }` and `Coolhand.without_capture { }` for scoped override of capture behavior within a block; uses thread-local storage
43
+ - **Exclude API Patterns** - New `config.exclude_api_patterns` deny-list checked after the `intercept_addresses` allow-list; default excludes `["/batchPredictionJobs/"]` to suppress Vertex AI batch job management noise
44
+
45
+ ### 🚫 Deprecated
46
+ - **Feedback `like` field** - The boolean `like` field is deprecated. Use `sentiment: 'like'` or `sentiment: 'dislike'` instead.
47
+
48
+ ### 🏗️ Architecture Improvements
49
+ - **Single Interceptor** - `NetHttpInterceptor` patches `Net::HTTP#request` and `Net::HTTPResponse#read_body`; removed ~1,400 lines of interceptor-specific code
50
+ - **Thread-Safe Streaming** - Uses `Thread.current[:coolhand_stream_buffer]` for streaming response capture
51
+ - **Capture Priority Hierarchy** - `debug_mode` (always capture) > thread-local override > global `capture` config
52
+
53
+ ### 🐛 Bug Fixes
54
+ - **Streaming Response Encoding** - Streamed response content is now force-encoded to UTF-8 before JSON parsing, eliminating noisy `BINARY` encoding warnings for multi-byte responses.
55
+ - **Double-Capture with `Net::HTTP.new` Pattern** - Fixed double-logging when callers use `Net::HTTP.new(host, port).request(req)` without an explicit `start` block. The re-entry guard is now per-connection-object (using a `compare_by_identity` Hash) rather than a boolean thread-local, so independent requests on a different `Net::HTTP` instance inside a callback are still captured.
56
+ - **Provider-Neutral Readiness Log** - Startup console message no longer names a single provider; it now reflects all monitored inference URIs.
57
+ - **Interceptor No Longer Silently Drops Logs on HTTP Errors** - Wrapped `Net::HTTP#request` in `begin/rescue/ensure` so `send_complete_request_log` is always called even when the SDK raises an exception (e.g., `Anthropic::Errors::NotFoundError` on a 404). Status is extracted from the exception via `.status`, `.response.status`, or message parsing.
58
+
59
+ ### 📦 Dependencies
60
+ - Bumped `faraday` from 2.14.0 to 2.14.1
61
+
62
+ ### 💔 Breaking Changes
63
+ - **`config.base_url` validation** - `Coolhand.configure` now raises `Coolhand::Error` if `base_url` is set to a plain `http://` URL (non-localhost). Previously any string was accepted silently. If you were pointing at an internal `http://` host (e.g. `http://logs.internal/api`), you will need to either enable TLS on that host or use an `https://` proxy in front of it.
64
+ - **Namespace Change** - `Coolhand::Ruby::*` references must be updated to `Coolhand::*`
65
+ - **Removed Files** - `faraday_interceptor.rb` and `anthropic_interceptor.rb` replaced by `net_http_interceptor.rb`
66
+ - **`environment` Config Behavior** - The `environment` attribute no longer controls whether requests are forwarded to the API. Use `config.debug_mode = true` instead if you previously relied on `environment: "development"` to suppress API calls.
67
+
68
+ ### 🔄 Migration Guide
69
+ 1. Update gem dependency to `~> 0.3.0`
70
+ 2. Replace `Coolhand::Ruby::` with `Coolhand::` in all class references
71
+ 3. If using `environment: "development"` to prevent API calls, switch to `config.debug_mode = true`
72
+ 4. If `config.base_url` was set to a plain `http://` (non-localhost) URL, switch to `https://` or use an `https://` proxy
73
+ 5. No other changes needed to `Coolhand.configure` blocks for basic usage
74
+
8
75
  ## [0.2.0] - 2025-12-16
9
76
 
10
77
  ### ✨ Major New Features
@@ -65,7 +132,7 @@ For users upgrading from v0.1.x:
65
132
  - **Collection Method Tracking** - Support for optional collection method suffix (`manual`, `auto-monitor`)
66
133
 
67
134
  ### 🏗️ Internal Improvements
68
- - **Added Collector Module** - New `Coolhand::Ruby::Collector` module for generating SDK identification strings
135
+ - **Added Collector Module** - New `Coolhand::Collector` module for generating SDK identification strings
69
136
  - **Updated ApiService** - Base service now automatically adds collector field to all API payloads
70
137
  - **Enhanced Logging** - Both LoggerService and FeedbackService now send collector information
71
138
 
data/CLAUDE.md ADDED
@@ -0,0 +1,34 @@
1
+ # Development Guidelines
2
+
3
+ ## Optional Provider Dependencies
4
+
5
+ Coolhand supports multiple LLM providers (OpenAI, Anthropic, Google Gemini, etc.). These provider gems should **never** be required at gem load time, as clients may not use all providers and shouldn't be forced to install unnecessary dependencies.
6
+
7
+ **Rule**: Any require for provider SDKs (openai, anthropic, google-generativeai, etc.) must be:
8
+ 1. Placed in the file where it's actually used (not in the main coolhand.rb)
9
+ 2. Only executed when that provider's functionality is accessed
10
+ 3. Not declared as a hard dependency in coolhand-ruby.gemspec
11
+
12
+ Example pattern:
13
+ ```ruby
14
+ # ❌ DON'T: In lib/coolhand.rb (loads unconditionally)
15
+ require "openai"
16
+
17
+ # ✅ DO: In lib/coolhand/open_ai/batch_result_processor.rb (only when needed)
18
+ require "openai"
19
+
20
+ module Coolhand
21
+ module OpenAi
22
+ class BatchResultProcessor
23
+ def client
24
+ @client ||= OpenAI::Client.new
25
+ end
26
+ end
27
+ end
28
+ end
29
+ ```
30
+
31
+ This ensures:
32
+ - Gem loads cleanly regardless of what providers are installed
33
+ - Apps using path gems (local development) don't break from missing optional dependencies
34
+ - Users only need gems for providers they actually use
data/README.md CHANGED
@@ -1,5 +1,7 @@
1
1
  # Coolhand Ruby Monitor
2
2
 
3
+ [![Gem Version](https://badge.fury.io/rb/coolhand.svg)](https://badge.fury.io/rb/coolhand)
4
+
3
5
  Monitor and log LLM API calls from multiple providers (OpenAI, Anthropic, Google AI, Cohere, and more) to the Coolhand analytics platform.
4
6
 
5
7
  ## Installation
@@ -24,7 +26,7 @@ gem 'coolhand'
24
26
 
25
27
  ```ruby
26
28
  # Add this configuration at the start of your application
27
- require 'coolhand/ruby'
29
+ require 'coolhand'
28
30
 
29
31
  Coolhand.configure do |config|
30
32
  config.api_key = 'your_api_key_here'
@@ -47,15 +49,38 @@ end
47
49
  - ⚡ **Performance optimized** - Negligible overhead via async logging
48
50
  - 🛡️ **Future-proof** - Automatically captures new AI calls added by your team
49
51
 
52
+ ## Self-Hosted Deployments
53
+
54
+ For compliance, data-residency, or cost reasons you can run your own Coolhand-compatible endpoint and point the SDK at it via `config.base_url`:
55
+
56
+ ```ruby
57
+ Coolhand.configure do |config|
58
+ config.api_key = ENV['COOLHAND_API_KEY']
59
+ config.base_url = ENV['COOLHAND_BASE_URL'] # e.g. "https://coolhand.internal.example.com/api"
60
+ end
61
+ ```
62
+
63
+ When `base_url` is unset the SDK defaults to `https://coolhandlabs.com/api` and behaviour is unchanged.
64
+
65
+ **Accepted values:**
66
+ - Any `https://` URL — required for production use
67
+ - `http://localhost` or `http://127.0.0.1` — accepted for local development only
68
+
69
+ **Trailing slashes** are stripped automatically, so `"https://example.com/api/"` and `"https://example.com/api"` are equivalent.
70
+
71
+ The SDK raises `Coolhand::Error` at configure time if `base_url` is set to a plain `http://` URL pointing at a non-localhost host.
72
+
50
73
  ## Feedback API
51
74
 
52
- Collect feedback on LLM responses to improve model performance:
75
+ Collect feedback on LLM responses to improve model performance.
76
+
77
+ > **Frontend Feedback Widget**: For browser-based feedback collection, see [coolhand-js](https://github.com/Coolhand-Labs/coolhand-js) - an accessible, lightweight JavaScript widget that leverages best UX practices to capture actionable user feedback on any AI output.
53
78
 
54
79
  ```ruby
55
- require 'coolhand/ruby'
80
+ require 'coolhand'
56
81
 
57
82
  # Create feedback for an LLM response
58
- feedback_service = Coolhand::Ruby::FeedbackService.new(Coolhand.configuration)
83
+ feedback_service = Coolhand::FeedbackService.new(Coolhand.configuration)
59
84
 
60
85
  feedback = feedback_service.create_feedback(
61
86
  llm_request_log_id: 123,
@@ -65,7 +90,7 @@ feedback = feedback_service.create_feedback(
65
90
  original_output: 'Here is the original LLM response!',
66
91
  revised_output: 'Here is the human edit of the original LLM response.',
67
92
  explanation: 'Tone of the original response read like AI-generated open source README docs',
68
- like: true
93
+ sentiment: 'dislike'
69
94
  )
70
95
  ```
71
96
 
@@ -80,8 +105,11 @@ feedback = feedback_service.create_feedback(
80
105
  ### Quality Data
81
106
  - **`revised_output`** ⭐ *Best Signal* - End user revision of the LLM response. The highest value data for improving quality scores.
82
107
  - **`explanation`** 💬 *Medium Signal* - End user explanation of why the response was good or bad. Valuable qualitative data.
83
- - **`like`** 👍 *Low Signal* - Boolean like/dislike. Lower quality signal but easy for users to provide.
108
+ - **`sentiment`** 🎭 *Preferred* - String sentiment: `'like'`, `'dislike'`, or `'neutral'`. Takes precedence over `like` if both are provided. The gem automatically converts `like` to `sentiment` before sending.
109
+ - **`like`** 👍 *Low Signal (Deprecated)* - Boolean: `true` = like, `false` = dislike. Use `sentiment` instead. Conversion: `true` → `"like"`, `false` → `"dislike"`.
110
+ - **`workload_hashid`** 🔗 *Workload Association* - Hashid of a workload to associate this feedback with.
84
111
  - **`creator_unique_id`** 👤 *User Tracking* - Unique ID to match feedback to the end user who created it
112
+ - **`creator_type`** 🧑‍🤝‍🤖 *Creator Type* - What kind of creator submitted the feedback: `'human'`, `'agent'`, or `'unknown'`.
85
113
 
86
114
  ## Rails Integration
87
115
 
@@ -100,7 +128,7 @@ Coolhand.configure do |config|
100
128
  config.silent = Rails.env.production?
101
129
 
102
130
  # Specify which LLM endpoints to intercept (array of strings)
103
- # Optional - defaults to ["api.openai.com", "api.anthropic.com"]
131
+ # Optional - defaults to OpenAI, Anthropic, ElevenLabs, Google Gemini, and GitHub Models
104
132
  # config.intercept_addresses = ["api.openai.com", "api.anthropic.com", "api.cohere.ai"]
105
133
  end
106
134
  ```
@@ -110,7 +138,7 @@ end
110
138
  ```ruby
111
139
  class ChatController < ApplicationController
112
140
  def create_feedback
113
- feedback_service = Coolhand::Ruby::FeedbackService.new(Coolhand.configuration)
141
+ feedback_service = Coolhand::FeedbackService.new(Coolhand.configuration)
114
142
 
115
143
  feedback = feedback_service.create_feedback(
116
144
  llm_request_log_id: params[:log_id],
@@ -118,7 +146,7 @@ class ChatController < ApplicationController
118
146
  original_output: params[:original_response],
119
147
  revised_output: params[:edited_response],
120
148
  explanation: params[:feedback_text],
121
- like: params[:thumbs_up]
149
+ sentiment: params[:sentiment]
122
150
  )
123
151
 
124
152
  if feedback
@@ -135,14 +163,14 @@ end
135
163
  ```ruby
136
164
  class FeedbackCollectionJob < ApplicationJob
137
165
  def perform(feedback_data)
138
- feedback_service = Coolhand::Ruby::FeedbackService.new(Coolhand.configuration)
166
+ feedback_service = Coolhand::FeedbackService.new(Coolhand.configuration)
139
167
 
140
168
  feedback_service.create_feedback(
141
169
  llm_provider_unique_id: feedback_data[:request_id],
142
170
  creator_unique_id: feedback_data[:user_id],
143
171
  original_output: feedback_data[:original],
144
172
  explanation: feedback_data[:reason],
145
- like: feedback_data[:positive]
173
+ sentiment: feedback_data[:sentiment]
146
174
  )
147
175
  end
148
176
  end
@@ -154,7 +182,8 @@ end
154
182
 
155
183
  | Option | Type | Default | Description |
156
184
  |--------|------|---------|-------------|
157
- | `api_key` | String | *required* | Your Coolhand API key for authentication |
185
+ | `api_key` | String | `nil` | Your Coolhand API key. If absent, intercepted requests are skipped with a warning log rather than raising at boot time |
186
+ | `enabled` | Boolean | `true` | Set to `false` to disable all patching and validation (e.g. `Rails.env.production?`) |
158
187
  | `silent` | Boolean | `false` | Whether to suppress console output |
159
188
  | `intercept_addresses` | Array | `["api.openai.com", "api.anthropic.com"]` | Array of API endpoint strings to monitor |
160
189
 
@@ -164,7 +193,7 @@ end
164
193
 
165
194
  ```ruby
166
195
  require 'openai'
167
- require 'coolhand/ruby'
196
+ require 'coolhand'
168
197
 
169
198
  # Configure Coolhand
170
199
  Coolhand.configure do |config|
@@ -275,28 +304,26 @@ The monitor works with multiple transport layers and Ruby libraries:
275
304
 
276
305
  **Native HTTP libraries:**
277
306
  - Official Anthropic Ruby SDK (using Net::HTTP)
307
+ - GitHub Models SDK / any client using `models.github.ai`
278
308
  - Any library using Net::HTTP directly
279
309
 
280
- **Auto-detection**: Coolhand automatically detects which transport layer your libraries use and applies the appropriate monitoring strategy.
310
+ **Universal Coverage**: Since most Ruby HTTP libraries use Net::HTTP under the hood, Coolhand's single interceptor provides comprehensive monitoring without needing library-specific integrations.
281
311
 
282
312
  ## How It Works
283
313
 
284
- Coolhand uses a dual-interceptor strategy to monitor different HTTP transport layers:
314
+ Coolhand uses a unified Net::HTTP interceptor to monitor all HTTP traffic to configured LLM endpoints:
285
315
 
286
- ### Faraday Interceptor
287
- - Patches Faraday connections using middleware injection
288
- - Monitors: OpenAI SDK, ruby-anthropic, LangChain.rb, and other Faraday-based libraries
289
- - Handles: Standard HTTP requests and Server-Sent Events (SSE) for streaming
290
-
291
- ### Anthropic Interceptor
292
- - Patches the official Anthropic gem's internal HTTP transport (Net::HTTP)
293
- - Monitors: Official Anthropic Ruby SDK requests
316
+ ### Net::HTTP Interceptor
317
+ - Patches Ruby's core `Net::HTTP` library using `Module#prepend`
318
+ - Monitors **all** HTTP libraries that use Net::HTTP under the hood (which is most of them)
319
+ - Handles both standard requests and streaming responses via `read_body` interception
320
+ - Thread-safe design using thread-local storage for streaming buffers
294
321
 
295
322
  ### Request Flow
296
323
  When a request matches configured LLM endpoints:
297
324
 
298
325
  1. The original request executes normally with zero performance impact
299
- 2. Request and response data (body, headers, status) are captured by the appropriate interceptor
326
+ 2. Request and response data (body, headers, status) are captured by the interceptor
300
327
  3. For streaming requests, the complete accumulated response is captured (not individual chunks)
301
328
  4. Data is sent to the Coolhand API asynchronously in a background thread
302
329
  5. Your application continues without interruption
@@ -336,7 +363,7 @@ For standard Ruby scripts or non-Rails applications:
336
363
 
337
364
  ```ruby
338
365
  #!/usr/bin/env ruby
339
- require 'coolhand/ruby'
366
+ require 'coolhand'
340
367
 
341
368
  Coolhand.configure do |config|
342
369
  config.api_key = 'your_api_key_here' # Store securely, don't commit to git
@@ -365,6 +392,102 @@ The monitor handles errors gracefully:
365
392
  - Invalid API keys will be reported but won't crash your app
366
393
  - Network issues are handled with appropriate error messages
367
394
 
395
+
396
+ ## Batch webhook handler (OpenAI)
397
+
398
+ Automatically handle OpenAI batch event logs (batch.completed, batch.failed, batch.expired, batch.cancelled)
399
+ by intercepting webhook requests and enqueuing your batch result processor.
400
+
401
+ Usage:
402
+ - Include the interceptor in your controller:
403
+ include Coolhand::WebhookInterceptor
404
+ - Add the before_action to validate and populate @validator payload:
405
+ before_action :intercept_batch_request, only: :openai
406
+ - Ensure you skip CSRF for the webhook endpoint:
407
+ skip_before_action :verify_authenticity_token
408
+ - Override the webhook_secret method to return your OpenAI webhook secret
409
+
410
+ Minimal example (only key lines shown):
411
+
412
+ ```ruby
413
+ # app/controllers/webhooks/batch_api_requests_controller.rb
414
+ # ...existing code...
415
+ include Coolhand::WebhookInterceptor
416
+
417
+ skip_before_action :verify_authenticity_token
418
+ before_action :intercept_batch_request, only: :openai
419
+
420
+ def openai
421
+ event = JSON.parse(@validator.payload)
422
+ case event["type"]
423
+ when "batch.completed", "batch.failed", "batch.expired", "batch.cancelled"
424
+ batch_id = event.dig("data", "id")
425
+ batch_request = BatchApiRequest.find_by(provider: "openai", provider_batch_id: batch_id)
426
+
427
+ if batch_request
428
+ OpenAi::BatchResultProcessor.perform_async(batch_request.id)
429
+ Rails.logger.info("Queued batch result processing for BatchApiRequest #{batch_request.id}")
430
+ else
431
+ Rails.logger.warn("Could not find BatchApiRequest for OpenAI batch ID: #{batch_id}")
432
+ end
433
+ else
434
+ Rails.logger.info("Unhandled OpenAI webhook event type: #{event["type"]}")
435
+ end
436
+
437
+ head :ok
438
+ rescue JSON::ParserError
439
+ head :bad_request
440
+ rescue StandardError => e
441
+ Rails.logger.error("OpenAI webhook error: #{e.message}")
442
+ head :internal_server_error
443
+ end
444
+
445
+ def webhook_secret
446
+ Rails.application.credentials.openai_webhook_secret
447
+ end
448
+ # ...existing code...
449
+ ```
450
+
451
+ ## Batch webhook handler (Vertex)
452
+
453
+ Automatically handle Vertex batch event logs.
454
+
455
+ Usage:
456
+ - call Coolhand::Vertex::BatchResultProcessor service with batch_info and download batch results
457
+
458
+ Minimal example (only key lines shown):
459
+
460
+ ```ruby
461
+ class Vertex::BatchCallbackProcessor < BaseService
462
+ option :batch_request, model: BatchApiRequest
463
+ option :batch_info
464
+
465
+ def call
466
+ case batch_info["state"]
467
+ when "JOB_STATE_PENDING"
468
+ nil
469
+ when "JOB_STATE_RUNNING", "JOB_STATE_QUEUED"
470
+ batch_request.update!(status: "processing")
471
+
472
+ Coolhand::Vertex::BatchResultProcessor.new(batch_info:).call
473
+ when "JOB_STATE_SUCCEEDED"
474
+ output_file_id = batch_info["outputInfo"]["gcsOutputDirectory"]
475
+ results = download_batch_results(output_file_id)
476
+ results.each { |batch_item| process_batch_result(batch_item) }
477
+
478
+ batch_request.update!(status: "completed", completed_at: Time.current, output_file_id:)
479
+
480
+ Coolhand::Vertex::BatchResultProcessor.new(batch_info:).call(results)
481
+
482
+ # Clean up GCS files after successful processing
483
+ cleanup_gcs_files(output_file_id)
484
+ when "JOB_STATE_FAILED"
485
+ handle_failed_batch(batch_info["error"]["message"])
486
+ end
487
+ end
488
+ end
489
+ ```
490
+
368
491
  ## Integration Guides
369
492
 
370
493
  - **[Anthropic Integration](docs/anthropic.md)** - Complete guide for both official and community Anthropic gems, including streaming, dual gem handling, and troubleshooting
@@ -376,10 +499,11 @@ The monitor handles errors gracefully:
376
499
  - No sensitive data is exposed in logs
377
500
  - All data is sent via HTTPS to Coolhand servers
378
501
 
379
- ## Other Languages
502
+ ## Related Packages
380
503
 
381
- - **Node.js**: [coolhand-node package](https://github.com/coolhand-io/coolhand-node) - Coolhand monitoring for Node.js applications
382
- - **API Docs**: [API Documentation](https://coolhandlabs.com/docs) - Direct API integration documentation
504
+ - **Frontend (Feedback Collection Widget)**: [coolhand-js](https://github.com/Coolhand-Labs/coolhand-js) - Frontend feedback widget for collecting user feedback on AI outputs
505
+ - **Node.js**: [coolhand-node package](https://github.com/Coolhand-Labs/coolhand-node) - Coolhand monitoring for Node.js applications
506
+ - **Python**: [coolhand package](https://github.com/Coolhand-Labs/coolhand-python) - Coolhand monitoring for Python applications
383
507
 
384
508
  ## Community
385
509
 
data/docs/anthropic.md CHANGED
@@ -15,7 +15,7 @@ Coolhand automatically detects which Anthropic gem you're using and applies the
15
15
  ### Basic Configuration
16
16
 
17
17
  ```ruby
18
- require 'coolhand/ruby'
18
+ require 'coolhand'
19
19
 
20
20
  Coolhand.configure do |config|
21
21
  config.api_key = 'your_coolhand_api_key_here'
@@ -39,7 +39,7 @@ gem 'anthropic'
39
39
 
40
40
  ```ruby
41
41
  require 'anthropic'
42
- require 'coolhand/ruby'
42
+ require 'coolhand'
43
43
 
44
44
  # Configure Coolhand
45
45
  Coolhand.configure do |config|
@@ -124,7 +124,7 @@ gem 'ruby-anthropic'
124
124
 
125
125
  ```ruby
126
126
  require 'ruby-anthropic'
127
- require 'coolhand/ruby'
127
+ require 'coolhand'
128
128
 
129
129
  # Configure Coolhand
130
130
  Coolhand.configure do |config|
@@ -172,7 +172,7 @@ When both gems are installed, Coolhand automatically handles the conflict:
172
172
  ```ruby
173
173
  require 'anthropic' # Official gem
174
174
  require 'ruby-anthropic' # Community gem
175
- require 'coolhand/ruby'
175
+ require 'coolhand'
176
176
 
177
177
  Coolhand.configure do |config|
178
178
  config.api_key = 'your_coolhand_api_key'
@@ -199,7 +199,7 @@ begin
199
199
  $LOAD_PATH.reject! { |path| path.include?('ruby-anthropic') }
200
200
 
201
201
  require 'anthropic'
202
- require 'coolhand/ruby'
202
+ require 'coolhand'
203
203
 
204
204
  Coolhand.configure do |config|
205
205
  config.api_key = 'your_coolhand_api_key'
@@ -359,10 +359,10 @@ request_id = Thread.current[:coolhand_current_request_id]
359
359
  puts "Logged to Coolhand with ID: #{request_id}"
360
360
 
361
361
  # Use this ID for feedback or debugging
362
- feedback_service = Coolhand::Ruby::FeedbackService.new(Coolhand.configuration)
362
+ feedback_service = Coolhand::FeedbackService.new(Coolhand.configuration)
363
363
  feedback_service.create_feedback(
364
364
  llm_request_log_id: request_id,
365
- like: true,
365
+ sentiment: "like",
366
366
  explanation: "Great response quality"
367
367
  )
368
368
  ```
@@ -380,7 +380,7 @@ Coolhand.configure do |config|
380
380
  end
381
381
 
382
382
  # Now you'll see console output like:
383
- # "✅ Coolhand ready - will log OpenAI and Anthropic (official gem) calls"
383
+ # "✅ Coolhand ready - will log inference calls on monitored URIs"
384
384
  # "COOLHAND: ⚠️ Warning: Both 'anthropic' and 'ruby-anthropic' gems are installed..."
385
385
  ```
386
386
 
@@ -393,8 +393,8 @@ end
393
393
  **Solution**: Ensure you require the gem before configuring Coolhand:
394
394
 
395
395
  ```ruby
396
- require 'anthropic' # Must come before coolhand/ruby
397
- require 'coolhand/ruby'
396
+ require 'anthropic' # Must come before coolhand
397
+ require 'coolhand'
398
398
 
399
399
  Coolhand.configure do |config|
400
400
  config.api_key = 'your_api_key'
@@ -515,4 +515,4 @@ For complete API documentation, see:
515
515
 
516
516
  - **Coolhand Issues**: [GitHub Issues](https://github.com/Coolhand-Labs/coolhand-ruby/issues)
517
517
  - **Anthropic API**: [Anthropic Documentation](https://docs.anthropic.com/)
518
- - **Gem Conflicts**: Check this guide's troubleshooting section
518
+ - **Gem Conflicts**: Check this guide's troubleshooting section
data/docs/elevenlabs.md CHANGED
@@ -213,11 +213,12 @@ class TranscriptsController < ApplicationController
213
213
 
214
214
  if feedback_data
215
215
  # Submit feedback to Coolhand using llm_provider_unique_id for matching
216
+ rating = feedback_data[:feedback_rating]
216
217
  coolhand_feedback = {
217
- like: feedback_data[:feedback_rating],
218
+ sentiment: rating.nil? ? nil : (rating ? "like" : "dislike"),
218
219
  explanation: feedback_data[:feedback_text],
219
220
  llm_provider_unique_id: conversation_id # Important: use this field for matching
220
- }
221
+ }.compact
221
222
 
222
223
  result = Coolhand.feedback_service.create_feedback(coolhand_feedback)
223
224
 
@@ -392,11 +393,12 @@ response = service.fetch_conversation(conversation_id)
392
393
  feedback = service.extract_feedback(response)
393
394
 
394
395
  # Test Coolhand submission
396
+ rating = feedback[:feedback_rating]
395
397
  coolhand_feedback = {
396
- like: feedback[:feedback_rating],
398
+ sentiment: rating.nil? ? nil : (rating ? "like" : "dislike"),
397
399
  explanation: feedback[:feedback_text],
398
400
  llm_provider_unique_id: conversation_id
399
- }
401
+ }.compact
400
402
  result = Coolhand.feedback_service.create_feedback(coolhand_feedback)
401
403
  ```
402
404