coolhand 0.5.0 → 0.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: daed2b7ac4fe321602eb90f673cb385bd930b9ff503bf7e5d7851cbf06422617
4
- data.tar.gz: 5d01a6386d142cdb31c490574191ac3867dabd30670c2a832c2aa18c7150c872
3
+ metadata.gz: 25393afdf95625b624ca2910cb3017685705529e9628a5d15cb3bdffdf122d52
4
+ data.tar.gz: 3e478c3b8494d5fef5bc87896fbf69cb9f93d088029b9a9745563c9a789b4473
5
5
  SHA512:
6
- metadata.gz: a0681a3419c92e13fbae7a1b81c6d81b7a1cf2f088571dd44b4d2694c0ae5bf5c4c4312d7bff53ccc740833290bc931b932c1179244f004090192b0baaeb19f0
7
- data.tar.gz: 9efd5e33528e0caeecc770c7286085320f9e4218f01fcf592ca82a27c96df71cf5ae52712a5a9aaeaf0702774be450bb7c3b7c71b0df8e57f56bd038f54f1b3d
6
+ metadata.gz: ad635f9377f6cb81967fc24b0c2aa9f9938c41e308c6b98511ee0139b9938e01ea766baf72a085700866c69ac50207da11e29f04d30a94ad3b487d3e22243d9e
7
+ data.tar.gz: d6ef0baf4e37d29b36a0d7a2d6a41822f551a0a753a2dc963c0da9eed60b90bf29b5f0015caa8a628c6f340ec9cb2b3ef2c83ecb9b59fbcec909d70f396603df
data/CHANGELOG.md CHANGED
@@ -7,6 +7,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.5.1] - 2026-08-02
11
+
12
+ ### Added
13
+ - **`Coolhand::Vertex::BatchResultProcessor.new` accepts an optional `model:` keyword** (falls back to a `"model"` key in `batch_info` when omitted) to populate the `model` field on batch-result logs — included only when one of those is available and non-blank, and omitted from the payload entirely otherwise (#76).
14
+ - **`Coolhand::BaseInterceptor.send_complete_request_log` accepts optional `source_api:`/`model:` keywords**, for synthetic/batch log types (like the Vertex processor above) that aren't backed by a real intercepted HTTP request (#76).
15
+
16
+ ### Changed
17
+ - **Vertex batch results with a missing or malformed job resource name or unparseable `startTime`/`endTime` are no longer logged at all.** Previously the processor still sent a request log using whatever was in `batch_info["name"]` (even if blank), just with a broken URL. It now validates these fields and skips the log entirely — with a `Rails.logger.error` explaining why — rather than sending something unusable (#76).
18
+ - **A batch whose `endTime` precedes its `startTime` (e.g. clock skew) is still logged**, with `duration_ms` clamped to `0` and a warning logged, rather than dropped — unlike the missing/malformed cases above, the request/response content itself is still valid here (#76).
19
+ - **A Vertex batch result item that isn't a `Hash` with a `"request"` and/or `"response"` key is now rejected and logged as an error**, instead of being sent as a log with `nil` request/response bodies (#76).
20
+ - **The published gem no longer packages `.claude/`, `.idea/`, or `CLAUDE.md`** — these are repo/agent tooling, not part of the library.
21
+
22
+ ### Security
23
+ - **A webhook processing error no longer leaves the request authenticated by default.** `Coolhand::WebhookInterceptor#intercept_batch_request` previously logged and swallowed *any* exception (a malformed webhook secret, a non-Hash JSON payload, a bug in `process_event`), returning normally without halting the Rails `before_action` chain — so the controller action still ran for a request whose signature was never confirmed valid. It now calls `head :unauthorized` in that rescue, matching the existing invalid-signature path, and validates the parsed webhook payload is a `Hash` before use.
24
+ - **A blank or whitespace-only OpenAI webhook secret is no longer usable as the HMAC key.** `Coolhand::OpenAi::WebhookValidator` previously only rejected `nil`/`false`, so `webhook_secret = ""` (e.g. an unset `ENV` var with a blank default) silently signed and verified requests with an empty key — a much weaker, easy-to-guess bypass than the already-known "secret not configured" path. It now uses the same `Coolhand.required_field?` blank-check used elsewhere in the gem.
25
+ - **`Cookie`/`Set-Cookie` headers are now redacted**, alongside the existing key/token/secret/signature/authorization patterns. Forwarding a Rails session cookie (a bearer credential) to Coolhand — e.g. via `LoggerService#forward_webhook(headers: request.headers)`, the pattern this gem's own docs show — was previously possible.
26
+ - **`BaseInterceptor.sanitize_url` now redacts a broader set of credential-shaped query parameters** (`sig`, `credential`, `password`, `auth`, in addition to the existing `key`/`token`/`secret`), matching the pattern-based approach already used for headers instead of a small exact-match list. This covers AWS SigV4 (`X-Amz-Signature`, `X-Amz-Credential`) and Google Cloud (`X-Goog-Signature`, `X-Goog-Credential`) presigned-URL parameters that were previously logged in full.
27
+ - **A failure capturing an intercepted request's body (e.g. an already-consumed `body_stream`) no longer prevents the real request from being attempted.** The capture happened before any error handling was in place, so an exception there escaped `NetHttpInterceptor#request` entirely — the host app's actual LLM call never happened. It's now rescued, logged, and treated as "body unavailable," and the real request still proceeds.
28
+ - **Streamed response bodies are no longer buffered into a thread-local for requests this gem never intercepted.** `ResponseInterceptor#read_body` previously buffered every block-form `read_body` call process-wide, regardless of whether the enclosing request matched `intercept_addresses` — an unbounded memory growth path on any thread that streams large non-LLM responses (file downloads, etc.). Buffering now only happens while a `NetHttpInterceptor#request` call is actively capturing.
29
+ - **A request made from inside another intercepted request's streaming callback no longer corrupts the outer request's logged response body.** The per-thread stream buffer is now saved and restored around each intercepted request instead of unconditionally cleared, so nested/reentrant LLM calls (e.g. a second provider call triggered from a chunk handler) no longer mix one request's content into another's log entry.
30
+ - **Coolhand's own request to its backend API is no longer re-intercepted and re-logged.** If a self-hosted `base_url` or custom `intercept_addresses` entry happens to match Coolhand's own API host, the log-shipping request itself would previously be captured and logged like any other request — which itself gets logged, recursively, without bound. It's now wrapped in `Coolhand.without_capture`.
31
+ - `Coolhand::OpenAi::WebhookValidator#webhook_secret` is no longer a public reader (it's only used internally) — reduces the chance of the raw secret ending up in error-tracker breadcrumbs or view-rendered controller state.
32
+ - `YAML.load_file` → `YAML.safe_load_file` for the gem's own bundled default-addresses/patterns YAML files — defense-in-depth against unsafe deserialization, consistent with modern Psych defaults.
33
+ - Added missing `require "openssl"` (`WebhookValidator`) and `require "stringio"` (`NetHttpInterceptor`) instead of relying on both being transitively loaded by something else first.
34
+
35
+ ### Fixed
36
+ - **Vertex batch result logging** — `Coolhand::Vertex::BatchResultProcessor` now sends a fully-qualified URL (`https://aiplatform.googleapis.com/v1/<resource name>`) instead of the bare Vertex job resource name, plus explicit `source_api` and `model` values, all inside `raw_request`. The Coolhand API treats `raw_request` as free-form JSON, so this improves the data available to its classification without depending on new top-level payload fields being accepted. Also fixed: `timestamp`/`completed_at` are now sent as proper ISO 8601 strings instead of raw `Time` objects, and the URL is now run through the same `sanitize_url` helper used by every other logging path in the gem, for consistency (a Vertex batch URL can't actually carry a redactable query string today, so this is defense-in-depth rather than a fix for a live leak) (#76).
37
+ - **`require "coolhand"` no longer requires `faraday`.** The gem hasn't used Faraday directly since the 0.4.0 unified `NetHttpInterceptor`, but a stale top-level `require "faraday"` meant loading the gem could raise `LoadError` for anyone without faraday installed — it was never a declared dependency. If your app relied on `require "coolhand"` transitively loading Faraday, require it yourself.
38
+ - `Coolhand::Vertex::BatchResultProcessor`, `Coolhand::OpenAi::BatchResultProcessor`, and `Coolhand::OpenAi::WebhookValidator` can now each be required independently (e.g. `require "coolhand/vertex/batch_result_processor"`) without first requiring `"coolhand"` — previously this could raise `NameError` depending on load order.
39
+ - `Coolhand::BaseInterceptor.send_complete_request_log` now runs `request_headers`/`response_headers` through `sanitize_headers` itself, rather than trusting every caller to have pre-sanitized them. `NetHttpInterceptor` already sanitized before calling it (`sanitize_headers` is idempotent, so no behavior change there); this closes the gap for any future caller that forgets.
40
+ - `Coolhand::Vertex::BatchResultProcessor#handle_failed_batch` no longer raises `NoMethodError` when a failed batch's `error` field is missing or not a Hash — it now logs the batch-failure message either way instead of falling through to the generic "failed to process" catch-all.
41
+ - **README's "Request Flow" section corrected** — it previously claimed data is sent to the Coolhand API "asynchronously in a background thread" with "zero performance impact"; it's actually sent inline, after the original response is available, bounded by the 5-second timeout described in 0.5.0's Security section.
42
+
10
43
  ## [0.5.0] - 2026-07-30
11
44
 
12
45
  ### Changed
data/README.md CHANGED
@@ -39,14 +39,14 @@ end
39
39
  # ✅ OpenAI SDK calls
40
40
  # ✅ Anthropic API calls
41
41
  # ✅ Direct HTTP requests to AI APIs
42
- # ✅ ANY library making AI API calls via Faraday
42
+ # ✅ Any library making AI API calls via Faraday's default adapter
43
43
 
44
44
  # NO code changes needed in your existing services!
45
45
  ```
46
46
 
47
47
  **✨ Why Automatic Monitoring:**
48
48
  - 🚫 **Zero refactoring** - No code changes to existing services
49
- - 📊 **Complete coverage** - Monitors ALL AI libraries using Faraday automatically
49
+ - 📊 **Complete coverage** - Monitors AI libraries using Net::HTTP or Faraday's default adapter automatically
50
50
  - 🔒 **Security built-in** - Automatic credential sanitization
51
51
  - ⚡ **Performance optimized** - Negligible overhead via async logging
52
52
  - 🛡️ **Future-proof** - Automatically captures new AI calls added by your team
@@ -297,19 +297,25 @@ The filtering is automatic and applies to all monitored API calls and webhook lo
297
297
 
298
298
  The monitor works with multiple transport layers and Ruby libraries:
299
299
 
300
- **Faraday-based libraries:**
300
+ **Faraday-based libraries** (via Faraday's default `net_http` adapter — a non-Net::HTTP adapter such as `faraday-typhoeus` is not covered):
301
301
  - OpenAI Ruby SDK
302
302
  - ruby-anthropic gem (community Anthropic gem)
303
303
  - ruby-openai gem
304
304
  - LangChain.rb
305
- - Direct Faraday requests
306
- - Any other Faraday-based HTTP client
305
+ - Direct Faraday requests using the default adapter
307
306
 
308
307
  **Native HTTP libraries:**
309
308
  - Official Anthropic Ruby SDK (using Net::HTTP)
310
309
  - GitHub Models SDK / any client using `models.github.ai`
311
310
  - Any library using Net::HTTP directly
312
311
 
312
+ **Other providers monitored out of the box:**
313
+ - Google Gemini (`generativelanguage.googleapis.com`)
314
+ - Google Vertex AI (`aiplatform.googleapis.com`) — see the [Vertex AI batch result logging guide](docs/vertex.md) for async batch jobs
315
+ - AWS Bedrock (OpenAI-compatible endpoint)
316
+ - Cloudflare AI Gateway
317
+ - OpenRouter
318
+
313
319
  **Universal Coverage**: Since most Ruby HTTP libraries use Net::HTTP under the hood, Coolhand's single interceptor provides comprehensive monitoring without needing library-specific integrations.
314
320
 
315
321
  ## How It Works
@@ -325,11 +331,11 @@ Coolhand uses a unified Net::HTTP interceptor to capture outgoing requests to co
325
331
  ### Request Flow
326
332
  When a request matches configured LLM endpoints:
327
333
 
328
- 1. The original request executes normally with zero performance impact
334
+ 1. The original request executes normally
329
335
  2. Request and response data (body, headers, status) are captured by the interceptor
330
336
  3. For streaming requests, the complete accumulated response is captured (not individual chunks)
331
- 4. Data is sent to the Coolhand API asynchronously in a background thread
332
- 5. Your application continues without interruption
337
+ 4. Data is sent to the Coolhand API inline, after the original response is available (bounded by a 5-second connect/read timeout so a slow or unreachable Coolhand backend can't hang your request indefinitely)
338
+ 5. Your application continues once the log is sent (or times out) — the original request/response is never blocked on Coolhand being reachable
333
339
 
334
340
  For non-matching endpoints, requests pass through unchanged.
335
341
 
@@ -395,108 +401,14 @@ The monitor handles errors gracefully:
395
401
  - Invalid API keys will be reported but won't crash your app
396
402
  - Network issues are handled with appropriate error messages
397
403
 
398
-
399
- ## Batch webhook handler (OpenAI)
400
-
401
- Automatically handle OpenAI batch event logs (batch.completed, batch.failed, batch.expired, batch.cancelled)
402
- by intercepting webhook requests and enqueuing your batch result processor.
403
-
404
- Usage:
405
- - Include the interceptor in your controller:
406
- include Coolhand::WebhookInterceptor
407
- - Add the before_action to validate and populate @validator payload:
408
- before_action :intercept_batch_request, only: :openai
409
- - Ensure you skip CSRF for the webhook endpoint:
410
- skip_before_action :verify_authenticity_token
411
- - Override the webhook_secret method to return your OpenAI webhook secret
412
-
413
- Minimal example (only key lines shown):
414
-
415
- ```ruby
416
- # app/controllers/webhooks/batch_api_requests_controller.rb
417
- # ...existing code...
418
- include Coolhand::WebhookInterceptor
419
-
420
- skip_before_action :verify_authenticity_token
421
- before_action :intercept_batch_request, only: :openai
422
-
423
- def openai
424
- event = JSON.parse(@validator.payload)
425
- case event["type"]
426
- when "batch.completed", "batch.failed", "batch.expired", "batch.cancelled"
427
- batch_id = event.dig("data", "id")
428
- batch_request = BatchApiRequest.find_by(provider: "openai", provider_batch_id: batch_id)
429
-
430
- if batch_request
431
- OpenAi::BatchResultProcessor.perform_async(batch_request.id)
432
- Rails.logger.info("Queued batch result processing for BatchApiRequest #{batch_request.id}")
433
- else
434
- Rails.logger.warn("Could not find BatchApiRequest for OpenAI batch ID: #{batch_id}")
435
- end
436
- else
437
- Rails.logger.info("Unhandled OpenAI webhook event type: #{event["type"]}")
438
- end
439
-
440
- head :ok
441
- rescue JSON::ParserError
442
- head :bad_request
443
- rescue StandardError => e
444
- Rails.logger.error("OpenAI webhook error: #{e.message}")
445
- head :internal_server_error
446
- end
447
-
448
- def webhook_secret
449
- Rails.application.credentials.openai_webhook_secret
450
- end
451
- # ...existing code...
452
- ```
453
-
454
- ## Batch webhook handler (Vertex)
455
-
456
- Automatically handle Vertex batch event logs.
457
-
458
- Usage:
459
- - call Coolhand::Vertex::BatchResultProcessor service with batch_info and download batch results
460
-
461
- Minimal example (only key lines shown):
462
-
463
- ```ruby
464
- class Vertex::BatchCallbackProcessor < BaseService
465
- option :batch_request, model: BatchApiRequest
466
- option :batch_info
467
-
468
- def call
469
- case batch_info["state"]
470
- when "JOB_STATE_PENDING"
471
- nil
472
- when "JOB_STATE_RUNNING", "JOB_STATE_QUEUED"
473
- batch_request.update!(status: "processing")
474
-
475
- Coolhand::Vertex::BatchResultProcessor.new(batch_info:).call
476
- when "JOB_STATE_SUCCEEDED"
477
- output_file_id = batch_info["outputInfo"]["gcsOutputDirectory"]
478
- results = download_batch_results(output_file_id)
479
- results.each { |batch_item| process_batch_result(batch_item) }
480
-
481
- batch_request.update!(status: "completed", completed_at: Time.current, output_file_id:)
482
-
483
- Coolhand::Vertex::BatchResultProcessor.new(batch_info:).call(results)
484
-
485
- # Clean up GCS files after successful processing
486
- cleanup_gcs_files(output_file_id)
487
- when "JOB_STATE_FAILED"
488
- handle_failed_batch(batch_info["error"]["message"])
489
- end
490
- end
491
- end
492
- ```
493
-
494
404
  ## Documentation
495
405
 
496
406
  - **[Configuration](docs/configuration.md)** — Self-hosted deployments, base_url rules, debug mode, custom intercept addresses
497
407
  - **[Feedback API](docs/feedback.md)** — Full field reference, matching strategies, sentiment values
498
408
  - **[Anthropic Integration](docs/anthropic.md)** — Official and community Anthropic Ruby gems, streaming, dual gem handling, and troubleshooting
499
409
  - **[ElevenLabs Integration](docs/elevenlabs.md)** — Webhook capture, feedback submission, and Rails integration
410
+ - **[OpenAI Batch Webhook Handler](docs/openai.md)** — Handle OpenAI batch job completion events via webhook interception
411
+ - **[Google Vertex AI Batch Result Logging](docs/vertex.md)** — Log completed Vertex AI batch prediction job results
500
412
 
501
413
  ## Security
502
414
 
data/docs/openai.md ADDED
@@ -0,0 +1,60 @@
1
+ # OpenAI Batch Webhook Handler
2
+
3
+ Automatically handle OpenAI batch event logs (`batch.completed`, `batch.failed`, `batch.expired`, `batch.cancelled`) by intercepting webhook requests and logging completed batch results to Coolhand.
4
+
5
+ For monitoring regular (non-batch) OpenAI API calls, see the [OpenAI client example](../README.md#with-openai-ruby-client) in the main README — no extra setup is required beyond `Coolhand.configure`.
6
+
7
+ Requires Rails — `Coolhand::WebhookInterceptor` and `Coolhand::OpenAi::BatchResultProcessor` log via `Rails.logger` internally. `config.capture = false` and `Coolhand.without_capture` do not suppress these logs: unlike the passive Net::HTTP interceptor, `intercept_batch_request` dispatching to `Coolhand::OpenAi::BatchResultProcessor` is an explicit, deliberate act, so it always sends.
8
+
9
+ ## Usage
10
+
11
+ - Add the `openai` gem to your Gemfile — `Coolhand::OpenAi::BatchResultProcessor` needs it to download batch result files.
12
+ - Include the interceptor in your controller: `include Coolhand::WebhookInterceptor`
13
+ - Add the `before_action` to validate and populate the `@validator` payload: `before_action :intercept_batch_request, only: :openai`
14
+ - Ensure you skip CSRF for the webhook endpoint: `skip_before_action :verify_authenticity_token`
15
+ - Override the `webhook_secret` method to return your OpenAI webhook secret
16
+
17
+ `intercept_batch_request` already validates the webhook and, for `batch.completed`/`batch.failed`/`batch.expired`/`batch.cancelled` events, calls `Coolhand::OpenAi::BatchResultProcessor` **synchronously**, inline in the `before_action` — that part requires no code in your controller action. It downloads two JSONL files from OpenAI and sends one request log per batch item, so a large batch can take long enough to trip your webhook endpoint's timeout (and OpenAI's retry behavior); consider that when sizing batches. The example below shows a controller action doing additional, app-specific work on top of that (e.g. updating your own `BatchApiRequest` record and enqueuing your own background job), which is optional.
18
+
19
+ ## Minimal example
20
+
21
+ Only the key lines are shown — wire this into your own controller and background job setup.
22
+
23
+ ```ruby
24
+ # app/controllers/webhooks/batch_api_requests_controller.rb
25
+ # ...existing code...
26
+ include Coolhand::WebhookInterceptor
27
+
28
+ skip_before_action :verify_authenticity_token
29
+ before_action :intercept_batch_request, only: :openai
30
+
31
+ def openai
32
+ event = JSON.parse(@validator.payload)
33
+ case event["type"]
34
+ when "batch.completed", "batch.failed", "batch.expired", "batch.cancelled"
35
+ batch_id = event.dig("data", "id")
36
+ batch_request = BatchApiRequest.find_by(provider: "openai", provider_batch_id: batch_id)
37
+
38
+ if batch_request
39
+ MyApp::BatchResultJob.perform_async(batch_request.id)
40
+ Rails.logger.info("Queued batch result processing for BatchApiRequest #{batch_request.id}")
41
+ else
42
+ Rails.logger.warn("Could not find BatchApiRequest for OpenAI batch ID: #{batch_id}")
43
+ end
44
+ else
45
+ Rails.logger.info("Unhandled OpenAI webhook event type: #{event["type"]}")
46
+ end
47
+
48
+ head :ok
49
+ rescue JSON::ParserError
50
+ head :bad_request
51
+ rescue StandardError => e
52
+ Rails.logger.error("OpenAI webhook error: #{e.message}")
53
+ head :internal_server_error
54
+ end
55
+
56
+ def webhook_secret
57
+ Rails.application.credentials.openai_webhook_secret
58
+ end
59
+ # ...existing code...
60
+ ```
data/docs/vertex.md ADDED
@@ -0,0 +1,54 @@
1
+ # Google Vertex AI Batch Result Logging
2
+
3
+ Log completed Google Vertex AI batch prediction job results the same way regular synchronous calls are logged. Unlike the [OpenAI batch handler](openai.md), this is not webhook-driven — you call `Coolhand::Vertex::BatchResultProcessor` directly from your own batch-job callback/polling code.
4
+
5
+ For monitoring regular (non-batch) Vertex AI calls, no extra setup is required beyond `Coolhand.configure` — `aiplatform.googleapis.com` is intercepted by default.
6
+
7
+ Requires Rails — `Coolhand::Vertex::BatchResultProcessor` logs via `Rails.logger` internally. `config.capture = false` and `Coolhand.without_capture` do not suppress these logs: unlike the passive Net::HTTP interceptor, calling this processor is an explicit, deliberate act, so it always sends.
8
+
9
+ ## Usage
10
+
11
+ ```ruby
12
+ require "coolhand/vertex/batch_result_processor"
13
+ ```
14
+
15
+ - Call the `Coolhand::Vertex::BatchResultProcessor` service with `batch_info` and the downloaded batch results.
16
+ - Optionally pass `model:` to populate the logged `model` field. It falls back to a `"model"` key in `batch_info` if present, and is omitted from the payload entirely when neither is available. If the value (from either source) looks like a Vertex model resource path (`publishers/<publisher>/models/<id>` or `projects/<project>/locations/<location>/models/<id>`), it's normalized down to just the trailing id/version before being logged; any other string is sent as-is.
17
+ - `batch_info["name"]` must be the exact Vertex job resource name (`projects/<project>/locations/<location>/batchPredictionJobs/<job-id>`), and `batch_info["startTime"]`/`["endTime"]` must be valid ISO 8601 timestamps — a batch with a missing or malformed resource name or timestamps is skipped entirely (logged as an error) rather than sent with a broken URL.
18
+ - If `endTime` precedes `startTime` (e.g. clock skew), the batch is still logged — `duration_ms` is clamped to `0` and a warning is logged — rather than dropping the results.
19
+ - Each element of the `batch_results` array passed to `.call` must be a `Hash` with a `"request"` and/or `"response"` key (the request/response body to log for that item — either key alone is accepted). Any other shape is treated as malformed and skipped, with the skip counted in a `Rails.logger.warn` summary rather than raised.
20
+
21
+ ## Minimal example
22
+
23
+ Only the key lines are shown — wire this into your own batch callback service.
24
+
25
+ ```ruby
26
+ class Vertex::BatchCallbackProcessor < BaseService
27
+ option :batch_request, model: BatchApiRequest
28
+ option :batch_info
29
+
30
+ def call
31
+ case batch_info["state"]
32
+ when "JOB_STATE_PENDING"
33
+ nil
34
+ when "JOB_STATE_RUNNING", "JOB_STATE_QUEUED"
35
+ batch_request.update!(status: "processing")
36
+
37
+ Coolhand::Vertex::BatchResultProcessor.new(batch_info:).call
38
+ when "JOB_STATE_SUCCEEDED"
39
+ output_file_id = batch_info["outputInfo"]["gcsOutputDirectory"]
40
+ results = download_batch_results(output_file_id)
41
+ results.each { |batch_item| process_batch_result(batch_item) }
42
+
43
+ batch_request.update!(status: "completed", completed_at: Time.current, output_file_id:)
44
+
45
+ Coolhand::Vertex::BatchResultProcessor.new(batch_info:, model: batch_request.llm_model).call(results)
46
+
47
+ # Clean up GCS files after successful processing
48
+ cleanup_gcs_files(output_file_id)
49
+ when "JOB_STATE_FAILED"
50
+ handle_failed_batch(batch_info["error"]["message"])
51
+ end
52
+ end
53
+ end
54
+ ```
@@ -92,7 +92,13 @@ module Coolhand
92
92
  request.body = json_body.force_encoding("UTF-8")
93
93
 
94
94
  begin
95
- response = http.request(request)
95
+ # This request goes through the same patched Net::HTTP as the host
96
+ # app's real LLM calls. Without this, a base_url/intercept_addresses
97
+ # configuration that also matches Coolhand's own API host would
98
+ # re-intercept this log-shipping call, generating a second log
99
+ # request that itself gets intercepted, and so on — unbounded
100
+ # request amplification.
101
+ response = Coolhand.without_capture { http.request(request) }
96
102
 
97
103
  if response.is_a?(Net::HTTPSuccess)
98
104
  result = JSON.parse(response.body, symbolize_names: true)
@@ -7,10 +7,11 @@ module Coolhand
7
7
 
8
8
  # Matches any header whose *name* signals sensitive content, regardless of
9
9
  # provider — covers known keys (x-api-key, x-goog-api-key, openai-api-key),
10
- # AWS SigV4 session tokens (x-amz-security-token), and future/unknown
11
- # providers using a similarly-named header. Shared with LoggerService so
12
- # the two logging paths (interceptor + webhook forwarding) stay consistent.
13
- SENSITIVE_HEADER_PATTERN = /key|token|secret|signature|authorization/i
10
+ # AWS SigV4 session tokens (x-amz-security-token), session/CSRF cookies
11
+ # (cookie, set-cookie), and future/unknown providers using a
12
+ # similarly-named header. Shared with LoggerService so the two logging
13
+ # paths (interceptor + webhook forwarding) stay consistent.
14
+ SENSITIVE_HEADER_PATTERN = /key|token|secret|signature|authorization|cookie/i
14
15
 
15
16
  def sanitize_headers(headers)
16
17
  return {} if headers.nil?
@@ -69,15 +70,22 @@ module Coolhand
69
70
  end
70
71
  end
71
72
 
73
+ # Matches query param *names* that signal sensitive content, the same way
74
+ # SENSITIVE_HEADER_PATTERN does for headers — covers exact params this
75
+ # gem already knew about (key/token/secret) as well as presigned-URL
76
+ # credential params used by AWS (X-Amz-Signature, X-Amz-Credential,
77
+ # X-Amz-Security-Token) and Google Cloud (X-Goog-Signature,
78
+ # X-Goog-Credential) storage APIs.
79
+ SENSITIVE_QUERY_PARAM_PATTERN = /key|token|secret|sig|credential|password|auth/i
80
+
72
81
  def sanitize_url(url)
73
82
  uri = URI.parse(url)
74
83
  return url unless uri.query
75
84
 
76
- sensitive = %w[key api_key apikey token access_token secret]
77
85
  params = URI.decode_www_form(uri.query)
78
86
  redacted = false
79
87
  params.map! do |n, v|
80
- if sensitive.include?(n.downcase)
88
+ if n.match?(SENSITIVE_QUERY_PARAM_PATTERN)
81
89
  redacted = true
82
90
  [n, "[REDACTED]"]
83
91
  else
@@ -96,23 +104,25 @@ module Coolhand
96
104
  end
97
105
 
98
106
  def send_complete_request_log(request_id:, method:, url:, request_headers:, request_body:, response_headers:,
99
- response_body:, status_code:, start_time:, end_time:, duration_ms:, is_streaming:)
100
- request_data = {
101
- raw_request: {
102
- id: request_id,
103
- timestamp: start_time.iso8601,
104
- method: method.to_s.downcase,
105
- url: sanitize_url(url),
106
- headers: request_headers,
107
- request_body: request_body,
108
- response_headers: response_headers,
109
- response_body: response_body,
110
- status_code: status_code,
111
- duration_ms: duration_ms,
112
- completed_at: end_time.iso8601,
113
- is_streaming: is_streaming
114
- }
107
+ response_body:, status_code:, start_time:, end_time:, duration_ms:, is_streaming:, source_api: nil, model: nil)
108
+ raw_request = {
109
+ id: request_id,
110
+ timestamp: start_time.iso8601,
111
+ method: method.to_s.downcase,
112
+ url: sanitize_url(url),
113
+ headers: sanitize_headers(request_headers),
114
+ request_body: request_body,
115
+ response_headers: sanitize_headers(response_headers),
116
+ response_body: response_body,
117
+ status_code: status_code,
118
+ duration_ms: duration_ms,
119
+ completed_at: end_time.iso8601,
120
+ is_streaming: is_streaming
115
121
  }
122
+ raw_request[:source_api] = source_api if Coolhand.required_field?(source_api)
123
+ raw_request[:model] = model if Coolhand.required_field?(model)
124
+
125
+ request_data = { raw_request: raw_request }
116
126
 
117
127
  api_service = Coolhand::ApiService.new
118
128
  api_service.send_llm_request_log(request_data)
@@ -6,11 +6,11 @@ require "uri"
6
6
  module Coolhand
7
7
  # Handles all configuration settings for the gem.
8
8
  class Configuration
9
- DEFAULT_EXCLUDE_API_PATTERNS = YAML.load_file(
9
+ DEFAULT_EXCLUDE_API_PATTERNS = YAML.safe_load_file(
10
10
  File.join(__dir__, "default_exclude_api_patterns.yml")
11
11
  ).freeze
12
12
 
13
- DEFAULT_INTERCEPT_ADDRESSES = YAML.load_file(
13
+ DEFAULT_INTERCEPT_ADDRESSES = YAML.safe_load_file(
14
14
  File.join(__dir__, "default_intercept_addresses.yml")
15
15
  ).freeze
16
16
 
@@ -1,5 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "stringio"
4
+
3
5
  module Coolhand
4
6
  module NetHttpInterceptor
5
7
  include BaseInterceptor
@@ -7,7 +9,12 @@ module Coolhand
7
9
  # Response streaming interceptor nested under NetHttpInterceptor
8
10
  module ResponseInterceptor
9
11
  def read_body(dest = nil, &block)
10
- return super unless block
12
+ # Only buffer while a #request call below is actively capturing —
13
+ # otherwise every block-form read_body in the process (including
14
+ # ones on responses this gem never intercepted) accumulates into a
15
+ # thread-local that's never freed, which is an unbounded memory
16
+ # leak on any thread that streams large non-LLM responses.
17
+ return super unless block && Thread.current[:coolhand_capturing_stream]
11
18
 
12
19
  super do |chunk|
13
20
  Thread.current[:coolhand_stream_buffer] ||= +""
@@ -49,8 +56,16 @@ module Coolhand
49
56
  return super unless should_capture?
50
57
 
51
58
  # Capture body before setting the guard — if this raises we skip logging cleanly
52
- # and the guard is never set, so there is no leak.
53
- captured_body = capture_request_body(req, body)
59
+ # and the guard is never set, so there is no leak. A failure here (e.g. an
60
+ # already-consumed body_stream) must never prevent the real request below
61
+ # from being attempted — this gem must never be the reason the host
62
+ # app's actual LLM call doesn't happen.
63
+ captured_body = begin
64
+ capture_request_body(req, body)
65
+ rescue StandardError => e
66
+ Coolhand.log "❌ Error capturing request body: #{e.message}"
67
+ nil
68
+ end
54
69
 
55
70
  active[self] = true
56
71
  start_time = Time.now
@@ -59,7 +74,14 @@ module Coolhand
59
74
  status_code = nil
60
75
  response_body = nil
61
76
 
77
+ # Save/restore rather than just nil-ing: a request made from inside
78
+ # this request's own streaming block (nested interception) would
79
+ # otherwise clobber this request's in-progress buffer with its own
80
+ # chunks, mixing one request's content into another's log.
81
+ previous_stream_buffer = Thread.current[:coolhand_stream_buffer]
82
+ previous_capturing_stream = Thread.current[:coolhand_capturing_stream]
62
83
  Thread.current[:coolhand_stream_buffer] = nil
84
+ Thread.current[:coolhand_capturing_stream] = true
63
85
 
64
86
  begin
65
87
  response = super
@@ -73,7 +95,8 @@ module Coolhand
73
95
  raise
74
96
  ensure
75
97
  active.delete(self)
76
- Thread.current[:coolhand_stream_buffer] = nil
98
+ Thread.current[:coolhand_stream_buffer] = previous_stream_buffer
99
+ Thread.current[:coolhand_capturing_stream] = previous_capturing_stream
77
100
  end_time = Time.now
78
101
  duration_ms = ((end_time - start_time) * 1000).round(2)
79
102
 
@@ -1,5 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative "../../coolhand"
4
+
3
5
  module Coolhand
4
6
  module OpenAi
5
7
  class BatchResultProcessor
@@ -1,9 +1,11 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "openssl"
4
+
3
5
  module Coolhand
4
6
  module OpenAi
5
7
  class WebhookValidator
6
- attr_reader :request, :errors, :payload, :webhook_secret
8
+ attr_reader :request, :errors, :payload
7
9
 
8
10
  def initialize(request, webhook_secret)
9
11
  @request = request
@@ -16,7 +18,7 @@ module Coolhand
16
18
  @payload = request.raw_post || request.body.read
17
19
 
18
20
  return false unless payload_valid?
19
- return validate_in_non_production_env unless webhook_secret
21
+ return validate_in_non_production_env unless Coolhand.required_field?(webhook_secret)
20
22
 
21
23
  secret_bytes = extract_secret_bytes
22
24
  webhook_signature, webhook_timestamp, webhook_id = extract_webhook_headers
@@ -32,6 +34,8 @@ module Coolhand
32
34
 
33
35
  private
34
36
 
37
+ attr_reader :webhook_secret
38
+
35
39
  def payload_valid?
36
40
  return true if @payload
37
41
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Coolhand
4
- VERSION = "0.5.0"
4
+ VERSION = "0.5.1"
5
5
  end
@@ -1,12 +1,27 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative "../../coolhand"
4
+
3
5
  module Coolhand
4
6
  module Vertex
5
7
  class BatchResultProcessor
6
- attr_reader :batch_info
8
+ # Global endpoint, not the job's region-specific one — matches the
9
+ # "aiplatform.googleapis.com" entry in default_intercept_addresses.yml
10
+ # so backend URL-shape classification stays consistent with the rest
11
+ # of the gem's Vertex traffic. The URL this produces always contains
12
+ # "/batchPredictionJobs/", which is also the default client-side
13
+ # exclude_api_patterns entry — that's harmless here since this class
14
+ # sends via BaseInterceptor/ApiService directly and never goes through
15
+ # NetHttpInterceptor's intercept/exclude filtering.
16
+ VERTEX_API_BASE_URL = "https://aiplatform.googleapis.com/v1/"
17
+ SOURCE_API = "vertex"
18
+ VALID_NAME_PATTERN = %r{\Aprojects/[^/?#\s]+/locations/[^/?#\s]+/batchPredictionJobs/[^/?#\s]+\z}
19
+
20
+ attr_reader :batch_info, :model
7
21
 
8
- def initialize(batch_info:)
22
+ def initialize(batch_info:, model: nil)
9
23
  @batch_info = batch_info
24
+ @model = model
10
25
  end
11
26
 
12
27
  def call(batch_results = [])
@@ -16,7 +31,7 @@ module Coolhand
16
31
  when "JOB_STATE_PENDING", "JOB_STATE_RUNNING", "JOB_STATE_QUEUED"
17
32
  Rails.logger.info("[Interceptor] Vertex batch #{batch_info} still processing")
18
33
  when "JOB_STATE_SUCCEEDED"
19
- batch_results.each { |batch_item| process_completed_batch(batch_item) }
34
+ process_completed_batch(batch_results)
20
35
  when "JOB_STATE_FAILED"
21
36
  handle_failed_batch
22
37
  else
@@ -28,56 +43,98 @@ module Coolhand
28
43
 
29
44
  private
30
45
 
31
- def process_completed_batch(batch_item)
32
- send_complete_request_log(request_id: SecureRandom.hex(16),
46
+ def process_completed_batch(batch_results)
47
+ name = batch_info["name"].to_s.sub(%r{\A/+}, "")
48
+ unless name.match?(VALID_NAME_PATTERN)
49
+ Rails.logger.error("[Interceptor] Vertex batch #{batch_info['displayName']} has a missing or " \
50
+ "invalid job resource name (#{batch_info['name'].inspect}); skipping request log")
51
+ return
52
+ end
53
+
54
+ begin
55
+ start_time = Time.iso8601(batch_info["startTime"])
56
+ end_time = Time.iso8601(batch_info["endTime"])
57
+ rescue TypeError, ArgumentError
58
+ Rails.logger.error("[Interceptor] Vertex batch #{batch_info['displayName']} has a missing or " \
59
+ "invalid startTime/endTime; skipping request log")
60
+ return
61
+ end
62
+
63
+ if end_time < start_time
64
+ Rails.logger.warn("[Interceptor] Vertex batch #{batch_info['displayName']} has an endTime before " \
65
+ "its startTime; sending request log(s) with duration_ms clamped to 0")
66
+ end_time = start_time
67
+ end
68
+
69
+ duration_ms = ((end_time - start_time) * 1000).to_i
70
+ url = "#{VERTEX_API_BASE_URL}#{name}"
71
+ resolved = resolved_model
72
+
73
+ sent = batch_results.count do |batch_item|
74
+ send_item_log(batch_item, url: url, model: resolved, start_time: start_time, end_time: end_time,
75
+ duration_ms: duration_ms)
76
+ end
77
+
78
+ # "Sent" here means dispatched to BaseInterceptor without a local
79
+ # error (e.g. a malformed batch_item) — BaseInterceptor swallows any
80
+ # downstream delivery failure itself, so this can't confirm the
81
+ # Coolhand API actually received the log.
82
+ if sent == batch_results.size
83
+ Rails.logger.info("[Interceptor] Dispatched #{sent} result(s) for Vertex batch " \
84
+ "#{batch_info['displayName']} for logging")
85
+ else
86
+ Rails.logger.warn("[Interceptor] Dispatched #{sent}/#{batch_results.size} result(s) for Vertex " \
87
+ "batch #{batch_info['displayName']} for logging — " \
88
+ "#{batch_results.size - sent} item(s) were malformed and skipped")
89
+ end
90
+ end
91
+
92
+ def send_item_log(batch_item, url:, model:, start_time:, end_time:, duration_ms:)
93
+ unless batch_item.is_a?(Hash) && (batch_item.key?("request") || batch_item.key?("response"))
94
+ Rails.logger.error("[Interceptor] Vertex batch #{batch_info['displayName']} has a malformed result " \
95
+ "item (#{batch_item.class}); skipping request log")
96
+ return false
97
+ end
98
+
99
+ BaseInterceptor.send_complete_request_log(
100
+ request_id: SecureRandom.hex(16),
33
101
  method: "POST",
34
- url: batch_info["name"],
102
+ url: url,
103
+ source_api: SOURCE_API,
104
+ model: model,
105
+ request_headers: {},
35
106
  request_body: batch_item["request"],
107
+ response_headers: {},
36
108
  response_body: batch_item["response"],
37
109
  status_code: 200,
38
- start_time: batch_info["startTime"],
39
- end_time: batch_info["endTime"])
40
-
41
- Rails.logger.info("[Interceptor] Successfully processed Vertex batch #{batch_info['displayName']}")
110
+ start_time: start_time,
111
+ end_time: end_time,
112
+ duration_ms: duration_ms,
113
+ is_streaming: false
114
+ )
115
+ true
42
116
  rescue StandardError => e
43
117
  Rails.logger.error("[Interceptor] Failed to send request log: #{e.message}")
118
+ false
44
119
  end
45
120
 
46
- def send_complete_request_log(request_id:, method:, url:, request_body:, response_body:, status_code:,
47
- start_time:, end_time:)
48
- start_time = Time.iso8601(start_time)
49
- end_time = Time.iso8601(end_time)
50
- duration_ms = ((end_time - start_time) * 1000).to_i
121
+ def resolved_model
122
+ candidate = [model, batch_info["model"]].find { |c| Coolhand.required_field?(c) }&.to_s&.strip
123
+ return if candidate.nil?
51
124
 
52
- request_data = {
53
- raw_request: {
54
- id: request_id,
55
- timestamp: start_time,
56
- method: method.to_s.downcase,
57
- url: url,
58
- headers: {},
59
- request_body: request_body,
60
- response_headers: {},
61
- response_body: response_body,
62
- status_code: status_code,
63
- duration_ms: duration_ms,
64
- completed_at: end_time,
65
- is_streaming: false
66
- }
67
- }
68
-
69
- api_service = Coolhand::ApiService.new
70
- api_service.send_llm_request_log(request_data)
71
-
72
- Coolhand.log "📤 Sent complete request/response log for #{request_id} (duration: #{duration_ms}ms)"
73
- rescue StandardError => e
74
- Coolhand.log "❌ Error sending complete request log: #{e.message}"
125
+ # Only normalize genuine Vertex model resource paths (e.g.
126
+ # "publishers/google/models/gemini-2.0-flash" or
127
+ # "projects/P/locations/L/models/M@1") down to their bare id. A
128
+ # caller-supplied `model:` might itself be a provider-qualified slug
129
+ # that happens to contain a slash (e.g. "meta-llama/Llama-3") —
130
+ # leave anything that isn't a resource path untouched.
131
+ candidate.match?(%r{\A(publishers|projects)/}) ? candidate.split("/").last : candidate
75
132
  end
76
133
 
77
134
  # TODO: implement API to handle failed batch results and display errors on dashboard page
78
135
  def handle_failed_batch
79
- Rails.logger.error("[Interceptor] Vertex batch for #{batch_info['displayName']} " \
80
- "failed: #{batch_info['error']['message']}")
136
+ message = batch_info["error"].is_a?(Hash) ? batch_info["error"]["message"] : batch_info["error"]
137
+ Rails.logger.error("[Interceptor] Vertex batch for #{batch_info['displayName']} failed: #{message}")
81
138
  end
82
139
  end
83
140
  end
@@ -1,5 +1,8 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative "open_ai/webhook_validator"
4
+ require_relative "open_ai/batch_result_processor"
5
+
3
6
  module Coolhand
4
7
  module WebhookInterceptor
5
8
  def intercept_batch_request
@@ -14,10 +17,18 @@ module Coolhand
14
17
  end
15
18
 
16
19
  payload = JSON.parse(@validator.payload)
20
+ raise TypeError, "webhook payload must be a JSON object, got #{payload.class}" unless payload.is_a?(Hash)
17
21
 
18
22
  process_event(payload)
19
23
  rescue StandardError => e
24
+ # Fail closed: any error here (malformed payload, a bug in
25
+ # process_event/BatchResultProcessor, etc.) must still halt the
26
+ # before_action chain. Falling through without calling `head` would
27
+ # let the controller action run for a request whose webhook
28
+ # signature was never confirmed valid.
20
29
  Rails.logger.error("[Interceptor] Failed to intercept batch request: #{e.message}")
30
+ head :unauthorized
31
+ false
21
32
  end
22
33
 
23
34
  def webhook_secret
data/lib/coolhand.rb CHANGED
@@ -1,10 +1,10 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "uri"
4
- require "faraday"
5
4
  require "securerandom"
6
5
  require "json"
7
6
  require "base64"
7
+ require "time"
8
8
 
9
9
  require_relative "coolhand/version"
10
10
  require_relative "coolhand/configuration"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: coolhand
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.5.0
4
+ version: 0.5.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Michael Carroll
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: exe
11
11
  cert_chain: []
12
- date: 2026-07-31 00:00:00.000000000 Z
12
+ date: 2026-08-02 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: base64
@@ -26,23 +26,20 @@ dependencies:
26
26
  - !ruby/object:Gem::Version
27
27
  version: '0.2'
28
28
  description: Automatically intercept and log LLM requests from Ruby applications.
29
- Supports OpenAI, official Anthropic gem, ruby-anthropic gem, and other Faraday-based
30
- libraries. Features dual interceptor architecture, streaming support, thread-safe
31
- operation, and automatic duplicate request prevention.
29
+ Supports OpenAI, official Anthropic gem, ruby-anthropic gem, Google Gemini, Google
30
+ Vertex AI, AWS Bedrock, OpenRouter, and any other library using Net::HTTP (directly,
31
+ or via Faraday's default adapter). Features a single unified Net::HTTP interceptor,
32
+ streaming support, thread-safe operation, and automatic duplicate request prevention.
32
33
  email:
33
34
  - mc@coolhandlabs.com
34
35
  executables: []
35
36
  extensions: []
36
37
  extra_rdoc_files: []
37
38
  files:
38
- - ".claude/skills/loop-review/SKILL.md"
39
- - ".claude/skills/prep-release/SKILL.md"
40
- - ".idea/coolhand-ruby.iml"
41
39
  - ".rspec"
42
40
  - ".rubocop.yml"
43
41
  - ".simplecov"
44
42
  - CHANGELOG.md
45
- - CLAUDE.md
46
43
  - LICENSE
47
44
  - README.md
48
45
  - Rakefile
@@ -51,6 +48,8 @@ files:
51
48
  - docs/configuration.md
52
49
  - docs/elevenlabs.md
53
50
  - docs/feedback.md
51
+ - docs/openai.md
52
+ - docs/vertex.md
54
53
  - lib/coolhand.rb
55
54
  - lib/coolhand/api_service.rb
56
55
  - lib/coolhand/base_interceptor.rb
@@ -1,112 +0,0 @@
1
- ---
2
- name: loop-review
3
- description: |
4
- Iteratively runs code review against the current diff, applies fixes, and
5
- re-reviews until a round comes back clean (or a safety cap is hit). Use
6
- when the user types /loop-review, asks to "loop the review", "review
7
- until clean", "keep reviewing and fixing until nothing's left", or wants
8
- a self-healing code review cycle instead of a single one-shot pass.
9
- user_invocable: true
10
- version: 0.3.0
11
- ---
12
-
13
- # Loop Review
14
-
15
- This skill runs `/code-review` repeatedly against the working diff, applies
16
- fixes between rounds, and re-reviews until a round finds nothing new or a
17
- safety cap is reached. Use it after a merge, a large diff, or whenever a
18
- single review pass isn't enough to converge on a clean state.
19
-
20
- ## Scope
21
-
22
- Default scope is the diff between the current branch and its merge-base
23
- with the repo's base branch (`git diff $(git merge-base origin/main
24
- HEAD)...HEAD`, or `origin/main` substituted with whatever base branch
25
- applies). If `$ARGUMENTS` names a path or a narrower scope, review only
26
- that instead of the full diff.
27
-
28
- This skill is deliberately diff-scoped. For a whole-package audit before a
29
- release (full-codebase security red-team, docs review, everything since
30
- the last tag) use `/prep-release` instead.
31
-
32
- `$ARGUMENTS` may also contain:
33
- - An effort level to pass through to `/code-review` (`low`/`medium`/
34
- `high`/`high→max`/`ultra`). Default: `medium`.
35
- - A round cap override, e.g. `--max-rounds 3`. Default: 5.
36
-
37
- ## The round loop
38
-
39
- 1. Invoke `/code-review <effort> --fix` (via the `Skill` tool) against the
40
- current scope.
41
- 2. **Dry round (0 findings) → converged.** Stop and move to verification.
42
- 3. **Findings found and fixed** → do not declare victory yet. Run another
43
- round to confirm the fixes didn't introduce a regression and that
44
- nothing was missed.
45
- 4. **No-progress detection**: if two consecutive rounds return the same
46
- non-empty set of findings, `--fix` isn't resolving them mechanically
47
- (likely a design/architecture call that needs a human). Stop looping,
48
- list the stuck findings, and hand them to the user instead of retrying
49
- forever.
50
- 5. **Safety cap**: if the round cap is reached without converging or
51
- getting stuck, stop and report the remaining findings — don't loop
52
- silently past the cap.
53
-
54
- Each round's fixes should stay reviewable: don't squash multiple rounds
55
- into one silent edit. Note per-round changes in the final summary so the
56
- user can inspect them with `git diff`.
57
-
58
- ## Review criteria
59
-
60
- Beyond whatever `/code-review` already checks for correctness bugs and
61
- reuse/simplification/efficiency, every round in a Ruby gem repo like this
62
- one should also flag:
63
-
64
- - **Ruby idiom / DRY**: semantic, expressive naming; no duplicated logic
65
- that should be extracted into a shared method; idiomatic use of
66
- Ruby/Enumerable over manual loops where it reads better; no needless
67
- boilerplate.
68
- - **Gem publishing discipline**: don't break public interfaces unless
69
- necessary.
70
- - If a break is necessary, it must come with: a `CHANGELOG.md` entry in
71
- Keep a Changelog format (this repo already follows that format — match
72
- the style of existing entries, e.g. plain-English migration notes like
73
- the `0.5.0` entry) and a version bump in `lib/coolhand/version.rb` that
74
- matches SemVer (patch = fix, minor = backward-compatible addition,
75
- minor = breaking change while pre-1.0, consistent with this repo's own
76
- versioning history).
77
- - Check the optional-provider-dependency rule from this repo's
78
- `CLAUDE.md`: provider SDK `require`s (`openai`, `anthropic`,
79
- `google-generativeai`, etc.) must stay scoped to the file that uses
80
- them and lazy-loaded, never added to `lib/coolhand.rb` or the gemspec
81
- as a hard dependency.
82
- - **Security**: injection risks, unsafe deserialization, secrets or
83
- credentials logged or committed, unvalidated input crossing a trust
84
- boundary, and anything touching how API keys/tokens are handled,
85
- stored, or transmitted.
86
-
87
- ## Post-loop verification
88
-
89
- Once the loop converges (or stops early per the rules above), run the
90
- project's lint/test command and include the result in the final summary.
91
- For this repo that's `bundle exec rake` (runs `rspec` then `rubocop`, per
92
- the `Rakefile`). If a fix round changed something outside this repo's
93
- usual toolchain, discover the right command instead of assuming.
94
-
95
- ## Rationalizations to resist
96
-
97
- - *"The first round already looked clean, I don't need a confirming
98
- round."* A fix round can introduce its own regression. Always re-review
99
- after applying fixes before declaring convergence.
100
- - *"Rubocop passed, so the review is done."* Lint passing is not the same
101
- as the review being clean — lint doesn't check the criteria above
102
- (interface breakage, changelog/version discipline, security). Run both.
103
- - *"This finding keeps coming back, I'll just keep re-running --fix and
104
- it'll eventually take."* If the same non-empty finding set repeats
105
- across two rounds, `--fix` isn't going to resolve it. Stop and surface
106
- it — looping past that point just burns rounds for no gain.
107
-
108
- ## Safety
109
-
110
- - Never force-push or amend existing commits as part of this loop.
111
- - The skill only edits the working tree; committing and pushing stays with
112
- the user.
@@ -1,160 +0,0 @@
1
- ---
2
- name: prep-release
3
- description: |
4
- Prepares this gem for a release: runs the full test suite, updates and
5
- cleans up the docs (README, CHANGELOG, docs/*.md) to reflect every change
6
- since the last tag, bumps the version if that hasn't already been done,
7
- and red-teams the whole package for security - not just this release's
8
- diff. Never tags or pushes. Use when the user types /prep-release, asks
9
- to "prep a release", "get ready to cut a release", "release checklist",
10
- or wants a pre-release audit before tagging/publishing a new version.
11
- user_invocable: true
12
- version: 0.2.0
13
- ---
14
-
15
- # Prep Release
16
-
17
- Three phases, run in order. This is a whole-package audit, not a diff
18
- review — do not scope any phase to just what changed since the last
19
- commit. For an iterative diff-scoped review during normal development, use
20
- `/loop-review` instead; this skill is for the release boundary.
21
-
22
- ## Phase 1: Run all tests
23
-
24
- Run `bundle exec rake` (`rspec` then `rubocop`, per the `Rakefile`). If a
25
- fix round changed something outside this repo's usual toolchain, discover
26
- the right command instead of assuming. All specs must pass and RuboCop
27
- must report zero offenses before continuing — a release doesn't ship on a
28
- red build. If either fails, stop here and report the failures; fixing
29
- genuine bugs takes priority over Phase 2/3 work.
30
-
31
- Then judge coverage on quality, not just the SimpleCov percentage the rake
32
- run reports (written to `coverage/`):
33
-
34
- 1. **Find the gaps.** List files/lines SimpleCov marks uncovered. Weight
35
- by risk: an uncovered error-handling branch or security check
36
- (signature/header validation) matters more than an uncovered
37
- `attr_reader`.
38
- 2. **Audit existing tests for meaningfulness, not just count.** Flag tests
39
- that only assert a stub returns what it was stubbed to return without
40
- exercising real conditional logic in the subject under test; missing
41
- negative/error-path cases (invalid input, malformed provider responses,
42
- network failure); missing domain edge cases (empty batch results,
43
- duplicate-request prevention, concurrent access, streaming vs
44
- non-streaming shapes).
45
- 3. **Recommend, don't pad.** Propose specific specs for the highest-risk
46
- gaps, named by `file:describe/context`. Don't add tests purely to move
47
- the percentage — a test with no failure mode it would catch adds
48
- maintenance cost without adding signal.
49
-
50
- ## Phase 2: Update and clean the docs
51
-
52
- 1. Find the last release tag: `git describe --tags --abbrev=0`.
53
- 2. Diff everything since that tag: `git log <last-tag>..HEAD --oneline` and
54
- `git diff <last-tag>..HEAD -- lib/` to see every behavioral change, not
55
- just the most recent commit.
56
- 3. For each change, check it's reflected in:
57
- - `CHANGELOG.md` — every notable change since the last tag needs an
58
- entry under `[Unreleased]` (or a new version heading), in Keep a
59
- Changelog format matching this repo's existing entries (see past
60
- entries for style — plain-English migration notes for anything
61
- behavior-affecting).
62
- - `README.md` / `docs/*.md` — any new config option, public method, or
63
- behavior change needs the relevant section updated. Follow this
64
- repo's docs philosophy from `CLAUDE.md`: the README stays a scannable
65
- landing page (basic config/feedback snippets only); anything needing
66
- more than one code block belongs in `docs/`.
67
- 4. **Clean, don't just append.** Look for docs that are now stale,
68
- contradictory, or redundant given the accumulated changes since the
69
- last tag — consolidate/rewrite rather than layering a new paragraph on
70
- top of an outdated one. Remove docs for anything removed from the gem.
71
- 5. **Bump the version if it hasn't already been done.** Check whether
72
- `lib/coolhand/version.rb` was already bumped for the changes
73
- accumulated since the last tag (e.g. by an earlier commit on this
74
- branch) — if so, leave it. If not, determine the SemVer bump this
75
- repo's convention implies (patch = fix, minor = backward-compatible
76
- addition or breaking change while pre-1.0), write it to
77
- `lib/coolhand/version.rb`, turn the `[Unreleased]` CHANGELOG heading
78
- into `## [X.Y.Z] - <today's date>`, and run `bundle install` so
79
- `Gemfile.lock`'s `coolhand (X.Y.Z)` line matches. State the version and
80
- bump rationale in the wrap-up summary so the user can override it if
81
- they'd have picked differently — don't ask before writing it, since
82
- this is a mechanical, reversible edit gated by Phase 1's green build.
83
-
84
- ## Phase 3: Red-team the whole package
85
-
86
- Adversarially review the entire `lib/` tree (not just this release's
87
- diff) for security issues. This gem intercepts outgoing LLM API traffic
88
- and logs it to Coolhand, so hunt specifically for:
89
-
90
- - **Credential/secret leakage**: does any interceptor, logger, or error
91
- handler write an API key, bearer token, or provider auth header value
92
- into a log line, exception message, or the payload sent to Coolhand?
93
- Check every header-sanitization path actually strips what it claims to
94
- (e.g. `WebhookValidator`, provider header redaction) rather than
95
- sanitizing a differently-cased or differently-named header.
96
- - **Webhook/signature validation**: can `WebhookValidator#valid?` (or
97
- equivalent) be bypassed — timing-unsafe comparison instead of a
98
- constant-time compare, an environment where an empty/missing signature
99
- is treated as valid, or a fallback path meant for development that's
100
- reachable in production.
101
- - **SSRF / address matching**: the default and configurable intercept
102
- address lists — can a crafted URL (redirect, unicode homograph,
103
- userinfo trick, subdomain confusion) match or evade the intended
104
- host-matching logic in a way that intercepts (or fails to intercept)
105
- the wrong destination?
106
- - **ReDoS**: any regex built from configurable or user-influenced input
107
- (intercept patterns, header names) — check for catastrophic backtracking
108
- shapes (nested quantifiers, overlapping alternation).
109
- - **Thread safety**: this gem documents thread-safe operation and
110
- duplicate-request prevention — look for unsynchronized shared mutable
111
- state (class-level `@@` vars, memoized `@client` on a shared instance)
112
- that a concurrent request could race on.
113
- - **Unsafe deserialization**: any `JSON.parse` without checking for
114
- `Marshal.load`/`YAML.load` (unsafe) usage, and any parsing of
115
- webhook/batch-result payloads that trusts attacker-controlled shape
116
- without validation.
117
- - **Fail-open vs fail-closed**: when Coolhand's API is unreachable, rate
118
- limited, or returns malformed data, does the gem fail open in a way that
119
- silently drops security-relevant logging, or fail in a way that breaks
120
- the host application's actual LLM call (the interceptor must never break
121
- the underlying request)?
122
-
123
- For each finding, report file, line, a concrete failure scenario, and
124
- severity. Apply safe, mechanical, low-risk fixes directly (e.g. a missing
125
- header-redaction pattern, a missing timeout). Flag but do not silently
126
- apply anything that's a behavior/architecture decision (e.g. changing a
127
- fail-open security default, adding replay protection, moving synchronous
128
- work to a background thread) — surface these to the user for a decision,
129
- the same "hand it to a human" rule `/loop-review` uses for stuck findings.
130
-
131
- ## Wrap-up
132
-
133
- Report one consolidated summary: test/lint result, coverage-quality gaps
134
- plus recommended specs, docs updated, the version (bumped or already
135
- current, and why), and security findings split into fixed vs.
136
- flagged-for-decision.
137
-
138
- ## Safety
139
-
140
- - Bumping `lib/coolhand/version.rb`, finalizing the CHANGELOG heading, and
141
- running `bundle install` for the lockfile are all in scope and don't need
142
- a stop-and-ask — they're mechanical, reversible, and gated on Phase 1
143
- already being green.
144
- - Never create or push a git tag, never push commits, and never run
145
- `rake release`, `gem push`, or anything else that publishes the gem or
146
- touches the remote. Tagging and publishing are the user's action once
147
- they've reviewed this skill's report, not something this skill does.
148
-
149
- ## Rationalizations to resist
150
-
151
- - *"The diff since the last tag is small, I'll skip the red-team."* Small
152
- diffs can still sit on top of latent issues in code nobody's touched
153
- recently — that's exactly what "whole package, not just the diff" means.
154
- - *"Tests pass, so coverage is fine."* Passing tests and meaningful
155
- coverage are different questions. A red build blocks release; a green
156
- build with hollow tests doesn't guarantee anything.
157
- - *"Docs are close enough, I'll skip the cleanup pass."* Accumulated
158
- changes since the last tag are exactly when docs drift from behavior —
159
- this phase exists because per-PR doc updates miss the cross-cutting
160
- view.
@@ -1,6 +0,0 @@
1
- <?xml version="1.0" encoding="UTF-8"?>
2
- <module version="4">
3
- <component name="ModuleRunConfigurationManager">
4
- <shared />
5
- </component>
6
- </module>
data/CLAUDE.md DELETED
@@ -1,47 +0,0 @@
1
- # Development Guidelines
2
-
3
- ## Optional Provider Dependencies
4
-
5
- Coolhand supports multiple LLM providers (OpenAI, Anthropic, Google Gemini, etc.). These provider gems should **never** be required at gem load time, as clients may not use all providers and shouldn't be forced to install unnecessary dependencies.
6
-
7
- **Rule**: Any require for provider SDKs (openai, anthropic, google-generativeai, etc.) must be:
8
- 1. Placed in the file where it's actually used (not in the main coolhand.rb)
9
- 2. Only executed when that provider's functionality is accessed
10
- 3. Not declared as a hard dependency in coolhand-ruby.gemspec
11
-
12
- Example pattern:
13
- ```ruby
14
- # ❌ DON'T: In lib/coolhand.rb (loads unconditionally)
15
- require "openai"
16
-
17
- # ✅ DO: In lib/coolhand/open_ai/batch_result_processor.rb (only when needed)
18
- require "openai"
19
-
20
- module Coolhand
21
- module OpenAi
22
- class BatchResultProcessor
23
- def client
24
- @client ||= OpenAI::Client.new
25
- end
26
- end
27
- end
28
- end
29
- ```
30
-
31
- This ensures:
32
- - Gem loads cleanly regardless of what providers are installed
33
- - Apps using path gems (local development) don't break from missing optional dependencies
34
- - Users only need gems for providers they actually use
35
-
36
- ## README and docs philosophy
37
-
38
- The README is a landing page — install, quick start, what it supports, where to go next. Keep it scannable. When in doubt, link rather than expand.
39
-
40
- **Three rules:**
41
- - **Config**: the basic `Coolhand.configure` snippet belongs in the README. Anything requiring more than one code block (self-hosted `base_url`, custom intercept addresses) goes in `docs/configuration.md`.
42
- - **Feedback**: the basic `create_feedback` snippet belongs in the README. The full field table, matching strategies, and sentiment conversion details go in `docs/feedback.md`.
43
- - **Integrations**: each integration gets its own `docs/<name>.md` file. The README links to them from the Documentation section.
44
-
45
- **Align with coolhand-node.** When adding a section that exists in the Node README, match its structure and tone. The two READMEs should feel like siblings.
46
-
47
- **Discoverability (SEO / AEO).** Write headings, the package description, and the supported-libraries list with search engines and AI agents in mind: use full provider/framework names (e.g. "OpenAI", "Anthropic", "Google Gemini", "Cohere") rather than abbreviations. The goal is that searches for "Ruby LLM monitoring", "Anthropic Ruby logging", or "OpenAI Ruby observability" surface this gem.