coolhand 0.10.0 → 0.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +32 -0
- data/README.md +10 -8
- data/docs/anthropic.md +28 -83
- data/docs/configuration.md +31 -5
- data/docs/elevenlabs.md +1 -1
- data/docs/help-center.md +4 -4
- data/docs/openai.md +63 -4
- data/docs/vertex.md +1 -0
- data/lib/coolhand/base_interceptor.rb +137 -37
- data/lib/coolhand/batch_body_cap.rb +36 -0
- data/lib/coolhand/body_redactor.rb +111 -0
- data/lib/coolhand/body_stream_tap.rb +121 -0
- data/lib/coolhand/default_intercept_addresses.yml +8 -0
- data/lib/coolhand/{public_content_service.rb → help_center_service.rb} +1 -1
- data/lib/coolhand/logger_service.rb +54 -5
- data/lib/coolhand/net_http_interceptor.rb +190 -28
- data/lib/coolhand/open_ai/batch_result_processor.rb +139 -53
- data/lib/coolhand/open_ai/webhook_id_store.rb +7 -0
- data/lib/coolhand/open_ai/webhook_validator.rb +43 -28
- data/lib/coolhand/path_normalizer.rb +36 -0
- data/lib/coolhand/string_body_redactor.rb +83 -0
- data/lib/coolhand/version.rb +1 -1
- data/lib/coolhand/vertex/batch_result_processor.rb +2 -2
- data/lib/coolhand/webhook_interceptor.rb +23 -0
- data/lib/coolhand.rb +7 -4
- metadata +8 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 822e11c64b550d4fda8ddd5488b4fe69a85d31dc36b52e71f2949823000b663a
|
|
4
|
+
data.tar.gz: 9edf58347f126a0a3211a426999b78f74a08bd48420a3aef4403798c5d74bb66
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e4e6d24e991a4fe5886acd1f525b62d8af122b06f048e637b0918086575f063a10a3fe3d4e8d69147c2ca37635993673787c39f8cff4a7b41897d8b0aaef9c1e
|
|
7
|
+
data.tar.gz: 93515c9f280274c9fecbf6152b135c069efc6246b15b8a73e91181a756c0c8d73e2ef26713763665b363a74340babb1dabe685e62ef36167c1ee1443018baa1e
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.10.1] - 2026-10-26
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
- **Amazon Bedrock FIPS, dual-stack and China-partition endpoints are now captured** ([#168](https://github.com/Coolhand-Labs/coolhand-ruby/pull/168)). The default intercept list held only `bedrock-runtime.*.amazonaws.com`, so calls to the FIPS hosts (`bedrock-runtime-fips.<region>.amazonaws.com`, the common case for GovCloud callers), the dual-stack IPv6 hosts (`bedrock-runtime.<region>.api.aws` and the FIPS variant) and the China partition (`bedrock-runtime.<region>.amazonaws.com.cn`) were silently not captured. A wildcard entry is an exact host match with no implicit subdomains, so each variant needs its own entry.
|
|
14
|
+
- Credentials in request and response bodies are redacted ([#178](https://github.com/Coolhand-Labs/coolhand-ruby/pull/178)). String values under `authorization`, `authorization_token`, `access_token`, `refresh_token`, `id_token`, `api_key`, `client_secret`, `private_key`, `secret_key` and `password` (any depth, matched by exact name) become `[REDACTED]`, as do the string values in the `headers` of an MCP tool and the ephemeral key returned by a Realtime `client_secret` response. `response_body` is now redacted too, not just `request_body`. **Migration note:** logged bodies that carry such fields now show `[REDACTED]` in Coolhand. See [Body Redaction](docs/configuration.md).
|
|
15
|
+
- `max_captured_body_bytes` now also caps **response** bodies ([#177](https://github.com/Coolhand-Labs/coolhand-ruby/pull/177)). A response larger than the cap (default 1 MB) is logged as a `{"_coolhand_capture_skipped": "body_too_large", ...}` marker instead of in full; your app always receives the complete response. A `body_stream` request that declares a `Content-Length` over the cap is no longer read for capture, and a non-Integer or negative cap falls back to the default instead of raising. See [Body Capture Limits](docs/configuration.md).
|
|
16
|
+
- `WebhookIdStore#release!(id)` and `WebhookValidator#release_claim!` ([#176](https://github.com/Coolhand-Labs/coolhand-ruby/pull/176)). A custom `webhook_id_store` may optionally implement `release!(id)`; stores that only implement `claim!` keep working unchanged.
|
|
17
|
+
- Batch results logged by `OpenAi::BatchResultProcessor` now carry `source_api: "openai"` and the request's `model` ([#171](https://github.com/Coolhand-Labs/coolhand-ruby/pull/171)).
|
|
18
|
+
|
|
19
|
+
- Redaction now also covers streamed and raw string bodies ([#197](https://github.com/Coolhand-Labs/coolhand-ruby/pull/197)). The JSON object or array on each `data:` line of a server-sent-event response, JSON text passed as a string, and the `request_body` / `response_body` given to `Coolhand.logger_service.forward_webhook` are parsed, redacted and re-serialized (compact output, a repeated key keeps its last value, numbers go to double precision). `[DONE]`, event and comment lines and plain text are left as received; a payload that cannot be redacted is replaced by a `{"_coolhand_capture_skipped": "redaction_failed"}` marker. **Migration note:** logged SSE and webhook bodies that carry credential-named fields now show `[REDACTED]`. See [Body Redaction](docs/configuration.md).
|
|
20
|
+
- Credentials in URL-valued body fields and more credential keys are redacted ([#193](https://github.com/Coolhand-Labs/coolhand-ruby/pull/193)). String values under `url`, `server_url`, `image_url`, `file_url` and `endpoint` that are a single absolute URL go through the URL sanitizer, and string values under `x-api-key`, `secret`, `aws_secret_access_key`, `session_token` and `bearer_token` are redacted. A secret in a URL path cannot be detected and is sent as is.
|
|
21
|
+
- Batch item bodies and chunked request bodies respect `max_captured_body_bytes` ([#198](https://github.com/Coolhand-Labs/coolhand-ruby/pull/198)). The OpenAI and Vertex batch processors cap each item's request and response body with the `body_too_large` marker and read OpenAI result files line by line, skipping malformed lines instead of dropping the file. A chunked `body_stream` request is no longer read ahead of the send: a read-only tap copies the bytes Net::HTTP reads, and the log entry is built after the send. New markers: `body_stream_incomplete`, `unsupported_body_stream`, `nesting_too_deep`, `unserializable_body`.
|
|
22
|
+
### Changed
|
|
23
|
+
- **Breaking:** `Coolhand.public_content_service` is now `Coolhand.help_center_service`, and `Coolhand::PublicContentService` is now `Coolhand::HelpCenterService` ([#167](https://github.com/Coolhand-Labs/coolhand-ruby/pull/167)). The old names are removed with no deprecated alias. **Migration note:** anyone calling `public_content_service` on v0.10.0 must switch to `help_center_service`; method signatures (`search_help_articles`, `get_help_article`, `list_help_categories`) and the requests sent are unchanged.
|
|
24
|
+
- OpenAI webhook ids are remembered for twice `webhook_replay_tolerance_seconds` ([#176](https://github.com/Coolhand-Labs/coolhand-ruby/pull/176)), and the claim is released when your `process_event` raises, so OpenAI's retry of the same `webhook-id` is processed instead of being rejected as a replay. The response is still 401 on failure. **Migration note:** a custom `webhook_id_store` now receives `2 * webhook_replay_tolerance_seconds` as `ttl_seconds`.
|
|
25
|
+
- Rate-limit headers (`x-ratelimit-remaining-tokens`, `anthropic-ratelimit-input-tokens-reset`, ...) are no longer redacted in captured logs ([#174](https://github.com/Coolhand-Labs/coolhand-ruby/pull/174)). The allow-list is closed-vocabulary: `x-ratelimit-api-key` and similar are still redacted.
|
|
26
|
+
- Relative batch input URLs (`/v1/chat/completions`) are logged as `https://api.openai.com/v1/chat/completions` ([#171](https://github.com/Coolhand-Labs/coolhand-ruby/pull/171)).
|
|
27
|
+
- Docs: `docs/anthropic.md` now describes the unified `NetHttpInterceptor` instead of the removed `FaradayInterceptor` ([#169](https://github.com/Coolhand-Labs/coolhand-ruby/pull/169)). CI: `ruby/setup-ruby` bumped to 1.335.0 for the Ruby 3.3.13 / 3.4.12 / 4.0.9 regexp security fixes ([#175](https://github.com/Coolhand-Labs/coolhand-ruby/pull/175)). Test coverage gaps closed (webhook secret handling, sanitizer negatives, batch wire payloads) ([#172](https://github.com/Coolhand-Labs/coolhand-ruby/pull/172)).
|
|
28
|
+
|
|
29
|
+
- Request paths are normalized before `exclude_api_patterns` and path-anchored `intercept_addresses` matching ([#195](https://github.com/Coolhand-Labs/coolhand-ruby/pull/195)): repeated `/` are collapsed, `.` and `..` segments resolved and unreserved percent-escapes decoded, so `/openai//files` or `/openai/%66iles` can no longer slip past an exclusion. An exclusion matches if the raw or the normalized path matches. Matching stays case-sensitive. The URL that is sent and logged is unchanged. **Migration note:** a custom anchor or pattern containing `//`, `..` or an unreserved escape no longer matches the normalized form.
|
|
30
|
+
- URL fragments are dropped by the URL sanitizer ([#191](https://github.com/Coolhand-Labs/coolhand-ruby/pull/191)), so `#access_token=...` no longer reaches Coolhand. **Migration note:** logged and forwarded URLs no longer include a `#fragment`.
|
|
31
|
+
### Security
|
|
32
|
+
- **`OpenAi::WebhookValidator` no longer accepts unsigned webhooks in `development` and `test` when a secret is set** ([#170](https://github.com/Coolhand-Labs/coolhand-ruby/pull/170)). Previously a request missing the `webhook-signature` / `webhook-timestamp` / `webhook-id` headers passed in those environments, so an app running with `RAILS_ENV=development` on a reachable host accepted forged `batch.completed` events. **Migration note:** unsigned test payloads posted to a local app that has a secret configured now get a 401. Sign the payload (snippet in [docs/openai.md](docs/openai.md#local-development-and-testing)), leave the secret unset in development, or replay a real delivery. With no secret configured, development and test still skip verification with a warning.
|
|
33
|
+
- `OpenAi::BatchResultProcessor` validates the batch id and the input/output file ids before using them in OpenAI API calls made with your `OPENAI_API_KEY` ([#171](https://github.com/Coolhand-Labs/coolhand-ruby/pull/171)), and skips a batch with missing or invalid `in_progress_at` / `completed_at` instead of logging epoch-0 timestamps. **Migration note:** events whose ids don't match `batch_...` / `file-...` are now ignored with an error in `Rails.logger`.
|
|
34
|
+
- `LoggerService#forward_webhook` no longer forwards the whole Rack env ([#173](https://github.com/Coolhand-Labs/coolhand-ruby/pull/173)). `headers: request.headers` iterates every Rack env entry, so session and request data (`rack.session`, `action_dispatch.*`) and CGI variables such as `QUERY_STRING` were shipped to Coolhand. Dotted keys and CGI/server variables are now skipped, while `HTTP_*`, `CONTENT_TYPE` and `CONTENT_LENGTH` are kept, and a caller-supplied `url:` is sanitized. **Migration note:** forwarded headers no longer include `remote-addr`, `request-method`, `query-string`, `path-info` and similar.
|
|
35
|
+
- URL and header redaction gaps closed ([#174](https://github.com/Coolhand-Labs/coolhand-ruby/pull/174)): the query params `code`, `sas`, `pwd` and `passwd` (exact name, case-insensitive), `;name=value` matrix params in paths and query values, and bare query segments with no `=` are redacted, and URL-form string values under `data_sources` (e.g. an `endpoint` with embedded credentials) are sanitized. Percent-escaped invalid UTF-8 in a param name no longer raises.
|
|
36
|
+
- A host streaming block that raises no longer leaks its message into the log ([#179](https://github.com/Coolhand-Labs/coolhand-ruby/pull/179)): `response_body.error.message` is now the fixed placeholder `"[redacted: raised in host streaming block]"` (transport errors keep their message). A non-intercepted request made inside an outer streaming block no longer has its chunks appended to the outer request's logged `response_body`.
|
|
37
|
+
- Release red-team hardening. The URL sanitizer now also redacts the query params `jwt`, `bearer`, `session`, `cookie`, `passphrase`, `ticket`, `assertion` (by substring) and `pass`, `sid`, `otp` (exact name). A JSON body nested deeper than 100 levels is logged as a `nesting_too_deep` marker instead of the raw string, which skipped body redaction. `BaseInterceptor.sanitize_headers` returns `{}` for an input shape it can't enumerate instead of logging its `to_s`. A failure while recording a response (or a streamed chunk) is now logged and swallowed instead of raising into the host app's already-successful request. **Migration note:** logged URLs with such params now show `[REDACTED]`.
|
|
38
|
+
- `OpenAi::WebhookValidator` rejects a nil payload in every `Rails.env` when a secret is set ([#192](https://github.com/Coolhand-Labs/coolhand-ruby/pull/192)), matching the missing-header rule above. An empty-string body still goes to the signature check. **Migration note:** a nil body posted to a local app that has a secret configured now gets a 401.
|
|
39
|
+
- `LoggerService#forward_webhook` forwards only `HTTP_*`, `CONTENT_TYPE` and `CONTENT_LENGTH` when the input looks like a whole Rack env ([#196](https://github.com/Coolhand-Labs/coolhand-ruby/pull/196)) (it has a `REQUEST_METHOD` key, a lowercase dotted key like `rack.input`, or only `UPPER_CASE` keys), so process env vars such as `DATABASE_URL` or `REDIS_URL` on CGI-style servers are no longer shipped. **Migration note:** a plain hash whose keys are all uppercase (`{"ETAG" => ...}`) is treated as an env and its non-`HTTP_*` entries are dropped; `HTTP_PROXY` is always dropped.
|
|
40
|
+
- `WebhookInterceptor#intercept_batch_request`, `#webhook_secret` and `#process_event` are now private and excluded from the controller's `action_methods` ([#194](https://github.com/Coolhand-Labs/coolhand-ruby/pull/194)), so they cannot be routed to as actions. **Migration note:** calling them from outside the controller raises `NoMethodError`, and the three names are reserved; define `webhook_secret` under `private` in your controller (see [docs/openai.md](docs/openai.md)).
|
|
41
|
+
|
|
10
42
|
## [0.10.0] - 2026-10-21
|
|
11
43
|
|
|
12
44
|
### Added
|
data/README.md
CHANGED
|
@@ -120,12 +120,12 @@ See [Reading Workloads →](docs/workload-search.md) and [Reading Logs →](docs
|
|
|
120
120
|
|
|
121
121
|
## Reading the Help Center
|
|
122
122
|
|
|
123
|
-
`Coolhand.
|
|
123
|
+
`Coolhand.help_center_service` reads the Coolhand help center (the same pages as coolhandlabs.com/help and /updates) with **no API key required**:
|
|
124
124
|
|
|
125
125
|
```ruby
|
|
126
126
|
require 'coolhand'
|
|
127
127
|
|
|
128
|
-
help_center = Coolhand.
|
|
128
|
+
help_center = Coolhand.help_center_service
|
|
129
129
|
|
|
130
130
|
result = help_center.search_help_articles(query: 'match rate', per_page: 5)
|
|
131
131
|
article = help_center.get_help_article(result.articles.first[:slug]) # full markdown :body
|
|
@@ -251,7 +251,7 @@ end
|
|
|
251
251
|
| `intercept_addresses` | Array | All [supported providers](#supported-libraries) (`DEFAULT_INTERCEPT_ADDRESSES`) | Array of API hosts to monitor (matched by host, optionally pinned to a port and/or path prefix). This is a required allow-list — `[]` is ignored (a warning is logged) rather than disabling capture; use `enabled` or `capture` for that. See [Configuration](docs/configuration.md) |
|
|
252
252
|
| `intercept_path_patterns` | Array | `[":generateContent", ":streamGenerateContent"]` | Path patterns to additionally monitor on Google API hosts — see [Configuration](docs/configuration.md) |
|
|
253
253
|
| `exclude_api_patterns` | Array | `["/batchPredictionJobs/"]` | Deny-list checked after `intercept_addresses`; matching paths are skipped. Unlike `intercept_addresses`, `exclude_api_patterns = []` genuinely disables exclusion. See [Configuration](docs/configuration.md) |
|
|
254
|
-
| `max_captured_body_bytes` | Integer | `1_000_000` | Maximum size of a captured JSON request body — oversized bodies are replaced with a placeholder. Non-JSON bodies (e.g. file/audio uploads) are always skipped regardless of size — see [Advanced Configuration](docs/configuration.md) |
|
|
254
|
+
| `max_captured_body_bytes` | Integer | `1_000_000` | Maximum size of a captured JSON request body or response body — oversized bodies are replaced with a placeholder (`nil` disables the cap). Non-JSON bodies (e.g. file/audio uploads) are always skipped regardless of size — see [Advanced Configuration](docs/configuration.md) |
|
|
255
255
|
| `async_logging` | Boolean | `true` | Send logs to Coolhand from a background thread so a slow Coolhand API never delays your LLM calls. Set to `false` to send inline — see [Advanced Configuration](docs/configuration.md#asynchronous-logging) |
|
|
256
256
|
| `auto_declare` | Array | `[]` | Rails only: `:active_job` and/or `:controller` to declare a workload and template for LLM calls made from jobs and controller actions — see [Declared Workloads](docs/declared_workloads.md#rails-auto-declare) |
|
|
257
257
|
|
|
@@ -273,7 +273,7 @@ client = OpenAI::Client.new(access_token: ENV['OPENAI_API_KEY'])
|
|
|
273
273
|
|
|
274
274
|
response = client.chat(
|
|
275
275
|
parameters: {
|
|
276
|
-
model: "gpt-
|
|
276
|
+
model: "gpt-4o-mini",
|
|
277
277
|
messages: [{ role: "user", content: "These pretzels are making me thirsty!"}],
|
|
278
278
|
temperature: 0.7
|
|
279
279
|
}
|
|
@@ -350,7 +350,8 @@ end
|
|
|
350
350
|
|
|
351
351
|
**Optional Parameters:**
|
|
352
352
|
- `event_type` - Event type to append to URL (e.g., "post_call_transcription" → `webhook://elevenlabs/post_call_transcription`)
|
|
353
|
-
- `headers` - Request headers (automatically sanitized)
|
|
353
|
+
- `headers` - Request headers (automatically sanitized). Passing Rails `request.headers` is safe: when the input looks like a Rack env (it has a `REQUEST_METHOD` key or a lowercase dotted key such as `rack.input`, or every key is `UPPER_CASE`; in a plain hash use hyphenated or mixed-case names), only `HTTP_*` headers, `CONTENT_TYPE` and `CONTENT_LENGTH` are forwarded, so entries such as `rack.*`, `REQUEST_METHOD`, `QUERY_STRING`, or process env vars (`DATABASE_URL`, ...) are dropped
|
|
354
|
+
- `url` - Optional URL override; credentials in userinfo and sensitive query params are redacted, and any `#fragment` is dropped
|
|
354
355
|
- `conversation_id`, `agent_id`, `metadata` - Custom fields for your tracking needs
|
|
355
356
|
|
|
356
357
|
**Error Handling:**
|
|
@@ -410,7 +411,7 @@ The monitor works with multiple transport layers and Ruby libraries:
|
|
|
410
411
|
- Google Gemini (`generativelanguage.googleapis.com`)
|
|
411
412
|
- Google Vertex AI (`aiplatform.googleapis.com`) — see the [Vertex AI batch result logging guide](docs/vertex.md) for async batch jobs
|
|
412
413
|
- Azure OpenAI Service, Azure AI Foundry, and Azure Machine Learning managed online endpoints (dedicated hosts, multi-service Azure AI Services hosts, and serverless/MaaS deployments) — see [Configuration](docs/configuration.md) for the full list of covered hosts
|
|
413
|
-
- AWS Bedrock (OpenAI-compatible endpoint)
|
|
414
|
+
- AWS Bedrock (`bedrock-runtime.<region>.amazonaws.com`, plus the `.amazonaws.com.cn`, `.api.aws` dual-stack and `bedrock-runtime-fips.<region>` variants — model invoke, Converse and the OpenAI-compatible endpoint)
|
|
414
415
|
- Cloudflare AI Gateway
|
|
415
416
|
- OpenRouter
|
|
416
417
|
- OpenCode Zen (`opencode.ai`)
|
|
@@ -518,7 +519,7 @@ The monitor handles errors gracefully:
|
|
|
518
519
|
- **[Reading the Help Center](docs/help-center.md)** — Search and read Coolhand's help center articles and list its sections, with no API key required
|
|
519
520
|
- **[Linking Feedback](docs/feedback-links.md)** — Link feedback to an optimization as evidence, singly or in bulk, using the private API key
|
|
520
521
|
- **[Declared Workloads](docs/declared_workloads.md)** — Declare the workload and template of captured LLM calls with `Coolhand.with_context`, and create or rename them with `upsert`
|
|
521
|
-
- **[Anthropic Integration](docs/anthropic.md)** — Official and community Anthropic Ruby gems, streaming,
|
|
522
|
+
- **[Anthropic Integration](docs/anthropic.md)** — Official and community Anthropic Ruby gems, streaming, using both gems, and troubleshooting
|
|
522
523
|
- **[ElevenLabs Integration](docs/elevenlabs.md)** — Webhook capture, feedback submission, and Rails integration
|
|
523
524
|
- **[OpenAI Batch Webhook Handler](docs/openai.md)** — Handle OpenAI batch job completion events via webhook interception
|
|
524
525
|
- **[Google Vertex AI Batch Result Logging](docs/vertex.md)** — Log completed Vertex AI batch prediction job results
|
|
@@ -526,7 +527,8 @@ The monitor handles errors gracefully:
|
|
|
526
527
|
|
|
527
528
|
## Security
|
|
528
529
|
|
|
529
|
-
- API keys in request headers are automatically redacted
|
|
530
|
+
- API keys in request headers and credential-like URL query params are automatically redacted (provider rate-limit headers such as `x-ratelimit-remaining-tokens` are kept)
|
|
531
|
+
- Credentials in request and response bodies (MCP `authorization`/`authorization_token`, Realtime ephemeral client secrets, `api_key`, and similar) are redacted; see [Body redaction](docs/configuration.md#body-redaction)
|
|
530
532
|
- No sensitive data is exposed in logs
|
|
531
533
|
- All data is sent via HTTPS to Coolhand servers
|
|
532
534
|
|
data/docs/anthropic.md
CHANGED
|
@@ -4,11 +4,13 @@ Coolhand provides comprehensive monitoring for Anthropic's Claude API through su
|
|
|
4
4
|
|
|
5
5
|
## Overview
|
|
6
6
|
|
|
7
|
-
Coolhand
|
|
7
|
+
Coolhand monitors Anthropic API calls (`api.anthropic.com`) with a single `NetHttpInterceptor` that patches `Net::HTTP`. Both Ruby gems are covered with no gem-specific setup:
|
|
8
8
|
|
|
9
|
-
- **Official Anthropic Gem** (`anthropic`): Uses Net::HTTP
|
|
10
|
-
- **Community Ruby-Anthropic Gem** (`ruby-anthropic`): Uses Faraday
|
|
11
|
-
- **
|
|
9
|
+
- **Official Anthropic Gem** (`anthropic`): Uses Net::HTTP directly, monitored via `NetHttpInterceptor`
|
|
10
|
+
- **Community Ruby-Anthropic Gem** (`ruby-anthropic`): Uses Faraday. Faraday's default `net_http` adapter delegates to Net::HTTP, so these calls are monitored by the same `NetHttpInterceptor`
|
|
11
|
+
- **Both gems installed**: Coolhand itself adds no conflict warnings or duplicate logging (see [Using Both Gems](#using-both-gems))
|
|
12
|
+
|
|
13
|
+
**Caveat:** A Faraday connection configured with a non-Net::HTTP adapter (such as `faraday-typhoeus`) bypasses `Net::HTTP` and is not covered.
|
|
12
14
|
|
|
13
15
|
## Quick Start
|
|
14
16
|
|
|
@@ -131,7 +133,7 @@ Coolhand.configure do |config|
|
|
|
131
133
|
config.api_key = 'your_coolhand_api_key'
|
|
132
134
|
end
|
|
133
135
|
|
|
134
|
-
# Use ruby-anthropic gem normally - monitored via Faraday
|
|
136
|
+
# Use ruby-anthropic gem normally - monitored via Faraday's default net_http adapter
|
|
135
137
|
client = Anthropic::Client.new(access_token: ENV['ANTHROPIC_API_KEY'])
|
|
136
138
|
|
|
137
139
|
response = client.messages(
|
|
@@ -141,13 +143,15 @@ response = client.messages(
|
|
|
141
143
|
)
|
|
142
144
|
|
|
143
145
|
puts response.dig("content", 0, "text")
|
|
144
|
-
# ✅ Automatically logged via
|
|
146
|
+
# ✅ Automatically logged via NetHttpInterceptor
|
|
145
147
|
```
|
|
146
148
|
|
|
149
|
+
> If your Faraday connection uses a non-Net::HTTP adapter (such as `faraday-typhoeus`), requests are not captured.
|
|
150
|
+
|
|
147
151
|
### Streaming with Ruby-Anthropic
|
|
148
152
|
|
|
149
153
|
```ruby
|
|
150
|
-
# Streaming
|
|
154
|
+
# Streaming is captured the same way as any other request
|
|
151
155
|
response = client.messages(
|
|
152
156
|
model: "claude-3-sonnet-20240229",
|
|
153
157
|
max_tokens: 1024,
|
|
@@ -163,53 +167,13 @@ end
|
|
|
163
167
|
# ✅ Complete conversation logged to Coolhand
|
|
164
168
|
```
|
|
165
169
|
|
|
166
|
-
##
|
|
167
|
-
|
|
168
|
-
### Automatic Conflict Resolution
|
|
169
|
-
|
|
170
|
-
When both gems are installed, Coolhand automatically handles the conflict:
|
|
171
|
-
|
|
172
|
-
```ruby
|
|
173
|
-
require 'anthropic' # Official gem
|
|
174
|
-
require 'ruby-anthropic' # Community gem
|
|
175
|
-
require 'coolhand'
|
|
170
|
+
## Using Both Gems
|
|
176
171
|
|
|
177
|
-
Coolhand.
|
|
178
|
-
config.api_key = 'your_coolhand_api_key'
|
|
179
|
-
end
|
|
172
|
+
Coolhand does not need to choose between the gems. Monitoring happens at the `Net::HTTP` layer through a single `NetHttpInterceptor`, so it does not depend on which gem makes the request. Coolhand does not disable monitoring, print conflict warnings, or log duplicates when both gems are installed.
|
|
180
173
|
|
|
181
|
-
|
|
182
|
-
# "Both 'anthropic' and 'ruby-anthropic' gems are installed.
|
|
183
|
-
# Coolhand will only monitor ruby-anthropic (Faraday-based) requests.
|
|
184
|
-
# Official anthropic gem monitoring has been disabled."
|
|
185
|
-
```
|
|
186
|
-
|
|
187
|
-
**Recommendation**: Use only one gem to avoid confusion. Choose based on your needs:
|
|
174
|
+
Both gems define an `Anthropic` namespace, so loading both in the same app can still conflict inside your own code. That is independent of Coolhand. Pick the gem that fits your needs:
|
|
188
175
|
- **Official gem**: Latest features, official support, Net::HTTP transport
|
|
189
|
-
- **Community gem**: Faraday-based, may have community contributions
|
|
190
|
-
|
|
191
|
-
### Manual Conflict Resolution
|
|
192
|
-
|
|
193
|
-
To explicitly use the official gem when both are installed:
|
|
194
|
-
|
|
195
|
-
```ruby
|
|
196
|
-
# Temporarily hide the community gem
|
|
197
|
-
begin
|
|
198
|
-
# Remove ruby-anthropic from the load path
|
|
199
|
-
$LOAD_PATH.reject! { |path| path.include?('ruby-anthropic') }
|
|
200
|
-
|
|
201
|
-
require 'anthropic'
|
|
202
|
-
require 'coolhand'
|
|
203
|
-
|
|
204
|
-
Coolhand.configure do |config|
|
|
205
|
-
config.api_key = 'your_coolhand_api_key'
|
|
206
|
-
end
|
|
207
|
-
|
|
208
|
-
# Now only the official gem will be monitored
|
|
209
|
-
client = Anthropic::Client.new
|
|
210
|
-
# ... use official gem
|
|
211
|
-
end
|
|
212
|
-
```
|
|
176
|
+
- **Community gem**: Faraday-based (default `net_http` adapter), may have community contributions
|
|
213
177
|
|
|
214
178
|
## Rails Integration
|
|
215
179
|
|
|
@@ -381,43 +345,24 @@ end
|
|
|
381
345
|
|
|
382
346
|
# Now you'll see console output like:
|
|
383
347
|
# "✅ Coolhand ready - will log inference calls on monitored URIs"
|
|
384
|
-
# "COOLHAND: ⚠️ Warning: Both 'anthropic' and 'ruby-anthropic' gems are installed..."
|
|
385
348
|
```
|
|
386
349
|
|
|
387
350
|
### Common Issues
|
|
388
351
|
|
|
389
|
-
#### 1.
|
|
390
|
-
|
|
391
|
-
**Problem**: Coolhand says "Anthropic gem not loaded"
|
|
352
|
+
#### 1. Requests Not Appearing
|
|
392
353
|
|
|
393
|
-
**
|
|
354
|
+
**Problem**: Anthropic calls are not showing up in Coolhand
|
|
394
355
|
|
|
395
|
-
|
|
396
|
-
require 'anthropic' # Must come before coolhand
|
|
397
|
-
require 'coolhand'
|
|
356
|
+
**Solution**: Check the following:
|
|
398
357
|
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
**Problem**: Seeing requests logged twice in dashboard
|
|
407
|
-
|
|
408
|
-
**Cause**: Usually indicates both gems are installed and conflicting
|
|
409
|
-
|
|
410
|
-
**Solution**: Choose one gem and uninstall the other:
|
|
411
|
-
|
|
412
|
-
```bash
|
|
413
|
-
# Keep official gem
|
|
414
|
-
gem uninstall ruby-anthropic
|
|
415
|
-
|
|
416
|
-
# OR keep community gem
|
|
417
|
-
gem uninstall anthropic
|
|
418
|
-
```
|
|
358
|
+
- `config.silent` is `false`, so Coolhand's debug and warning output is visible while you check the rest.
|
|
359
|
+
- `config.api_key` is set. Intercepted requests are skipped, with a warning log, when the key is missing.
|
|
360
|
+
- `config.enabled` is not `false`.
|
|
361
|
+
- `config.capture` is not `false`, the call is not inside `Coolhand.without_capture`, and the path does not match `exclude_api_patterns`.
|
|
362
|
+
- The request goes to a monitored address (`api.anthropic.com` is monitored by default).
|
|
363
|
+
- If you use `ruby-anthropic`, the Faraday connection uses the default `net_http` adapter. A non-Net::HTTP adapter such as `faraday-typhoeus` is not covered.
|
|
419
364
|
|
|
420
|
-
####
|
|
365
|
+
#### 2. Missing Streaming Data
|
|
421
366
|
|
|
422
367
|
**Problem**: Only seeing individual chunks, not complete response
|
|
423
368
|
|
|
@@ -436,7 +381,7 @@ response = client.messages(parameters: { stream: true })
|
|
|
436
381
|
response.each { |chunk| print chunk.delta.text }
|
|
437
382
|
```
|
|
438
383
|
|
|
439
|
-
####
|
|
384
|
+
#### 3. Performance Concerns
|
|
440
385
|
|
|
441
386
|
**Problem**: Worried about monitoring overhead
|
|
442
387
|
|
|
@@ -496,7 +441,7 @@ end
|
|
|
496
441
|
|
|
497
442
|
## Best Practices
|
|
498
443
|
|
|
499
|
-
1. **
|
|
444
|
+
1. **Pick One Gem**: Both `anthropic` and `ruby-anthropic` define an `Anthropic` namespace, so avoid loading both in the same app. Coolhand monitors either one
|
|
500
445
|
2. **Secure API Keys**: Use environment variables or Rails credentials, never commit keys
|
|
501
446
|
3. **Silent in Production**: Set `config.silent = true` in production environments
|
|
502
447
|
4. **Monitor Performance**: Use fast models (haiku) for high-frequency requests
|
|
@@ -515,4 +460,4 @@ For complete API documentation, see:
|
|
|
515
460
|
|
|
516
461
|
- **Coolhand Issues**: [GitHub Issues](https://github.com/Coolhand-Labs/coolhand-ruby/issues)
|
|
517
462
|
- **Anthropic API**: [Anthropic Documentation](https://docs.anthropic.com/)
|
|
518
|
-
- **
|
|
463
|
+
- **Missing Requests**: Check this guide's troubleshooting section
|
data/docs/configuration.md
CHANGED
|
@@ -35,10 +35,30 @@ end
|
|
|
35
35
|
**What changes when `debug_mode` is `true`:**
|
|
36
36
|
- No HTTP request is made — `create_log`, `create_feedback`, and `send_llm_request_log` all return `nil` and print the payload via `JSON.pretty_generate` instead.
|
|
37
37
|
- Capture is forced on for every request, including inside a `Coolhand.without_capture` block and when `config.capture = false`. This is intentional — debug mode is meant to show you everything, not respect capture suppression — so don't leave it enabled in an environment where you rely on `without_capture` to keep specific calls out of the logs.
|
|
38
|
-
- Header and URL sanitization (`[REDACTED]` API keys/tokens, redacted `key=`/`token=` query params) still applies to the printed payload, exactly as it would to a real request.
|
|
38
|
+
- Header and URL sanitization (`[REDACTED]` API keys/tokens, redacted `key=`/`token=`/`code=` query params, `;key=` matrix params and bare `?value` query segments; URL fragments are dropped; rate-limit headers such as `x-ratelimit-remaining-tokens` are kept) still applies to the printed payload, exactly as it would to a real request.
|
|
39
39
|
|
|
40
40
|
Because it forces capture unconditionally and prints full request/response bodies to the console, only enable `debug_mode` in development — never in production or in an environment handling real user data.
|
|
41
41
|
|
|
42
|
+
## Body Redaction
|
|
43
|
+
|
|
44
|
+
Besides headers and URLs, Coolhand redacts credentials in the JSON body of both the request and the response before anything is sent. A redacted value becomes `[REDACTED]`; the rest of the body is untouched.
|
|
45
|
+
|
|
46
|
+
- **String values under these keys, at any depth:** `authorization`, `authorization_token`, `access_token`, `refresh_token`, `id_token`, `api_key`, `client_secret`, `private_key`, `secret_key`, `password`, `x-api-key`, `secret`, `secret_access_key`, `aws_secret_access_key`, `session_token`, `bearer_token`. Names match exactly, ignoring case and any `_`, `-` or space (so `apiKey` and `API-Key` match, `total_tokens` does not). Only string values are redacted, so a tool JSON Schema with a property named `api_key` or `token` is left as is.
|
|
47
|
+
- **URL-valued fields:** a string under the key `url`, `server_url`, `image_url`, `file_url` or `endpoint` (same name matching, any depth) that is a single absolute URL has embedded credentials scrubbed: userinfo (`https://user:pw@host`) and query parameters whose name looks secret (matched as a case-insensitive substring, so `key=`, `X-Amz-Signature=` and `X-Amz-Credential=` match but so do `keyword=` and `author=`). A URL that cannot be parsed loses its userinfo, query and fragment. The URL is otherwise kept. Prose that merely contains a URL, `data:` URIs and relative paths are left as is. A bare `token` key is never redacted, since logprobs use `{"token": "Hello"}`.
|
|
48
|
+
- **Remote MCP servers:** OpenAI `tools[].authorization` and Anthropic `mcp_servers[].authorization_token`. Every string value in the `headers` map of an OpenAI MCP tool (`type: "mcp"` or a `server_url` is present) is redacted too.
|
|
49
|
+
- **Realtime ephemeral keys:** the `value` of a `client_secret` object (`expires_at` is kept), and the `value` of a client-secret response that has a numeric `expires_at` beside it.
|
|
50
|
+
- **Azure OpenAI On Your Data:** under a `data_sources` (or `dataSources`) key at any depth, key/token/secret/password-style fields are redacted, and credentials embedded in URL values (userinfo, `key=`/`sas=` query params) are scrubbed.
|
|
51
|
+
|
|
52
|
+
Limits:
|
|
53
|
+
- A secret that is part of a URL's path (for example `https://actions.zapier.com/mcp/SECRET/sse`) cannot be detected generically and is sent as is.
|
|
54
|
+
- Parsed JSON objects and arrays are walked as described above. Subtrees nested more than 100 levels deep are replaced with `[REDACTED]`.
|
|
55
|
+
- A raw String body is checked in two cases: JSON text (e.g. a webhook body that is a JSON string) is parsed, redacted and re-serialized, and the JSON object/array payload on each `data:` line of a streamed (server-sent event) response is redacted the same way. A JSON payload is always re-serialized (compact, a repeated key keeps its last value), even when nothing in it was redacted, so its whitespace and string escapes may differ from the original and numbers are normalized to double precision (integers of any size are kept). Provider streams are compact JSON already, so they usually come through identical.
|
|
56
|
+
- Everything else in a String is sent as received: plain text, `[DONE]`, `event:`/`id:`/`retry:` and comment lines, a `data:` line that is not a JSON object or array or that Ruby's JSON parser rejects (even if it still looks like JSON, e.g. an unescaped control character or trailing text), a JSON payload split across several `data:` lines, and newline-delimited JSON streams (no `data:` prefix). Secrets inside free text such as model output are never detected.
|
|
57
|
+
- A String JSON payload nested more than 100 levels deep is replaced with `{"_coolhand_capture_skipped": "nesting_too_deep"}` (as JSON text, since it replaces text). If redacting a payload (or a whole String body) fails for any reason, it is replaced with `{"_coolhand_capture_skipped": "redaction_failed"}` rather than logged unchecked.
|
|
58
|
+
- Streamed bodies are capped by `max_captured_body_bytes` (see [Body Capture Limits](#body-capture-limits)) before redaction, so an over-limit stream is logged as the `body_too_large` placeholder and never scanned.
|
|
59
|
+
|
|
60
|
+
Webhooks sent with `Coolhand.logger_service.forward_webhook` get the same redaction on `webhook_body` and `response_body`, after the existing binary-field filtering. Webhook bodies are not subject to the capture cap.
|
|
61
|
+
|
|
42
62
|
## Asynchronous Logging
|
|
43
63
|
|
|
44
64
|
By default Coolhand ships each captured request to its API from background worker threads (one for per-request logs, a separate one for batch jobs), so a slow or unreachable Coolhand backend never adds latency to your own LLM calls. The request-path work is limited to capturing and sanitizing the payload.
|
|
@@ -58,9 +78,9 @@ end
|
|
|
58
78
|
- `config.debug_mode` always runs inline so printed payloads appear in order.
|
|
59
79
|
- Logs queued but not yet sent are lost if the process is killed hard (`kill -9`, OOM). Set `async_logging = false` if you need delivery before each call returns.
|
|
60
80
|
|
|
61
|
-
##
|
|
81
|
+
## Body Capture Limits
|
|
62
82
|
|
|
63
|
-
To avoid holding large uploads (batch JSONL files, fine-tune corpora, audio for transcription) in memory and shipping them to Coolhand in full, `config.max_captured_body_bytes` caps how much of a request body gets captured:
|
|
83
|
+
To avoid holding large uploads (batch JSONL files, fine-tune corpora, audio for transcription) and large responses in memory and shipping them to Coolhand in full, `config.max_captured_body_bytes` caps how much of a request body and of a response body gets captured. Set it to `nil` to disable the cap (a non-Integer or negative value is ignored and the default is used):
|
|
64
84
|
|
|
65
85
|
```ruby
|
|
66
86
|
Coolhand.configure do |config|
|
|
@@ -71,8 +91,12 @@ end
|
|
|
71
91
|
**What happens at the limits:**
|
|
72
92
|
- A request body whose `Content-Type` doesn't look like JSON (e.g. `multipart/form-data`, `audio/mpeg`) is never read into memory for capture — the log entry gets a placeholder (`{"_coolhand_capture_skipped" => "non_json_content_type", ...}`) instead of the body. This is intentional: an opaque binary blob isn't human/LLM-readable in Coolhand's UI anyway.
|
|
73
93
|
- A JSON (or content-type-unset) body larger than the configured limit is replaced with a placeholder (`{"_coolhand_capture_skipped" => "body_too_large", ...}`) instead of being logged in full.
|
|
94
|
+
- A request body streamed via `body_stream` whose `Content-Length` is already over the limit is skipped without being read into memory at all.
|
|
95
|
+
- A chunked `body_stream` request (no `Content-Length`) is never read ahead for capture. Coolhand sits in front of the stream and copies the bytes Net::HTTP reads for the wire, keeping at most `max_captured_body_bytes` of them; once the limit is crossed it keeps only a byte count, so memory stays bounded however large the upload is. If the body is over the limit the log gets the `body_too_large` placeholder (`size_bytes` is the bytes sent). If the stream was not read to its end (for example the connection was refused, or the server rejected the request before the body was sent), the log gets `{"_coolhand_capture_skipped" => "body_stream_incomplete", "size_bytes" => ...}`, since there is no complete body to log. A chunked source that `IO.copy_stream` reads by path (a `String` path or a `Pathname`) is not tapped; its log gets `{"_coolhand_capture_skipped" => "unsupported_body_stream"}`. Your stream is read once, by Net::HTTP, exactly as without Coolhand, and is back on the request object afterwards. While the request is being sent, `req.body_stream` is a read-only proxy offering only `read` (and `readpartial` when your stream has it); anything else called on it during the send (such as `rewind` or `size`, from other HTTP instrumentation running inside Coolhand's `request`) raises `NoMethodError`.
|
|
96
|
+
- OpenAI and Vertex batch results ([OpenAI](openai.md), [Vertex](vertex.md)) are capped per item: an item's request body and its response body are each replaced by the same `body_too_large` placeholder when larger than the limit, so one huge item doesn't inflate the others or the memory held while a batch is logged. A body that can't be measured is replaced by `{"_coolhand_capture_skipped" => "nesting_too_deep"}` (nested more than 100 levels) or `{"_coolhand_capture_skipped" => "unserializable_body"}` (for example `NaN`/`Infinity` values) instead, and the rest of the batch is still logged.
|
|
97
|
+
- A response body larger than the limit is replaced with the same kind of placeholder (`{"_coolhand_capture_skipped" => "body_too_large", "size_bytes" => ..., "max_bytes" => ..., "content_type" => ...}`). For a streamed response (`read_body` with a block) Coolhand stops buffering once the limit is crossed, so memory use stays bounded; `size_bytes` is the number of bytes seen while streaming.
|
|
74
98
|
|
|
75
|
-
In
|
|
99
|
+
In every case, only the *captured log entry* is affected — the real request to the LLM provider always sends the complete, untruncated body, and your code always receives the complete response (every streamed chunk is passed to your block untouched).
|
|
76
100
|
|
|
77
101
|
## Custom Intercept Addresses
|
|
78
102
|
|
|
@@ -93,7 +117,7 @@ Setting `intercept_addresses` **replaces** the default list entirely, so include
|
|
|
93
117
|
|
|
94
118
|
The default list can be found in `Coolhand::Configuration::DEFAULT_INTERCEPT_ADDRESSES`.
|
|
95
119
|
|
|
96
|
-
Each entry is matched against the request's parsed host (optionally with a port and path prefix, see below) — an exact match, or a dot-boundary suffix match (`my-llm-proxy.internal` also matches `eu.my-llm-proxy.internal`) — case-insensitively. A single `*` in an entry matches exactly one host label, e.g. `bedrock-runtime.*.amazonaws.com`. Entries are never matched against the full URL, so query strings that merely mention a host can't trigger a false match.
|
|
120
|
+
Each entry is matched against the request's parsed host (optionally with a port and path prefix, see below) — an exact match, or a dot-boundary suffix match (`my-llm-proxy.internal` also matches `eu.my-llm-proxy.internal`) — case-insensitively. A single `*` in an entry matches exactly one host label, e.g. `bedrock-runtime.*.amazonaws.com`. A wildcard entry is an exact host match with no implicit subdomains, so each Bedrock endpoint variant is listed separately (`bedrock-runtime.*.amazonaws.com`, `.amazonaws.com.cn`, `.api.aws`, plus the `bedrock-runtime-fips.*` hosts on `amazonaws.com` and `api.aws`); a VPC interface endpoint host (`vpce-….bedrock-runtime.…`) needs its own entry. Entries are never matched against the full URL, so query strings that merely mention a host can't trigger a false match.
|
|
97
121
|
|
|
98
122
|
An entry can also pin a port (`host:port`) and/or a path (`host/path`, or `host:port/path`), and then a request must match all of them. Cohere, TypeSafe and Ollama are defaulted this way: `api.cohere.com/v2/chat`, `api.typesafe.ai/v1/systemone`, and `*:11434/api/chat` (plus `/api/generate`, `/api/embed`, `/api/embeddings`, also for `127.0.0.1`). Ollama has no fixed host, so its defaults require Ollama's default port *and* an Ollama path — an app's own unrelated `/api/chat` on another host or port is not captured. To capture an Ollama server on a dotted host or IP, or a non-default port, add entries such as `192.168.1.5:11434/api/chat` or `ollama.internal:8080/api/chat`. An IPv6 loopback server (`[::1]:11434`) is already covered by the defaults.
|
|
99
123
|
|
|
@@ -117,6 +141,8 @@ Coolhand.configure do |config|
|
|
|
117
141
|
end
|
|
118
142
|
```
|
|
119
143
|
|
|
144
|
+
Before `exclude_api_patterns`, path-anchored `intercept_addresses` entries (like `api.cohere.com/v2/chat`, see above) and `intercept_path_patterns` are compared, the request path is normalized for matching only (the URL that is sent and logged is unchanged): repeated `/` are collapsed, `.` and `..` segments are resolved (never above the root), and percent-escapes of unreserved characters (`A-Z a-z 0-9 - . _ ~`) are decoded, so `/openai//files`, `/openai/./files` and `/openai/%66iles` are excluded just like `/openai/files`. Reserved escapes such as `%2F` stay encoded, and path matching remains case-sensitive. Your own patterns are compared literally, so write them in normalized form; an exclude pattern also fails closed, matching if either the raw or the normalized path contains it.
|
|
145
|
+
|
|
120
146
|
Unlike `intercept_addresses`, `exclude_api_patterns` genuinely supports `= []` — since it only narrows an allow-list that already passed, an empty deny-list just means "exclude nothing," which is safe.
|
|
121
147
|
|
|
122
148
|
## Capture Control
|
data/docs/elevenlabs.md
CHANGED
|
@@ -35,7 +35,7 @@ class WebhooksController < ApplicationController
|
|
|
35
35
|
webhook_body: request.raw_post || request.body.read,
|
|
36
36
|
source: "elevenlabs",
|
|
37
37
|
event_type: request.headers["X-ElevenLabs-Event"] || "conversation",
|
|
38
|
-
headers: request.headers
|
|
38
|
+
headers: request.headers # only HTTP_* headers, CONTENT_TYPE and CONTENT_LENGTH are forwarded from a Rack env
|
|
39
39
|
)
|
|
40
40
|
|
|
41
41
|
if result
|
data/docs/help-center.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Reading the Help Center
|
|
2
2
|
|
|
3
|
-
`Coolhand::
|
|
3
|
+
`Coolhand::HelpCenterService` reads Coolhand's help center: the articles served at
|
|
4
4
|
`coolhandlabs.com/help/:slug` and the posts at `/updates/:slug`. "Help center" is the user-facing name;
|
|
5
5
|
on the wire the resource is `public_content` (`GET /api/v2/public_content`, `/:slug` and `/categories`).
|
|
6
6
|
It is read-only. Creating, updating and archiving articles is Coolhand-internal and not wrapped.
|
|
@@ -10,7 +10,7 @@ It is read-only. Creating, updating and archiving articles is Coolhand-internal
|
|
|
10
10
|
```ruby
|
|
11
11
|
require "coolhand"
|
|
12
12
|
|
|
13
|
-
help_center = Coolhand.
|
|
13
|
+
help_center = Coolhand.help_center_service # works with Coolhand.configure never called
|
|
14
14
|
```
|
|
15
15
|
|
|
16
16
|
- No key configured: no `X-API-Key` header is sent, and the calls work. Anonymous callers are limited to
|
|
@@ -111,10 +111,10 @@ unknown slug, `429` when the anonymous rate limit is exceeded.
|
|
|
111
111
|
|
|
112
112
|
## Verifying against a live server
|
|
113
113
|
|
|
114
|
-
`spec/live/
|
|
114
|
+
`spec/live/coolhand_help_center_live_spec.rb` runs against a real server with no key:
|
|
115
115
|
|
|
116
116
|
```bash
|
|
117
|
-
COOLHAND_SPEC_LIVE_BASE_URL=http://127.0.0.1:3000/api bundle exec rspec spec/live/
|
|
117
|
+
COOLHAND_SPEC_LIVE_BASE_URL=http://127.0.0.1:3000/api bundle exec rspec spec/live/coolhand_help_center_live_spec.rb
|
|
118
118
|
```
|
|
119
119
|
|
|
120
120
|
It makes about nine anonymous requests, so running it repeatedly within a minute can hit the `429`.
|
data/docs/openai.md
CHANGED
|
@@ -12,7 +12,7 @@ Requires Rails — `Coolhand::WebhookInterceptor` and `Coolhand::OpenAi::BatchRe
|
|
|
12
12
|
- Include the interceptor in your controller: `include Coolhand::WebhookInterceptor`
|
|
13
13
|
- Add the `before_action` to validate and populate the `@validator` payload: `before_action :intercept_batch_request, only: :openai`
|
|
14
14
|
- Ensure you skip CSRF for the webhook endpoint: `skip_before_action :verify_authenticity_token`
|
|
15
|
-
- Override the `webhook_secret` method to return your OpenAI webhook secret
|
|
15
|
+
- Override the `webhook_secret` method to return your OpenAI webhook secret. Define it under `private` so it can't be routed as an action
|
|
16
16
|
|
|
17
17
|
`intercept_batch_request` already validates the webhook and, for `batch.completed`/`batch.failed`/`batch.expired`/`batch.cancelled` events, calls `Coolhand::OpenAi::BatchResultProcessor` — that part requires no code in your controller action. The batch's status is checked inline in the `before_action`, but for a completed batch the two JSONL file downloads and the one-request-log-per-item sends run on Coolhand's [background log thread](configuration.md#asynchronous-logging), so a large batch doesn't hold up your webhook response. (With `config.async_logging = false` this all runs inline, and a large batch can trip your webhook endpoint's timeout and OpenAI's retry behavior.) The example below shows a controller action doing additional, app-specific work on top of that (e.g. updating your own `BatchApiRequest` record and enqueuing your own background job), which is optional.
|
|
18
18
|
|
|
@@ -53,19 +53,68 @@ rescue StandardError => e
|
|
|
53
53
|
head :internal_server_error
|
|
54
54
|
end
|
|
55
55
|
|
|
56
|
+
private # keep helpers like this below your public actions
|
|
57
|
+
|
|
56
58
|
def webhook_secret
|
|
57
59
|
Rails.application.credentials.openai_webhook_secret
|
|
58
60
|
end
|
|
59
61
|
# ...existing code...
|
|
60
62
|
```
|
|
61
63
|
|
|
64
|
+
`intercept_batch_request`, `webhook_secret` and `process_event` are private in `Coolhand::WebhookInterceptor`, so Rails doesn't treat them as controller actions, and `before_action :intercept_batch_request` works with a private method. Code outside the controller can no longer call them directly (it gets `NoMethodError`); `super` from your own override still works.
|
|
65
|
+
|
|
66
|
+
If your controller re-defines `webhook_secret` or `process_event` as public, they are public methods again, but `Coolhand::WebhookInterceptor` still removes all three names from the controller's `action_methods`, so they stay unroutable. That also means these names are reserved: a controller action with one of those names would not be routable. Defining your overrides under `private` is still the recommended form.
|
|
67
|
+
|
|
68
|
+
## What gets logged for a completed batch
|
|
69
|
+
|
|
70
|
+
Each batch item is logged with `source_api: "openai"` and the `model` from the item's request body (falling back to the response body). Batch input files use relative URLs such as `/v1/chat/completions`; these are logged as `https://api.openai.com/v1/chat/completions` (the host is not configurable, so Azure OpenAI or proxy batches are logged with that host too). The batch's `in_progress_at` and `completed_at` timestamps become each log's start and completion times; if OpenAI reports either as missing, the whole batch is skipped with an error in `Rails.logger` rather than logged with bogus times. If `completed_at` is earlier than `in_progress_at`, a warning is logged and `duration_ms` is clamped to 0.
|
|
71
|
+
|
|
72
|
+
Each item's request body and response body are capped separately by [`max_captured_body_bytes`](configuration.md#body-capture-limits): one over the limit is logged as a `{"_coolhand_capture_skipped" => "body_too_large", ...}` placeholder (the item itself is still logged, and `model` is still taken from the full request body). See [Body Capture Limits](configuration.md#body-capture-limits) for the other placeholders. When the OpenAI client returns a result file as a String or IO, it is read and parsed one JSONL line at a time; a client that returns the file already parsed (`ruby-openai` does) has loaded it in full before Coolhand sees it. A malformed line is skipped and counted in an error log, and the other lines are still processed.
|
|
73
|
+
|
|
74
|
+
The batch ID from the webhook payload and the batch's input/output file IDs are used in calls to the OpenAI API with your `OPENAI_API_KEY`, so they are format-checked first (`batch_...` and `file-...`/`file_...`, letters, digits, `_` and `-` only). An event or batch whose IDs don't match is ignored with an error in `Rails.logger`.
|
|
75
|
+
|
|
62
76
|
## Signature header and secret rotation
|
|
63
77
|
|
|
64
78
|
The `webhook-signature` header may carry several space-separated signatures (`v1,sigA v1,sigB`), as senders produce while a webhook secret is being rotated. `Coolhand::OpenAi::WebhookValidator` accepts the request if any `v1,` entry matches your secret, comparing each in constant time and ignoring entries with any other version prefix. The timestamp and replay checks below still run, and only after a signature matches.
|
|
65
79
|
|
|
80
|
+
## Local development and testing
|
|
81
|
+
|
|
82
|
+
Signature headers (`webhook-signature`, `webhook-timestamp`, `webhook-id`) are required whenever a webhook secret is configured, in every `Rails.env`, including `development` and `test`. A request missing any of them is rejected, so your controller responds with 401. Earlier versions let unsigned requests through in development and test, which meant an app running with `RAILS_ENV=development` on a reachable host accepted forged webhooks.
|
|
83
|
+
|
|
84
|
+
The same applies to a missing (`nil`) request body: with a secret configured it is rejected in every `Rails.env`, since there is nothing to verify the signature against. An empty-string body is not treated as missing; it goes through the normal signature check and passes only if it was actually signed.
|
|
85
|
+
|
|
86
|
+
Signature verification (and the missing-body check) is skipped, with a logged warning, only when no secret is configured at all, and only in `development` and `test`. Every other environment rejects the request.
|
|
87
|
+
|
|
88
|
+
If you post unsigned test payloads to a local app that has a secret set, pick one of these:
|
|
89
|
+
|
|
90
|
+
- **Sign the test payload** with the same secret:
|
|
91
|
+
|
|
92
|
+
```ruby
|
|
93
|
+
require "openssl"
|
|
94
|
+
require "base64"
|
|
95
|
+
require "json"
|
|
96
|
+
require "securerandom"
|
|
97
|
+
|
|
98
|
+
# Same key handling as the validator: a `whsec_` secret is base64-decoded, anything else is used as-is.
|
|
99
|
+
secret = ENV.fetch("OPENAI_WEBHOOK_SECRET")
|
|
100
|
+
secret_bytes = secret.start_with?("whsec_") ? Base64.strict_decode64(secret.delete_prefix("whsec_")) : secret
|
|
101
|
+
id = "wh_local_#{SecureRandom.hex(4)}"
|
|
102
|
+
timestamp = Time.now.to_i.to_s
|
|
103
|
+
body = { type: "batch.completed", data: { id: "batch_123" } }.to_json
|
|
104
|
+
signature = Base64.strict_encode64(OpenSSL::HMAC.digest("sha256", secret_bytes, "#{id}.#{timestamp}.#{body}"))
|
|
105
|
+
|
|
106
|
+
# Send `body` with these headers:
|
|
107
|
+
# webhook-id: <id>
|
|
108
|
+
# webhook-timestamp: <timestamp>
|
|
109
|
+
# webhook-signature: v1,<signature>
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
- **Leave the secret unset in development**, for example have `webhook_secret` return `nil` unless `Rails.env.production?`.
|
|
113
|
+
- **Replay a real delivery** to your local app through a tunnel.
|
|
114
|
+
|
|
66
115
|
## Replay protection
|
|
67
116
|
|
|
68
|
-
`Coolhand::OpenAi::WebhookValidator` rejects requests whose `webhook-timestamp` is more than **300 seconds** away from the current time, and rejects a `webhook-id` it has already seen within that window — this prevents a captured request (from a log aggregator, APM trace, or proxy log) from being replayed to re-trigger `Coolhand::OpenAi::BatchResultProcessor`.
|
|
117
|
+
`Coolhand::OpenAi::WebhookValidator` rejects requests whose `webhook-timestamp` is more than **300 seconds** away from the current time, and rejects a `webhook-id` it has already seen within twice that window (a request dated at the edge of the tolerance stays fresh for up to two tolerances, so the id is remembered at least as long) — this prevents a captured request (from a log aggregator, APM trace, or proxy log) from being replayed to re-trigger `Coolhand::OpenAi::BatchResultProcessor`.
|
|
69
118
|
|
|
70
119
|
Both are configurable:
|
|
71
120
|
|
|
@@ -76,7 +125,7 @@ Coolhand.configure do |config|
|
|
|
76
125
|
end
|
|
77
126
|
```
|
|
78
127
|
|
|
79
|
-
The default `webhook-id` dedup store is in-memory and per-process, so it only protects a single process — replays across multiple app processes/dynos within the
|
|
128
|
+
The default `webhook-id` dedup store is in-memory and per-process, so it only protects a single process — replays across multiple app processes/dynos within the replay-protection window aren't caught by default. If you run more than one process, supply your own store via `config.webhook_id_store`; any object responding to `claim!(id, ttl_seconds)` works (`ttl_seconds` is twice `webhook_replay_tolerance_seconds`), e.g. one backed by `Rails.cache`. `claim!` must atomically check-and-record the id in one operation (not a separate read then write) — otherwise two requests racing on the same id can both "win" — so lean on your cache backend's compare-and-set primitive (`unless_exist:` below maps to Redis `SET NX` when using `RedisCacheStore`):
|
|
80
129
|
|
|
81
130
|
```ruby
|
|
82
131
|
class RailsCacheWebhookIdStore
|
|
@@ -92,4 +141,14 @@ Coolhand.configure do |config|
|
|
|
92
141
|
end
|
|
93
142
|
```
|
|
94
143
|
|
|
95
|
-
Note: the `webhook-id` is claimed as soon as the signature and timestamp are confirmed valid — before your controller's `process_event` runs. If something downstream
|
|
144
|
+
Note: the `webhook-id` is claimed as soon as the signature and timestamp are confirmed valid — before your controller's `process_event` runs. If something downstream raises (e.g. an error in your own `process_event` override, or an unexpectedly-shaped payload), the interceptor releases the claim before responding, so OpenAI's retry of the same `webhook-id` is processed rather than rejected as a replay. (`Coolhand::OpenAi::BatchResultProcessor` rescues and logs its own errors, and does its heavy work in the background, so this mainly matters for exceptions raised in your own code.)
|
|
145
|
+
|
|
146
|
+
Releasing is optional for custom stores: if your store also responds to `release!(id)`, it is called to forget the claim; a store that only implements `claim!` behaves as before and keeps the id claimed for its TTL even after a failure. To opt in, add `release!` to the store above:
|
|
147
|
+
|
|
148
|
+
```ruby
|
|
149
|
+
class RailsCacheWebhookIdStore
|
|
150
|
+
def release!(id)
|
|
151
|
+
Rails.cache.delete("coolhand:openai_webhook:#{id}")
|
|
152
|
+
end
|
|
153
|
+
end
|
|
154
|
+
```
|
data/docs/vertex.md
CHANGED
|
@@ -19,6 +19,7 @@ require "coolhand/vertex/batch_result_processor"
|
|
|
19
19
|
- Optionally pass `model:` to populate the logged `model` field. It falls back to a `"model"` key in `batch_info` if present, and is omitted from the payload entirely when neither is available. If the value (from either source) looks like a Vertex model resource path (`publishers/<publisher>/models/<id>` or `projects/<project>/locations/<location>/models/<id>`), it's normalized down to just the trailing id/version before being logged; any other string is sent as-is.
|
|
20
20
|
- `batch_info["name"]` must be the exact Vertex job resource name (`projects/<project>/locations/<location>/batchPredictionJobs/<job-id>`), and `batch_info["startTime"]`/`["endTime"]` must be valid ISO 8601 timestamps — a batch with a missing or malformed resource name or timestamps is skipped entirely (logged as an error) rather than sent with a broken URL.
|
|
21
21
|
- If `endTime` precedes `startTime` (e.g. clock skew), the batch is still logged — `duration_ms` is clamped to `0` and a warning is logged — rather than dropping the results.
|
|
22
|
+
- Each item's `"request"` and `"response"` are capped separately by [`max_captured_body_bytes`](configuration.md#body-capture-limits): one over the limit is logged as a `{"_coolhand_capture_skipped" => "body_too_large", ...}` placeholder. (See [Body Capture Limits](configuration.md#body-capture-limits) for the other placeholders.) The results array is supplied by your code, so it is already in memory; only what is sent to Coolhand is capped.
|
|
22
23
|
- Each element of the `batch_results` array passed to `.call` must be a `Hash` with a `"request"` and/or `"response"` key (the request/response body to log for that item — either key alone is accepted). Any other shape is treated as malformed and skipped, with the skip counted in a `Rails.logger.warn` summary rather than raised.
|
|
23
24
|
|
|
24
25
|
## Minimal example
|