coolhand 0.8.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +28 -0
- data/README.md +46 -2
- data/Rakefile +7 -0
- data/docs/declared_workloads.md +156 -0
- data/docs/help-center.md +120 -0
- data/docs/openai.md +4 -0
- data/lib/coolhand/api_service.rb +5 -1
- data/lib/coolhand/base_interceptor.rb +3 -1
- data/lib/coolhand/configuration.rb +14 -1
- data/lib/coolhand/declarable.rb +178 -0
- data/lib/coolhand/declared_context.rb +106 -0
- data/lib/coolhand/log_service.rb +1 -7
- data/lib/coolhand/logger_service.rb +11 -19
- data/lib/coolhand/net_http_interceptor.rb +3 -2
- data/lib/coolhand/open_ai/webhook_validator.rb +27 -15
- data/lib/coolhand/public_content_service.rb +69 -0
- data/lib/coolhand/rails_auto_declare.rb +55 -0
- data/lib/coolhand/railtie.rb +11 -0
- data/lib/coolhand/read_query.rb +14 -0
- data/lib/coolhand/read_requests.rb +11 -2
- data/lib/coolhand/response_warnings.rb +39 -0
- data/lib/coolhand/template_service.rb +26 -10
- data/lib/coolhand/version.rb +1 -1
- data/lib/coolhand/workload_service.rb +12 -8
- data/lib/coolhand/write_requests.rb +14 -0
- data/lib/coolhand.rb +41 -12
- metadata +10 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 301fa9e603334f3c6b5349755239cdf06418b9b78f1fae3de44abc352b8f1828
|
|
4
|
+
data.tar.gz: 0b687769a058ff206f50deaf46ad1e28909e7f4e4229d217ad5b36b7350ff9ce
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 9a4653638e2362853b90baf55bc1df97f73ee1144357524c61be21798b58fcffe383d9bbde1b9cb33ac7c9d07064377eff681667be8ed2a341103cdb06ab42ff
|
|
7
|
+
data.tar.gz: 464a1d38d383ebf98f3f42b9f60fccfe65d13fcc3a1ea45d5df0c7fc1f26aad721a9b2650f1c41277a09dbf5a92a24f32f2223f961c9940de592fecf54d1057d
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.10.0] - 2026-10-21
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
- **Read the Coolhand help center** ([#164](https://github.com/Coolhand-Labs/coolhand-ruby/pull/164)) — `Coolhand.public_content_service` wraps the read-only help center endpoints: `search_help_articles(query:, category:, page:, per_page:)` returns a `Coolhand::HelpArticleSearchResult` (`articles`, `pagination`), `get_help_article(slug)` returns one article with its markdown `body` and `toc`, and `list_help_categories` lists the sections. Unlike every other read service it needs **no API key**: with none configured no `X-API-Key` header is sent (anonymous callers are limited to 30 requests per minute per IP), and a configured `config.api_key` is sent and lifts the limit. Like the other read methods, these raise `Coolhand::Error` / `Coolhand::HttpError` rather than logging and returning `nil`. See [Reading the Help Center](docs/help-center.md).
|
|
14
|
+
|
|
15
|
+
### Security
|
|
16
|
+
- `LoggerService#forward_webhook` now redacts credentials (userinfo and sensitive query parameters such as `token=`) in a caller-supplied `url:` before logging it, the same way intercepted request URLs are. **Migration note:** if you pass a `url:` containing such values, they now appear as `[REDACTED]` in Coolhand.
|
|
17
|
+
|
|
18
|
+
## [0.9.0] - 2026-10-18
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
- **Declare the workload and template of captured LLM calls** ([#153](https://github.com/Coolhand-Labs/coolhand-ruby/pull/153)) — `Coolhand.with_context(reporter:, workload:, template:) { ... }` stamps every LLM call captured inside the block with the workload and template it belongs to, sent as `llm_request_log.declared`. The context is snapshotted on the calling thread, so it survives `async_logging`. Nested blocks override per field (innermost wins). `Coolhand::Declarable` (`coolhand_workload`, `coolhand_declare`, `coolhand_declare_all`) declares it once on a class, inherited by subclasses, and opt-in `config.auto_declare = [:active_job, :controller]` does the same for Rails jobs and controller actions (the gem still has no Rails dependency). `Coolhand.workload_service.upsert` and `Coolhand.template_service.upsert` create or update records by key with your **private** API key and raise `Coolhand::HttpError` on failure, and `warnings` returned with a logged request (e.g. the public-key cap) are now logged. Keys longer than 255 characters, with surrounding whitespace or a leading, trailing or doubled `/`, and a template `key` with no workload raise `ArgumentError` up front, because the server would reject and drop the whole log. Older servers ignore `declared`. See [Declared Workloads](docs/declared_workloads.md).
|
|
22
|
+
- `bin/live-examples` (also `bundle exec rake examples:live`) runs every `examples/*.rb` against real provider keys and prints one `PASS` / `SKIP` / `FAIL` line per script ([#155](https://github.com/Coolhand-Labs/coolhand-ruby/pull/155)). Development tooling only: `bin/` and `examples/` are not packaged in the gem. See [Examples](examples/README.md).
|
|
23
|
+
|
|
24
|
+
### Changed
|
|
25
|
+
- The `🔗 Net::HTTP interceptor patched` log line is no longer printed when the interceptor is installed ([#154](https://github.com/Coolhand-Labs/coolhand-ruby/pull/154)). **Migration note:** anything that matched on that line in log output should stop doing so; use `Coolhand::NetHttpInterceptor.patched?` instead.
|
|
26
|
+
- Internal de-duplication with no behavior change ([#156](https://github.com/Coolhand-Labs/coolhand-ruby/pull/156)): list reads share `ReadQuery#fetch_list`, `LoggerService` shares one webhook-rejection path, `OpenAi::WebhookValidator` shares one enforce-or-allow path (same error messages and same strict/lenient behavior by `Rails.env`), and `Coolhand.with_capture` / `without_capture` share one override helper. `contract/` (declared, intentional duplication) is excluded from the gem package.
|
|
27
|
+
- CI: added the `exhale` duplication and complexity gate ([#156](https://github.com/Coolhand-Labs/coolhand-ruby/pull/156)), and bumped `ruby/setup-ruby` to 1.329.0 ([#150](https://github.com/Coolhand-Labs/coolhand-ruby/pull/150)) and `actions/upload-artifact` to 7.0.2 ([#151](https://github.com/Coolhand-Labs/coolhand-ruby/pull/151)).
|
|
28
|
+
|
|
29
|
+
### Fixed
|
|
30
|
+
- `OpenAi::WebhookValidator` now accepts a `webhook-signature` header carrying several space-separated signatures (`v1,sigA v1,sigB`) and passes if any `v1,` entry matches ([#161](https://github.com/Coolhand-Labs/coolhand-ruby/pull/161)). Previously the whole remainder after `v1,` was compared as one value, so a legitimate webhook sent during secret rotation was rejected with a 401. Every candidate is compared in constant time, non-`v1` entries are ignored, a header that is not valid UTF-8 is rejected rather than raising, and the timestamp and replay-id checks still run only after a match. Single-signature headers behave as before. See [OpenAI Batch Webhook Handler](docs/openai.md#signature-header-and-secret-rotation).
|
|
31
|
+
|
|
32
|
+
### Security
|
|
33
|
+
- Warnings returned in a Coolhand log response are now sanitized before they are printed ([#162](https://github.com/Coolhand-Labs/coolhand-ruby/pull/162)): entries are coerced with `to_s`, invalid UTF-8 is scrubbed, control characters and line separators become a space, invisible Unicode format characters are removed, entries over 500 characters are truncated, and at most 5 are printed per response. Well-formed warnings print as before.
|
|
34
|
+
|
|
35
|
+
### Documentation
|
|
36
|
+
- `docs/declared_workloads.md` and the README now state that `Coolhand.with_context`, `with_capture` and `without_capture` are fiber-local, so work started in a new Fiber or Async task inside the block is not covered, and show opening the block inside the Fiber instead ([#160](https://github.com/Coolhand-Labs/coolhand-ruby/pull/160)).
|
|
37
|
+
|
|
10
38
|
## [0.8.0] - 2026-10-09
|
|
11
39
|
|
|
12
40
|
### Added
|
data/README.md
CHANGED
|
@@ -118,6 +118,24 @@ Coolhand.log_service.search_logs(order: 'cost_desc', min_cost: 1.0, per: 10, **w
|
|
|
118
118
|
|
|
119
119
|
See [Reading Workloads →](docs/workload-search.md) and [Reading Logs →](docs/log-search.md).
|
|
120
120
|
|
|
121
|
+
## Reading the Help Center
|
|
122
|
+
|
|
123
|
+
`Coolhand.public_content_service` reads the Coolhand help center (the same pages as coolhandlabs.com/help and /updates) with **no API key required**:
|
|
124
|
+
|
|
125
|
+
```ruby
|
|
126
|
+
require 'coolhand'
|
|
127
|
+
|
|
128
|
+
help_center = Coolhand.public_content_service
|
|
129
|
+
|
|
130
|
+
result = help_center.search_help_articles(query: 'match rate', per_page: 5)
|
|
131
|
+
article = help_center.get_help_article(result.articles.first[:slug]) # full markdown :body
|
|
132
|
+
categories = help_center.list_help_categories
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
Anonymous callers are limited to 30 requests per minute per IP. If `config.api_key` is set it is sent and lifts the limit; a key the server rejects raises a 401 `Coolhand::HttpError`. These are read methods, so they **raise** where the write methods log and return `nil`.
|
|
136
|
+
|
|
137
|
+
See [Reading the Help Center →](docs/help-center.md) for parameters, the category filter and errors.
|
|
138
|
+
|
|
121
139
|
## Linking Feedback to an Optimization
|
|
122
140
|
|
|
123
141
|
Attach feedback to an optimization as evidence, one at a time or in bulk. These require your **private** API key and **raise** `Coolhand::Error` on failure (`Coolhand::HttpError` for non-2xx responses).
|
|
@@ -132,6 +150,29 @@ links.unlink_feedback('optimizationHashid', link[:id])
|
|
|
132
150
|
|
|
133
151
|
For batching, error semantics, and details, see [Linking Feedback →](docs/feedback-links.md).
|
|
134
152
|
|
|
153
|
+
## Declaring Workloads and Templates
|
|
154
|
+
|
|
155
|
+
Tell Coolhand which workload and template an LLM call belongs to, instead of waiting for it to be matched. Calls captured inside the block carry the declared identity:
|
|
156
|
+
|
|
157
|
+
```ruby
|
|
158
|
+
Coolhand.with_context(reporter: 'my-app', workload: { key: 'Parser#call' }, template: { key: 'Parser#call/summarize' }) do
|
|
159
|
+
client.chat(...) # captured as usual
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
Coolhand.workload_service.upsert(key: 'Parser#call', name: 'Parser') # private key; create or rename
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Or declare it once on the class and let every capture inside it pick it up, subclasses included:
|
|
166
|
+
|
|
167
|
+
```ruby
|
|
168
|
+
class ApplicationService
|
|
169
|
+
include Coolhand::Declarable
|
|
170
|
+
coolhand_declare_all # each public method declares "ClassName#method"
|
|
171
|
+
end
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
For nesting rules, class-level declarations, Rails jobs and controllers, upserts, and public-key limits, see [Declared Workloads →](docs/declared_workloads.md).
|
|
175
|
+
|
|
135
176
|
## Rails Integration
|
|
136
177
|
|
|
137
178
|
### Configuration
|
|
@@ -212,6 +253,7 @@ end
|
|
|
212
253
|
| `exclude_api_patterns` | Array | `["/batchPredictionJobs/"]` | Deny-list checked after `intercept_addresses`; matching paths are skipped. Unlike `intercept_addresses`, `exclude_api_patterns = []` genuinely disables exclusion. See [Configuration](docs/configuration.md) |
|
|
213
254
|
| `max_captured_body_bytes` | Integer | `1_000_000` | Maximum size of a captured JSON request body — oversized bodies are replaced with a placeholder. Non-JSON bodies (e.g. file/audio uploads) are always skipped regardless of size — see [Advanced Configuration](docs/configuration.md) |
|
|
214
255
|
| `async_logging` | Boolean | `true` | Send logs to Coolhand from a background thread so a slow Coolhand API never delays your LLM calls. Set to `false` to send inline — see [Advanced Configuration](docs/configuration.md#asynchronous-logging) |
|
|
256
|
+
| `auto_declare` | Array | `[]` | Rails only: `:active_job` and/or `:controller` to declare a workload and template for LLM calls made from jobs and controller actions — see [Declared Workloads](docs/declared_workloads.md#rails-auto-declare) |
|
|
215
257
|
|
|
216
258
|
## Usage Examples
|
|
217
259
|
|
|
@@ -273,7 +315,7 @@ Coolhand.without_capture do
|
|
|
273
315
|
end
|
|
274
316
|
```
|
|
275
317
|
|
|
276
|
-
Both methods are thread-safe. Blocks nest correctly — the innermost block takes precedence, and the outer setting is restored when the block exits.
|
|
318
|
+
Both methods are thread-safe. Blocks nest correctly — the innermost block takes precedence, and the outer setting is restored when the block exits. The override is fiber-local, so it does not follow into new Threads, Fibers or Async tasks started inside the block; see [Threads and background logging](docs/declared_workloads.md#threads-and-background-logging).
|
|
277
319
|
|
|
278
320
|
## Logging Inbound Webhooks
|
|
279
321
|
|
|
@@ -464,7 +506,7 @@ The monitor handles errors gracefully:
|
|
|
464
506
|
- Failed API logging attempts are logged to console but don't interrupt your application
|
|
465
507
|
- Invalid API keys will be reported but won't crash your app
|
|
466
508
|
- Network issues are handled with appropriate error messages
|
|
467
|
-
- The read methods (`search_templates`, `get_template`, `search_workloads`, `search_logs`, `get_log`)
|
|
509
|
+
- The read methods (`search_templates`, `get_template`, `search_workloads`, `search_logs`, `get_log`, `search_help_articles`, `get_help_article`, `list_help_categories`), the feedback-link methods and the `upsert` methods are the exception — they raise `Coolhand::Error` (`Coolhand::HttpError` for non-2xx responses); see [Reading Templates →](docs/template-search.md), [Reading Workloads →](docs/workload-search.md), [Reading Logs →](docs/log-search.md), [Linking Feedback →](docs/feedback-links.md) and [Declared Workloads →](docs/declared_workloads.md)
|
|
468
510
|
|
|
469
511
|
## Documentation
|
|
470
512
|
|
|
@@ -473,7 +515,9 @@ The monitor handles errors gracefully:
|
|
|
473
515
|
- **[Reading Templates](docs/template-search.md)** — Search LLM request templates and fetch a single one, prompt patterns included, using the private API key
|
|
474
516
|
- **[Reading Workloads](docs/workload-search.md)** — List workloads with optional cost and performance metrics over a rolling or explicit time window
|
|
475
517
|
- **[Reading Logs](docs/log-search.md)** — Search logs with per-log cost, filter and sort by cost, and fetch one log with its cost breakdown
|
|
518
|
+
- **[Reading the Help Center](docs/help-center.md)** — Search and read Coolhand's help center articles and list its sections, with no API key required
|
|
476
519
|
- **[Linking Feedback](docs/feedback-links.md)** — Link feedback to an optimization as evidence, singly or in bulk, using the private API key
|
|
520
|
+
- **[Declared Workloads](docs/declared_workloads.md)** — Declare the workload and template of captured LLM calls with `Coolhand.with_context`, and create or rename them with `upsert`
|
|
477
521
|
- **[Anthropic Integration](docs/anthropic.md)** — Official and community Anthropic Ruby gems, streaming, dual gem handling, and troubleshooting
|
|
478
522
|
- **[ElevenLabs Integration](docs/elevenlabs.md)** — Webhook capture, feedback submission, and Rails integration
|
|
479
523
|
- **[OpenAI Batch Webhook Handler](docs/openai.md)** — Handle OpenAI batch job completion events via webhook interception
|
data/Rakefile
CHANGED
|
@@ -9,4 +9,11 @@ require "rubocop/rake_task"
|
|
|
9
9
|
|
|
10
10
|
RuboCop::RakeTask.new
|
|
11
11
|
|
|
12
|
+
namespace :examples do
|
|
13
|
+
desc "Run every live script in examples/ against real provider keys (loads keys via zsh -ic)"
|
|
14
|
+
task :live do
|
|
15
|
+
sh RbConfig.ruby, File.expand_path("bin/live-examples", __dir__)
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
|
|
12
19
|
task default: %i[spec rubocop]
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
# Declaring workloads and templates
|
|
2
|
+
|
|
3
|
+
By default Coolhand works out which workload and template an LLM call belongs to by matching the log against known templates. If your code already knows (a service object, a prompt library, an adapter), declare it instead. A declared log skips matching entirely and is filed under the workload and template you named.
|
|
4
|
+
|
|
5
|
+
There are two parts:
|
|
6
|
+
|
|
7
|
+
- `Coolhand.with_context` stamps the LLM calls the gem captures automatically.
|
|
8
|
+
- `Coolhand.workload_service.upsert` and `Coolhand.template_service.upsert` create or update workloads and templates explicitly. They need your **private** API key.
|
|
9
|
+
|
|
10
|
+
## Declaring on captured calls
|
|
11
|
+
|
|
12
|
+
```ruby
|
|
13
|
+
Coolhand.with_context(
|
|
14
|
+
reporter: "squishling",
|
|
15
|
+
workload: { key: "Parser#call", name: "Parser#call", description: "Parses incoming tickets" },
|
|
16
|
+
template: { key: "Parser#call/squish:L2", name: "Parser#call · squish L2" }
|
|
17
|
+
) do
|
|
18
|
+
client.chat(...) # captured as usual, and sent with the declared identity
|
|
19
|
+
end
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
| Field | Accepts | Notes |
|
|
23
|
+
|---|---|---|
|
|
24
|
+
| `reporter` | String | Who is making the declaration, e.g. the library or app. Used when a record is first created. |
|
|
25
|
+
| `workload` | Hash: `id`, `key`, `name`, `description` | Needs an `id` or a `key`. Other keys are ignored. |
|
|
26
|
+
| `template` | Hash: `id`, `key`, `name` | Needs an `id` or a `key`. Other keys are ignored. |
|
|
27
|
+
|
|
28
|
+
- `id` refers to an existing record. `key` is a stable string of your choosing; Coolhand finds or creates the record with that key. On a template, `id` wins over `key`.
|
|
29
|
+
- `name`, `description` and `reporter` are used only when a record is **first created**. Declaring never renames or re-attributes an existing record; use `upsert` for that. A display name that is already taken gets a numeric suffix, and names such as "Unmatched" are reserved.
|
|
30
|
+
- A declaration with only a `workload` gets a default template keyed `<workload key>/default`. A template given by `key` needs a workload alongside it, either in the same block or an enclosing one.
|
|
31
|
+
- Keys are at most 255 characters, printable, and cannot start or end with whitespace or `/` or contain `//`. `#`, `:` and `·` are fine. Descriptions are cut to 1000 characters.
|
|
32
|
+
- Keys must be stable, so never put a request id or timestamp in one: each distinct key creates a record.
|
|
33
|
+
- A `workload` or `template` that is not a Hash, has neither `id` nor `key`, has an invalid key, or a template `key` with no workload raises `ArgumentError` before your block runs. The server would answer these with a `422` and **not save the log**.
|
|
34
|
+
- An unknown `id` (or one from another client) gets the same `422` and the log is not saved, so prefer `key`.
|
|
35
|
+
|
|
36
|
+
### Nesting
|
|
37
|
+
|
|
38
|
+
Blocks nest. Each field (`reporter`, `workload`, `template`) is taken from the innermost block that sets it, and fields an inner block leaves out are inherited (with the one exception below):
|
|
39
|
+
|
|
40
|
+
```ruby
|
|
41
|
+
Coolhand.with_context(reporter: "my-app", workload: { key: "Parser#call" }) do
|
|
42
|
+
Coolhand.with_context(template: { key: "Parser#call/summarize" }) do
|
|
43
|
+
# reporter: "my-app", workload: Parser#call, template: Parser#call/summarize
|
|
44
|
+
end
|
|
45
|
+
end
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
A template belongs to a workload, so an inner block that declares a `workload` without a `template` does not inherit the outer block's template. Declare both together when you change workload.
|
|
49
|
+
|
|
50
|
+
A `workload` or `template` Hash is replaced as a whole, not merged key by key. An inner `workload: { key: "b" }` inside an outer `workload: { id: "a" }` declares only `b`, rather than a mix of `id` and `key`.
|
|
51
|
+
|
|
52
|
+
### Threads and background logging
|
|
53
|
+
|
|
54
|
+
The context is thread-local, like `Coolhand.with_capture`. It is also fiber-local, so it does not carry into threads, fibers, `Enumerator#next` blocks or async tasks you start inside the block; calls made there are captured without a declaration. The identity is read on the thread that made the LLM call, before the log is queued, so it survives `async_logging` (the default).
|
|
55
|
+
|
|
56
|
+
This matters for Fiber-scheduler concurrency such as [Async](https://github.com/socketry/async). Nothing raises: the call is logged, just without a workload or template.
|
|
57
|
+
|
|
58
|
+
```ruby
|
|
59
|
+
Coolhand.with_context(workload: { key: "Parser#call" }) do
|
|
60
|
+
Async do
|
|
61
|
+
client.chat(...) # runs in a new Fiber: captured with no declared workload
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
`Coolhand.with_capture` and `Coolhand.without_capture` work the same way, so a `without_capture` block does not suppress calls made from a Fiber started inside it, and with `config.capture = false` a `with_capture` block does not enable them. To cover work that runs in another Fiber, open the block inside that Fiber:
|
|
67
|
+
|
|
68
|
+
```ruby
|
|
69
|
+
Async do
|
|
70
|
+
Coolhand.with_context(workload: { key: "Parser#call" }) do
|
|
71
|
+
client.chat(...) # captured with the declared workload
|
|
72
|
+
end
|
|
73
|
+
end
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Public keys
|
|
77
|
+
|
|
78
|
+
Declarations made with a public key are capped per client at 200 workloads and 1,000 templates. Past the cap, Coolhand ignores the declaration, ingests the log normally, and returns a warning, which the gem prints unless `config.silent` is on. Server-supplied warnings are sanitized before printing (control and invisible characters removed, each entry cut at 500 characters, at most 5 printed per response). Declarations from a private key are not capped.
|
|
79
|
+
|
|
80
|
+
Older Coolhand servers ignore `declared`, so it is safe to use against them.
|
|
81
|
+
|
|
82
|
+
## Declaring on a class
|
|
83
|
+
|
|
84
|
+
Wrapping every call site in `with_context` does not suit passive capture. Include `Coolhand::Declarable` and declare the identity once, on the class. Wrapped methods run inside `with_context`, so everything above (nesting, precedence, async) applies unchanged.
|
|
85
|
+
|
|
86
|
+
```ruby
|
|
87
|
+
class ApplicationService
|
|
88
|
+
include Coolhand::Declarable
|
|
89
|
+
|
|
90
|
+
coolhand_workload reporter: "my-app"
|
|
91
|
+
coolhand_declare_all
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
class Parser < ApplicationService
|
|
95
|
+
def call(ticket) = client.chat(...) # workload "Parser#call", template "Parser#call"
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
class Summariser < ApplicationService
|
|
99
|
+
coolhand_workload key: "Summariser" # one workload for every method; templates stay per method
|
|
100
|
+
|
|
101
|
+
def call(ticket) = client.chat(...) # workload "Summariser", template "Summariser#call"
|
|
102
|
+
end
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
- `coolhand_declare :call, :other` wraps just those methods, whether they are defined before or after the line. `template:` takes a key or a `{ key:, name: }` Hash instead of the default `"ClassName#method"`: `coolhand_declare :call, template: "Parser#call/summarise"`.
|
|
106
|
+
- `coolhand_declare_all` wraps every public instance method defined after it, in the class and its subclasses, so put it at the top of the class body. Private and protected methods are left alone, however they are made private (`private def`, `private :name`, or a `private` section), as are attribute readers and writers and object-protocol methods such as `to_s`, `==` and `method_missing`. A method you name in `coolhand_declare` is wrapped regardless.
|
|
107
|
+
- The key uses the **runtime** class, so a method inherited from `ApplicationService` declares as `Parser#call` when called on a `Parser`. Anonymous classes are not declared.
|
|
108
|
+
- `coolhand_workload(key:, name:, description:, reporter:)` sets class-level defaults and is inherited by subclasses, field by field, with the nearest class winning. Without a `key`, each method gets its own workload, `ClassName#method`. `name` and `description` need a `key`, since one name shared by per-method workloads would collide.
|
|
109
|
+
- Services that call each other nest: the innermost wrapped method wins, and it brings its own template (see [Nesting](#nesting)).
|
|
110
|
+
- It only works where the LLM call runs on the same thread and fiber as the wrapped method (see [Threads and background logging](#threads-and-background-logging)).
|
|
111
|
+
|
|
112
|
+
## Rails auto-declare
|
|
113
|
+
|
|
114
|
+
For Rails apps, Coolhand can declare jobs and controller actions without touching their classes. It is off by default, because every distinct job and action becomes a workload.
|
|
115
|
+
|
|
116
|
+
```ruby
|
|
117
|
+
Coolhand.configure do |config|
|
|
118
|
+
config.api_key = ENV.fetch("COOLHAND_API_KEY")
|
|
119
|
+
config.auto_declare = [:active_job, :controller]
|
|
120
|
+
end
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
| Target | Wraps | Declares |
|
|
124
|
+
|---|---|---|
|
|
125
|
+
| `:active_job` | `around_perform` on `ActiveJob::Base` | `SummariseJob#perform`, reporter `rails` |
|
|
126
|
+
| `:controller` | `around_action` on `ActionController::Base` and `::API` | `UsersController#show`, reporter `rails` |
|
|
127
|
+
|
|
128
|
+
Coolhand must be loaded after Rails (the default with `Bundler.require` in a Rails app), otherwise the hooks are never registered. They are installed after your initializers have run, so `Coolhand.configure` can come before or after anything else. Anything declared inside the job or action (a `Declarable` service, a `with_context` block) takes precedence over the job or action declaration. With a public API key the per-client declaration cap applies (see [Public keys](#public-keys)), so enable only the targets you want as workloads.
|
|
129
|
+
|
|
130
|
+
## Creating and renaming explicitly
|
|
131
|
+
|
|
132
|
+
```ruby
|
|
133
|
+
Coolhand.configure { |config| config.api_key = ENV.fetch("COOLHAND_PRIVATE_API_KEY") }
|
|
134
|
+
|
|
135
|
+
workload = Coolhand.workload_service.upsert(
|
|
136
|
+
key: "Parser#call", name: "Parser", description: "Parses incoming tickets", reporter: "my-app"
|
|
137
|
+
)
|
|
138
|
+
|
|
139
|
+
Coolhand.template_service.upsert(
|
|
140
|
+
key: "Parser#call/summarize",
|
|
141
|
+
workload: { key: "Parser#call" }, # or { id: workload[:id] }
|
|
142
|
+
name: "Parser · summarize",
|
|
143
|
+
reporter: "my-app"
|
|
144
|
+
)
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
- Both are `PUT .../by_key/<key>`: they create the record if the key is new and update it otherwise. The server answers `201` on create and `200` on update; both return the record as a Hash.
|
|
148
|
+
- The key never changes. Calling `upsert` with the same key and a new `name` renames the record, and an update changes only the fields you send.
|
|
149
|
+
- A template needs a `workload` (`{ id: }` or `{ key: }`, sent to the server as `workload_id` / `workload_key`) when it is new. For an existing template, naming a different workload is a `422`.
|
|
150
|
+
- A key may contain `/`, `#`, `:` and other characters; the gem escapes it for the URL.
|
|
151
|
+
|
|
152
|
+
## Errors
|
|
153
|
+
|
|
154
|
+
`upsert` raises, unlike the logging writes. A non-2xx response raises `Coolhand::HttpError` whose `status` is the HTTP code (`401` for a missing or public key, `422` for an invalid key, a taken name, a missing or unknown workload, or a key held by a deprecated template, `504` if the server timed out). A blank key, or one made of `.` / `..` path segments, and transport failures raise `Coolhand::Error`; a `workload` that is not a Hash raises `ArgumentError`.
|
|
155
|
+
|
|
156
|
+
`config.debug_mode` does not suppress `upsert`: it changes server state, so it is always sent.
|
data/docs/help-center.md
ADDED
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
# Reading the Help Center
|
|
2
|
+
|
|
3
|
+
`Coolhand::PublicContentService` reads Coolhand's help center: the articles served at
|
|
4
|
+
`coolhandlabs.com/help/:slug` and the posts at `/updates/:slug`. "Help center" is the user-facing name;
|
|
5
|
+
on the wire the resource is `public_content` (`GET /api/v2/public_content`, `/:slug` and `/categories`).
|
|
6
|
+
It is read-only. Creating, updating and archiving articles is Coolhand-internal and not wrapped.
|
|
7
|
+
|
|
8
|
+
**Auth:** none required. Unlike every other service, no API key is needed:
|
|
9
|
+
|
|
10
|
+
```ruby
|
|
11
|
+
require "coolhand"
|
|
12
|
+
|
|
13
|
+
help_center = Coolhand.public_content_service # works with Coolhand.configure never called
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
- No key configured: no `X-API-Key` header is sent, and the calls work. Anonymous callers are limited to
|
|
17
|
+
**30 requests per minute per IP**; beyond that the server answers `429`.
|
|
18
|
+
- A configured `config.api_key` is sent as `X-API-Key` and callers with a valid key are not rate limited.
|
|
19
|
+
This gem reads the key from configuration only, not from the environment.
|
|
20
|
+
- A key the server does not recognise is a `401` (`{"error":"Invalid API key."}`) even though the endpoint is
|
|
21
|
+
public. It is raised, not retried anonymously.
|
|
22
|
+
|
|
23
|
+
## `search_help_articles`
|
|
24
|
+
|
|
25
|
+
```ruby
|
|
26
|
+
result = help_center.search_help_articles(query: "match rate", category: "feedback-and-prompts", per_page: 10)
|
|
27
|
+
|
|
28
|
+
result.articles.each { |a| puts "#{a[:title]}: #{a[:search_headline]}" }
|
|
29
|
+
result.pagination.total_count
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
### Keywords
|
|
33
|
+
|
|
34
|
+
All optional.
|
|
35
|
+
|
|
36
|
+
| keyword | type | notes |
|
|
37
|
+
|---|---|---|
|
|
38
|
+
| `query` | String | Relevance search (Postgres full text, the same as the website's). Supports quoted phrases, `-exclusions` and `or`. Omit for a full listing. A blank string counts as omitted. |
|
|
39
|
+
| `category` | String | Restrict to one section by slug (see `list_help_categories`). A blank string counts as omitted. |
|
|
40
|
+
| `page` | Integer | 1-based. |
|
|
41
|
+
| `per_page` | Integer | Default 25, max 100, both enforced server-side. |
|
|
42
|
+
|
|
43
|
+
The endpoint has no `category` parameter. `category` is sent in its structured form instead, which also
|
|
44
|
+
moves the relevance search to a different key; the plain `q=...` string and the `q[...]` form cannot be
|
|
45
|
+
mixed:
|
|
46
|
+
|
|
47
|
+
| `query` | `category` | Sent |
|
|
48
|
+
|---|---|---|
|
|
49
|
+
| set | unset | `q=<query>` |
|
|
50
|
+
| unset | set | `q[help_category_slug_eq]=<category>` |
|
|
51
|
+
| set | set | `q[help_category_slug_eq]=<category>&q[full_text]=<query>` |
|
|
52
|
+
|
|
53
|
+
### Return value
|
|
54
|
+
|
|
55
|
+
A `Coolhand::HelpArticleSearchResult` with `articles` (an Array of Symbol-keyed Hashes) and `pagination`.
|
|
56
|
+
Each article is a summary; the markdown `:body` is omitted from lists:
|
|
57
|
+
|
|
58
|
+
```ruby
|
|
59
|
+
{
|
|
60
|
+
slug: "what-is-feedback-match-rate",
|
|
61
|
+
title: "What is feedback match rate?",
|
|
62
|
+
answer: "Match rate is the share of ...", # String or nil
|
|
63
|
+
article_kind: "concept", # reference | concept | how_to | troubleshooting | faq_answer | release_note
|
|
64
|
+
tags: ["feedback", "matching"],
|
|
65
|
+
category: "feedback-and-prompts", # section slug
|
|
66
|
+
url: "https://coolhandlabs.com/help/what-is-feedback-match-rate",
|
|
67
|
+
markdown_url: "https://coolhandlabs.com/help/what-is-feedback-match-rate.md",
|
|
68
|
+
reading_time_minutes: 10, # Integer or nil
|
|
69
|
+
search_headline: "<mark>Match</mark> rate ...", # nil on a plain listing
|
|
70
|
+
published_at: "2026-08-30T13:57:52Z", # String or nil
|
|
71
|
+
updated_content_at: "2026-08-30T13:57:52Z" # String or nil
|
|
72
|
+
}
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
`:search_headline` is the matching passage with `<mark>` tags around the matched terms, so you can see
|
|
76
|
+
why something matched without fetching the body.
|
|
77
|
+
|
|
78
|
+
`pagination` is a `Coolhand::Pagination`, the same as the other read methods, built from the `X-Page`,
|
|
79
|
+
`X-Per-Page`, `X-Total-Count` and `X-Total-Pages` response headers (`current_page`, `per_page`,
|
|
80
|
+
`total_count`, `total_pages`, `has_next_page`, `has_prev_page`).
|
|
81
|
+
|
|
82
|
+
## `get_help_article(slug)`
|
|
83
|
+
|
|
84
|
+
```ruby
|
|
85
|
+
article = help_center.get_help_article("what-is-feedback-match-rate")
|
|
86
|
+
article[:body] # full markdown
|
|
87
|
+
article[:toc] # [{ id:, text:, level:, children: }]
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Returns a Hash: the summary fields minus `:search_headline`, plus `:body` (full markdown), `:noindex`
|
|
91
|
+
(Boolean) and `:toc` (the heading structure, h3s nested under their h2). The slug is escaped as a single
|
|
92
|
+
path segment. Raises `Coolhand::Error` before any request if `slug` is blank or a bare `.`/`..`. An unknown
|
|
93
|
+
or unpublished slug raises `Coolhand::HttpError` with `status` 404.
|
|
94
|
+
|
|
95
|
+
## `list_help_categories`
|
|
96
|
+
|
|
97
|
+
```ruby
|
|
98
|
+
help_center.list_help_categories
|
|
99
|
+
# [{ slug: "providers", name: "Provider guides", description: "...", url_prefix: "/help", allowed_tags: nil }, ...]
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Returns an Array of Hashes: `:slug` (the `category` filter above), `:name`, `:description`, `:url_prefix`
|
|
103
|
+
(`/help` or `/updates`) and `:allowed_tags` (the only tags the section accepts; `nil` means any tag).
|
|
104
|
+
|
|
105
|
+
## Errors
|
|
106
|
+
|
|
107
|
+
These are read methods, so they raise where the logging writes log and return `nil`. A network failure or
|
|
108
|
+
a non-JSON body raises `Coolhand::Error`. A non-2xx response raises `Coolhand::HttpError`, whose `status`
|
|
109
|
+
holds the HTTP status code and `body` the (truncated) response body: `401` for an invalid key, `404` for an
|
|
110
|
+
unknown slug, `429` when the anonymous rate limit is exceeded.
|
|
111
|
+
|
|
112
|
+
## Verifying against a live server
|
|
113
|
+
|
|
114
|
+
`spec/live/coolhand_public_content_live_spec.rb` runs against a real server with no key:
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
COOLHAND_SPEC_LIVE_BASE_URL=http://127.0.0.1:3000/api bundle exec rspec spec/live/coolhand_public_content_live_spec.rb
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
It makes about nine anonymous requests, so running it repeatedly within a minute can hit the `429`.
|
data/docs/openai.md
CHANGED
|
@@ -59,6 +59,10 @@ end
|
|
|
59
59
|
# ...existing code...
|
|
60
60
|
```
|
|
61
61
|
|
|
62
|
+
## Signature header and secret rotation
|
|
63
|
+
|
|
64
|
+
The `webhook-signature` header may carry several space-separated signatures (`v1,sigA v1,sigB`), as senders produce while a webhook secret is being rotated. `Coolhand::OpenAi::WebhookValidator` accepts the request if any `v1,` entry matches your secret, comparing each in constant time and ignoring entries with any other version prefix. The timestamp and replay checks below still run, and only after a signature matches.
|
|
65
|
+
|
|
62
66
|
## Replay protection
|
|
63
67
|
|
|
64
68
|
`Coolhand::OpenAi::WebhookValidator` rejects requests whose `webhook-timestamp` is more than **300 seconds** away from the current time, and rejects a `webhook-id` it has already seen within that window — this prevents a captured request (from a log aggregator, APM trace, or proxy log) from being replayed to re-trigger `Coolhand::OpenAi::BatchResultProcessor`.
|
data/lib/coolhand/api_service.rb
CHANGED
|
@@ -6,12 +6,14 @@ require "json"
|
|
|
6
6
|
require_relative "collector"
|
|
7
7
|
require_relative "errors"
|
|
8
8
|
require_relative "read_requests"
|
|
9
|
+
require_relative "response_warnings"
|
|
9
10
|
|
|
10
11
|
module Coolhand
|
|
11
12
|
class ApiService
|
|
12
13
|
# The GET half of this class. See Coolhand::ReadRequests for why reads raise where writes
|
|
13
14
|
# log and return nil.
|
|
14
15
|
include ReadRequests
|
|
16
|
+
include ResponseWarnings
|
|
15
17
|
|
|
16
18
|
# Caps how much of a failed response body is interpolated into a log line or an exception
|
|
17
19
|
# message. Without it an oversized body from a proxy or gateway becomes the message.
|
|
@@ -36,7 +38,9 @@ module Coolhand
|
|
|
36
38
|
log JSON.pretty_generate(sanitize_payload_for_json(payload))
|
|
37
39
|
nil
|
|
38
40
|
else
|
|
39
|
-
send_request(payload, "✅ Successfully sent request metadata")
|
|
41
|
+
result = send_request(payload, "✅ Successfully sent request metadata")
|
|
42
|
+
log_response_warnings(result)
|
|
43
|
+
result
|
|
40
44
|
end
|
|
41
45
|
end
|
|
42
46
|
|
|
@@ -149,7 +149,8 @@ module Coolhand
|
|
|
149
149
|
end
|
|
150
150
|
|
|
151
151
|
def send_complete_request_log(request_id:, method:, url:, request_headers:, request_body:, response_headers:,
|
|
152
|
-
response_body:, status_code:, start_time:, end_time:, duration_ms:, is_streaming:, source_api: nil, model: nil
|
|
152
|
+
response_body:, status_code:, start_time:, end_time:, duration_ms:, is_streaming:, source_api: nil, model: nil,
|
|
153
|
+
declared: nil)
|
|
153
154
|
raw_request = {
|
|
154
155
|
id: request_id,
|
|
155
156
|
timestamp: start_time.iso8601,
|
|
@@ -168,6 +169,7 @@ module Coolhand
|
|
|
168
169
|
raw_request[:model] = model if Coolhand.required_field?(model)
|
|
169
170
|
|
|
170
171
|
request_data = { raw_request: raw_request }
|
|
172
|
+
request_data[:declared] = declared if declared && !declared.empty?
|
|
171
173
|
|
|
172
174
|
LogQueue.submit do
|
|
173
175
|
Coolhand::ApiService.new.send_llm_request_log(request_data)
|
|
@@ -30,7 +30,7 @@ module Coolhand
|
|
|
30
30
|
|
|
31
31
|
attr_accessor :api_key, :environment, :silent, :debug_mode, :capture, :exclude_api_patterns, :enabled,
|
|
32
32
|
:max_captured_body_bytes, :webhook_replay_tolerance_seconds, :webhook_id_store, :async_logging
|
|
33
|
-
attr_reader :intercept_addresses, :intercept_path_patterns, :base_url
|
|
33
|
+
attr_reader :intercept_addresses, :intercept_path_patterns, :base_url, :auto_declare
|
|
34
34
|
|
|
35
35
|
def initialize
|
|
36
36
|
# Set defaults
|
|
@@ -45,6 +45,7 @@ module Coolhand
|
|
|
45
45
|
@exclude_api_patterns = DEFAULT_EXCLUDE_API_PATTERNS.dup
|
|
46
46
|
@enabled = true
|
|
47
47
|
@async_logging = true
|
|
48
|
+
@auto_declare = []
|
|
48
49
|
@max_captured_body_bytes = DEFAULT_MAX_CAPTURED_BODY_BYTES
|
|
49
50
|
@webhook_replay_tolerance_seconds = 300
|
|
50
51
|
@webhook_id_store = Coolhand::OpenAi::WebhookIdStore.new
|
|
@@ -74,6 +75,18 @@ module Coolhand
|
|
|
74
75
|
@intercept_path_patterns = value.is_a?(Array) ? value : [value]
|
|
75
76
|
end
|
|
76
77
|
|
|
78
|
+
# Rails entry points to declare a workload/template for automatically: any of
|
|
79
|
+
# RailsAutoDeclare::TARGETS (:active_job, :controller). Off by default. See docs/declared_workloads.md.
|
|
80
|
+
def auto_declare=(value)
|
|
81
|
+
targets = Array(value).map { |target| target.to_s.to_sym }
|
|
82
|
+
unknown = targets - RailsAutoDeclare::TARGETS
|
|
83
|
+
unless unknown.empty?
|
|
84
|
+
raise Error, "auto_declare supports #{RailsAutoDeclare::TARGETS.inspect}, got #{unknown.inspect}"
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
@auto_declare = targets.uniq
|
|
88
|
+
end
|
|
89
|
+
|
|
77
90
|
def base_url=(value)
|
|
78
91
|
stripped = value&.sub(%r{/+\z}, "")
|
|
79
92
|
raise Error, BASE_URL_ERROR_MSG unless stripped.nil? || valid_base_url?(stripped)
|