coolhand 0.7.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8f38b442e23ef4733417edbaa234c3459734a11f861caeb4261ea034f7a464dd
-  data.tar.gz: c3d6aee98d5eea898ff53bedba1f43497c82517ad335d330d5e6619a5035e13f
+  metadata.gz: 48857d802183f91a163c82c3e1dd89ce34ac7a8f1f3d1079c5ae6d5d594ad300
+  data.tar.gz: a852465d5b9dced73580eff8c81f45093a97c48c79a66579cff27c2dbedc7cbe
 SHA512:
-  metadata.gz: feafe561a70cd33c6c995f1ebd20c1d93e93ca9bb092dc6214f5b52464eb23e9ac68797d2c64377a2e215ed9ca0d50fa1b4b43f91dd2b355577ca33a2c80483f
-  data.tar.gz: 749542ecea37f0d9f08840bf6e0eda16fe5aa65f4484abb1e720c3358667190137509e99f80d798e0b3292809f95efd7823e7655a8c05db1b0747ebdcf784b02
+  metadata.gz: 0d4e6bc5e9ef29365afaba5b9c4a4edbadb552f77268adfe27ac6276f23cd44e4c75fe6df85b1ddef8bdf25d99d88204a46a127dcce7c3a2183edc09e5019508
+  data.tar.gz: ee192c55d280b47abc3f6a9af7f2ddd48d4966493955cee5614557feeea2d87d71a28c16f488fee4dc9fddd916b88fc9ee98621e058e3ad0de80431fc6bd871c

data/AGENTS.harness.md

@@ -39,8 +39,8 @@ node <workspaceRoot>/coolhand/harness/harness.mjs my-issue --run <RUN_DIR> --rep
 2. `<workspaceRoot>/coolhand/harness/RESIST_RULES.md` — the refuse list.
 3. The API definition at `specPath`. It is your only source of truth **for the endpoint's
    contract** — paths, params, response fields, status codes.
-4. `coolhand-ruby/CLAUDE.md` — this repo's own rulebook. **It is authoritative.** Where it
-   disagrees with this harness file, CLAUDE.md wins — follow it and say so in your PR.
+4. `coolhand-ruby/AGENTS.md` — this repo's own rulebook. **It is authoritative.** Where it
+   disagrees with this harness file, AGENTS.md wins — follow it and say so in your PR.
 
 **Your issue links node's PR as the reference implementation. Use it for structure, not
 for facts.** Node went first so you do not have to rediscover how a REST method fits into
@@ -70,7 +70,7 @@ inside the current architecture, escalate and STOP.
 Require it inside the file that actually uses it, executed only when that provider's
 functionality is accessed. Clients may not use every provider and must not be forced to
 install gems they do not need — apps consuming this as a path gem break on missing optional
-dependencies. See `coolhand-ruby/CLAUDE.md`.
+dependencies. See `coolhand-ruby/AGENTS.md`.
 
 ## 3. Prove it against the real server
 
@@ -107,13 +107,44 @@ node <workspaceRoot>/coolhand/harness/harness.mjs wait --run <RUN_DIR> --channel
 
 Name the rule number (`R1`–`R5`). Do not guess. Do not stub. Do not work around it.
 
-## 5. Run your review skill
-
-**Mandatory, before you push.** Follow `<workspaceRoot>/coolhand/harness/RESIST_RULES.md`
-→ "Before you push: run your repo's review skill" (R8) — load
-`.claude/skills/loop-review/SKILL.md` off disk and run it for real against your diff. Do
-not approximate its steps manually; if you cannot spawn the reviewer subagent it calls
-for, escalate to node (R8) and STOP rather than substitute a self-review.
+## 5. Get your review — you cannot run it yourself
+
+**Mandatory, before you push. You do not run this section yourself.** You are three
+levels deep in this tree (server → node → you), and the platform does not let an agent at
+this depth spawn the reviewer subagent the review skill needs — see
+`<workspaceRoot>/coolhand/harness/RESIST_RULES.md` → R8's depth table. This is not something
+to confirm by trying; go straight to the request below.
+
+1. Commit locally first — build, prove it against the local server, run `bundle exec rake`,
+   all of section 3, all done before this step.
+2. Send an R8 escalation to node naming your repo path, branch, and `HEAD` sha:
+   ```
+   node <workspaceRoot>/coolhand/harness/harness.mjs send --run <RUN_DIR> --channel ruby \
+     --from ruby --to node --kind escalation \
+     --text "R8: ready for review. repo=<workspaceRoot>/coolhand-ruby branch=<branch> sha=<sha>"
+   ```
+3. `wait --after` the `messageId` your `send` in step 2 just printed. A timeout is silence,
+   not permission — call `wait` again, `--after` that same id, rather than pushing
+   (`RESIST_RULES.md` → "What escalate and stop means, mechanically"). If you loop back here
+   from step 5, use the fresh escalation's `messageId` instead.
+4. **The first reply is an `ack`, not the answer** — it means node has started, not that it
+   is done. `wait` again, `--after` that ack's `messageId`, for the `resolution` that
+   actually follows.
+5. **If what follows is a second `ack` instead of a `resolution`, node is asking you to do
+   something before it can start — most often that your checkout is not clean** (see
+   `RESIST_RULES.md` → R8 → "Serving a review for a child" step 2). Do what it asks, then
+   send a fresh R8 escalation with your current `HEAD` sha (it may have changed) and go back
+   to step 3. Do not just `wait` again — nothing arrives until you re-escalate.
+6. **The `resolution` that eventually arrives always carries the full Iteration Breakdown
+   table plus an explicit `CLEAN`/`capped` verdict** — the same `result` value node has
+   already recorded (it records before it replies) — from a review node ran against your
+   actual repo. Not just one or the other, not your own read of the diff, and not an ack.
+   This is what you post as your own PR comment below, unedited plus the "Reviewed by node
+   against `<sha>`" line.
+7. **Then move on to section 6 (Open your PR).** If node's review applied fixes,
+   it already committed them directly into your checkout while you waited — whatever is at
+   `HEAD` when you push there already includes them, and you do not need to reproduce or
+   look for them separately.
 
 ## 6. Open your PR — then STOP
 
@@ -123,8 +154,11 @@ for, escalate to node (R8) and STOP rather than substitute a self-review.
 2. Body must reference your issue with `Closes #N` so it auto-closes on merge, and must
    say: **depends on the server PR — deploy that first.**
 3. Record it: `node <workspaceRoot>/coolhand/harness/harness.mjs pr --run <RUN_DIR> --repo ruby --url <url>`
-4. Post your review skill's Iteration Breakdown table (section 5) as a comment on this PR
-   — see `RESIST_RULES.md` → "After the loop exits."
+4. Post node's Iteration Breakdown table (section 5) as a comment on this PR — see
+   `RESIST_RULES.md` → "After the loop exits" — and add one line the table itself does not
+   carry: `Reviewed by node against <sha>`, since you did not run it yourself. **Do not call
+   `harness.mjs loop-review` here — node already recorded it**, against the sha that
+   resulted from its own fix commits, not the one you originally escalated with.
 5. **Stop.** You launch no one. The tree ends with you on this branch.
 
 ## 7. Done means
@@ -133,7 +167,8 @@ for, escalate to node (R8) and STOP rather than substitute a self-review.
 - [ ] `bundle exec rake` passes (RSpec + RuboCop)
 - [ ] At least one spec hit the real local server, not a mock
 - [ ] PR opened, recorded, references its issue, states its dependency on the server PR
-- [ ] Your review skill ran for real (not approximated) and its Iteration Breakdown table
-      is posted as a comment on your PR
+- [ ] Node ran your review for real (not approximated by you) and its Iteration Breakdown
+      table is posted as a comment on your PR, naming node and the reviewed sha, and
+      recorded with `harness.mjs loop-review`
 - [ ] Every field and status code came from the definition, not from node's code
 - [ ] You launched no child agents

data/CHANGELOG.md

@@ -7,6 +7,47 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.9.0] - 2026-10-18
+
+### Added
+- **Declare the workload and template of captured LLM calls** ([#153](https://github.com/Coolhand-Labs/coolhand-ruby/pull/153)) — `Coolhand.with_context(reporter:, workload:, template:) { ... }` stamps every LLM call captured inside the block with the workload and template it belongs to, sent as `llm_request_log.declared`. The context is snapshotted on the calling thread, so it survives `async_logging`. Nested blocks override per field (innermost wins). `Coolhand::Declarable` (`coolhand_workload`, `coolhand_declare`, `coolhand_declare_all`) declares it once on a class, inherited by subclasses, and opt-in `config.auto_declare = [:active_job, :controller]` does the same for Rails jobs and controller actions (the gem still has no Rails dependency). `Coolhand.workload_service.upsert` and `Coolhand.template_service.upsert` create or update records by key with your **private** API key and raise `Coolhand::HttpError` on failure, and `warnings` returned with a logged request (e.g. the public-key cap) are now logged. Keys longer than 255 characters, with surrounding whitespace or a leading, trailing or doubled `/`, and a template `key` with no workload raise `ArgumentError` up front, because the server would reject and drop the whole log. Older servers ignore `declared`. See [Declared Workloads](docs/declared_workloads.md).
+- `bin/live-examples` (also `bundle exec rake examples:live`) runs every `examples/*.rb` against real provider keys and prints one `PASS` / `SKIP` / `FAIL` line per script ([#155](https://github.com/Coolhand-Labs/coolhand-ruby/pull/155)). Development tooling only: `bin/` and `examples/` are not packaged in the gem. See [Examples](examples/README.md).
+
+### Changed
+- The `🔗 Net::HTTP interceptor patched` log line is no longer printed when the interceptor is installed ([#154](https://github.com/Coolhand-Labs/coolhand-ruby/pull/154)). **Migration note:** anything that matched on that line in log output should stop doing so; use `Coolhand::NetHttpInterceptor.patched?` instead.
+- Internal de-duplication with no behavior change ([#156](https://github.com/Coolhand-Labs/coolhand-ruby/pull/156)): list reads share `ReadQuery#fetch_list`, `LoggerService` shares one webhook-rejection path, `OpenAi::WebhookValidator` shares one enforce-or-allow path (same error messages and same strict/lenient behavior by `Rails.env`), and `Coolhand.with_capture` / `without_capture` share one override helper. `contract/` (declared, intentional duplication) is excluded from the gem package.
+- CI: added the `exhale` duplication and complexity gate ([#156](https://github.com/Coolhand-Labs/coolhand-ruby/pull/156)), and bumped `ruby/setup-ruby` to 1.329.0 ([#150](https://github.com/Coolhand-Labs/coolhand-ruby/pull/150)) and `actions/upload-artifact` to 7.0.2 ([#151](https://github.com/Coolhand-Labs/coolhand-ruby/pull/151)).
+
+### Fixed
+- `OpenAi::WebhookValidator` now accepts a `webhook-signature` header carrying several space-separated signatures (`v1,sigA v1,sigB`) and passes if any `v1,` entry matches ([#161](https://github.com/Coolhand-Labs/coolhand-ruby/pull/161)). Previously the whole remainder after `v1,` was compared as one value, so a legitimate webhook sent during secret rotation was rejected with a 401. Every candidate is compared in constant time, non-`v1` entries are ignored, a header that is not valid UTF-8 is rejected rather than raising, and the timestamp and replay-id checks still run only after a match. Single-signature headers behave as before. See [OpenAI Batch Webhook Handler](docs/openai.md#signature-header-and-secret-rotation).
+
+### Security
+- Warnings returned in a Coolhand log response are now sanitized before they are printed ([#162](https://github.com/Coolhand-Labs/coolhand-ruby/pull/162)): entries are coerced with `to_s`, invalid UTF-8 is scrubbed, control characters and line separators become a space, invisible Unicode format characters are removed, entries over 500 characters are truncated, and at most 5 are printed per response. Well-formed warnings print as before.
+
+### Documentation
+- `docs/declared_workloads.md` and the README now state that `Coolhand.with_context`, `with_capture` and `without_capture` are fiber-local, so work started in a new Fiber or Async task inside the block is not covered, and show opening the block inside the Fiber instead ([#160](https://github.com/Coolhand-Labs/coolhand-ruby/pull/160)).
+
+## [0.8.0] - 2026-10-09
+
+### Added
+- **Read workloads and logs, with cost and performance metrics** ([#138](https://github.com/Coolhand-Labs/coolhand-ruby/pull/138)) — `Coolhand.workload_service.search_workloads` lists workloads (`GET /api/v2/workloads`) and `Coolhand.log_service` reads logs back out: `search_logs` (filters including `since`, `until`, `min_cost`, `order`, `project_path`, `source_application`, `include_total`, `sort`) and `get_log`, which returns `cost` and `cost_breakdown`. `search_templates` and `get_template` gain `include_metrics`, `days_back`, `since` and `until`. `since:`/`until:` accept a `Time`, `DateTime`, `Date` or ISO8601 String; anything else raises `ArgumentError` before a request is made. These need your **private** API key and raise `Coolhand::Error` (`Coolhand::HttpError` for non-2xx responses) rather than logging and returning `nil`. **Deploy note:** requires the matching Coolhand server release (Coolhand-Labs/coolhand#1753). See [Reading Workloads](docs/workload-search.md) and [Reading Logs](docs/log-search.md).
+- **Link feedback to an optimization** ([#133](https://github.com/Coolhand-Labs/coolhand-ruby/pull/133)) — `Coolhand.optimization_feedback_link_service` provides `link_feedback`, `bulk_link_feedback` (sent in batches of 100, results summed) and `unlink_feedback` for `/api/v2/optimizations/:id/feedback_links`. Private API key required; failures raise `Coolhand::Error` / `Coolhand::HttpError`. See [Linking Feedback](docs/feedback-links.md).
+- **Logs are sent from a background queue by default** ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148)) — the Coolhand POST no longer runs inline in the intercepted LLM call, so a slow or unreachable Coolhand backend can no longer add up to about 10s to each call. `Coolhand::LogQueue` holds up to 1,000 jobs per lane and drops the oldest when full, restarts itself in forked processes, and flushes at exit (waiting up to 5s). OpenAI and Vertex batch results are processed as one background job per batch, on their own lane so a long batch cannot delay or evict per-request logs. New `config.async_logging` (default `true`) and `Coolhand.flush(timeout: 5)`; `examples/benchmark_async_logging.rb` is a pass/fail benchmark. See [Asynchronous Logging](docs/configuration.md#asynchronous-logging). **Migration note:** anything that relied on the log POST finishing before the intercepted call returned (tests asserting on a stubbed `ApiService` inline, scripts that end with `exit!`, a process killed by `kill -9` or the OOM killer, which loses queued logs) should set `config.async_logging = false` or call `Coolhand.flush`. `debug_mode` always runs inline.
+
+### Changed
+- Repo guidance moved from `CLAUDE.md` to `AGENTS.md` ([#130](https://github.com/Coolhand-Labs/coolhand-ruby/pull/130)), and the agent-harness review handshake in `AGENTS.harness.md` was rewritten ([#134](https://github.com/Coolhand-Labs/coolhand-ruby/pull/134)). Neither ships in the gem, and `AGENTS.md` is excluded from the gem package in place of `CLAUDE.md`.
+- CI now uses `ruby/setup-ruby` 1.327.0 ([#135](https://github.com/Coolhand-Labs/coolhand-ruby/pull/135)).
+- Shared read-URL building moved into `Coolhand::ReadQuery`, and POST/DELETE helpers that raise on failure into `Coolhand::WriteRequests`; `TemplateService` now uses `ReadQuery`. Internal refactor with no behavior change to `search_templates`/`get_template` beyond the new options.
+- `Coolhand::NetHttpInterceptor` no longer includes `BaseInterceptor` ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148)); its helpers moved to `Coolhand::NetHttpInterceptor::Helpers`, so `Net::HTTP` gains only `#request` instead of about 15 generically named private methods (`intercept?`, `parse_json`, `sanitize_url`, ...). **Migration note:** internal API, but if you called those helpers on a `Net::HTTP` instance, use `Coolhand::NetHttpInterceptor::Helpers` or `Coolhand::BaseInterceptor`.
+
+### Fixed
+- Disabling the Net::HTTP interceptor logged a stale "Faraday monitoring disabled" message ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148)); it now says "Net::HTTP interceptor unpatched (capture stopped)".
+
+### Security
+- **Release-time hardening from a whole-package security review** ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148), plus fixes made on the release branch):
+  - Requests forwarded by the OpenAI batch webhook handler now go through the same request-body redaction as live interception, so Azure OpenAI "On Your Data" datastore credentials in a batch request body are replaced with `[REDACTED]` before being sent to Coolhand.
+  - The OpenAI batch handler now matches requests to results with a one-pass index instead of a scan per result, so a large batch no longer takes quadratic time. When a `custom_id` is duplicated the first request still wins, as before.
+
 ## [0.7.0] - 2026-09-20
 
 ### Added
@@ -22,7 +63,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Security
 - **Azure OpenAI "On Your Data" datastore credentials are now redacted from captured request bodies** ([#115](https://github.com/Coolhand-Labs/coolhand-ruby/pull/115)) — a request's `data_sources`/`dataSources` entries can carry live credentials (Azure AI Search API keys, Cosmos/Mongo connection strings, Elasticsearch encoded keys) that header and URL sanitization never looked at. `BaseInterceptor.sanitize_body` now replaces any value under those entries whose key name contains `key`, `token`, `secret`, `password`, `credential`, `connectionstring` or `signature` (separators and case ignored) with `[REDACTED]` before the log is forwarded. Message content and tool definitions outside `data_sources` are untouched.
-- **Release-time hardening from a whole-package security review:**
+- **Release-time hardening from a whole-package security review** ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148), plus fixes made on the release branch):
   - `Coolhand.log` no longer raises when stdout is unwritable. Previously a failing `puts` inside the interceptor's `ensure` block could replace the host application's own LLM response with an `IOError`.
   - A bad `intercept_addresses`/`exclude_api_patterns` value (e.g. a String or Symbol instead of an array of Strings) now degrades to "don't capture" instead of raising into every intercepted request.
   - `BaseInterceptor.sanitize_url` now fails closed on an unparseable URL, dropping userinfo, query and fragment rather than returning the raw URL with credentials intact.

data/README.md

@@ -103,6 +103,58 @@ Search is a parameter on the list endpoint, not a route of its own, and the `Unm
 
 For the full filter reference, pagination, and error handling, see [Reading Templates →](docs/template-search.md).
 
+### Cost and performance metrics
+
+Pass `include_metrics: true` to `search_templates` (or `search_workloads`) for the dashboard's cost and performance numbers, over a rolling `days_back` window or an explicit `since:` / `until:` one. `search_logs` carries each log's `cost` and can sort and filter by it.
+
+```ruby
+window = { since: Time.utc(2026, 9, 1), until: Time.utc(2026, 10, 1) }
+
+Coolhand.workload_service.search_workloads(include_metrics: true, **window).workloads
+  .map { |w| [w[:name], w[:metrics][:total_cost]] }
+
+Coolhand.log_service.search_logs(order: 'cost_desc', min_cost: 1.0, per: 10, **window).logs
+```
+
+See [Reading Workloads →](docs/workload-search.md) and [Reading Logs →](docs/log-search.md).
+
+## Linking Feedback to an Optimization
+
+Attach feedback to an optimization as evidence, one at a time or in bulk. These require your **private** API key and **raise** `Coolhand::Error` on failure (`Coolhand::HttpError` for non-2xx responses).
+
+```ruby
+links = Coolhand.optimization_feedback_link_service
+
+link = links.link_feedback('optimizationHashid', 'feedbackHashid', note: 'why')
+links.bulk_link_feedback('optimizationHashid', %w[fb1 fb2 fb3]) # => { linked:, already_linked:, errored:, not_found: }
+links.unlink_feedback('optimizationHashid', link[:id])
+```
+
+For batching, error semantics, and details, see [Linking Feedback →](docs/feedback-links.md).
+
+## Declaring Workloads and Templates
+
+Tell Coolhand which workload and template an LLM call belongs to, instead of waiting for it to be matched. Calls captured inside the block carry the declared identity:
+
+```ruby
+Coolhand.with_context(reporter: 'my-app', workload: { key: 'Parser#call' }, template: { key: 'Parser#call/summarize' }) do
+  client.chat(...) # captured as usual
+end
+
+Coolhand.workload_service.upsert(key: 'Parser#call', name: 'Parser') # private key; create or rename
+```
+
+Or declare it once on the class and let every capture inside it pick it up, subclasses included:
+
+```ruby
+class ApplicationService
+  include Coolhand::Declarable
+  coolhand_declare_all # each public method declares "ClassName#method"
+end
+```
+
+For nesting rules, class-level declarations, Rails jobs and controllers, upserts, and public-key limits, see [Declared Workloads →](docs/declared_workloads.md).
+
 ## Rails Integration
 
 ### Configuration
@@ -182,6 +234,8 @@ end
 | `intercept_path_patterns` | Array | `[":generateContent", ":streamGenerateContent"]` | Path patterns to additionally monitor on Google API hosts — see [Configuration](docs/configuration.md) |
 | `exclude_api_patterns` | Array | `["/batchPredictionJobs/"]` | Deny-list checked after `intercept_addresses`; matching paths are skipped. Unlike `intercept_addresses`, `exclude_api_patterns = []` genuinely disables exclusion. See [Configuration](docs/configuration.md) |
 | `max_captured_body_bytes` | Integer | `1_000_000` | Maximum size of a captured JSON request body — oversized bodies are replaced with a placeholder. Non-JSON bodies (e.g. file/audio uploads) are always skipped regardless of size — see [Advanced Configuration](docs/configuration.md) |
+| `async_logging` | Boolean | `true` | Send logs to Coolhand from a background thread so a slow Coolhand API never delays your LLM calls. Set to `false` to send inline — see [Advanced Configuration](docs/configuration.md#asynchronous-logging) |
+| `auto_declare` | Array | `[]` | Rails only: `:active_job` and/or `:controller` to declare a workload and template for LLM calls made from jobs and controller actions — see [Declared Workloads](docs/declared_workloads.md#rails-auto-declare) |
 
 ## Usage Examples
 
@@ -243,7 +297,7 @@ Coolhand.without_capture do
 end
 ```
 
-Both methods are thread-safe. Blocks nest correctly — the innermost block takes precedence, and the outer setting is restored when the block exits.
+Both methods are thread-safe. Blocks nest correctly — the innermost block takes precedence, and the outer setting is restored when the block exits. The override is fiber-local, so it does not follow into new Threads, Fibers or Async tasks started inside the block; see [Threads and background logging](docs/declared_workloads.md#threads-and-background-logging).
 
 ## Logging Inbound Webhooks
 
@@ -434,13 +488,17 @@ The monitor handles errors gracefully:
 - Failed API logging attempts are logged to console but don't interrupt your application
 - Invalid API keys will be reported but won't crash your app
 - Network issues are handled with appropriate error messages
-- The read methods (`search_templates`, `get_template`) are the exception — they raise `Coolhand::HttpError`; see [Reading Templates →](docs/template-search.md)
+- The read methods (`search_templates`, `get_template`, `search_workloads`, `search_logs`, `get_log`), the feedback-link methods and the `upsert` methods are the exception — they raise `Coolhand::Error` (`Coolhand::HttpError` for non-2xx responses); see [Reading Templates →](docs/template-search.md), [Reading Workloads →](docs/workload-search.md), [Reading Logs →](docs/log-search.md), [Linking Feedback →](docs/feedback-links.md) and [Declared Workloads →](docs/declared_workloads.md)
 
 ## Documentation
 
 - **[Configuration](docs/configuration.md)** — Self-hosted deployments, base_url rules, debug mode, custom intercept addresses
 - **[Feedback API](docs/feedback.md)** — Full field reference, matching strategies, sentiment values
 - **[Reading Templates](docs/template-search.md)** — Search LLM request templates and fetch a single one, prompt patterns included, using the private API key
+- **[Reading Workloads](docs/workload-search.md)** — List workloads with optional cost and performance metrics over a rolling or explicit time window
+- **[Reading Logs](docs/log-search.md)** — Search logs with per-log cost, filter and sort by cost, and fetch one log with its cost breakdown
+- **[Linking Feedback](docs/feedback-links.md)** — Link feedback to an optimization as evidence, singly or in bulk, using the private API key
+- **[Declared Workloads](docs/declared_workloads.md)** — Declare the workload and template of captured LLM calls with `Coolhand.with_context`, and create or rename them with `upsert`
 - **[Anthropic Integration](docs/anthropic.md)** — Official and community Anthropic Ruby gems, streaming, dual gem handling, and troubleshooting
 - **[ElevenLabs Integration](docs/elevenlabs.md)** — Webhook capture, feedback submission, and Rails integration
 - **[OpenAI Batch Webhook Handler](docs/openai.md)** — Handle OpenAI batch job completion events via webhook interception

data/Rakefile

@@ -9,4 +9,11 @@ require "rubocop/rake_task"
 
 RuboCop::RakeTask.new
 
+namespace :examples do
+  desc "Run every live script in examples/ against real provider keys (loads keys via zsh -ic)"
+  task :live do
+    sh RbConfig.ruby, File.expand_path("bin/live-examples", __dir__)
+  end
+end
+
 task default: %i[spec rubocop]

data/docs/configuration.md

@@ -39,6 +39,25 @@ end
 
 Because it forces capture unconditionally and prints full request/response bodies to the console, only enable `debug_mode` in development — never in production or in an environment handling real user data.
 
+## Asynchronous Logging
+
+By default Coolhand ships each captured request to its API from background worker threads (one for per-request logs, a separate one for batch jobs), so a slow or unreachable Coolhand backend never adds latency to your own LLM calls. The request-path work is limited to capturing and sanitizing the payload.
+
+```ruby
+Coolhand.configure do |config|
+  config.async_logging = false # default: true — send inline, in the intercepted call's request path
+end
+```
+
+**How it behaves:**
+- The queue is bounded at 1,000 pending logs. If Coolhand is down long enough to fill it, the *oldest* pending log is dropped (with a log line) rather than blocking your app or growing memory.
+- The bound is on the *number* of pending logs, not their size: each holds its captured response body in memory until sent, so a long Coolhand outage under heavy traffic with large responses can hold a lot of memory. Batch jobs use their own lane, so a long-running batch never delays or evicts per-request logs.
+- A forked process (Puma/Unicorn workers, Spring) starts its own worker automatically.
+- Pending logs are flushed at process exit (up to 5 seconds in total, so a batch job still running at exit may be only partly logged). In a short-lived script, or to wait explicitly, call `Coolhand.flush(timeout: 5)` — it returns `true` once everything has been sent.
+- OpenAI and Vertex batch results are processed as a single background job per batch.
+- `config.debug_mode` always runs inline so printed payloads appear in order.
+- Logs queued but not yet sent are lost if the process is killed hard (`kill -9`, OOM). Set `async_logging = false` if you need delivery before each call returns.
+
 ## Request Body Capture Limits
 
 To avoid holding large uploads (batch JSONL files, fine-tune corpora, audio for transcription) in memory and shipping them to Coolhand in full, `config.max_captured_body_bytes` caps how much of a request body gets captured:

data/docs/declared_workloads.md

@@ -0,0 +1,156 @@
+# Declaring workloads and templates
+
+By default Coolhand works out which workload and template an LLM call belongs to by matching the log against known templates. If your code already knows (a service object, a prompt library, an adapter), declare it instead. A declared log skips matching entirely and is filed under the workload and template you named.
+
+There are two parts:
+
+- `Coolhand.with_context` stamps the LLM calls the gem captures automatically.
+- `Coolhand.workload_service.upsert` and `Coolhand.template_service.upsert` create or update workloads and templates explicitly. They need your **private** API key.
+
+## Declaring on captured calls
+
+```ruby
+Coolhand.with_context(
+  reporter: "squishling",
+  workload: { key: "Parser#call", name: "Parser#call", description: "Parses incoming tickets" },
+  template: { key: "Parser#call/squish:L2", name: "Parser#call · squish L2" }
+) do
+  client.chat(...) # captured as usual, and sent with the declared identity
+end
+```
+
+| Field | Accepts | Notes |
+|---|---|---|
+| `reporter` | String | Who is making the declaration, e.g. the library or app. Used when a record is first created. |
+| `workload` | Hash: `id`, `key`, `name`, `description` | Needs an `id` or a `key`. Other keys are ignored. |
+| `template` | Hash: `id`, `key`, `name` | Needs an `id` or a `key`. Other keys are ignored. |
+
+- `id` refers to an existing record. `key` is a stable string of your choosing; Coolhand finds or creates the record with that key. On a template, `id` wins over `key`.
+- `name`, `description` and `reporter` are used only when a record is **first created**. Declaring never renames or re-attributes an existing record; use `upsert` for that. A display name that is already taken gets a numeric suffix, and names such as "Unmatched" are reserved.
+- A declaration with only a `workload` gets a default template keyed `<workload key>/default`. A template given by `key` needs a workload alongside it, either in the same block or an enclosing one.
+- Keys are at most 255 characters, printable, and cannot start or end with whitespace or `/` or contain `//`. `#`, `:` and `·` are fine. Descriptions are cut to 1000 characters.
+- Keys must be stable, so never put a request id or timestamp in one: each distinct key creates a record.
+- A `workload` or `template` that is not a Hash, has neither `id` nor `key`, has an invalid key, or a template `key` with no workload raises `ArgumentError` before your block runs. The server would answer these with a `422` and **not save the log**.
+- An unknown `id` (or one from another client) gets the same `422` and the log is not saved, so prefer `key`.
+
+### Nesting
+
+Blocks nest. Each field (`reporter`, `workload`, `template`) is taken from the innermost block that sets it, and fields an inner block leaves out are inherited (with the one exception below):
+
+```ruby
+Coolhand.with_context(reporter: "my-app", workload: { key: "Parser#call" }) do
+  Coolhand.with_context(template: { key: "Parser#call/summarize" }) do
+    # reporter: "my-app", workload: Parser#call, template: Parser#call/summarize
+  end
+end
+```
+
+A template belongs to a workload, so an inner block that declares a `workload` without a `template` does not inherit the outer block's template. Declare both together when you change workload.
+
+A `workload` or `template` Hash is replaced as a whole, not merged key by key. An inner `workload: { key: "b" }` inside an outer `workload: { id: "a" }` declares only `b`, rather than a mix of `id` and `key`.
+
+### Threads and background logging
+
+The context is thread-local, like `Coolhand.with_capture`. It is also fiber-local, so it does not carry into threads, fibers, `Enumerator#next` blocks or async tasks you start inside the block; calls made there are captured without a declaration. The identity is read on the thread that made the LLM call, before the log is queued, so it survives `async_logging` (the default).
+
+This matters for Fiber-scheduler concurrency such as [Async](https://github.com/socketry/async). Nothing raises: the call is logged, just without a workload or template.
+
+```ruby
+Coolhand.with_context(workload: { key: "Parser#call" }) do
+  Async do
+    client.chat(...) # runs in a new Fiber: captured with no declared workload
+  end
+end
+```
+
+`Coolhand.with_capture` and `Coolhand.without_capture` work the same way, so a `without_capture` block does not suppress calls made from a Fiber started inside it, and with `config.capture = false` a `with_capture` block does not enable them. To cover work that runs in another Fiber, open the block inside that Fiber:
+
+```ruby
+Async do
+  Coolhand.with_context(workload: { key: "Parser#call" }) do
+    client.chat(...) # captured with the declared workload
+  end
+end
+```
+
+### Public keys
+
+Declarations made with a public key are capped per client at 200 workloads and 1,000 templates. Past the cap, Coolhand ignores the declaration, ingests the log normally, and returns a warning, which the gem prints unless `config.silent` is on. Server-supplied warnings are sanitized before printing (control and invisible characters removed, each entry cut at 500 characters, at most 5 printed per response). Declarations from a private key are not capped.
+
+Older Coolhand servers ignore `declared`, so it is safe to use against them.
+
+## Declaring on a class
+
+Wrapping every call site in `with_context` does not suit passive capture. Include `Coolhand::Declarable` and declare the identity once, on the class. Wrapped methods run inside `with_context`, so everything above (nesting, precedence, async) applies unchanged.
+
+```ruby
+class ApplicationService
+  include Coolhand::Declarable
+
+  coolhand_workload reporter: "my-app"
+  coolhand_declare_all
+end
+
+class Parser < ApplicationService
+  def call(ticket) = client.chat(...) # workload "Parser#call", template "Parser#call"
+end
+
+class Summariser < ApplicationService
+  coolhand_workload key: "Summariser" # one workload for every method; templates stay per method
+
+  def call(ticket) = client.chat(...) # workload "Summariser", template "Summariser#call"
+end
+```
+
+- `coolhand_declare :call, :other` wraps just those methods, whether they are defined before or after the line. `template:` takes a key or a `{ key:, name: }` Hash instead of the default `"ClassName#method"`: `coolhand_declare :call, template: "Parser#call/summarise"`.
+- `coolhand_declare_all` wraps every public instance method defined after it, in the class and its subclasses, so put it at the top of the class body. Private and protected methods are left alone, however they are made private (`private def`, `private :name`, or a `private` section), as are attribute readers and writers and object-protocol methods such as `to_s`, `==` and `method_missing`. A method you name in `coolhand_declare` is wrapped regardless.
+- The key uses the **runtime** class, so a method inherited from `ApplicationService` declares as `Parser#call` when called on a `Parser`. Anonymous classes are not declared.
+- `coolhand_workload(key:, name:, description:, reporter:)` sets class-level defaults and is inherited by subclasses, field by field, with the nearest class winning. Without a `key`, each method gets its own workload, `ClassName#method`. `name` and `description` need a `key`, since one name shared by per-method workloads would collide.
+- Services that call each other nest: the innermost wrapped method wins, and it brings its own template (see [Nesting](#nesting)).
+- It only works where the LLM call runs on the same thread and fiber as the wrapped method (see [Threads and background logging](#threads-and-background-logging)).
+
+## Rails auto-declare
+
+For Rails apps, Coolhand can declare jobs and controller actions without touching their classes. It is off by default, because every distinct job and action becomes a workload.
+
+```ruby
+Coolhand.configure do |config|
+  config.api_key = ENV.fetch("COOLHAND_API_KEY")
+  config.auto_declare = [:active_job, :controller]
+end
+```
+
+| Target | Wraps | Declares |
+|---|---|---|
+| `:active_job` | `around_perform` on `ActiveJob::Base` | `SummariseJob#perform`, reporter `rails` |
+| `:controller` | `around_action` on `ActionController::Base` and `::API` | `UsersController#show`, reporter `rails` |
+
+Coolhand must be loaded after Rails (the default with `Bundler.require` in a Rails app), otherwise the hooks are never registered. They are installed after your initializers have run, so `Coolhand.configure` can come before or after anything else. Anything declared inside the job or action (a `Declarable` service, a `with_context` block) takes precedence over the job or action declaration. With a public API key the per-client declaration cap applies (see [Public keys](#public-keys)), so enable only the targets you want as workloads.
+
+## Creating and renaming explicitly
+
+```ruby
+Coolhand.configure { |config| config.api_key = ENV.fetch("COOLHAND_PRIVATE_API_KEY") }
+
+workload = Coolhand.workload_service.upsert(
+  key: "Parser#call", name: "Parser", description: "Parses incoming tickets", reporter: "my-app"
+)
+
+Coolhand.template_service.upsert(
+  key: "Parser#call/summarize",
+  workload: { key: "Parser#call" }, # or { id: workload[:id] }
+  name: "Parser · summarize",
+  reporter: "my-app"
+)
+```
+
+- Both are `PUT .../by_key/<key>`: they create the record if the key is new and update it otherwise. The server answers `201` on create and `200` on update; both return the record as a Hash.
+- The key never changes. Calling `upsert` with the same key and a new `name` renames the record, and an update changes only the fields you send.
+- A template needs a `workload` (`{ id: }` or `{ key: }`, sent to the server as `workload_id` / `workload_key`) when it is new. For an existing template, naming a different workload is a `422`.
+- A key may contain `/`, `#`, `:` and other characters; the gem escapes it for the URL.
+
+## Errors
+
+`upsert` raises, unlike the logging writes. A non-2xx response raises `Coolhand::HttpError` whose `status` is the HTTP code (`401` for a missing or public key, `422` for an invalid key, a taken name, a missing or unknown workload, or a key held by a deprecated template, `504` if the server timed out). A blank key, or one made of `.` / `..` path segments, and transport failures raise `Coolhand::Error`; a `workload` that is not a Hash raises `ArgumentError`.
+
+`config.debug_mode` does not suppress `upsert`: it changes server state, so it is always sent.

data/docs/feedback-links.md

@@ -0,0 +1,48 @@
+# Linking feedback to an optimization
+
+Attach feedback to an optimization as supporting evidence. `Coolhand::OptimizationFeedbackLinkService`
+wraps `POST /api/v2/optimizations/{optimization_id}/feedback_links` (single and bulk share the route,
+chosen by the body) and `DELETE /api/v2/optimizations/{optimization_id}/feedback_links/{id}`.
+
+These methods require your **private** API key; the public key gets a `401`.
+
+```ruby
+require "coolhand"
+
+Coolhand.configure do |config|
+  config.api_key = ENV.fetch("COOLHAND_PRIVATE_API_KEY")
+end
+
+links = Coolhand.optimization_feedback_link_service
+
+# One feedback. link[:id] is the link's hashid, not the feedback's.
+link = links.link_feedback("optimizationHashid", "feedbackHashid", note: "why")
+
+# Many feedbacks.
+result = links.bulk_link_feedback("optimizationHashid", %w[fb1 fb2 fb3])
+result # => { linked: 2, already_linked: 1, errored: 0, not_found: [] }
+
+# Remove a link.
+links.unlink_feedback("optimizationHashid", link[:id])
+```
+
+## Bulk behavior
+
+- The server accepts at most 100 ids per request. `bulk_link_feedback` splits longer lists into
+  batches of 100, sums `linked` / `already_linked` / `errored` and concatenates `not_found`.
+- Already-linked ids count as `already_linked`, not errors. Unknown, malformed and other-client ids
+  are all reported in `not_found`.
+- If a batch fails, the call raises and earlier batches stay applied. Repeating the call is safe.
+- `link_feedback` is not retry-safe: after a transport error, a `422` "already linked" on retry may mean
+  the first attempt succeeded. Use `bulk_link_feedback` with a single id for retry-safe semantics.
+- An empty list or a blank id raises `Coolhand::Error` before any request is made.
+- Ids are trimmed of surrounding whitespace but not de-duplicated client-side.
+
+## Errors
+
+Unlike the logging writes, all three methods raise. A non-2xx response raises `Coolhand::HttpError`
+whose `status` is the HTTP code: `401` missing or public key, `404` unknown optimization, feedback or
+link, `422` invalid input, an already-linked feedback (single mode) or a `note` that is too long.
+Client-side validation and transport failures raise `Coolhand::Error`.
+
+`config.debug_mode` does not suppress these calls: they change server state, so they are always sent.

data/docs/log-search.md

@@ -0,0 +1,102 @@
+# Reading Logs (Search + Get)
+
+`Coolhand::LogService` reads logs back out of Coolhand, with the per-log `cost` the dashboard shows.
+The write side, `Coolhand.logger_service`, is a different class. It wraps two read-only endpoints:
+
+| method | endpoint |
+|---|---|
+| `search_logs` | `GET /api/v2/llm_request_logs` |
+| `get_log` | `GET /api/v2/llm_request_logs/{id}` |
+
+Both require your **private** API key. The public key is write-only on this API and is rejected exactly
+like an invalid one.
+
+```ruby
+require "coolhand"
+
+Coolhand.configure do |config|
+  config.api_key = ENV.fetch("COOLHAND_PRIVATE_API_KEY")
+end
+
+logs = Coolhand.log_service
+
+# The ten most expensive logs of the last 30 days
+result = logs.search_logs(since: Time.now.utc - (30 * 86_400), order: "cost_desc", per: 10)
+result.logs.each { |log| puts "#{log[:id]} #{log[:model]} $#{log[:cost]}" }
+
+detail = logs.get_log(result.logs.first[:id])
+detail[:cost_breakdown][:input_cost]
+```
+
+## `search_logs`
+
+All keywords are optional, and named after the wire parameters (`sort` maps to `q[s]`). Two wire
+parameters are not exposed: the `q[source_api_in][]` array filter, so use the single-value `source_api`
+keyword, and the `per_page` alias, since this gem only sends `per`.
+
+| keyword | type | notes |
+|---|---|---|
+| `template_id`, `workload_id` | String | Hashids. One that does not decode, or that belongs to another client, is a `422`. |
+| `system_prompt_contains`, `user_prompt_contains` | String | Case-insensitive substring. |
+| `model`, `source_api`, `source_api_result`, `source_application` | String | Exact match. `source_api_result` is `success`, `failed`, `operational` or `unmatched`. |
+| `project_path` | String | Exact match against `metadata.project_path`. |
+| `unmatched_only` | Boolean | Only logs with no template. |
+| `days_back` | Integer | Last N days. Unset means unrestricted. Ignored when `since` is given; with only `until`, the window is `days_back` long ending at `until`. |
+| `since`, `until` | Time, DateTime, Date or String | Bounds on `created_at`: `since` inclusive, `until` exclusive. See [Windows](workload-search.md#windows-since-and-until). |
+| `min_cost` | Numeric | Only logs whose `cost` (USD) is at least this. `0` is sent. |
+| `order` | String | `"cost_desc"` sorts by `cost`, highest first, replacing `sort`. Any other value is a `422` from the server. |
+| `include_prompts` | Boolean | Adds `:system_prompt` and `:user_prompt`, each truncated to 500 characters. |
+| `include_total` | Boolean | Asks the server for the `X-Total-*` headers. Off by default there (it skips a `COUNT(*)`). |
+| `sort` | String | Ransack sort expression, sent as `q[s]`, e.g. `"created_at desc"`. |
+| `page`, `per` | Integer | 1-based; `per` defaults to 25, max 100. |
+
+`min_cost` and `order: "cost_desc"` only consider logs that can be priced (non-nil `cost`). The cost is
+computed per log, so on a client with a lot of history combine them with `since` or `days_back`, or the
+query can time out (`504`).
+
+### Return value
+
+A `Coolhand::LogSearchResult`, a `Struct` with `#logs` and `#pagination`. The response body is a bare
+array, never wrapped. Rows are Symbol-keyed Hashes: `:id`, `:collector`, `:source_api`,
+`:source_application`, `:metadata`, `:source_api_result`, `:model`, `:template_id`, `:template_name`,
+`:input_tokens`, `:output_tokens`, `:latency_ms`, `:created_at`, `:updated_at`, `:ingest_evidence` and
+`:cost`.
+
+`:cost` is a Float in USD, or `nil` when the log has no tokens or its model has no pricing.
+
+**Pagination without totals.** `X-Page` and `X-Per-Page` are always sent, but `X-Total-Count` and
+`X-Total-Pages` only come with `include_total: true`. Without it, `pagination.total_count` and
+`total_pages` are lower bounds the gem derives, not counts, and `has_next_page` is true when the page
+came back full. Pass `include_total: true` when you need real totals.
+
+## `get_log(id, section: nil, max_chars: nil, search_query: nil, include_thinking: nil)`
+
+`id` is the log hashid. Returns the full log as a Symbol-keyed Hash: the list fields plus the prompts and
+`:output`, and:
+
+| field | type |
+|---|---|
+| `:url` | String: the log's dashboard path (`/c/<client>/llm_request_logs/<id>`) |
+| `:cost` | Float or nil |
+| `:cost_breakdown` | Hash or nil: `:total_cost`, `:input_cost`, `:output_cost`, `:cached_input_cost`, `:cache_creation_input_cost`, `:reasoning_output_cost` |
+
+`section` (`"full"`, `"beginning"`, `"end"`), `max_chars` and `search_query` bound or search the content;
+`include_thinking` adds `:thinking_response`. Truncation and search are reported back as `:truncated`,
+`:total_chars`, `:search_query` and `:matches`. Only directly-collected logs are fetchable; internally
+generated records are a `404`.
+
+## Errors
+
+The read methods raise where the write methods log and return `nil`. `Coolhand::HttpError` carries
+`#status` and `#body`.
+
+| status | when |
+|---|---|
+| `401` | No API key, an invalid key, or the public key. |
+| `404` | `get_log` only: unknown id, another client's log, or an internally generated record. |
+| `422` | Bad `template_id`, `workload_id`, `days_back`, `since`, `until`, `min_cost`, `order` or `max_chars`. |
+| `504` | The search exceeded the statement timeout. Narrow the window and retry. |
+
+`Coolhand::Error` covers no API key configured, a transport failure, a body that is not valid JSON, a
+non-array list body, and a blank `id` passed to `get_log`. An unsupported `since`/`until` type raises
+`ArgumentError` before any request.