coolhand 0.7.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8f38b442e23ef4733417edbaa234c3459734a11f861caeb4261ea034f7a464dd
-  data.tar.gz: c3d6aee98d5eea898ff53bedba1f43497c82517ad335d330d5e6619a5035e13f
+  metadata.gz: 256c9237619b3f7116aa9751f78f0d0e367381b12afa694742da07ed1d7d3123
+  data.tar.gz: 149ddf8c4cd1800b105382ade5d2a2e9e57a360884e34e1b988b926739e90f30
 SHA512:
-  metadata.gz: feafe561a70cd33c6c995f1ebd20c1d93e93ca9bb092dc6214f5b52464eb23e9ac68797d2c64377a2e215ed9ca0d50fa1b4b43f91dd2b355577ca33a2c80483f
-  data.tar.gz: 749542ecea37f0d9f08840bf6e0eda16fe5aa65f4484abb1e720c3358667190137509e99f80d798e0b3292809f95efd7823e7655a8c05db1b0747ebdcf784b02
+  metadata.gz: 2ccbf54c2787a8df93178120583d83666553454a1b86385ec1c4ea10e228b859b995a35bed1450e9ab6d38dbb5d145d9b7a269be89f5562870d51a63630e5653
+  data.tar.gz: 4e6db88c4530737370030809a9d8e094d791761a6306a63701edc4413ed81f61a716d975953799c3d46b0cebbb3ddd9e1d172b22ee5d7b29c18bdd65ffe9e25c

data/AGENTS.harness.md

@@ -39,8 +39,8 @@ node <workspaceRoot>/coolhand/harness/harness.mjs my-issue --run <RUN_DIR> --rep
 2. `<workspaceRoot>/coolhand/harness/RESIST_RULES.md` — the refuse list.
 3. The API definition at `specPath`. It is your only source of truth **for the endpoint's
    contract** — paths, params, response fields, status codes.
-4. `coolhand-ruby/CLAUDE.md` — this repo's own rulebook. **It is authoritative.** Where it
-   disagrees with this harness file, CLAUDE.md wins — follow it and say so in your PR.
+4. `coolhand-ruby/AGENTS.md` — this repo's own rulebook. **It is authoritative.** Where it
+   disagrees with this harness file, AGENTS.md wins — follow it and say so in your PR.
 
 **Your issue links node's PR as the reference implementation. Use it for structure, not
 for facts.** Node went first so you do not have to rediscover how a REST method fits into
@@ -70,7 +70,7 @@ inside the current architecture, escalate and STOP.
 Require it inside the file that actually uses it, executed only when that provider's
 functionality is accessed. Clients may not use every provider and must not be forced to
 install gems they do not need — apps consuming this as a path gem break on missing optional
-dependencies. See `coolhand-ruby/CLAUDE.md`.
+dependencies. See `coolhand-ruby/AGENTS.md`.
 
 ## 3. Prove it against the real server
 
@@ -107,13 +107,44 @@ node <workspaceRoot>/coolhand/harness/harness.mjs wait --run <RUN_DIR> --channel
 
 Name the rule number (`R1`–`R5`). Do not guess. Do not stub. Do not work around it.
 
-## 5. Run your review skill
-
-**Mandatory, before you push.** Follow `<workspaceRoot>/coolhand/harness/RESIST_RULES.md`
-→ "Before you push: run your repo's review skill" (R8) — load
-`.claude/skills/loop-review/SKILL.md` off disk and run it for real against your diff. Do
-not approximate its steps manually; if you cannot spawn the reviewer subagent it calls
-for, escalate to node (R8) and STOP rather than substitute a self-review.
+## 5. Get your review — you cannot run it yourself
+
+**Mandatory, before you push. You do not run this section yourself.** You are three
+levels deep in this tree (server → node → you), and the platform does not let an agent at
+this depth spawn the reviewer subagent the review skill needs — see
+`<workspaceRoot>/coolhand/harness/RESIST_RULES.md` → R8's depth table. This is not something
+to confirm by trying; go straight to the request below.
+
+1. Commit locally first — build, prove it against the local server, run `bundle exec rake`,
+   all of section 3, all done before this step.
+2. Send an R8 escalation to node naming your repo path, branch, and `HEAD` sha:
+   ```
+   node <workspaceRoot>/coolhand/harness/harness.mjs send --run <RUN_DIR> --channel ruby \
+     --from ruby --to node --kind escalation \
+     --text "R8: ready for review. repo=<workspaceRoot>/coolhand-ruby branch=<branch> sha=<sha>"
+   ```
+3. `wait --after` the `messageId` your `send` in step 2 just printed. A timeout is silence,
+   not permission — call `wait` again, `--after` that same id, rather than pushing
+   (`RESIST_RULES.md` → "What escalate and stop means, mechanically"). If you loop back here
+   from step 5, use the fresh escalation's `messageId` instead.
+4. **The first reply is an `ack`, not the answer** — it means node has started, not that it
+   is done. `wait` again, `--after` that ack's `messageId`, for the `resolution` that
+   actually follows.
+5. **If what follows is a second `ack` instead of a `resolution`, node is asking you to do
+   something before it can start — most often that your checkout is not clean** (see
+   `RESIST_RULES.md` → R8 → "Serving a review for a child" step 2). Do what it asks, then
+   send a fresh R8 escalation with your current `HEAD` sha (it may have changed) and go back
+   to step 3. Do not just `wait` again — nothing arrives until you re-escalate.
+6. **The `resolution` that eventually arrives always carries the full Iteration Breakdown
+   table plus an explicit `CLEAN`/`capped` verdict** — the same `result` value node has
+   already recorded (it records before it replies) — from a review node ran against your
+   actual repo. Not just one or the other, not your own read of the diff, and not an ack.
+   This is what you post as your own PR comment below, unedited plus the "Reviewed by node
+   against `<sha>`" line.
+7. **Then move on to section 6 (Open your PR).** If node's review applied fixes,
+   it already committed them directly into your checkout while you waited — whatever is at
+   `HEAD` when you push there already includes them, and you do not need to reproduce or
+   look for them separately.
 
 ## 6. Open your PR — then STOP
 
@@ -123,8 +154,11 @@ for, escalate to node (R8) and STOP rather than substitute a self-review.
 2. Body must reference your issue with `Closes #N` so it auto-closes on merge, and must
    say: **depends on the server PR — deploy that first.**
 3. Record it: `node <workspaceRoot>/coolhand/harness/harness.mjs pr --run <RUN_DIR> --repo ruby --url <url>`
-4. Post your review skill's Iteration Breakdown table (section 5) as a comment on this PR
-   — see `RESIST_RULES.md` → "After the loop exits."
+4. Post node's Iteration Breakdown table (section 5) as a comment on this PR — see
+   `RESIST_RULES.md` → "After the loop exits" — and add one line the table itself does not
+   carry: `Reviewed by node against <sha>`, since you did not run it yourself. **Do not call
+   `harness.mjs loop-review` here — node already recorded it**, against the sha that
+   resulted from its own fix commits, not the one you originally escalated with.
 5. **Stop.** You launch no one. The tree ends with you on this branch.
 
 ## 7. Done means
@@ -133,7 +167,8 @@ for, escalate to node (R8) and STOP rather than substitute a self-review.
 - [ ] `bundle exec rake` passes (RSpec + RuboCop)
 - [ ] At least one spec hit the real local server, not a mock
 - [ ] PR opened, recorded, references its issue, states its dependency on the server PR
-- [ ] Your review skill ran for real (not approximated) and its Iteration Breakdown table
-      is posted as a comment on your PR
+- [ ] Node ran your review for real (not approximated by you) and its Iteration Breakdown
+      table is posted as a comment on your PR, naming node and the reviewed sha, and
+      recorded with `harness.mjs loop-review`
 - [ ] Every field and status code came from the definition, not from node's code
 - [ ] You launched no child agents

data/CHANGELOG.md

@@ -7,6 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.8.0] - 2026-10-09
+
+### Added
+- **Read workloads and logs, with cost and performance metrics** ([#138](https://github.com/Coolhand-Labs/coolhand-ruby/pull/138)) — `Coolhand.workload_service.search_workloads` lists workloads (`GET /api/v2/workloads`) and `Coolhand.log_service` reads logs back out: `search_logs` (filters including `since`, `until`, `min_cost`, `order`, `project_path`, `source_application`, `include_total`, `sort`) and `get_log`, which returns `cost` and `cost_breakdown`. `search_templates` and `get_template` gain `include_metrics`, `days_back`, `since` and `until`. `since:`/`until:` accept a `Time`, `DateTime`, `Date` or ISO8601 String; anything else raises `ArgumentError` before a request is made. These need your **private** API key and raise `Coolhand::Error` (`Coolhand::HttpError` for non-2xx responses) rather than logging and returning `nil`. **Deploy note:** requires the matching Coolhand server release (Coolhand-Labs/coolhand#1753). See [Reading Workloads](docs/workload-search.md) and [Reading Logs](docs/log-search.md).
+- **Link feedback to an optimization** ([#133](https://github.com/Coolhand-Labs/coolhand-ruby/pull/133)) — `Coolhand.optimization_feedback_link_service` provides `link_feedback`, `bulk_link_feedback` (sent in batches of 100, results summed) and `unlink_feedback` for `/api/v2/optimizations/:id/feedback_links`. Private API key required; failures raise `Coolhand::Error` / `Coolhand::HttpError`. See [Linking Feedback](docs/feedback-links.md).
+- **Logs are sent from a background queue by default** ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148)) — the Coolhand POST no longer runs inline in the intercepted LLM call, so a slow or unreachable Coolhand backend can no longer add up to about 10s to each call. `Coolhand::LogQueue` holds up to 1,000 jobs per lane and drops the oldest when full, restarts itself in forked processes, and flushes at exit (waiting up to 5s). OpenAI and Vertex batch results are processed as one background job per batch, on their own lane so a long batch cannot delay or evict per-request logs. New `config.async_logging` (default `true`) and `Coolhand.flush(timeout: 5)`; `examples/benchmark_async_logging.rb` is a pass/fail benchmark. See [Asynchronous Logging](docs/configuration.md#asynchronous-logging). **Migration note:** anything that relied on the log POST finishing before the intercepted call returned (tests asserting on a stubbed `ApiService` inline, scripts that end with `exit!`, a process killed by `kill -9` or the OOM killer, which loses queued logs) should set `config.async_logging = false` or call `Coolhand.flush`. `debug_mode` always runs inline.
+
+### Changed
+- Repo guidance moved from `CLAUDE.md` to `AGENTS.md` ([#130](https://github.com/Coolhand-Labs/coolhand-ruby/pull/130)), and the agent-harness review handshake in `AGENTS.harness.md` was rewritten ([#134](https://github.com/Coolhand-Labs/coolhand-ruby/pull/134)). Neither ships in the gem, and `AGENTS.md` is excluded from the gem package in place of `CLAUDE.md`.
+- CI now uses `ruby/setup-ruby` 1.327.0 ([#135](https://github.com/Coolhand-Labs/coolhand-ruby/pull/135)).
+- Shared read-URL building moved into `Coolhand::ReadQuery`, and POST/DELETE helpers that raise on failure into `Coolhand::WriteRequests`; `TemplateService` now uses `ReadQuery`. Internal refactor with no behavior change to `search_templates`/`get_template` beyond the new options.
+- `Coolhand::NetHttpInterceptor` no longer includes `BaseInterceptor` ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148)); its helpers moved to `Coolhand::NetHttpInterceptor::Helpers`, so `Net::HTTP` gains only `#request` instead of about 15 generically named private methods (`intercept?`, `parse_json`, `sanitize_url`, ...). **Migration note:** internal API, but if you called those helpers on a `Net::HTTP` instance, use `Coolhand::NetHttpInterceptor::Helpers` or `Coolhand::BaseInterceptor`.
+
+### Fixed
+- Disabling the Net::HTTP interceptor logged a stale "Faraday monitoring disabled" message ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148)); it now says "Net::HTTP interceptor unpatched (capture stopped)".
+
+### Security
+- **Release-time hardening from a whole-package security review** ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148), plus fixes made on the release branch):
+  - Requests forwarded by the OpenAI batch webhook handler now go through the same request-body redaction as live interception, so Azure OpenAI "On Your Data" datastore credentials in a batch request body are replaced with `[REDACTED]` before being sent to Coolhand.
+  - The OpenAI batch handler now matches requests to results with a one-pass index instead of a scan per result, so a large batch no longer takes quadratic time. When a `custom_id` is duplicated the first request still wins, as before.
+
 ## [0.7.0] - 2026-09-20
 
 ### Added
@@ -22,7 +43,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Security
 - **Azure OpenAI "On Your Data" datastore credentials are now redacted from captured request bodies** ([#115](https://github.com/Coolhand-Labs/coolhand-ruby/pull/115)) — a request's `data_sources`/`dataSources` entries can carry live credentials (Azure AI Search API keys, Cosmos/Mongo connection strings, Elasticsearch encoded keys) that header and URL sanitization never looked at. `BaseInterceptor.sanitize_body` now replaces any value under those entries whose key name contains `key`, `token`, `secret`, `password`, `credential`, `connectionstring` or `signature` (separators and case ignored) with `[REDACTED]` before the log is forwarded. Message content and tool definitions outside `data_sources` are untouched.
-- **Release-time hardening from a whole-package security review:**
+- **Release-time hardening from a whole-package security review** ([#148](https://github.com/Coolhand-Labs/coolhand-ruby/pull/148), plus fixes made on the release branch):
   - `Coolhand.log` no longer raises when stdout is unwritable. Previously a failing `puts` inside the interceptor's `ensure` block could replace the host application's own LLM response with an `IOError`.
   - A bad `intercept_addresses`/`exclude_api_patterns` value (e.g. a String or Symbol instead of an array of Strings) now degrades to "don't capture" instead of raising into every intercepted request.
   - `BaseInterceptor.sanitize_url` now fails closed on an unparseable URL, dropping userinfo, query and fragment rather than returning the raw URL with credentials intact.

data/README.md

@@ -103,6 +103,35 @@ Search is a parameter on the list endpoint, not a route of its own, and the `Unm
 
 For the full filter reference, pagination, and error handling, see [Reading Templates →](docs/template-search.md).
 
+### Cost and performance metrics
+
+Pass `include_metrics: true` to `search_templates` (or `search_workloads`) for the dashboard's cost and performance numbers, over a rolling `days_back` window or an explicit `since:` / `until:` one. `search_logs` carries each log's `cost` and can sort and filter by it.
+
+```ruby
+window = { since: Time.utc(2026, 9, 1), until: Time.utc(2026, 10, 1) }
+
+Coolhand.workload_service.search_workloads(include_metrics: true, **window).workloads
+  .map { |w| [w[:name], w[:metrics][:total_cost]] }
+
+Coolhand.log_service.search_logs(order: 'cost_desc', min_cost: 1.0, per: 10, **window).logs
+```
+
+See [Reading Workloads →](docs/workload-search.md) and [Reading Logs →](docs/log-search.md).
+
+## Linking Feedback to an Optimization
+
+Attach feedback to an optimization as evidence, one at a time or in bulk. These require your **private** API key and **raise** `Coolhand::Error` on failure (`Coolhand::HttpError` for non-2xx responses).
+
+```ruby
+links = Coolhand.optimization_feedback_link_service
+
+link = links.link_feedback('optimizationHashid', 'feedbackHashid', note: 'why')
+links.bulk_link_feedback('optimizationHashid', %w[fb1 fb2 fb3]) # => { linked:, already_linked:, errored:, not_found: }
+links.unlink_feedback('optimizationHashid', link[:id])
+```
+
+For batching, error semantics, and details, see [Linking Feedback →](docs/feedback-links.md).
+
 ## Rails Integration
 
 ### Configuration
@@ -182,6 +211,7 @@ end
 | `intercept_path_patterns` | Array | `[":generateContent", ":streamGenerateContent"]` | Path patterns to additionally monitor on Google API hosts — see [Configuration](docs/configuration.md) |
 | `exclude_api_patterns` | Array | `["/batchPredictionJobs/"]` | Deny-list checked after `intercept_addresses`; matching paths are skipped. Unlike `intercept_addresses`, `exclude_api_patterns = []` genuinely disables exclusion. See [Configuration](docs/configuration.md) |
 | `max_captured_body_bytes` | Integer | `1_000_000` | Maximum size of a captured JSON request body — oversized bodies are replaced with a placeholder. Non-JSON bodies (e.g. file/audio uploads) are always skipped regardless of size — see [Advanced Configuration](docs/configuration.md) |
+| `async_logging` | Boolean | `true` | Send logs to Coolhand from a background thread so a slow Coolhand API never delays your LLM calls. Set to `false` to send inline — see [Advanced Configuration](docs/configuration.md#asynchronous-logging) |
 
 ## Usage Examples
 
@@ -434,13 +464,16 @@ The monitor handles errors gracefully:
 - Failed API logging attempts are logged to console but don't interrupt your application
 - Invalid API keys will be reported but won't crash your app
 - Network issues are handled with appropriate error messages
-- The read methods (`search_templates`, `get_template`) are the exception — they raise `Coolhand::HttpError`; see [Reading Templates →](docs/template-search.md)
+- The read methods (`search_templates`, `get_template`, `search_workloads`, `search_logs`, `get_log`) and the feedback-link methods are the exception — they raise `Coolhand::Error` (`Coolhand::HttpError` for non-2xx responses); see [Reading Templates →](docs/template-search.md), [Reading Workloads →](docs/workload-search.md), [Reading Logs →](docs/log-search.md) and [Linking Feedback →](docs/feedback-links.md)
 
 ## Documentation
 
 - **[Configuration](docs/configuration.md)** — Self-hosted deployments, base_url rules, debug mode, custom intercept addresses
 - **[Feedback API](docs/feedback.md)** — Full field reference, matching strategies, sentiment values
 - **[Reading Templates](docs/template-search.md)** — Search LLM request templates and fetch a single one, prompt patterns included, using the private API key
+- **[Reading Workloads](docs/workload-search.md)** — List workloads with optional cost and performance metrics over a rolling or explicit time window
+- **[Reading Logs](docs/log-search.md)** — Search logs with per-log cost, filter and sort by cost, and fetch one log with its cost breakdown
+- **[Linking Feedback](docs/feedback-links.md)** — Link feedback to an optimization as evidence, singly or in bulk, using the private API key
 - **[Anthropic Integration](docs/anthropic.md)** — Official and community Anthropic Ruby gems, streaming, dual gem handling, and troubleshooting
 - **[ElevenLabs Integration](docs/elevenlabs.md)** — Webhook capture, feedback submission, and Rails integration
 - **[OpenAI Batch Webhook Handler](docs/openai.md)** — Handle OpenAI batch job completion events via webhook interception

data/docs/configuration.md

@@ -39,6 +39,25 @@ end
 
 Because it forces capture unconditionally and prints full request/response bodies to the console, only enable `debug_mode` in development — never in production or in an environment handling real user data.
 
+## Asynchronous Logging
+
+By default Coolhand ships each captured request to its API from background worker threads (one for per-request logs, a separate one for batch jobs), so a slow or unreachable Coolhand backend never adds latency to your own LLM calls. The request-path work is limited to capturing and sanitizing the payload.
+
+```ruby
+Coolhand.configure do |config|
+  config.async_logging = false # default: true — send inline, in the intercepted call's request path
+end
+```
+
+**How it behaves:**
+- The queue is bounded at 1,000 pending logs. If Coolhand is down long enough to fill it, the *oldest* pending log is dropped (with a log line) rather than blocking your app or growing memory.
+- The bound is on the *number* of pending logs, not their size: each holds its captured response body in memory until sent, so a long Coolhand outage under heavy traffic with large responses can hold a lot of memory. Batch jobs use their own lane, so a long-running batch never delays or evicts per-request logs.
+- A forked process (Puma/Unicorn workers, Spring) starts its own worker automatically.
+- Pending logs are flushed at process exit (up to 5 seconds in total, so a batch job still running at exit may be only partly logged). In a short-lived script, or to wait explicitly, call `Coolhand.flush(timeout: 5)` — it returns `true` once everything has been sent.
+- OpenAI and Vertex batch results are processed as a single background job per batch.
+- `config.debug_mode` always runs inline so printed payloads appear in order.
+- Logs queued but not yet sent are lost if the process is killed hard (`kill -9`, OOM). Set `async_logging = false` if you need delivery before each call returns.
+
 ## Request Body Capture Limits
 
 To avoid holding large uploads (batch JSONL files, fine-tune corpora, audio for transcription) in memory and shipping them to Coolhand in full, `config.max_captured_body_bytes` caps how much of a request body gets captured:

data/docs/feedback-links.md

@@ -0,0 +1,48 @@
+# Linking feedback to an optimization
+
+Attach feedback to an optimization as supporting evidence. `Coolhand::OptimizationFeedbackLinkService`
+wraps `POST /api/v2/optimizations/{optimization_id}/feedback_links` (single and bulk share the route,
+chosen by the body) and `DELETE /api/v2/optimizations/{optimization_id}/feedback_links/{id}`.
+
+These methods require your **private** API key; the public key gets a `401`.
+
+```ruby
+require "coolhand"
+
+Coolhand.configure do |config|
+  config.api_key = ENV.fetch("COOLHAND_PRIVATE_API_KEY")
+end
+
+links = Coolhand.optimization_feedback_link_service
+
+# One feedback. link[:id] is the link's hashid, not the feedback's.
+link = links.link_feedback("optimizationHashid", "feedbackHashid", note: "why")
+
+# Many feedbacks.
+result = links.bulk_link_feedback("optimizationHashid", %w[fb1 fb2 fb3])
+result # => { linked: 2, already_linked: 1, errored: 0, not_found: [] }
+
+# Remove a link.
+links.unlink_feedback("optimizationHashid", link[:id])
+```
+
+## Bulk behavior
+
+- The server accepts at most 100 ids per request. `bulk_link_feedback` splits longer lists into
+  batches of 100, sums `linked` / `already_linked` / `errored` and concatenates `not_found`.
+- Already-linked ids count as `already_linked`, not errors. Unknown, malformed and other-client ids
+  are all reported in `not_found`.
+- If a batch fails, the call raises and earlier batches stay applied. Repeating the call is safe.
+- `link_feedback` is not retry-safe: after a transport error, a `422` "already linked" on retry may mean
+  the first attempt succeeded. Use `bulk_link_feedback` with a single id for retry-safe semantics.
+- An empty list or a blank id raises `Coolhand::Error` before any request is made.
+- Ids are trimmed of surrounding whitespace but not de-duplicated client-side.
+
+## Errors
+
+Unlike the logging writes, all three methods raise. A non-2xx response raises `Coolhand::HttpError`
+whose `status` is the HTTP code: `401` missing or public key, `404` unknown optimization, feedback or
+link, `422` invalid input, an already-linked feedback (single mode) or a `note` that is too long.
+Client-side validation and transport failures raise `Coolhand::Error`.
+
+`config.debug_mode` does not suppress these calls: they change server state, so they are always sent.

data/docs/log-search.md

@@ -0,0 +1,102 @@
+# Reading Logs (Search + Get)
+
+`Coolhand::LogService` reads logs back out of Coolhand, with the per-log `cost` the dashboard shows.
+The write side, `Coolhand.logger_service`, is a different class. It wraps two read-only endpoints:
+
+| method | endpoint |
+|---|---|
+| `search_logs` | `GET /api/v2/llm_request_logs` |
+| `get_log` | `GET /api/v2/llm_request_logs/{id}` |
+
+Both require your **private** API key. The public key is write-only on this API and is rejected exactly
+like an invalid one.
+
+```ruby
+require "coolhand"
+
+Coolhand.configure do |config|
+  config.api_key = ENV.fetch("COOLHAND_PRIVATE_API_KEY")
+end
+
+logs = Coolhand.log_service
+
+# The ten most expensive logs of the last 30 days
+result = logs.search_logs(since: Time.now.utc - (30 * 86_400), order: "cost_desc", per: 10)
+result.logs.each { |log| puts "#{log[:id]} #{log[:model]} $#{log[:cost]}" }
+
+detail = logs.get_log(result.logs.first[:id])
+detail[:cost_breakdown][:input_cost]
+```
+
+## `search_logs`
+
+All keywords are optional, and named after the wire parameters (`sort` maps to `q[s]`). Two wire
+parameters are not exposed: the `q[source_api_in][]` array filter, so use the single-value `source_api`
+keyword, and the `per_page` alias, since this gem only sends `per`.
+
+| keyword | type | notes |
+|---|---|---|
+| `template_id`, `workload_id` | String | Hashids. One that does not decode, or that belongs to another client, is a `422`. |
+| `system_prompt_contains`, `user_prompt_contains` | String | Case-insensitive substring. |
+| `model`, `source_api`, `source_api_result`, `source_application` | String | Exact match. `source_api_result` is `success`, `failed`, `operational` or `unmatched`. |
+| `project_path` | String | Exact match against `metadata.project_path`. |
+| `unmatched_only` | Boolean | Only logs with no template. |
+| `days_back` | Integer | Last N days. Unset means unrestricted. Ignored when `since` is given; with only `until`, the window is `days_back` long ending at `until`. |
+| `since`, `until` | Time, DateTime, Date or String | Bounds on `created_at`: `since` inclusive, `until` exclusive. See [Windows](workload-search.md#windows-since-and-until). |
+| `min_cost` | Numeric | Only logs whose `cost` (USD) is at least this. `0` is sent. |
+| `order` | String | `"cost_desc"` sorts by `cost`, highest first, replacing `sort`. Any other value is a `422` from the server. |
+| `include_prompts` | Boolean | Adds `:system_prompt` and `:user_prompt`, each truncated to 500 characters. |
+| `include_total` | Boolean | Asks the server for the `X-Total-*` headers. Off by default there (it skips a `COUNT(*)`). |
+| `sort` | String | Ransack sort expression, sent as `q[s]`, e.g. `"created_at desc"`. |
+| `page`, `per` | Integer | 1-based; `per` defaults to 25, max 100. |
+
+`min_cost` and `order: "cost_desc"` only consider logs that can be priced (non-nil `cost`). The cost is
+computed per log, so on a client with a lot of history combine them with `since` or `days_back`, or the
+query can time out (`504`).
+
+### Return value
+
+A `Coolhand::LogSearchResult`, a `Struct` with `#logs` and `#pagination`. The response body is a bare
+array, never wrapped. Rows are Symbol-keyed Hashes: `:id`, `:collector`, `:source_api`,
+`:source_application`, `:metadata`, `:source_api_result`, `:model`, `:template_id`, `:template_name`,
+`:input_tokens`, `:output_tokens`, `:latency_ms`, `:created_at`, `:updated_at`, `:ingest_evidence` and
+`:cost`.
+
+`:cost` is a Float in USD, or `nil` when the log has no tokens or its model has no pricing.
+
+**Pagination without totals.** `X-Page` and `X-Per-Page` are always sent, but `X-Total-Count` and
+`X-Total-Pages` only come with `include_total: true`. Without it, `pagination.total_count` and
+`total_pages` are lower bounds the gem derives, not counts, and `has_next_page` is true when the page
+came back full. Pass `include_total: true` when you need real totals.
+
+## `get_log(id, section: nil, max_chars: nil, search_query: nil, include_thinking: nil)`
+
+`id` is the log hashid. Returns the full log as a Symbol-keyed Hash: the list fields plus the prompts and
+`:output`, and:
+
+| field | type |
+|---|---|
+| `:url` | String: the log's dashboard path (`/c/<client>/llm_request_logs/<id>`) |
+| `:cost` | Float or nil |
+| `:cost_breakdown` | Hash or nil: `:total_cost`, `:input_cost`, `:output_cost`, `:cached_input_cost`, `:cache_creation_input_cost`, `:reasoning_output_cost` |
+
+`section` (`"full"`, `"beginning"`, `"end"`), `max_chars` and `search_query` bound or search the content;
+`include_thinking` adds `:thinking_response`. Truncation and search are reported back as `:truncated`,
+`:total_chars`, `:search_query` and `:matches`. Only directly-collected logs are fetchable; internally
+generated records are a `404`.
+
+## Errors
+
+The read methods raise where the write methods log and return `nil`. `Coolhand::HttpError` carries
+`#status` and `#body`.
+
+| status | when |
+|---|---|
+| `401` | No API key, an invalid key, or the public key. |
+| `404` | `get_log` only: unknown id, another client's log, or an internally generated record. |
+| `422` | Bad `template_id`, `workload_id`, `days_back`, `since`, `until`, `min_cost`, `order` or `max_chars`. |
+| `504` | The search exceeded the statement timeout. Narrow the window and retry. |
+
+`Coolhand::Error` covers no API key configured, a transport failure, a body that is not valid JSON, a
+non-array list body, and a blank `id` passed to `get_log`. An unsupported `since`/`until` type raises
+`ArgumentError` before any request.

data/docs/openai.md

@@ -14,7 +14,7 @@ Requires Rails — `Coolhand::WebhookInterceptor` and `Coolhand::OpenAi::BatchRe
 - Ensure you skip CSRF for the webhook endpoint: `skip_before_action :verify_authenticity_token`
 - Override the `webhook_secret` method to return your OpenAI webhook secret
 
-`intercept_batch_request` already validates the webhook and, for `batch.completed`/`batch.failed`/`batch.expired`/`batch.cancelled` events, calls `Coolhand::OpenAi::BatchResultProcessor` **synchronously**, inline in the `before_action` — that part requires no code in your controller action. It downloads two JSONL files from OpenAI and sends one request log per batch item, so a large batch can take long enough to trip your webhook endpoint's timeout (and OpenAI's retry behavior); consider that when sizing batches. The example below shows a controller action doing additional, app-specific work on top of that (e.g. updating your own `BatchApiRequest` record and enqueuing your own background job), which is optional.
+`intercept_batch_request` already validates the webhook and, for `batch.completed`/`batch.failed`/`batch.expired`/`batch.cancelled` events, calls `Coolhand::OpenAi::BatchResultProcessor` — that part requires no code in your controller action. The batch's status is checked inline in the `before_action`, but for a completed batch the two JSONL file downloads and the one-request-log-per-item sends run on Coolhand's [background log thread](configuration.md#asynchronous-logging), so a large batch doesn't hold up your webhook response. (With `config.async_logging = false` this all runs inline, and a large batch can trip your webhook endpoint's timeout and OpenAI's retry behavior.) The example below shows a controller action doing additional, app-specific work on top of that (e.g. updating your own `BatchApiRequest` record and enqueuing your own background job), which is optional.
 
 ## Minimal example
 

data/docs/template-search.md

@@ -41,7 +41,7 @@ This is **not** a port of the `search_templates` MCP tool, and the two do not ag
 Template *creation, update and deprecation* stay on the MCP surface. This REST surface is
 read-only, and there is no version-history sub-resource.
 
-## `search_templates(search: nil, workload_id: nil, status: nil, include_deprecated: nil, include_system: nil, page: nil, per: nil)`
+## `search_templates(search: nil, workload_id: nil, status: nil, include_deprecated: nil, include_system: nil, include_metrics: nil, days_back: nil, since: nil, until: nil, page: nil, per: nil)`
 
 Search is a *parameter* on the list endpoint rather than a route of its own, so this is one method
 rather than a list/search pair.
@@ -56,6 +56,10 @@ and this API's already agree, so there is no second spelling to translate throug
 | `status` | String | `"draft"`, `"published"` or `"failure"`. Any other non-empty value is a `422` from the server; empty is treated as no filter. |
 | `include_deprecated` | Boolean | Include templates with a non-null `deprecated_at`. Defaults to `false` server-side. |
 | `include_system` | Boolean | Include the `Unmatched` / `Ignored API Calls` buckets. Defaults to `false` server-side. |
+| `include_metrics` | Boolean | Adds a `:metrics` Hash to every row. See [Metrics](#metrics). |
+| `days_back` | Integer | Rolling metrics window in days (server default 28, max 365). Ignored unless metrics are requested, and when `since` is given. |
+| `since` | Time, DateTime, Date or String | Metrics window start, inclusive. Overrides `days_back`. |
+| `until` | Time, DateTime, Date or String | Metrics window end, exclusive. Defaults to now. |
 | `page` | Integer | 1-based. |
 | `per` | Integer | Default 25, max 100, both enforced server-side. |
 
@@ -138,11 +142,14 @@ unmatched = Coolhand.template_service
 puts unmatched[:log_count]
 ```
 
-## `get_template(id)`
+## `get_template(id, include_metrics: nil, days_back: nil, since: nil, until: nil)`
 
 `id` is the template hashid — the `:id` field from `search_templates`.
 
-Returns a Hash with every list field above, **plus** the full untruncated regexes the list omits:
+Returns a Hash with every list field above, **plus** the full untruncated regexes the list omits, and
+a `:metrics` Hash. Unlike the list, metrics are **on by default** here: pass `include_metrics: false` to
+omit them, which also skips the window validation. `days_back`, `since` and `until` set the window
+exactly as they do on the list, and an invalid one is a `422` unless metrics are off.
 
 | field | type |
 |---|---|
@@ -153,6 +160,41 @@ Unlike the list, this filters on nothing but client ownership: a deprecated or s
 reachable by id with **no opt-in flag**, because inspecting one of those is the usual reason to
 fetch a template directly.
 
+## Metrics
+
+With `include_metrics: true` every row (and by default the `get_template` result) carries `:metrics`,
+computed by the same SQL as the dashboard, so tiered pricing, cached-token discounts and reasoning tokens
+are applied. `Coolhand.workload_service` returns the same object per workload, rolled up across its
+templates ([Reading Workloads](workload-search.md)).
+
+Averages and counters cover non-failed, directly-collected client logs inside the window. Keys are the
+wire names, as Symbols.
+
+| field | type | notes |
+|---|---|---|
+| `:days_back` | Integer or nil | `nil` when `since` defined the window. |
+| `:since`, `:until` | String | The resolved window, ISO-8601 UTC. |
+| `:request_count` | Integer | |
+| `:failure_count` | Integer | Failed logs. |
+| `:error_rate` | Float or nil | `nil` when `failure_count` is 0. |
+| `:error_rate_change` | Float or nil | Against the window of the same length immediately before. |
+| `:avg_cost_per_request` | Float or nil | `nil` when no log in the window could be priced. |
+| `:total_cost` | Float or nil | |
+| `:priced_request_count` | Integer | The logs `total_cost` covers: non-failed, with tokens, model has pricing. |
+| `:long_context_request_count` | Integer | Priced logs that crossed their model's input-token pricing tier. |
+| `:total_input_tokens`, `:total_output_tokens` | Integer | Raw token columns summed over non-failed logs, not cache-adjusted. |
+| `:avg_input_tokens`, `:avg_output_tokens` | Integer or nil | |
+| `:avg_latency_ms` | Float or nil | |
+| `:correctness_score`, `:sentiment_score`, `:revision_score` | Float or nil | `sentiment_score` and `revision_score` are all-time, not windowed. |
+| `:first_request_at`, `:last_request_at` | String or nil | Lifetime, not windowed. |
+
+Summing the per-log `cost` from [Reading Logs](log-search.md) can exceed `total_cost`, because the log
+`cost` is also computed for failed logs and logs with a zero token count.
+
+Windows follow the rules in [Reading Workloads](workload-search.md#windows-since-and-until). A window
+error is a `422` on the `since` or `until` key. Metrics add aggregate statements, so expect a slower
+response, and a `504` on a large page.
+
 ## Errors
 
 The write methods in this gem (`create_feedback`, `create_log`, `send_llm_request_log`) log a
@@ -186,7 +228,7 @@ end
 |---|---|
 | `401` | Missing, invalid, or public API key. |
 | `404` | `get_template` only. Unknown id, **or** one belonging to another client — existence is not disclosed, so this is never a `403`. |
-| `422` | Unrecognised `status`, or a `workload_id` that does not decode or belongs to another client. |
+| `422` | Unrecognised `status`, a `workload_id` that does not decode or belongs to another client, or a bad `days_back`, `since` or `until` when metrics are requested. |
 | `504` | See below. |
 
 ### `504` is expected, and retryable

data/docs/vertex.md

@@ -15,6 +15,7 @@ require "coolhand/vertex/batch_result_processor"
 ```
 
 - Call the `Coolhand::Vertex::BatchResultProcessor` service with `batch_info` and the downloaded batch results.
+- For a succeeded job, the per-item request logs are sent from Coolhand's [background log thread](configuration.md#asynchronous-logging) as a single job, so `call` returns without waiting on the Coolhand API (set `config.async_logging = false` to send inline).
 - Optionally pass `model:` to populate the logged `model` field. It falls back to a `"model"` key in `batch_info` if present, and is omitted from the payload entirely when neither is available. If the value (from either source) looks like a Vertex model resource path (`publishers/<publisher>/models/<id>` or `projects/<project>/locations/<location>/models/<id>`), it's normalized down to just the trailing id/version before being logged; any other string is sent as-is.
 - `batch_info["name"]` must be the exact Vertex job resource name (`projects/<project>/locations/<location>/batchPredictionJobs/<job-id>`), and `batch_info["startTime"]`/`["endTime"]` must be valid ISO 8601 timestamps — a batch with a missing or malformed resource name or timestamps is skipped entirely (logged as an error) rather than sent with a broken URL.
 - If `endTime` precedes `startTime` (e.g. clock skew), the batch is still logged — `duration_ms` is clamped to `0` and a warning is logged — rather than dropping the results.

data/docs/workload-search.md

@@ -0,0 +1,116 @@
+# Reading Workloads (Search)
+
+`Coolhand::WorkloadService#search_workloads` lists your workloads, the groups of prompt templates that
+make up one task or agent, optionally with a cost and performance `metrics` rollup per workload. It
+wraps `GET /api/v2/workloads`. It is read-only, and there is no per-workload endpoint.
+
+**Auth:** requires your **private** API key. The public key is write-only on this API and is rejected
+exactly like an invalid key.
+
+**IDs:** a workload's `:id` is its **hashid**, a String. It is what the `workload_id` filter on
+`search_templates` and `search_logs` expects.
+
+**Scoping:** the client is always derived from your API key. There is deliberately no `client_id`
+keyword, so passing one raises `ArgumentError`.
+
+## `search_workloads`
+
+```ruby
+require "coolhand"
+
+Coolhand.configure do |config|
+  config.api_key = ENV.fetch("COOLHAND_PRIVATE_API_KEY")
+end
+
+result = Coolhand.workload_service.search_workloads(
+  include_metrics: true,
+  since: Time.utc(2026, 9, 1),
+  until: Time.utc(2026, 10, 1),
+  per: 50
+)
+
+result.workloads.each do |workload|
+  puts "#{workload[:name]}: $#{workload[:metrics][:total_cost]}"
+end
+```
+
+### Keywords
+
+All optional, and named exactly like the wire parameters.
+
+| keyword | type | notes |
+|---|---|---|
+| `search` | String | Case-insensitive substring match on the workload name. |
+| `include_archived` | Boolean | Defaults to `false` server-side. |
+| `include_system` | Boolean | `Unmatched`, `Embedding Requests` and the like. Defaults to `false` server-side. |
+| `include_templates` | Boolean | Adds each workload's active templates and their routing patterns as `:templates`. |
+| `include_metrics` | Boolean | Adds a `:metrics` rollup across all of the workload's templates. |
+| `days_back` | Integer | Rolling metrics window ending now (server default 28, max 365). Ignored when `since` is given. |
+| `since` | Time, DateTime, Date or String | Metrics window start, inclusive. Overrides `days_back`. |
+| `until` | Time, DateTime, Date or String | Metrics window end, exclusive. Defaults to now. |
+| `page` | Integer | 1-based. |
+| `per` | Integer | Default 25, max 100, both enforced server-side. This gem only sends `per`, never its `per_page` alias. |
+
+`false` and `0` are sent as given; only an unset (`nil`) keyword is left off the request.
+
+### Windows: `since` and `until`
+
+`Time`, `DateTime` and `Date` values are serialised as UTC ISO8601 (a `Date` as a date alone, which the
+server reads as midnight UTC). A String is passed through untouched, so the server stays the one
+validator. Anything else raises `ArgumentError` before a request is made.
+
+```ruby
+search_workloads(include_metrics: true, since: "2026-09-01")                 # date alone: midnight UTC
+search_workloads(include_metrics: true, since: "2026-09-01T12:00:00+02:00")  # the + is encoded as %2B for you
+```
+
+Server-side rules, which only apply when `include_metrics` is set:
+
+- A missing offset means UTC. `since` is inclusive and `until` is exclusive.
+- With `since`, `days_back` is ignored and `metrics[:days_back]` is `nil`. With only `until`, the window
+  is `days_back` long ending there.
+- The prior window behind `error_rate_change` is the same length, immediately before the window.
+- A malformed value, a `since` not before `until`, or a window over 365 days is a `422` with the error on
+  the `since` or `until` key.
+
+### Return value
+
+A `Coolhand::WorkloadSearchResult`, a `Struct` with `#workloads` and `#pagination`. Rows are plain
+Hashes with Symbol keys, handed back as the API rendered them so a field the server adds later reaches
+you.
+
+| field | type | notes |
+|---|---|---|
+| `:id` | String | Hashid. |
+| `:name` | String | |
+| `:description` | String or nil | |
+| `:archived`, `:system`, `:merged` | Boolean | |
+| `:template_count` | Integer | Active templates. |
+| `:draft_template_count` | Integer | |
+| `:log_count` | Integer | Every log on any of its templates, all generators, so it can exceed the sum of the templates' own `log_count`. |
+| `:last_activity` | String or nil | ISO-8601 UTC. |
+| `:templates` | Array | Only with `include_templates`. Each has `:id`, `:name`, `:status`, `:user_prompt_pattern`, `:system_prompt_pattern`. |
+| `:metrics` | Hash | Only with `include_metrics`. See below. |
+
+Rows are ordered by name. `pagination` is a `Coolhand::Pagination` built from the `X-Page`,
+`X-Per-Page`, `X-Total-Count` and `X-Total-Pages` response headers, which this endpoint always sends,
+never from the length of the array.
+
+### `metrics`
+
+The same object `search_templates` returns per template, computed by the same SQL as the dashboard, so
+tiered pricing, cached-token discounts and reasoning tokens are applied. See
+[Metrics](template-search.md#metrics) for every field.
+
+## Errors
+
+The read methods raise where the write methods log and return `nil`. `Coolhand::HttpError` carries
+`#status` and `#body` (`{"errors": {"<field>": ["msg"]}}` on a `422`).
+
+| status | when |
+|---|---|
+| `401` | No API key, an invalid key, or the public key. |
+| `422` | A bad `days_back`, `since` or `until`. Only checked when `include_metrics` is set. |
+| `504` | An aggregate exceeded the server's statement timeout. Retryable: narrow with `search` or a smaller `per`. |
+
+`Coolhand::Error` covers no API key configured, a transport failure, and a body that is not a JSON array.

data/lib/coolhand/api_service.rb

@@ -80,9 +80,10 @@ module Coolhand
       uri = URI.parse(@api_endpoint)
       http = Net::HTTP.new(uri.host, uri.port)
       http.use_ssl = (uri.scheme == "https")
-      # Bound worst-case latency: this call happens inline in the intercepted
-      # request's path, so a slow/unreachable Coolhand backend must not hang
-      # the host app's real LLM call for Ruby's ~60s Net::HTTP defaults.
+      # Bound worst-case latency: with async_logging off this call happens inline in
+      # the intercepted request's path (otherwise it ties up the background log
+      # worker), so a slow/unreachable Coolhand backend must not hang for Ruby's
+      # ~60s Net::HTTP defaults.
       http.open_timeout = 5
       http.read_timeout = 5