coolhand 0.6.0 → 0.8.0

data/docs/workload-search.md

@@ -0,0 +1,116 @@
+# Reading Workloads (Search)
+
+`Coolhand::WorkloadService#search_workloads` lists your workloads, the groups of prompt templates that
+make up one task or agent, optionally with a cost and performance `metrics` rollup per workload. It
+wraps `GET /api/v2/workloads`. It is read-only, and there is no per-workload endpoint.
+
+**Auth:** requires your **private** API key. The public key is write-only on this API and is rejected
+exactly like an invalid key.
+
+**IDs:** a workload's `:id` is its **hashid**, a String. It is what the `workload_id` filter on
+`search_templates` and `search_logs` expects.
+
+**Scoping:** the client is always derived from your API key. There is deliberately no `client_id`
+keyword, so passing one raises `ArgumentError`.
+
+## `search_workloads`
+
+```ruby
+require "coolhand"
+
+Coolhand.configure do |config|
+  config.api_key = ENV.fetch("COOLHAND_PRIVATE_API_KEY")
+end
+
+result = Coolhand.workload_service.search_workloads(
+  include_metrics: true,
+  since: Time.utc(2026, 9, 1),
+  until: Time.utc(2026, 10, 1),
+  per: 50
+)
+
+result.workloads.each do |workload|
+  puts "#{workload[:name]}: $#{workload[:metrics][:total_cost]}"
+end
+```
+
+### Keywords
+
+All optional, and named exactly like the wire parameters.
+
+| keyword | type | notes |
+|---|---|---|
+| `search` | String | Case-insensitive substring match on the workload name. |
+| `include_archived` | Boolean | Defaults to `false` server-side. |
+| `include_system` | Boolean | `Unmatched`, `Embedding Requests` and the like. Defaults to `false` server-side. |
+| `include_templates` | Boolean | Adds each workload's active templates and their routing patterns as `:templates`. |
+| `include_metrics` | Boolean | Adds a `:metrics` rollup across all of the workload's templates. |
+| `days_back` | Integer | Rolling metrics window ending now (server default 28, max 365). Ignored when `since` is given. |
+| `since` | Time, DateTime, Date or String | Metrics window start, inclusive. Overrides `days_back`. |
+| `until` | Time, DateTime, Date or String | Metrics window end, exclusive. Defaults to now. |
+| `page` | Integer | 1-based. |
+| `per` | Integer | Default 25, max 100, both enforced server-side. This gem only sends `per`, never its `per_page` alias. |
+
+`false` and `0` are sent as given; only an unset (`nil`) keyword is left off the request.
+
+### Windows: `since` and `until`
+
+`Time`, `DateTime` and `Date` values are serialised as UTC ISO8601 (a `Date` as a date alone, which the
+server reads as midnight UTC). A String is passed through untouched, so the server stays the one
+validator. Anything else raises `ArgumentError` before a request is made.
+
+```ruby
+search_workloads(include_metrics: true, since: "2026-09-01")                 # date alone: midnight UTC
+search_workloads(include_metrics: true, since: "2026-09-01T12:00:00+02:00")  # the + is encoded as %2B for you
+```
+
+Server-side rules, which only apply when `include_metrics` is set:
+
+- A missing offset means UTC. `since` is inclusive and `until` is exclusive.
+- With `since`, `days_back` is ignored and `metrics[:days_back]` is `nil`. With only `until`, the window
+  is `days_back` long ending there.
+- The prior window behind `error_rate_change` is the same length, immediately before the window.
+- A malformed value, a `since` not before `until`, or a window over 365 days is a `422` with the error on
+  the `since` or `until` key.
+
+### Return value
+
+A `Coolhand::WorkloadSearchResult`, a `Struct` with `#workloads` and `#pagination`. Rows are plain
+Hashes with Symbol keys, handed back as the API rendered them so a field the server adds later reaches
+you.
+
+| field | type | notes |
+|---|---|---|
+| `:id` | String | Hashid. |
+| `:name` | String | |
+| `:description` | String or nil | |
+| `:archived`, `:system`, `:merged` | Boolean | |
+| `:template_count` | Integer | Active templates. |
+| `:draft_template_count` | Integer | |
+| `:log_count` | Integer | Every log on any of its templates, all generators, so it can exceed the sum of the templates' own `log_count`. |
+| `:last_activity` | String or nil | ISO-8601 UTC. |
+| `:templates` | Array | Only with `include_templates`. Each has `:id`, `:name`, `:status`, `:user_prompt_pattern`, `:system_prompt_pattern`. |
+| `:metrics` | Hash | Only with `include_metrics`. See below. |
+
+Rows are ordered by name. `pagination` is a `Coolhand::Pagination` built from the `X-Page`,
+`X-Per-Page`, `X-Total-Count` and `X-Total-Pages` response headers, which this endpoint always sends,
+never from the length of the array.
+
+### `metrics`
+
+The same object `search_templates` returns per template, computed by the same SQL as the dashboard, so
+tiered pricing, cached-token discounts and reasoning tokens are applied. See
+[Metrics](template-search.md#metrics) for every field.
+
+## Errors
+
+The read methods raise where the write methods log and return `nil`. `Coolhand::HttpError` carries
+`#status` and `#body` (`{"errors": {"<field>": ["msg"]}}` on a `422`).
+
+| status | when |
+|---|---|
+| `401` | No API key, an invalid key, or the public key. |
+| `422` | A bad `days_back`, `since` or `until`. Only checked when `include_metrics` is set. |
+| `504` | An aggregate exceeded the server's statement timeout. Retryable: narrow with `search` or a smaller `per`. |
+
+`Coolhand::Error` covers no API key configured, a transport failure, and a body that is not a JSON array.

data/lib/coolhand/api_service.rb

@@ -80,9 +80,10 @@ module Coolhand
       uri = URI.parse(@api_endpoint)
       http = Net::HTTP.new(uri.host, uri.port)
       http.use_ssl = (uri.scheme == "https")
-      # Bound worst-case latency: this call happens inline in the intercepted
-      # request's path, so a slow/unreachable Coolhand backend must not hang
-      # the host app's real LLM call for Ruby's ~60s Net::HTTP defaults.
+      # Bound worst-case latency: with async_logging off this call happens inline in
+      # the intercepted request's path (otherwise it ties up the background log
+      # worker), so a slow/unreachable Coolhand backend must not hang for Ruby's
+      # ~60s Net::HTTP defaults.
       http.open_timeout = 5
       http.read_timeout = 5
 

data/lib/coolhand/base_interceptor.rb

@@ -11,7 +11,7 @@ module Coolhand
     # (cookie, set-cookie), and future/unknown providers using a
     # similarly-named header. Shared with LoggerService so the two logging
     # paths (interceptor + webhook forwarding) stay consistent.
-    SENSITIVE_HEADER_PATTERN = /key|token|secret|signature|authorization|cookie/i
+    SENSITIVE_HEADER_PATTERN = /key|token|secret|signature|auth|passw|credential|bearer|jwt|session|cookie/i
 
     def sanitize_headers(headers)
       return {} if headers.nil?
@@ -110,7 +110,42 @@ module Coolhand
 
       modified ? uri.to_s : url
     rescue URI::InvalidURIError
-      url
+      # Fail closed: an unparseable URL can't be checked param-by-param, so drop everything
+      # that could carry a credential (userinfo, query, fragment) rather than pass it through.
+      url.to_s.sub(%r{//[^/?#]*@}, "//REDACTED@").split(/[?#]/, 2).first
+    end
+
+    # Key names inside an Azure OpenAI "On Your Data" data_sources/dataSources entry that carry
+    # live datastore credentials in the request body, where header/query-param sanitization never
+    # looks. Matched against a separator-stripped, lowercased key name so both `connection_string`
+    # and `connectionString` (Cosmos/Mongo, embeds `AccountKey=...`) and `encoded_api_key`
+    # (Elasticsearch) are caught — not just exact or snake_case-only names.
+    SENSITIVE_BODY_KEY_PATTERN = /key|token|secret|passw|pwd|credential|bearer|jwt|connectionstring|signature/i
+
+    def sanitize_body(body)
+      return body unless body.is_a?(Hash)
+
+      sanitized = body.dup
+      sanitized.each_key do |key|
+        next unless key.to_s.delete("_-").casecmp?("datasources")
+
+        sanitized[key] = redact_sensitive_body_values(sanitized[key])
+      end
+      sanitized
+    end
+
+    def redact_sensitive_body_values(node)
+      case node
+      when Hash
+        node.each_with_object({}) do |(k, v), acc|
+          normalized = k.to_s.gsub(/[_\-\s]/, "").downcase
+          acc[k] = normalized.match?(SENSITIVE_BODY_KEY_PATTERN) ? "[REDACTED]" : redact_sensitive_body_values(v)
+        end
+      when Array
+        node.map { |v| redact_sensitive_body_values(v) }
+      else
+        node
+      end
     end
 
     def send_complete_request_log(request_id:, method:, url:, request_headers:, request_body:, response_headers:,
@@ -121,7 +156,7 @@ module Coolhand
         method: method.to_s.downcase,
         url: sanitize_url(url),
         headers: sanitize_headers(request_headers),
-        request_body: request_body,
+        request_body: sanitize_body(request_body),
         response_headers: sanitize_headers(response_headers),
         response_body: response_body,
         status_code: status_code,
@@ -134,10 +169,10 @@ module Coolhand
 
       request_data = { raw_request: raw_request }
 
-      api_service = Coolhand::ApiService.new
-      api_service.send_llm_request_log(request_data)
-
-      Coolhand.log "📤 Sent complete request/response log for #{request_id} (duration: #{duration_ms}ms)"
+      LogQueue.submit do
+        Coolhand::ApiService.new.send_llm_request_log(request_data)
+        Coolhand.log "📤 Sent complete request/response log for #{request_id} (duration: #{duration_ms}ms)"
+      end
     rescue StandardError => e
       Coolhand.log "❌ Error sending complete request log: #{e.message}"
     end

data/lib/coolhand/configuration.rb

@@ -29,7 +29,7 @@ module Coolhand
     DEFAULT_MAX_CAPTURED_BODY_BYTES = 1_000_000
 
     attr_accessor :api_key, :environment, :silent, :debug_mode, :capture, :exclude_api_patterns, :enabled,
-      :max_captured_body_bytes, :webhook_replay_tolerance_seconds, :webhook_id_store
+      :max_captured_body_bytes, :webhook_replay_tolerance_seconds, :webhook_id_store, :async_logging
     attr_reader :intercept_addresses, :intercept_path_patterns, :base_url
 
     def initialize
@@ -44,6 +44,7 @@ module Coolhand
       @capture = true
       @exclude_api_patterns = DEFAULT_EXCLUDE_API_PATTERNS.dup
       @enabled = true
+      @async_logging = true
       @max_captured_body_bytes = DEFAULT_MAX_CAPTURED_BODY_BYTES
       @webhook_replay_tolerance_seconds = 300
       @webhook_id_store = Coolhand::OpenAi::WebhookIdStore.new

data/lib/coolhand/default_exclude_api_patterns.yml

@@ -8,3 +8,14 @@
 # Users can disable:           c.exclude_api_patterns = []
 
 - "/batchPredictionJobs/"
+# Azure OpenAI control-plane paths (not inference) — both spellings are needed
+# since substring matching is contiguous: "/openai/files" does not match
+# "/openai/v1/files".
+- "/openai/files"
+- "/openai/batches"
+- "/openai/fine_tuning"
+- "/openai/models"
+- "/openai/v1/files"
+- "/openai/v1/batches"
+- "/openai/v1/fine_tuning"
+- "/openai/v1/models"

data/lib/coolhand/default_intercept_addresses.yml

@@ -4,6 +4,10 @@
 # should be captured and forwarded as an llm_request_log. A single "*" in an
 # entry matches exactly one host label — e.g. "bedrock-runtime.*.amazonaws.com"
 # matches "bedrock-runtime.us-east-1.amazonaws.com" but nothing else.
+# An entry may also pin a port and/or a path prefix: "host:port", "host/path",
+# or "host:port/path" (the path match is on a segment boundary) — useful for
+# multi-service hosts where unrelated APIs (e.g. Azure Speech, Vision, Language)
+# share a domain with the LLM inference path.
 #
 # Users can extend defaults:   c.intercept_addresses << "my.custom.api.com"
 # Users can override entirely: c.intercept_addresses = ["only.this.com"]
@@ -20,5 +24,55 @@
 - "aiplatform.googleapis.com"
 - "gateway.ai.cloudflare.com"
 - "bedrock-runtime.*.amazonaws.com"
+- "api.deepseek.com"
+- "api.mistral.ai"
+- "api.perplexity.ai"
+- "api.x.ai"
+# Cohere is path-scoped, never host-wide: the server only ingests v2 chat and
+# v1/v2 embed, and its other endpoints (v1 chat, rerank, tokenize, classify...)
+# have different envelopes that would be recorded as empty "successes".
+- "api.cohere.com/v2/chat"
+- "api.cohere.com/v1/embed"
+- "api.cohere.com/v2/embed"
+- "api.cohere.ai/v2/chat"
+- "api.cohere.ai/v1/embed"
+- "api.cohere.ai/v2/embed"
+# TypeSafe Jev (a "System One" model, not an LLM) — only /v1/systemone is ingested.
+- "api.typesafe.ai/v1/systemone"
+# Ollama (self-hosted) has no fixed host, so an entry must pin BOTH the default
+# port (11434) and the path — a bare "/api/chat" would capture any app's own
+# unrelated chat endpoint. "*" matches one host label (localhost, an "ollama"
+# compose service, a bare hostname); a LAN IP or dotted host needs its own
+# entry, e.g. "192.168.1.5:11434/api/chat".
+- "*:11434/api/chat"
+- "*:11434/api/generate"
+- "*:11434/api/embed"
+- "*:11434/api/embeddings"
+- "127.0.0.1:11434/api/chat"
+- "127.0.0.1:11434/api/generate"
+- "127.0.0.1:11434/api/embed"
+- "127.0.0.1:11434/api/embeddings"
 - "openrouter.ai"
 - "opencode.ai"
+# Azure OpenAI dedicated hosts (commercial, US Gov, China/21Vianet)
+- "openai.azure.com"
+- "openai.azure.us"
+- "openai.azure.cn"
+# Azure AI Services / Foundry multi-service hosts — path-anchored so Speech,
+# Vision, Language and Content Safety on the same host are NOT captured
+- "cognitiveservices.azure.com/openai/"
+- "cognitiveservices.azure.com/models/"
+- "cognitiveservices.azure.us/openai/"
+- "cognitiveservices.azure.us/models/"
+- "cognitiveservices.azure.cn/openai/"
+- "cognitiveservices.azure.cn/models/"
+- "services.ai.azure.com/openai/"
+- "services.ai.azure.com/models/"
+- "services.ai.azure.us/openai/"
+- "services.ai.azure.us/models/"
+# Serverless / MaaS + Azure ML managed online endpoints. inference.ml.azure.*
+# is deliberately unanchored — see docs/configuration.md for why.
+- "inference.ai.azure.com"
+- "models.ai.azure.com"
+- "inference.ml.azure.com"
+- "inference.ml.azure.us"

data/lib/coolhand/errors.rb

@@ -3,7 +3,8 @@
 module Coolhand
   class Error < StandardError; end
 
-  # Raised by the gem's read methods when the Coolhand API answers with a non-2xx status.
+  # Raised by the gem's raising API methods (template reads, feedback links) when the Coolhand API
+  # answers with a non-2xx status.
   #
   # `status` is carried so callers can branch on it (404 vs retryable 504) without matching the
   # message text.

data/lib/coolhand/log_queue.rb

@@ -0,0 +1,179 @@
+# frozen_string_literal: true
+
+module Coolhand
+  # Runs log-shipping work on background threads so a slow or unreachable Coolhand backend can't
+  # add latency to the host app's own LLM calls.
+  #
+  # Work is spread over two lanes, each with its own bounded queue and worker thread: `:default`
+  # for the one-log-per-intercepted-call sends, and `:batch` for whole-batch jobs (download +
+  # one send per item) that can run for minutes. Keeping them apart stops a long batch from
+  # delaying — or, once the queue fills, evicting — the real-time logs.
+  #
+  # Each queue is bounded (oldest job dropped when full), restarts itself in a forked child, and
+  # is flushed at process exit. Work runs inline instead when `config.async_logging` is false,
+  # when `config.debug_mode` is on (debug output should appear in order), or when the caller is
+  # already a worker thread (a queued job's own sends don't re-enter a queue).
+  module LogQueue
+    MAX_QUEUE_SIZE = 1000
+    DEFAULT_FLUSH_TIMEOUT = 5
+
+    # One bounded FIFO plus the thread that drains it.
+    class Lane
+      attr_reader :thread
+
+      def initialize
+        @mutex = Mutex.new
+        @wakeup = ConditionVariable.new
+        @idle = ConditionVariable.new
+        @jobs = []
+        @thread = nil
+        @pid = Process.pid
+        @busy = false
+      end
+
+      def enqueue(job)
+        @mutex.synchronize do
+          restart_after_fork
+          drop_oldest if @jobs.size >= MAX_QUEUE_SIZE
+          @jobs << job
+          ensure_worker
+          @wakeup.signal
+        end
+      end
+
+      # True once every queued job has finished, false if the timeout elapsed first.
+      def flush(deadline)
+        @mutex.synchronize do
+          # A forked child's copy of the queue belongs to the parent's worker; nothing to wait on.
+          return true if @pid != Process.pid
+          # A dead worker will never drain the queue, so don't wait on it.
+          return @jobs.empty? unless @thread&.alive?
+
+          while @jobs.any? || @busy
+            remaining = deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
+            return false unless remaining.positive?
+
+            @idle.wait(@mutex, remaining)
+          end
+          true
+        end
+      end
+
+      def reset!
+        thread = nil
+        @mutex.synchronize do
+          thread = @thread
+          @jobs.clear
+          @thread = nil
+          @busy = false
+          @wakeup.broadcast
+          @idle.broadcast
+        end
+        thread&.join(1)
+        thread&.kill if thread&.alive?
+      end
+
+      private
+
+      def drop_oldest
+        @jobs.shift
+        Coolhand.log "⚠️ Log queue full (#{MAX_QUEUE_SIZE}); dropping the oldest pending log"
+      end
+
+      # A forked child inherits the queue's memory but not its worker thread.
+      def restart_after_fork
+        return if @pid == Process.pid
+
+        @pid = Process.pid
+        @jobs = []
+        @thread = nil
+        @busy = false
+      end
+
+      def ensure_worker
+        return if @thread&.alive?
+
+        @thread = Thread.new { work }
+        @thread.report_on_exception = false
+      end
+
+      def work
+        loop do
+          job = next_job
+          break unless job
+
+          begin
+            run(job)
+          ensure
+            @mutex.synchronize do
+              @busy = false
+              @idle.broadcast
+            end
+          end
+        end
+      end
+
+      def next_job
+        @mutex.synchronize do
+          @wakeup.wait(@mutex) while @jobs.empty? && @thread == Thread.current
+          return nil unless @thread == Thread.current
+
+          @busy = true
+          @jobs.shift
+        end
+      end
+
+      def run(job)
+        job.call
+      rescue StandardError => e
+        Coolhand.log "❌ Error in background log job: #{e.message}"
+      end
+    end
+
+    LANES = { default: Lane.new, batch: Lane.new }.freeze
+
+    @at_exit_mutex = Mutex.new
+    @at_exit_registered = false
+
+    class << self
+      def submit(lane: :default, &job)
+        return yield unless async?
+
+        register_at_exit
+        LANES.fetch(lane).enqueue(job)
+        nil
+      end
+
+      # Blocks until every queued job has finished or the timeout elapses. Returns true when
+      # every lane drained in time.
+      def flush(timeout = DEFAULT_FLUSH_TIMEOUT)
+        deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + timeout
+        LANES.values.map { |lane| lane.flush(deadline) }.all?
+      end
+
+      # Testing-only: discard queued work and stop the workers.
+      def reset!
+        LANES.each_value(&:reset!)
+      end
+
+      private
+
+      def async?
+        Coolhand.configuration.async_logging && !Coolhand.configuration.debug_mode && !worker_thread?
+      end
+
+      def worker_thread?
+        LANES.each_value.any? { |lane| lane.thread == Thread.current }
+      end
+
+      def register_at_exit
+        @at_exit_mutex.synchronize do
+          return if @at_exit_registered
+
+          @at_exit_registered = true
+          at_exit { flush }
+        end
+      end
+    end
+  end
+end

data/lib/coolhand/log_service.rb

@@ -0,0 +1,67 @@
+# frozen_string_literal: true
+
+require_relative "api_service"
+require_relative "pagination"
+require_relative "read_query"
+
+module Coolhand
+  # Rows stay plain Symbol-keyed Hashes so a field the server adds later is not dropped in transit.
+  LogSearchResult = Struct.new(:logs, :pagination, keyword_init: true)
+
+  # Reads logs back out of Coolhand. The write side is {LoggerService}. Requires the client's
+  # **private** key. See docs/log-search.md.
+  class LogService < ApiService
+    include ReadQuery
+
+    ERROR_NOUN = "Log"
+    BLANK_ID_MESSAGE = "get_log: id must be a non-empty log hashid"
+
+    def initialize
+      super("v2/llm_request_logs")
+    end
+
+    # Filters, cost ordering, error semantics and server behaviour: docs/log-search.md.
+    def search_logs(template_id: nil, workload_id: nil, system_prompt_contains: nil, user_prompt_contains: nil,
+      model: nil, source_api: nil, source_api_result: nil, source_application: nil, project_path: nil,
+      unmatched_only: nil, days_back: nil, since: nil, until: nil, min_cost: nil, order: nil,
+      include_prompts: nil, include_total: nil, sort: nil, page: nil, per: nil)
+      query = {
+        template_id: template_id,
+        workload_id: workload_id,
+        system_prompt_contains: system_prompt_contains,
+        user_prompt_contains: user_prompt_contains,
+        model: model,
+        source_api: source_api,
+        source_api_result: source_api_result,
+        source_application: source_application,
+        project_path: project_path,
+        unmatched_only: unmatched_only,
+        days_back: days_back,
+        **window_params(since, binding.local_variable_get(:until)),
+        min_cost: min_cost,
+        order: order,
+        include_prompts: include_prompts,
+        include_total: include_total,
+        "q[s]": sort,
+        page: page,
+        per: per
+      }.compact
+
+      logs, response = get_json_with_headers(list_url(query), ERROR_NOUN)
+      raise Error, "#{ERROR_NOUN} response was not a JSON array" unless logs.is_a?(Array)
+
+      LogSearchResult.new(
+        logs: logs,
+        pagination: Pagination.from_headers(response, items: logs, page: page, per: per)
+      ).freeze
+    end
+
+    # The full log, with `cost` and `cost_breakdown`. Both are nil when the log has no tokens or its
+    # model has no pricing.
+    def get_log(id, section: nil, max_chars: nil, search_query: nil, include_thinking: nil)
+      query = { section: section, max_chars: max_chars, search_query: search_query,
+                include_thinking: include_thinking }.compact
+      get_json(resource_url(id, BLANK_ID_MESSAGE, query), ERROR_NOUN)
+    end
+  end
+end