batchwatch 0.2.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: bfefddebcc183a5e63b03b570caa3d183103d4b661f90a617779241306201d6b
+  data.tar.gz: 02de063b1c8f22d8d2a5f93f2c3dda990b482af905ab811c5a630eb029289279
+SHA512:
+  metadata.gz: e06032ef12b72d4d8213ae27271723afcbbb87a17087304ecda1ea999c8fe65777fc87a39b773cfc58faed151e4d685e94127e4186c4792d59bfd08af553e6fc
+  data.tar.gz: 8134786ba93b55272cbdf87db36e02581b600d229aa5ef08d8275810c93c29344ae583193b6488d8036f14787dac812a39eb3b356bc61132a7c7fc700c728908

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Andreas Graae
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,176 @@
+# batchwatch — Ruby client
+
+Client for [batchwatch.dev](https://batchwatch.dev): crowdsourced measurement
+of queue time on LLM batch APIs.
+
+Batch endpoints cost 50% of the synchronous ones, but "completes within 24
+hours" is impossible to plan around. batchwatch measures what the queue
+actually does and answers one question: *should I use batch for this job?*
+
+No dependencies. The standard library — `net/http`, `json`, `uri`, `socket`,
+`tmpdir` — and nothing else.
+
+## Install
+
+Point your `Gemfile` at the subdirectory:
+
+```ruby
+gem "batchwatch", git: "https://github.com/batchwatch/client",
+                  glob: "clients/ruby/*.gemspec"
+```
+
+or vendor the three files under `lib/` — they have no third-party imports. A
+RubyGems release is on the way.
+
+## Two lines
+
+```ruby
+require "batchwatch"
+
+bw = Batchwatch::Client.new(token: "bw_...")  # token optional; falls back to $BATCHWATCH_TOKEN
+
+# 1. before you submit — does this belong in the queue?
+if bw.should_batch("gpt-5.6-sol", max_wait: "15m")
+  job = client.batches.create(...)
+else
+  answer = client.chat.completions.create(...)
+end
+
+# 2. measure it, so the next person gets a better answer
+bw.track("gpt-5.6-sol", input_tokens: 9720) do |t|
+  result = wait_for(job)
+  t.done(output_tokens: result.usage.completion_tokens)
+end
+```
+
+With a block, `track` closes the measurement for you, including on the error
+path: an exception raised inside your block is recorded as `failed` and
+re-raised untouched. Without a block it returns the tracking handle and you
+call `t.done(...)` yourself.
+
+Get a key with no email and no card:
+
+    curl -X POST https://batchwatch.dev/v1/keys -d '{"label":"my pipeline"}'
+
+## It fails open, always
+
+If batchwatch is down, slow, or broken, your job must not notice. That is the
+first requirement, ahead of collecting any data at all.
+
+- Every submission runs on a background thread. `track()` and `t.done()` do no
+  network I/O on your thread.
+- Two-second timeout by default (`BATCHWATCH_TIMEOUT`), applied to both the
+  connect and the read, so a server that accepts but never answers cannot hold
+  you.
+- Every batchwatch error is swallowed and passed to the optional `logger` at
+  `debug` level. Nothing is printed unless you wire one up.
+- `should_batch()` is the one synchronous call, because you are waiting for the
+  answer. If it cannot answer, you get your own `default` back — never a guess.
+  The default is `false`, "run it synchronously": being wrong that way costs
+  money, being wrong the other way blows a deadline.
+- An exception raised inside your own `track` block is recorded as `failed` and
+  re-raised untouched. We swallow our errors, never yours.
+
+`test/test_fail_open.rb` proves it against a port nothing listens on and
+against a socket that accepts but never answers.
+
+## It never sends your content
+
+No prompts, no completions, no system prompts, no tool calls, no file names.
+The request body is built from a fixed allowlist — provider, model, mode,
+endpoint, request count, token counts, timestamps, status — and everything
+else is dropped by `Batchwatch.sanitize` on the way out. There is no field to put
+text in.
+
+`test/test_no_content.rb` asserts it on the bytes a real HTTP server received,
+and includes a positive control so the test cannot pass by the client simply
+sending nothing.
+
+## `output_tokens` defaults to `nil`, never `0`
+
+You know your input tokens. You cannot know your output tokens before the model
+has answered. So the default is absence (`nil`), not zero.
+
+Zero is not a harmless placeholder here: output costs five to six times as much
+as input, so a saving computed on zero output is systematically too low —
+measured at 3.4x too low on a real model — and nothing in the response would
+tell you. If you know a ceiling, pass `max_tokens` instead and the answer comes
+back labelled as a ceiling.
+
+Passing `output_tokens: 0` really does send `0`: zero is a measurement, absence
+is not.
+
+## Spooling
+
+When a measurement cannot be delivered, the completed record is appended to a
+JSONL file and replayed later through `POST /v1/calls/complete`. Losing
+measurements exactly when the network is bad means losing them exactly when
+they are most interesting.
+
+- Default path: `$BATCHWATCH_SPOOL`, or `batchwatch-spool.jsonl` in the system
+  temp directory (`Dir.tmpdir`). Set `BATCHWATCH_SPOOL=""` or pass
+  `spool: nil` to turn it off.
+- The spool is replayed automatically, at most once a minute, right after a
+  successful call — that is the moment we know the network is up. Call
+  `bw.flush_spool` yourself from a shutdown hook if you want it drained on
+  exit.
+- **Spooling requires a token.** `/v1/calls/complete` takes your own
+  timestamps, so it is closed to anonymous callers; without a key a spool file
+  could never be sent, and writing one would just leak disk. `bw.spool` is
+  `nil` when no token is set.
+- The file is capped at 5 MB. Beyond that, measurements are dropped rather than
+  filling your disk.
+- A replayed measurement can arrive twice if the original `PATCH` reached the
+  server but the response did not. That is deliberate: a duplicate is visible
+  in the dataset, a lost measurement is not.
+- The file format is identical across the Python, TypeScript, Go and Ruby
+  clients, so a spool written by one can be flushed by another.
+- Threads are handled by a mutex. Two *processes* sharing one spool file may
+  send a record twice — give each process its own `BATCHWATCH_SPOOL` if that
+  matters.
+
+## Configuration
+
+| Argument | Environment | Default |
+|---|---|---|
+| `token` | `BATCHWATCH_TOKEN` | none (anonymous) |
+| `base_url` | `BATCHWATCH_URL` | `https://batchwatch.dev` |
+| `timeout` | `BATCHWATCH_TIMEOUT` | `2.0` seconds |
+| `spool` | `BATCHWATCH_SPOOL` | `<tempdir>/batchwatch-spool.jsonl` |
+| `enabled` | — | `true` |
+| `logger` | — | `nil` (nothing logged) |
+
+`enabled: false` turns every network call into a no-op, which is what you want
+in CI.
+
+## API
+
+- `should_batch(model, max_wait: nil, default: false, **kw) -> true/false`
+- `advice(model, max_wait: nil, provider: "openai", input_tokens: nil, output_tokens: nil, max_tokens: nil, risk: "p90") -> Hash | nil`
+- `wait_now(model, provider: "openai", mode: "batch") -> Hash | nil`
+- `track(model, provider: "openai", mode: "batch", requests: 1, input_tokens: nil, endpoint: nil) { |t| ... }` — with or without a block
+  - `t.done(output_tokens: nil, status: "completed", ttfb_ms: nil)`
+  - `t.failed(status: "failed")`
+  - `t.started(input_tokens: ...)` when the count is only known after submission
+- `flush(timeout: 5.0) -> true/false` — wait for outstanding submissions before exit
+- `flush_spool(timeout: nil) -> Integer` — send what is on disk, returns accepted
+
+## Tests
+
+    rake test
+    # or a single suite:
+    ruby -Ilib -Itest test/test_fail_open.rb
+
+26 tests, no network beyond loopback. They start real HTTP servers on
+ephemeral ports (raw `TCPServer`, port 0) rather than stubbing `Net::HTTP`:
+the thing under test is network behaviour, so the network should be in the
+test. The allowlist and fail-open tests carry positive controls, so a client
+that sent nothing at all would fail them rather than pass.
+
+## Requirements
+
+Requires Ruby 3.0 or newer. Tested on 3.1.
+
+## Licence
+
+MIT

data/lib/batchwatch/client.rb

@@ -0,0 +1,670 @@
+# frozen_string_literal: true
+
+# batchwatch client.
+#
+# Two rules shape this file.
+#
+# **It never fails upward.** If batchwatch is down, slow, or broken, the
+# caller's batch job must not notice. Every network call runs on a background
+# thread with a short timeout, and every error is swallowed and logged at
+# DEBUG. The only exception that ever leaves this module is the caller's own.
+#
+# **It never sends content.** Prompts, completions, system prompts, tool
+# calls, file names - none of it. The body is built from a fixed allowlist of
+# fields: provider, model, mode, endpoint, request count, token counts and
+# timestamps. There is no field to put text in, by construction.
+#
+#     require "batchwatch"
+#
+#     bw = Batchwatch::Client.new(token: "bw_...")   # token is optional
+#
+#     # 1) before you submit: does this belong in the queue?
+#     if bw.should_batch("gpt-5.6-sol", max_wait: "15m")
+#       job = client.batches.create(...)
+#     else
+#       answer = client.chat.completions.create(...)
+#     end
+#
+#     # 2) measure it
+#     bw.track("gpt-5.6-sol", input_tokens: 9720) do |t|
+#       # ...
+#       t.done(output_tokens: 4519)
+#     end
+require "digest"
+require "json"
+require "net/http"
+require "uri"
+require "tmpdir"
+require "time"
+
+require_relative "spool"
+
+module Batchwatch
+  VERSION = "0.2.1"
+
+  # How often, at most, we try to drain the spool on our own.
+  SPOOL_INTERVAL_S = 60.0
+
+  # How long completion waits for the start call's id to land. Only the
+  # background thread waits - the caller is long gone.
+  ID_WAIT_S = 3.0
+
+  # Field names allowed to leave the machine. Everything else is not in the
+  # body. The test test_no_content.rb pins this list. Same fields as the
+  # Python, Go, TypeScript and .NET clients.
+  #
+  # The last four (acted_verdict, deadline_s, quoted_p50_s, quoted_p90_s) are
+  # the outcome measurement (#101): the advice we gave ourselves, attached to
+  # the later completion so the SERVER can compare its own measured duration
+  # with what we promised. All of it is numbers and a decision we made
+  # ourselves - no new PII.
+  ALLOWED_FIELDS = %w[
+    provider model mode endpoint requests
+    input_tokens output_tokens started_at ended_at
+    status ttfb_ms source
+    acted_verdict deadline_s quoted_p50_s quoted_p90_s
+  ].freeze
+
+  def self.default_url
+    ENV["BATCHWATCH_URL"] || "https://batchwatch.dev"
+  end
+
+  def self.default_timeout
+    (ENV["BATCHWATCH_TIMEOUT"] || "2.0").to_f
+  end
+
+  def self.default_spool
+    v = ENV["BATCHWATCH_SPOOL"]
+    unless v.nil?
+      # An empty string turns the spool off.
+      return v.empty? ? nil : v
+    end
+    File.join(Dir.tmpdir, "batchwatch-spool.jsonl")
+  end
+
+  # Drop everything not on the allowlist.
+  #
+  # The last stop before the network. Even though no public method accepts free
+  # text, THIS function is the place you can point to when someone asks "how do
+  # you know a prompt cannot get out".
+  def self.sanitize(body)
+    body.each_with_object({}) do |(k, v), out|
+      out[k.to_s] = v if ALLOWED_FIELDS.include?(k.to_s)
+    end
+  end
+
+  def self.iso(time)
+    time.utc.strftime("%Y-%m-%dT%H:%M:%SZ")
+  end
+
+  # ---------------------------------------------------------- idempotency (#30)
+  #
+  # The die-and-reflush problem: the process dies with a completed measurement
+  # on disk, restarts, and replays it. If the first attempt already reached the
+  # server, the replay must NOT count the same job twice - a duplicate row drags
+  # the percentiles (the whole product) toward the slow tail, because the slow
+  # calls are exactly the ones that time out and get resent.
+  #
+  # The fix is an Idempotency-Key that the server dedupes on. The rule that
+  # makes it work: the key IS DERIVED FROM THE MEASUREMENT, deterministically,
+  # so the replay reconstructs exactly the same key as the first attempt. A
+  # fresh key at send time (a UUID per call) would give the replay a DIFFERENT
+  # key, and nothing would be deduped. The key is a pure function of the
+  # measurement's content - the record IS its own stored key material.
+  #
+  # The scheme follows backfill.py's send(): a readable, named string built from
+  # the identifying fields; printable ASCII with no control characters.
+
+  def self.idem_field(body, key)
+    v = body[key] || body[key.to_sym]
+    v.nil? ? "" : v.to_s
+  end
+  private_class_method :idem_field
+
+  # A short, stable content hash of a record. Hex SHA-256 (first 16 bytes) of
+  # the record's canonical JSON, serialised in the same insertion order the
+  # client sends on the wire - so the same record hashes to the same value
+  # every time, on every run, and a replay reconstructs it exactly. Printable
+  # ASCII, no control characters, well within the server's 255-char key limit.
+  def self.content_hash(body)
+    Digest::SHA256.hexdigest(JSON.generate(body))[0, 32]
+  end
+  private_class_method :content_hash
+
+  # Per-record key for POST /v1/calls (the start call).
+  #
+  # Keeps the readable provider/model prefix for logs, then a content hash of
+  # the full (sanitized) start body. started_at is second-resolution, so two
+  # DISTINCT measurements of the same provider+model within one second would
+  # otherwise share a key - and the server refuses a reused key that carries a
+  # different body (409 body_differs), silently dropping the second measurement.
+  # Hashing the whole body keeps distinct measurements apart; an identical body
+  # (a retry or a die-and-reflush replay) still hashes the same and dedupes, so
+  # #30 holds. This key is only ever computed live at the start POST, never
+  # reconstructed from a spool line, so idem_complete's cross-client
+  # spool-replay key format is untouched.
+  def self.idem_start(body)
+    "bw-start-#{idem_field(body, 'provider')}-#{idem_field(body, 'model')}-" \
+      "#{content_hash(body)}"
+  end
+
+  # Per-request key for POST /v1/calls/complete. Derived from the group being
+  # sent, exactly like backfill.py: a single record is just a group of one.
+  # A replay of the SAME records chunks them identically (the spool preserves
+  # the order), so the same request carries the same key.
+  def self.idem_complete(group)
+    return nil if group.nil? || group.empty?
+
+    first = group.first
+    last = group.last
+    "bw-complete-#{idem_field(first, 'provider')}-#{idem_field(first, 'started_at')}-" \
+      "#{group.length}-#{idem_field(last, 'ended_at')}"
+  end
+
+  # Translate a max_wait into seconds, or nil if we cannot make sense of it.
+  #
+  # The caller typically writes "15m", "1h", "30s" or "2d" - the same language
+  # /v1/should-i-batch itself accepts. A bare number is read as seconds. If we
+  # cannot parse it, we send nil rather than a guess (rule #30).
+  def self.seconds(max_wait)
+    return nil if max_wait.nil?
+    return max_wait.to_f if max_wait.is_a?(Numeric)
+
+    s = max_wait.to_s.strip.downcase
+    return nil if s.empty?
+
+    factor = { "s" => 1.0, "m" => 60.0, "h" => 3600.0, "d" => 86_400.0 }
+    unit = s[-1]
+    if factor.key?(unit)
+      num = s[0..-2]
+      mult = factor[unit]
+    else
+      num = s
+      mult = 1.0
+    end
+    num = num.strip
+    return nil unless num.match?(/\A-?\d+(\.\d+)?\z/)
+
+    num.to_f * mult
+  end
+
+  # The fields from a stored advice that may be attached to a completion (#101).
+  #
+  # Only non-nil values come along: if a percentile or a deadline is missing,
+  # the field is OMITTED entirely - we do not invent a zero (rule #30). With no
+  # advice at all the result is an empty hash, so nothing is attached.
+  def self.outcome_fields(advice)
+    return {} if advice.nil?
+
+    out = { "acted_verdict" => advice["acted_verdict"] }
+    %w[deadline_s quoted_p50_s quoted_p90_s].each do |name|
+      out[name] = advice[name] unless advice[name].nil?
+    end
+    out
+  end
+
+  # A non-2xx response. Never reaches the caller of a public method - only the
+  # optional debug logger.
+  class HTTPError < StandardError
+    attr_reader :status, :body
+
+    def initialize(status, body)
+      @status = status
+      @body = body
+      super("batchwatch #{status}")
+    end
+  end
+
+  # The client. Every submission is non-blocking and fails open.
+  #
+  # Args:
+  #   token: API key. Falls back to $BATCHWATCH_TOKEN. Optional for
+  #     measurement, required to replay a spool.
+  #   base_url: Default $BATCHWATCH_URL or https://batchwatch.dev
+  #   timeout: Seconds per HTTP call. Default $BATCHWATCH_TIMEOUT or 2.0.
+  #   enabled: false turns every network call into a no-op.
+  #   spool: Path to the spool file, nil to turn it off. Default
+  #     $BATCHWATCH_SPOOL or a file in the temp directory. Spooling is inactive
+  #     without a token, because the replay route requires one.
+  #   logger: Optional logger. All swallowed errors go to logger.debug.
+  class Client
+    attr_reader :token, :base, :timeout, :enabled, :spool
+
+    UNSET = Object.new
+    private_constant :UNSET
+
+    def initialize(token: nil, base_url: nil, timeout: nil, enabled: true,
+                   spool: UNSET, logger: nil)
+      @token = token || ENV["BATCHWATCH_TOKEN"]
+      @base = (base_url || Batchwatch.default_url).sub(%r{/+\z}, "")
+      @timeout = timeout.nil? ? Batchwatch.default_timeout : timeout
+      @enabled = enabled
+      @logger = logger
+      @threads = []
+      @threads_lock = Mutex.new
+
+      path = spool.equal?(UNSET) ? Batchwatch.default_spool : spool
+      # Without a key /v1/calls/complete cannot accept it, so a spool file could
+      # never be sent. So we do not write it.
+      @spool = (path && @token) ? Spool.new(path, logger: logger) : nil
+      if path && !@token
+        debug("batchwatch: no token - spool disabled " \
+              "(/v1/calls/complete requires a key)")
+      end
+      @spool_last = 0.0
+      @spool_lock = Mutex.new
+      # Last advice per model (#101). When should_batch answers, we store here
+      # what we recommended + the quoted percentiles, so the NEXT completion for
+      # the same model can attach them. The correlation is a documented
+      # approximation: latest-advice-per-model, not per-job.
+      @advice = {}
+      @advice_lock = Mutex.new
+    end
+
+    # ----------------------------------------------------------------- core
+
+    # One HTTP call. Raises on error - only public methods swallow.
+    #
+    # +idem+ is an optional Idempotency-Key. On the write routes (/v1/calls and
+    # /v1/calls/complete) it lets the server dedupe a resend: the same body +
+    # the same key writes the measurement once, no matter how many times a
+    # die-and-reflush sends it (#30). The key is derived from the measurement,
+    # so the replay carries the same key as the first attempt. nil = no header.
+    def call(path, method: "GET", body: nil, timeout: nil, idem: nil)
+      uri = URI.parse(@base + path)
+      deadline = timeout || @timeout
+
+      req_class = case method
+                  when "GET" then Net::HTTP::Get
+                  when "POST" then Net::HTTP::Post
+                  when "PATCH" then Net::HTTP::Patch
+                  else raise ArgumentError, "unknown method #{method}"
+                  end
+      req = req_class.new(uri)
+      req["user-agent"] = "batchwatch-ruby/#{VERSION}"
+      unless body.nil?
+        req["content-type"] = "application/json"
+        req.body = JSON.generate(body)
+      end
+      req["authorization"] = "Bearer #{@token}" if @token
+      req["idempotency-key"] = idem if idem && !idem.empty?
+
+      http = Net::HTTP.new(uri.host, uri.port)
+      http.use_ssl = (uri.scheme == "https")
+      # Both the connect and the read timeout, so a server that accepts but
+      # never answers cannot hold us longer than the deadline.
+      http.open_timeout = deadline
+      http.read_timeout = deadline
+
+      res = http.request(req)
+      code = res.code.to_i
+      raw = res.body || ""
+      raise HTTPError.new(code, raw) if code < 200 || code >= 300
+
+      raw.strip.empty? ? nil : JSON.parse(raw)
+    end
+
+    # Run on a background thread. Errors are swallowed - this must never take
+    # the caller's own call down with it.
+    def in_background(&block)
+      return nil unless @enabled
+
+      t = Thread.new do
+        begin
+          block.call
+        rescue HTTPError => e
+          debug("batchwatch #{e.status}: #{e.body}")
+        rescue StandardError => e
+          debug("batchwatch unavailable: #{e}")
+        end
+      end
+      # Clean up finished threads, so the list does not grow without bound in a
+      # long-running process.
+      @threads_lock.synchronize do
+        @threads.reject! { |x| !x.alive? }
+        @threads << t
+      end
+      t
+    end
+
+    # Wait for outstanding submissions. Call it before the process exits.
+    def flush(timeout: 5.0)
+      deadline = monotonic + timeout
+      snapshot = @threads_lock.synchronize { @threads.dup }
+      snapshot.each do |t|
+        rest = deadline - monotonic
+        t.join(rest > 0 ? rest : 0)
+      end
+      @threads_lock.synchronize { @threads.reject! { |t| !t.alive? } }
+      @threads_lock.synchronize { @threads.empty? }
+    end
+
+    # ---------------------------------------------------------- decision
+
+    # What the queue is doing right now, or nil if we cannot say.
+    def wait_now(model, provider: "openai", mode: "batch")
+      q = URI.encode_www_form(provider: provider, model: model, mode: mode)
+      r = call("/v1/wait?#{q}")
+      return nil if !r || r["verdict"] == "insufficient_data"
+
+      r
+    rescue StandardError => e
+      debug("batchwatch wait failed: #{e}")
+      nil
+    end
+
+    # The whole verdict. Returns nil if we cannot answer.
+    #
+    # output_tokens is usually UNKNOWN at this point - the model decides them.
+    # The default is therefore nil, not zero. Sending zero would make the
+    # server compute the saving on zero output, and output costs five to six
+    # times as much as input: the answer would be systematically too low,
+    # without anyone seeing it. If you know a ceiling, pass max_tokens.
+    def advice(model, max_wait: nil, provider: "openai",
+               input_tokens: nil, output_tokens: nil, max_tokens: nil,
+               risk: "p90")
+      q = { "provider" => provider, "model" => model, "risk" => risk }
+      { "input_tokens" => input_tokens, "output_tokens" => output_tokens,
+        "max_tokens" => max_tokens }.each do |name, value|
+        q[name] = value.to_i unless value.nil?
+      end
+      q["max_wait"] = max_wait unless max_wait.nil?
+      call("/v1/should-i-batch?#{URI.encode_www_form(q)}")
+    rescue StandardError => e
+      debug("batchwatch advice failed: #{e}")
+      nil
+    end
+
+    # true/false. On any doubt you get your default back.
+    #
+    # We never guess on the caller's behalf: if we cannot answer, the caller
+    # gets their own predetermined value. The default is false - "run it
+    # synchronously" - which is the safe way to be wrong, because a synchronous
+    # call just costs more, while an unexpected eight-hour queue can take a
+    # product down.
+    def should_batch(model, max_wait: nil, default: false, **kw)
+      r = advice(model, max_wait: max_wait, **kw)
+      return default unless r
+
+      answer = case r["verdict"]
+               when "run_batch" then true
+               when "run_sync", "batch_at" then false
+               else default
+               end
+      remember_advice(model, answer, max_wait, r)
+      answer
+    end
+
+    # Store the advice we just gave for THIS model (#101). Only numbers and our
+    # own decision end up here - no user data. The quoted percentiles are taken
+    # from the server's response; if one of them is missing, we store nil and
+    # simply do not attach it (rule #30: nothing invented as 0).
+    def remember_advice(model, acted_verdict, max_wait, answer)
+      num = ->(v) { v.is_a?(Numeric) ? v.to_f : nil }
+      advice = {
+        "acted_verdict" => acted_verdict,
+        "deadline_s" => Batchwatch.seconds(max_wait),
+        "quoted_p50_s" => num.call(answer["p50_s"]),
+        "quoted_p90_s" => num.call(answer["p90_s"])
+      }
+      @advice_lock.synchronize { @advice[model] = advice }
+    end
+
+    # The latest advice for a model, or nil. The approximation is
+    # latest-advice-per-model: the newest should_batch attaches to the next
+    # completion of the same model.
+    def get_advice(model)
+      @advice_lock.synchronize { @advice[model] }
+    end
+
+    # ------------------------------------------------------------- measurement
+
+    # Measure one call. The submission happens in the background.
+    #
+    # No exception from batchwatch ever reaches the caller. An exception from
+    # the caller's own block is recorded as "failed" and re-raised untouched.
+    #
+    # With a block it acts like the Python context manager: it closes the
+    # measurement for you, including on the error path. Without a block it
+    # returns the tracking handle and you call done() yourself.
+    def track(model, provider: "openai", mode: "batch", requests: 1,
+              input_tokens: nil, endpoint: nil)
+      t = Tracking.new(self, provider, model, mode, requests, input_tokens,
+                       endpoint)
+      t.start
+      return t unless block_given?
+
+      begin
+        yield t
+      rescue Exception # rubocop:disable Lint/RescueException
+        # Everything - including Interrupt/SignalException - is recorded as
+        # failed and re-raised untouched. We swallow our own errors, never the
+        # caller's.
+        t.done(status: "failed")
+        raise
+      else
+        t.done unless t.finished?
+      end
+    end
+
+    # --------------------------------------------------------------- spool
+
+    # Send everything waiting on disk. Returns the number accepted.
+    #
+    # Synchronous and safe to call from a shutdown hook. Never raises. Records
+    # the server rejects as invalid are dropped - they will never become valid
+    # - and the count is logged.
+    def flush_spool(timeout: nil)
+      return 0 if !@spool || !@enabled
+
+      records = @spool.take
+      return 0 if records.empty?
+
+      sent = 0
+      rest = records.dup
+      until rest.empty?
+        group = rest.shift(MAX_BATCH)
+        begin
+          # The key is derived from the group itself: a replay after a crash
+          # chunks the records identically (the spool preserves the order), so
+          # the key is the same and the server dedupes the double write (#30).
+          r = call("/v1/calls/complete", method: "POST", body: group,
+                                         timeout: timeout || [@timeout, 10.0].max,
+                                         idem: Batchwatch.idem_complete(group))
+        rescue StandardError => e
+          debug("batchwatch: spool could not be sent: #{e}")
+          # The group that failed stays put along with the rest.
+          @spool.keep(group + rest)
+          return sent
+        end
+        sent += (r || {}).fetch("accepted", 0)
+        rejected = (r || {}).fetch("rejected", 0)
+        if rejected && rejected > 0
+          debug("batchwatch: #{rejected} spooled measurements were rejected and dropped")
+        end
+      end
+      @spool.keep([])
+      debug("batchwatch: #{sent} spooled measurements sent")
+      sent
+    end
+
+    # Called from the background thread AFTER a successful call - so we know the
+    # network is up right now, and we avoid hammering a server that is not
+    # answering anyway.
+    def maybe_flush_spool
+      return unless @spool
+
+      now = monotonic
+      @spool_lock.synchronize do
+        return if now - @spool_last < SPOOL_INTERVAL_S
+
+        @spool_last = now
+      end
+      flush_spool
+    rescue StandardError => e
+      debug("batchwatch: spool drain failed: #{e}")
+    end
+
+    # Store a COMPLETED measurement that could not be delivered.
+    def spool_measurement(body)
+      if @spool
+        @spool.append(body)
+      else
+        debug("batchwatch: the measurement was lost (no spool)")
+      end
+    end
+
+    private
+
+    def monotonic
+      Process.clock_gettime(Process::CLOCK_MONOTONIC)
+    end
+
+    def debug(msg)
+      @logger&.debug(msg)
+    end
+  end
+
+  # Handle for one measurement in flight. Returned by track().
+  class Tracking
+    attr_reader :id
+
+    def initialize(bw, provider, model, mode, requests, input_tokens, endpoint)
+      @bw = bw
+      @body = { "provider" => provider, "model" => model, "mode" => mode,
+                "requests" => requests, "endpoint" => endpoint }
+      @input_tokens = input_tokens
+      @id = nil
+      @t0 = Time.now
+      @finished = false
+      @lock = Mutex.new
+      @id_ready = false
+      @id_cond = ConditionVariable.new
+    end
+
+    def finished?
+      @lock.synchronize { @finished }
+    end
+
+    def start
+      body = @body.dup
+      body["input_tokens"] = @input_tokens
+      body["started_at"] = Batchwatch.iso(@t0)
+
+      start_body = Batchwatch.sanitize(body)
+      @bw.in_background do
+        begin
+          r = @bw.call("/v1/calls", method: "POST", body: start_body,
+                                    idem: Batchwatch.idem_start(start_body))
+          @lock.synchronize do
+            @id = r && r["id"]
+            @id_ready = true
+            @id_cond.broadcast
+          end
+          @bw.maybe_flush_spool if r
+        rescue StandardError
+          @lock.synchronize do
+            @id_ready = true
+            @id_cond.broadcast
+          end
+          raise
+        end
+      end
+    end
+
+    # Update the token count when it is only known after submission.
+    def started(input_tokens: nil)
+      @input_tokens = input_tokens unless input_tokens.nil?
+    end
+
+    # Close the measurement.
+    #
+    # output_tokens stays nil when you do not know it. It is never defaulted to
+    # zero: zero is a measurement, absence is not, and the server prices them
+    # differently on purpose.
+    def done(output_tokens: nil, status: "completed", ttfb_ms: nil)
+      @lock.synchronize do
+        return if @finished
+
+        @finished = true
+      end
+      ended = Time.now
+
+      # The outcome measurement (#101): the latest advice for THIS model,
+      # attached to the completion. No advice -> empty hash -> the fields are
+      # omitted entirely.
+      advice = Batchwatch.outcome_fields(@bw.get_advice(@body["model"]))
+
+      @bw.in_background do
+        # Wait briefly for the id to land from the start call.
+        wait_for_id
+        id = @lock.synchronize { @id }
+
+        full = @body.dup
+        full.merge!("input_tokens" => @input_tokens,
+                    "output_tokens" => output_tokens, "status" => status,
+                    "started_at" => Batchwatch.iso(@t0),
+                    "ended_at" => Batchwatch.iso(ended))
+        full["ttfb_ms"] = ttfb_ms unless ttfb_ms.nil?
+        full.merge!(advice)
+
+        if id
+          begin
+            @bw.call("/v1/calls/#{id}", method: "PATCH", body: Batchwatch.sanitize(
+              { "status" => status, "output_tokens" => output_tokens,
+                "ttfb_ms" => ttfb_ms, "ended_at" => Batchwatch.iso(ended) }.merge(advice)
+            ))
+            next
+          rescue StandardError
+            # The server already has the start. A spooled resend can therefore
+            # produce a duplicate if the PATCH reached the server anyway -
+            # chosen on purpose: a duplicate can be seen in the dataset, a lost
+            # measurement cannot.
+            @bw.spool_measurement(Batchwatch.sanitize(full))
+            next
+          end
+        end
+
+        # The start call never reached the server. Send the whole measurement
+        # in one go instead of losing it. The key is derived from the
+        # measurement, so a later spool replay of exactly this record carries
+        # the SAME key and is deduped (#30).
+        cleaned = Batchwatch.sanitize(full)
+        begin
+          @bw.call("/v1/calls/complete", method: "POST", body: cleaned,
+                                         idem: Batchwatch.idem_complete([cleaned]))
+        rescue StandardError
+          @bw.spool_measurement(cleaned)
+        end
+      end
+    end
+
+    # Record the job as not-completed. An unfinished wait is not a wait.
+    def failed(status: "failed")
+      done(status: status)
+    end
+
+    private
+
+    def wait_for_id
+      deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + ID_WAIT_S
+      @lock.synchronize do
+        until @id_ready
+          rest = deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
+          break if rest <= 0
+
+          @id_cond.wait(@lock, rest)
+        end
+      end
+    end
+  end
+
+  @default = nil
+  @default_lock = Mutex.new
+
+  # Shortcut that uses a shared default client, to get started quickly.
+  def self.track(model, **kw, &blk)
+    @default_lock.synchronize { @default ||= Client.new }
+    @default.track(model, **kw, &blk)
+  end
+end

data/lib/batchwatch/spool.rb

@@ -0,0 +1,186 @@
+# frozen_string_literal: true
+
+# On-disk spool for measurements that could not be delivered.
+#
+# A measurement is worth the most exactly when the network is misbehaving, so
+# that is the worst possible moment to lose one. Undeliverable *completed*
+# measurements are therefore written to a JSONL file and replayed later through
+# POST /v1/calls/complete.
+#
+# The file format is one JSON object per line, in exactly the form
+# /v1/calls/complete accepts. It is the same format in the Python, Go,
+# TypeScript and .NET clients, so a spool file written by one can be drained
+# by another.
+#
+# Replay requires an API key: /v1/calls/complete takes the caller's own
+# timestamps and is closed to anonymous callers for that reason. A client
+# without a token therefore does not spool at all - a file no one can send is
+# just a disk leak.
+#
+# Threads are handled with a mutex; separate processes sharing one spool file
+# are not. Appending from multiple processes almost always goes fine, but two
+# processes draining the same file at the same time may send the same
+# measurement twice. Give each process its own BATCHWATCH_SPOOL if it matters.
+require "json"
+
+module Batchwatch
+  # The server's cap on one POST /v1/calls/complete.
+  MAX_BATCH = 500
+
+  # Cap on the spool file. If batchwatch is down for a week, a busy pipeline
+  # must not fill the user's disk. When the cap is reached, the measurement is
+  # dropped - and that is the right choice: his machine is not our storage.
+  MAX_BYTES = 5 * 1024 * 1024
+
+  # Append-only JSONL file of completed measurements waiting to be sent.
+  class Spool
+    attr_reader :path, :pending, :max_bytes
+
+    def initialize(path, max_bytes: MAX_BYTES, logger: nil)
+      @path = path
+      @pending = "#{path}.pending"
+      @max_bytes = max_bytes
+      @logger = logger
+      # The measurements come from background threads, one per completed call.
+      # Without the lock, two concurrent completions lose one of themselves:
+      # "find the end, write" is not atomic. Seen in a test with three at once:
+      # two lines made it.
+      @lock = Mutex.new
+    end
+
+    # ------------------------------------------------------------------ write
+
+    # Store one completed measurement. Returns true if it was stored.
+    #
+    # Never raises: failing a spool must not be worse than the network error
+    # that triggered the spool.
+    def append(record)
+      @lock.synchronize { write_one(record) }
+    rescue StandardError => e
+      debug("batchwatch: could not spool: #{e}")
+      false
+    end
+
+    # ------------------------------------------------------------------- read
+
+    # Move everything spooled into the pending file and return it.
+    #
+    # Returns a list of records. An empty list means there is nothing to send -
+    # also when the spool simply could not be read.
+    def take
+      @lock.synchronize { take_all }
+    rescue StandardError => e
+      debug("batchwatch: could not read spool: #{e}")
+      []
+    end
+
+    # Put records back after a partial or failed drain.
+    def keep(remaining)
+      @lock.synchronize do
+        if remaining && !remaining.empty?
+          write_all(@pending, remaining)
+        elsif File.exist?(@pending)
+          File.delete(@pending)
+        end
+      end
+    rescue StandardError => e
+      debug("batchwatch: could not write spool back: #{e}")
+    end
+
+    # Number of records waiting on disk. Best effort, never raises.
+    def size
+      @lock.synchronize { read_all(@pending).length + read_all(@path).length }
+    rescue StandardError => e
+      debug("batchwatch: could not count spool: #{e}")
+      0
+    end
+
+    private
+
+    def write_one(record)
+      if File.exist?(@path) && File.size(@path) >= @max_bytes
+        debug("batchwatch: spool is full (#{@path}) - the measurement is dropped")
+        return false
+      end
+      dir = File.dirname(@path)
+      mkdir_p(dir) if dir && !dir.empty? && dir != "."
+
+      line = JSON.generate(record)
+      # "r+b"/"a+b"-like behaviour: we need to be able to READ the last byte,
+      # and a pure append mode is write-only. If the file ends mid-line after a
+      # crash, the next measurement must not be glued onto it - otherwise we
+      # lose not just the half line, but the whole one too.
+      File.open(@path, File::WRONLY | File::CREAT, 0o644) do |f|
+        f.seek(0, IO::SEEK_END)
+        size = f.pos
+        prefix = ""
+        if size > 0
+          # Read the last byte through a separate handle - WRONLY cannot read.
+          # If the trailing newline is missing, we push one in front.
+          last = File.open(@path, "rb") { |r| r.seek(-1, IO::SEEK_END); r.read(1) }
+          prefix = "\n" if last != "\n"
+        end
+        # ONE write. If it is split, another writer can slip its line in
+        # between the two halves.
+        f.write("#{prefix}#{line}\n")
+      end
+      debug("batchwatch: measurement spooled to #{@path}")
+      true
+    end
+
+    def take_all
+      records = read_all(@pending) + read_all(@path)
+      return [] if records.empty?
+
+      # The order is deliberate: write pending BEFORE the source is removed. A
+      # crash in between must produce duplicates, not loss - a duplicate can be
+      # seen and filtered out, a lost measurement does not exist.
+      write_all(@pending, records)
+      File.delete(@path) if File.exist?(@path)
+      records
+    end
+
+    def read_all(path)
+      return [] unless File.exist?(path)
+
+      out = []
+      File.foreach(path) do |line|
+        line = line.strip
+        next if line.empty?
+
+        begin
+          out << JSON.parse(line)
+        rescue JSON::ParserError
+          # A half-written line after a crash. Skip it instead of losing the
+          # rest of the file.
+          debug("batchwatch: unusable line in spool skipped")
+        end
+      end
+      out
+    end
+
+    def write_all(path, records)
+      File.open(path, "w") do |f|
+        records.each { |p| f.write("#{JSON.generate(p)}\n") }
+      end
+    end
+
+    # We avoid a dependency on 'fileutils' in the library by rolling a small
+    # recursive mkdir ourselves - the standard library can do all of it.
+    def mkdir_p(dir)
+      return if File.directory?(dir)
+
+      parent = File.dirname(dir)
+      mkdir_p(parent) if parent != dir && !File.directory?(parent)
+      begin
+        Dir.mkdir(dir)
+      rescue Errno::EEXIST
+        # Another thread got to make it. Fine.
+      end
+    end
+
+    def debug(msg)
+      @logger&.debug(msg)
+    end
+  end
+end

data/lib/batchwatch.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+# batchwatch - Ruby client for batchwatch.dev.
+#
+# Crowdsourced measurement of queue time on LLM batch APIs. Batch endpoints
+# cost 50% of the synchronous ones, but "completes within 24 hours" is
+# impossible to plan around. batchwatch measures what the queue actually does
+# and answers one question: *should I use batch for this job?*
+#
+# Standard library only. No dependencies, and none planned.
+#
+#     require "batchwatch"
+#
+#     bw = Batchwatch::Client.new(token: "bw_...")
+#     if bw.should_batch("gpt-5.6-sol", max_wait: "15m")
+#       # ... send batch ...
+#     end
+#     bw.track("gpt-5.6-sol", input_tokens: 9720) do |t|
+#       t.done(output_tokens: 4519)
+#     end
+require_relative "batchwatch/spool"
+require_relative "batchwatch/client"

metadata

@@ -0,0 +1,82 @@
+--- !ruby/object:Gem::Specification
+name: batchwatch
+version: !ruby/object:Gem::Version
+  version: 0.2.1
+platform: ruby
+authors:
+- Andreas Graae
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2026-08-29 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.15'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.15'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+description: Crowdsourced measurement of queue time on LLM batch APIs. Fails open
+  (a batchwatch outage never touches your job) and never sends content - only a fixed
+  allowlist of provider, model, mode, endpoint, request count, token counts and timestamps.
+  Standard library only, no dependencies.
+email:
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- LICENSE
+- README.md
+- lib/batchwatch.rb
+- lib/batchwatch/client.rb
+- lib/batchwatch/spool.rb
+homepage: https://batchwatch.dev
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://batchwatch.dev
+  source_code_uri: https://github.com/batchwatch/client
+  rubygems_mfa_required: 'true'
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.3.27
+signing_key:
+specification_version: 4
+summary: Client for batchwatch.dev - measure queue time on LLM batch APIs without
+  ever blocking your job
+test_files: []