batchwatch 0.2.1 → 0.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: bfefddebcc183a5e63b03b570caa3d183103d4b661f90a617779241306201d6b
4
- data.tar.gz: 02de063b1c8f22d8d2a5f93f2c3dda990b482af905ab811c5a630eb029289279
3
+ metadata.gz: d76f91aceeae04301676b3b43f48d94ee8cac41b83dede45b38a7d770c8aab1b
4
+ data.tar.gz: '0493dd22486b9768eb7910e8ed595fecf51efc6419111acd1f0ca8e4d3f8838e'
5
5
  SHA512:
6
- metadata.gz: e06032ef12b72d4d8213ae27271723afcbbb87a17087304ecda1ea999c8fe65777fc87a39b773cfc58faed151e4d685e94127e4186c4792d59bfd08af553e6fc
7
- data.tar.gz: 8134786ba93b55272cbdf87db36e02581b600d229aa5ef08d8275810c93c29344ae583193b6488d8036f14787dac812a39eb3b356bc61132a7c7fc700c728908
6
+ metadata.gz: 64057bdf78674a4152d026ad69fecec08c549fb0dbd84cd8d04fd48dc858f2700a4fe76b9a2f9bbcf9eaede4ad6b963a8876931405dab47f5153e766d8ccead4
7
+ data.tar.gz: 2223af0f8679cb1219994f2f246af43616e4fc64f175cbed5e8f590e8739c18f585f8e5a7ad5c04cfccf98360536c4da67ca54065fbc4e8f22a79b37414983be
data/README.md CHANGED
@@ -12,15 +12,11 @@ No dependencies. The standard library — `net/http`, `json`, `uri`, `socket`,
12
12
 
13
13
  ## Install
14
14
 
15
- Point your `Gemfile` at the subdirectory:
15
+ gem install batchwatch
16
16
 
17
- ```ruby
18
- gem "batchwatch", git: "https://github.com/batchwatch/client",
19
- glob: "clients/ruby/*.gemspec"
20
- ```
21
-
22
- or vendor the three files under `lib/` — they have no third-party imports. A
23
- RubyGems release is on the way.
17
+ On [RubyGems](https://rubygems.org/gems/batchwatch), or add `gem "batchwatch"`
18
+ to your `Gemfile`. Or vendor the three files under `lib/` — they have no
19
+ third-party imports.
24
20
 
25
21
  ## Two lines
26
22
 
@@ -52,6 +48,118 @@ Get a key with no email and no card:
52
48
 
53
49
  curl -X POST https://batchwatch.dev/v1/keys -d '{"label":"my pipeline"}'
54
50
 
51
+ ## The deadline guard — batch when you can, sync when you must
52
+
53
+ Batch is half the price, but a queue that misses your deadline can take down a
54
+ product. `bw.batch(...)` gets you both: it runs your batch, watches the clock,
55
+ and if the batch has not finished by your deadline it cancels it and runs your
56
+ synchronous fallback instead — so your job always gets an answer, on time.
57
+
58
+ ```ruby
59
+ job = bw.batch("gpt-5.6-sol", deadline: "15m",
60
+ on_deadline: -> { client.chat.completions.create(...) })
61
+ job.submit(-> { client.batches.create(...) })
62
+ result = job.result
63
+ ```
64
+
65
+ You hand over two callables — the batch-create and the sync fallback — and the
66
+ client runs them. It never sees or builds your provider payload; there is no
67
+ field for it, exactly as with `track`. If the batch finishes in time you get its
68
+ result; if the deadline fires you get the fallback's result.
69
+
70
+ Every fallback is a **measured prediction outcome**: "batching would have missed
71
+ — running sync was right." It goes down the same accuracy path `should_batch`
72
+ already feeds, so the server can score how often the guard was needed. Nothing
73
+ new leaves the machine.
74
+
75
+ `deadline` speaks the same duration language as `should_batch`'s `max_wait` —
76
+ `"15m"`, `"6h"`, `"30s"`, a bare number of seconds, or `nil` for no guard. The
77
+ defaults duck-type the OpenAI / Anthropic batch shape; a different provider
78
+ passes `poll:` / `cancel:` / `result_of:` callables.
79
+
80
+ ### The poll loop is ours, not yours
81
+
82
+ `result` owns the wait so you do not write the same sleep/backoff loop everyone
83
+ else does. It polls with **exponential backoff and jitter**, never faster than a
84
+ **rate-limit floor** (a naive one-second loop against a 24-hour job is 86,400
85
+ requests and an angry provider) and never slower than a ceiling. The **first**
86
+ interval is informed by the model's own **measured p50** — no reason to poll
87
+ every five seconds against a model whose median queue time is forty minutes —
88
+ and reading that p50 **fails open**: if batchwatch is unreachable, polling simply
89
+ continues on the fixed fallback schedule.
90
+
91
+ The 24-hour **expiry is a distinct terminal state**, never silently a timeout:
92
+ `job.expired` is `true` when the batch hit the cutoff, so you can tell "the queue
93
+ was slow" from "the batch failed".
94
+
95
+ Tune the cadence if you need to; the defaults are sane:
96
+
97
+ ```ruby
98
+ job = bw.batch("gpt-5.6-sol", deadline: "6h",
99
+ on_deadline: -> { client.chat.completions.create(...) },
100
+ poll_base: 30, poll_floor: 5, poll_ceiling: 900)
101
+ job.submit(-> { client.batches.create(...) })
102
+ result = job.result
103
+ ```
104
+
105
+ Driving an async or worker loop yourself instead of blocking a thread? Step the
106
+ same state machine one poll at a time:
107
+
108
+ ```ruby
109
+ state = job.poll_once # PollState::RUNNING / DONE / EXPIRED / FAILED
110
+ if state == Batchwatch::PollState::RUNNING
111
+ sleep(job.next_interval) # backoff + jitter, already applied
112
+ end
113
+ ```
114
+
115
+ ### Partial completion — landed, failed, expired
116
+
117
+ A batch of 20,000 requests is not binary: some land, some fail per-request, and
118
+ some never return before the 24-hour expiry. `job.split` separates the three,
119
+ mapped back to your own objects **by `custom_id`** (never by index — provider
120
+ ordering is not guaranteed):
121
+
122
+ ```ruby
123
+ result = job.split(downloaded_lines, every_submitted_id)
124
+ result.landed # count that came back clean
125
+ result.failed # count that failed per-request
126
+ result.expired # count still outstanding at the 24h cutoff
127
+ result.complete? # true only if EVERYTHING landed — never a silent success
128
+ result.result_for("my-request-42") # your object for one id, mapped correctly
129
+
130
+ # retry only what failed — idempotent, so a second call submits nothing new
131
+ child = job.retry_failed(->(failed_ids) { client.batches.create(...) })
132
+ ```
133
+
134
+ An expired job is reported to batchwatch as `status="expired"`, which is recorded
135
+ but kept out of the percentiles (only `completed` rows count) — so a slow queue
136
+ neither pollutes p90 nor loses the "the queue was slow" signal. That measurement
137
+ rule lives with the ingest contract on the server, under `/v1/calls/complete` →
138
+ "Partial completion", not only here in the client.
139
+
140
+ ## Subscribe to outcome alerts
141
+
142
+ Get told when a model's queue degrades, on your own webhook or Slack. All three
143
+ calls are keyed to your own token, so — like `my_calls` and unlike the
144
+ measurement path — they do **not** fail open: without a token they raise
145
+ `Batchwatch::AuthError` rather than silently pretend you subscribed.
146
+
147
+ ```ruby
148
+ bw = Batchwatch::Client.new(token: "bw_...")
149
+
150
+ # webhook: omit the secret and the server mints one, returned ONCE — read it here
151
+ sub = bw.subscribe("webhook", "https://example.com/hook",
152
+ providers: "openai", min_severity: "severe")
153
+ puts sub["secret"] # shown only on creation, never again
154
+
155
+ # slack needs no secret
156
+ bw.subscribe("slack", "https://hooks.slack.com/services/...")
157
+
158
+ # list your active subscriptions (never the secret), then revoke one by id
159
+ bw.subscriptions.each { |s| puts [s["id"], s["channel"], s["target"]].join(" ") }
160
+ bw.unsubscribe(sub["id"])
161
+ ```
162
+
55
163
  ## It fails open, always
56
164
 
57
165
  If batchwatch is down, slow, or broken, your job must not notice. That is the
@@ -100,6 +208,35 @@ back labelled as a ceiling.
100
208
  Passing `output_tokens: 0` really does send `0`: zero is a measurement, absence
101
209
  is not.
102
210
 
211
+ ## Read your own contributions and key status
212
+
213
+ Two read routes, both keyed to your own token. They are the readback for
214
+ `track()`: there is no route to a single call by id, so `my_calls` is how you
215
+ confirm a measurement landed.
216
+
217
+ ```ruby
218
+ bw = Batchwatch::Client.new(token: "bw_...")
219
+
220
+ # everything this key has contributed
221
+ mine = bw.my_calls
222
+ puts "#{mine['count']} calls"
223
+ mine["calls"].each { |c| puts [c["model"], c["status"], c["duration_s"]].join(" ") }
224
+
225
+ # page through — pagination is keyed on started_server, never an offset, so a
226
+ # row arriving mid-walk cannot make you skip anything. Follow "next" (or pass
227
+ # after: <unix seconds>) until it is null.
228
+ mine = bw.my_calls(after: 1787666964, limit: 100)
229
+
230
+ # your tier, whether you are contributing, and your quota
231
+ status = bw.key_status
232
+ puts [status["tier"], status["contributing"], status["quota"]["calls_left"]].join(" ")
233
+ ```
234
+
235
+ Both require a key and, like `subscribe`, do **not** fail open — without a token
236
+ there is nothing to read, so they raise `Batchwatch::AuthError` rather than
237
+ return an empty answer that reads like "no contributions". The server's row comes
238
+ back verbatim; no key is renamed.
239
+
103
240
  ## Spooling
104
241
 
105
242
  When a measurement cannot be delivered, the completed record is appended to a
@@ -152,6 +289,20 @@ in CI.
152
289
  - `t.done(output_tokens: nil, status: "completed", ttfb_ms: nil)`
153
290
  - `t.failed(status: "failed")`
154
291
  - `t.started(input_tokens: ...)` when the count is only known after submission
292
+ - `batch(model, deadline: nil, on_deadline: nil, provider: "openai", **kw) -> BatchJob` — the deadline-guarded, self-polling job
293
+ - `job.submit(create)` — run the caller's batch-create callable (a block/proc/lambda), remember the handle
294
+ - `job.result` — block, polling with backoff + jitter, or fall back to `on_deadline` at the deadline
295
+ - `job.poll_once -> PollState` — one non-blocking step for an async/worker loop; `job.next_interval` is the seconds to sleep before the next
296
+ - `job.fell_back` — `true` once the guard has fired; `job.expired` — `true` if the batch hit the 24h expiry; `job.poll_count` — polls made
297
+ - `job.split(results = nil, custom_ids = nil) -> BatchResult` — split landed / failed / expired, mapped by `custom_id` (`result.complete?` / `result.result_for(id)`)
298
+ - `job.retry_failed(resubmit, result: nil) -> BatchJob | nil` — resubmit only the failed subset, idempotently
299
+ - poll cadence: `poll_base:` / `poll_floor:` / `poll_ceiling:` / `poll_backoff:` / `poll_jitter:` / `use_p50_cadence:`
300
+ - override the provider shape with `poll:` / `cancel:` / `result_of:` / `succeeded:` / `expired:`
301
+ - `subscribe(channel, target, secret: nil, providers: nil, models: nil, min_severity: nil) -> Hash` — outcome alerts for this key (`POST /v1/subscriptions`); requires a key
302
+ - `subscriptions -> Array` — this key's active subscriptions (`GET /v1/subscriptions`); requires a key
303
+ - `unsubscribe(sub_id) -> Hash` — revoke one of your own subscriptions (`DELETE /v1/subscriptions/{id}`); requires a key
304
+ - `my_calls(after: nil, limit: nil) -> Hash` — this key's own contributions (`GET /v1/calls/mine`); requires a key, follows `next`/`after` for pagination
305
+ - `key_status -> Hash` — this key's tier / contribution status / quota (`GET /v1/keys/current`); requires a key
155
306
  - `flush(timeout: 5.0) -> true/false` — wait for outstanding submissions before exit
156
307
  - `flush_spool(timeout: nil) -> Integer` — send what is on disk, returns accepted
157
308
 
@@ -161,11 +312,12 @@ in CI.
161
312
  # or a single suite:
162
313
  ruby -Ilib -Itest test/test_fail_open.rb
163
314
 
164
- 26 tests, no network beyond loopback. They start real HTTP servers on
165
- ephemeral ports (raw `TCPServer`, port 0) rather than stubbing `Net::HTTP`:
166
- the thing under test is network behaviour, so the network should be in the
167
- test. The allowlist and fail-open tests carry positive controls, so a client
168
- that sent nothing at all would fail them rather than pass.
315
+ No network beyond loopback. They start real HTTP servers on ephemeral ports (raw
316
+ `TCPServer`, port 0) rather than stubbing `Net::HTTP`: the thing under test is
317
+ network behaviour, so the network should be in the test. The allowlist and
318
+ fail-open tests carry positive controls, so a client that sent nothing at all
319
+ would fail them rather than pass. The poll-loop tests inject a clock, sleep and
320
+ rng, so the backoff schedule is pinned exactly with no real sleeps.
169
321
 
170
322
  ## Requirements
171
323
 
@@ -40,7 +40,7 @@ require "time"
40
40
  require_relative "spool"
41
41
 
42
42
  module Batchwatch
43
- VERSION = "0.2.1"
43
+ VERSION = "0.2.2"
44
44
 
45
45
  # How often, at most, we try to drain the spool on our own.
46
46
  SPOOL_INTERVAL_S = 60.0
@@ -215,6 +215,16 @@ module Batchwatch
215
215
  end
216
216
  end
217
217
 
218
+ # Raised when an action REQUIRES a key and none is set.
219
+ #
220
+ # The measurement path fails open - with no key it degrades silently, because
221
+ # a missing telemetry call must never stand in the way of the caller's job.
222
+ # But reading your own contributions is an EXPLICIT action against a per-key
223
+ # route: here a silent empty answer would be a lie - the caller reads "no
224
+ # contributions" when in truth there was nothing to read. So we say it out
225
+ # loud instead.
226
+ class AuthError < StandardError; end
227
+
218
228
  # The client. Every submission is non-blocking and fails open.
219
229
  #
220
230
  # Args:
@@ -278,6 +288,7 @@ module Batchwatch
278
288
  when "GET" then Net::HTTP::Get
279
289
  when "POST" then Net::HTTP::Post
280
290
  when "PATCH" then Net::HTTP::Patch
291
+ when "DELETE" then Net::HTTP::Delete
281
292
  else raise ArgumentError, "unknown method #{method}"
282
293
  end
283
294
  req = req_class.new(uri)
@@ -417,6 +428,98 @@ module Batchwatch
417
428
  @advice_lock.synchronize { @advice[model] }
418
429
  end
419
430
 
431
+ # ---------------------------------------------------------- subscription
432
+
433
+ # Subscribe to outcome alerts for THIS key. Returns the row.
434
+ #
435
+ # +channel+ is "webhook" or "slack", +target+ an https URL. +secret+ applies
436
+ # only to webhook: if you omit it, the server makes one itself and returns
437
+ # the plaintext ONCE in +secret+ - read it here and configure your receiver,
438
+ # it never comes again. +providers+ and +models+ are comma-strings (or omit
439
+ # = all), +min_severity+ is "degraded" or "severe".
440
+ #
441
+ # Unlike the measurement path, this one does NOT fail open: subscribing is an
442
+ # explicit action against an authorized route, so without a key we raise
443
+ # AuthError rather than silently pretend.
444
+ def subscribe(channel, target, secret: nil, providers: nil, models: nil,
445
+ min_severity: nil)
446
+ require_key("subscribe")
447
+ body = { "channel" => channel, "target" => target }
448
+ { "secret" => secret, "providers" => providers, "models" => models,
449
+ "min_severity" => min_severity }.each do |name, value|
450
+ body[name] = value unless value.nil?
451
+ end
452
+ call("/v1/subscriptions", method: "POST", body: body)
453
+ end
454
+
455
+ # THIS key's active subscriptions, as an array. Never the secret.
456
+ def subscriptions
457
+ require_key("subscriptions")
458
+ r = call("/v1/subscriptions")
459
+ (r || {}).fetch("subscriptions", [])
460
+ end
461
+
462
+ # Revoke one of your OWN subscriptions by numeric id. Returns the response.
463
+ def unsubscribe(sub_id)
464
+ require_key("unsubscribe")
465
+ call("/v1/subscriptions/#{sub_id}", method: "DELETE")
466
+ end
467
+
468
+ # --------------------------------------------------------------- job (#178)
469
+
470
+ # A high-level batch job: deadline guard (#178) + poll loop (#179) + partial
471
+ # completion (#180).
472
+ #
473
+ # The epic's promise: *we take the callable, never the payload*. You give us
474
+ # two callables - the batch-create and the synchronous fallback - and we run
475
+ # them, watch the deadline, poll the batch, and hand back a result. We never
476
+ # see or construct the provider payload, exactly as with track().
477
+ #
478
+ # job = bw.batch("gpt-5.6-sol", deadline: "15m",
479
+ # on_deadline: -> { client.chat.completions.create(...) })
480
+ # job.submit(-> { client.batches.create(...) })
481
+ # result = job.result
482
+ #
483
+ # If the batch finishes before +deadline+ you get its result. If the deadline
484
+ # arrives first we cancel the batch (best-effort), run +on_deadline+ and
485
+ # return that instead - so the caller's job always gets an answer, on time.
486
+ # Every fallback is reported down the existing #101 accuracy path
487
+ # (acted_verdict=false on a completion) so the server can score the
488
+ # prediction; nothing new is sent.
489
+ #
490
+ # See Batchwatch::BatchJob for the full contract (poll cadence, poll_once,
491
+ # split, retry_failed, and the injectable clock/sleep/rng seams).
492
+ def batch(model, deadline: nil, on_deadline: nil, provider: "openai", **kw)
493
+ BatchJob.new(self, model, deadline: deadline, on_deadline: on_deadline,
494
+ provider: provider, **kw)
495
+ end
496
+
497
+ # Report a deadline-guard outcome down the existing #101 path.
498
+ #
499
+ # The deadline guard is a should_batch-style prediction carried out: when it
500
+ # falls back, the outcome is the server's "run_sync" case ("batching would
501
+ # have missed"). #101 already carries exactly that - a completion measurement
502
+ # bearing acted_verdict and deadline_s, which the server judges against its
503
+ # own measured duration and rolls into /v1/accuracy. So we seed the same
504
+ # per-model advice slot should_batch writes and emit a completion through the
505
+ # SAME track() path - no second reporter, no new route.
506
+ #
507
+ # A completion in time carries acted_verdict=true (the batch held the
508
+ # deadline); a fallback carries false. Both sides are reported, which is what
509
+ # keeps the accuracy number honest. Percentiles are omitted, not invented as
510
+ # zero (rule #30): the guard did not quote a p90.
511
+ def record_job_outcome(model, provider, acted_verdict, deadline_s)
512
+ @advice_lock.synchronize do
513
+ @advice[model] = {
514
+ "acted_verdict" => acted_verdict,
515
+ "deadline_s" => deadline_s,
516
+ "quoted_p50_s" => nil,
517
+ "quoted_p90_s" => nil
518
+ }
519
+ end
520
+ track(model, provider: provider, mode: "batch") { |t| t.done }
521
+ end
522
+
420
523
  # ------------------------------------------------------------- measurement
421
524
 
422
525
  # Measure one call. The submission happens in the background.
@@ -447,6 +550,52 @@ module Batchwatch
447
550
  end
448
551
  end
449
552
 
553
+ # ------------------------------------------------------- read your own key
554
+
555
+ # The measurements THIS key has contributed. Requires a key.
556
+ #
557
+ # GET /v1/calls/mine returns everything the service holds that came from
558
+ # your key - the only readback there is: there is no route to a single call
559
+ # by id, and none to anyone else's rows. Use it to confirm a measurement
560
+ # landed after track() / flush_spool().
561
+ #
562
+ # Returns the server's row verbatim: {"label", "count", "next", "calls":
563
+ # [...], "note"}. +next+ is a ready-made URL for the following page (null on
564
+ # the last), keyed on started_server so a walk cannot skip a row that
565
+ # arrives mid-walk.
566
+ #
567
+ # after: unix seconds; only rows with a later started_server come back.
568
+ # Omit for the first page.
569
+ # limit: rows per page, clamped server-side to 1-1000 (default 500).
570
+ #
571
+ # Unlike the measurement path this does NOT fail open: the route is per-key,
572
+ # so without one we raise AuthError rather than pretend.
573
+ def my_calls(after: nil, limit: nil)
574
+ require_key("my_calls")
575
+ q = {}
576
+ q["after"] = after.to_i unless after.nil?
577
+ q["limit"] = limit.to_i unless limit.nil?
578
+ path = "/v1/calls/mine"
579
+ path += "?#{URI.encode_www_form(q)}" unless q.empty?
580
+ call(path)
581
+ end
582
+
583
+ # THIS key's tier, contribution status and quota. Requires a key.
584
+ #
585
+ # GET /v1/keys/current returns the row verbatim: {"label", "tier",
586
+ # "contributing", "recent_measurements", "required", "window_days",
587
+ # "delayed_by_s", "live", "quota": {...}}. +tier+ is the effective tier
588
+ # derived from your measurements (free / contributor; only paid is
589
+ # operator-assigned), and +quota+ reports the calls window for the gated
590
+ # routes.
591
+ #
592
+ # Unlike the measurement path this does NOT fail open: without a key there
593
+ # is no key to describe, so we raise AuthError.
594
+ def key_status
595
+ require_key("key_status")
596
+ call("/v1/keys/current")
597
+ end
598
+
450
599
  # --------------------------------------------------------------- spool
451
600
 
452
601
  # Send everything waiting on disk. Returns the number accepted.
@@ -516,6 +665,17 @@ module Batchwatch
516
665
 
517
666
  private
518
667
 
668
+ # Fail loudly if there is no key. The read routes are per-key - without one
669
+ # there is nothing to read, and a silent empty answer would be misread as
670
+ # "no contributions".
671
+ def require_key(action)
672
+ return if @token
673
+
674
+ raise AuthError,
675
+ "batchwatch: #{action} requires a key - set token: ... or " \
676
+ "$BATCHWATCH_TOKEN"
677
+ end
678
+
519
679
  def monotonic
520
680
  Process.clock_gettime(Process::CLOCK_MONOTONIC)
521
681
  end
@@ -0,0 +1,741 @@
1
+ # frozen_string_literal: true
2
+
3
+ # The high-level batch job - the client does the annoying parts.
4
+ #
5
+ # The epic's core promise is *"we take the callable, never the payload"*. Every
6
+ # other batchwatch surface answers a question and hands the decision back to
7
+ # you; this one carries it out. You hand us two callables - the batch-create and
8
+ # the synchronous fallback - and we run them, watch the deadline, poll the
9
+ # batch, and give you a result. We never see or construct the provider payload:
10
+ # there is no field for it, by construction, exactly as with track().
11
+ #
12
+ # job = bw.batch("gpt-5.6-sol", deadline: "15m",
13
+ # on_deadline: -> { client.chat.completions.create(...) })
14
+ # job.submit(-> { client.batches.create(...) })
15
+ # result = job.result
16
+ #
17
+ # What happens:
18
+ #
19
+ # * job.submit(fn) runs your batch-create callable and remembers the handle it
20
+ # returned and the time it started. We never look inside the handle beyond the
21
+ # questions we have to ask it - is it done, did it succeed, what did it return
22
+ # - and those go through callables you can override.
23
+ # * job.result waits for the batch. If it finishes before the deadline you get
24
+ # its result. If the deadline arrives first, we cancel the batch (best-effort),
25
+ # run on_deadline and give you *that* result instead - your job still gets an
26
+ # answer, on time.
27
+ # * **Every fallback is a measured prediction outcome.** A deadline miss is the
28
+ # server's #101 "run_sync" case, so we report it down the SAME accuracy path
29
+ # track() already uses: a completion carrying acted_verdict=false and the
30
+ # deadline_s we were given. No second reporter, no new route, no payload.
31
+ #
32
+ # The wait itself (#179) owns the poll loop so the caller does not: exponential
33
+ # backoff with jitter, a rate-limit floor so a naive one-second loop against a
34
+ # 24-hour job cannot fire 86,400 requests, a ceiling so the interval does not
35
+ # run away, and a first cadence informed by our own measured p50 for the model.
36
+ # All of it fails open: if batchwatch cannot tell us the p50, we poll on the
37
+ # fixed fallback schedule rather than stopping.
38
+ #
39
+ # Partial completion (#180) is the third promise: a real batch of 20,000
40
+ # requests comes back with some landed, some failed per-request, and some never
41
+ # returned before the 24-hour expiry. BatchJob#split hands you a BatchResult
42
+ # that separates those three, mapped back to your own objects by custom_id
43
+ # (never by index - provider ordering is not guaranteed), and
44
+ # BatchJob#retry_failed resubmits only the failed subset with a stable
45
+ # idempotency key so a retry cannot duplicate the job.
46
+
47
+ module Batchwatch
48
+ # Raised for a misuse of the job API that we must not paper over.
49
+ #
50
+ # The measurement path fails open, because a lost telemetry call must never
51
+ # stand in the way of the user's job. The job path is different: it IS the
52
+ # user's job. If we cannot run it as asked - no submit before a result, no
53
+ # fallback to fall back to - a silent no-op would strand the caller waiting on
54
+ # a result that never comes. So we say it out loud instead.
55
+ class BatchJobError < StandardError; end
56
+
57
+ # ------------------------------------------------------------- poll cadence
58
+ #
59
+ # The floor is the rate-limit guard: never poll faster than this, no matter
60
+ # how eager the schedule, because a one-second loop against a 24-hour job is
61
+ # 86,400 requests and an angry provider. The ceiling stops the backoff running
62
+ # away - once the interval reaches it we keep polling at that cadence rather
63
+ # than doubling forever. The base is where the schedule starts when we have no
64
+ # p50 to inform it. All three are seconds; every one is overridable on
65
+ # bw.batch(...).
66
+ POLL_FLOOR_S = 5.0 # rate-limit floor: never poll faster than this
67
+ POLL_BASE_S = 5.0 # first interval when we have no measured p50
68
+ POLL_CEILING_S = 300.0 # backoff ceiling: never poll slower than this
69
+ POLL_BACKOFF = 2.0 # multiply the interval by this each miss
70
+ POLL_JITTER = 0.5 # full-jitter fraction: sleep in [(1-j)*d, d]
71
+
72
+ # How much of a model's measured p50 we spend before the first poll. A median
73
+ # queue time of forty minutes means the batch is very unlikely to be done in
74
+ # the first few minutes, so polling then only burns rate limit. We wait a
75
+ # fraction of the p50, clamped to the ceiling, so a slow model gets a patient
76
+ # first poll and a fast one is not starved. This is the use of our own dataset
77
+ # that the card calls a genuine selling point - nobody without the
78
+ # measurements can do it.
79
+ POLL_P50_FRACTION = 0.5
80
+
81
+ # The outcome of a single non-blocking poll - the async/worker driver's view.
82
+ #
83
+ # BatchJob#poll_once returns one of these constants so a caller driving the
84
+ # loop from their own event loop (rather than letting #result block a thread)
85
+ # can decide what to do next without reading provider internals:
86
+ #
87
+ # * RUNNING - not terminal yet; sleep next_interval and poll again.
88
+ # * DONE - terminal and successful; call #result for the value.
89
+ # * EXPIRED - terminal via the 24h expiry; a DISTINCT state, never a silent
90
+ # timeout, so the caller can tell "the queue was slow" from "the batch
91
+ # failed".
92
+ # * FAILED - terminal but not successful (failed / cancelled).
93
+ #
94
+ # EXPIRED and FAILED both fall back the same way inside #result; they are kept
95
+ # separate here because the caller - and #180's measurement - must be able to
96
+ # tell them apart.
97
+ module PollState
98
+ RUNNING = "running"
99
+ DONE = "done"
100
+ EXPIRED = "expired"
101
+ FAILED = "failed"
102
+ end
103
+
104
+ # Terminal batch statuses across the providers we have seen. A job in one of
105
+ # these is not going to change again, so waiting longer is pointless. Mirrors
106
+ # the status maps in the backfill tool (completed / failed / expired /
107
+ # cancelled), plus OpenAI's "finalizing"-then-"completed" and Anthropic's
108
+ # "ended".
109
+ TERMINAL_STATUSES = %w[
110
+ completed complete succeeded success
111
+ failed error errored
112
+ expired cancelled canceled ended
113
+ ].freeze
114
+
115
+ # The 24h expiry is a TERMINAL state, not "keep waiting" - and it is a
116
+ # distinct one. Counting an expired job as completed pollutes the accuracy
117
+ # number; counting it as a plain failure loses the signal that the queue was
118
+ # slow. So we name it separately (both here for the poll loop and in #180's
119
+ # split).
120
+ EXPIRED_STATUSES = %w[expired].freeze
121
+
122
+ SUCCESS_STATUSES = %w[completed complete succeeded success ended].freeze
123
+
124
+ # Markers for the coarse whole-batch split and an unmappable result line. Kept
125
+ # as frozen sentinels rather than magic strings so they cannot collide with a
126
+ # real caller custom_id.
127
+ WHOLE_MARKER = "__batchwatch_whole_batch__"
128
+ UNMAPPED_MARKER = "__batchwatch_unmapped__"
129
+
130
+ # The lower-cased status string of a handle, or "". Duck-typed.
131
+ #
132
+ # Reads handle.status (a method, or a [:status]/["status"] hash key) and
133
+ # nothing else. Never touches the payload.
134
+ def self.handle_status(handle)
135
+ status =
136
+ if handle.respond_to?(:status)
137
+ handle.status
138
+ elsif handle.respond_to?(:[])
139
+ handle[:status] || handle["status"]
140
+ end
141
+ status.nil? ? "" : status.to_s.downcase
142
+ end
143
+
144
+ # Is this batch finished? Duck-typed against the common provider shape. The
145
+ # OpenAI / Anthropic batch objects both expose a status string; we read it and
146
+ # nothing else. "Finished" means a terminal status; anything else means keep
147
+ # waiting.
148
+ def self.default_poll(handle)
149
+ TERMINAL_STATUSES.include?(handle_status(handle))
150
+ end
151
+
152
+ # Did the batch finish *successfully* (as opposed to failed/expired)?
153
+ def self.default_succeeded(handle)
154
+ SUCCESS_STATUSES.include?(handle_status(handle))
155
+ end
156
+
157
+ # Did the batch reach the 24h expiry? Kept separate from default_succeeded so
158
+ # an expiry is a distinct terminal state the caller and the measurement can
159
+ # see - never silently a timeout.
160
+ def self.default_expired(handle)
161
+ EXPIRED_STATUSES.include?(handle_status(handle))
162
+ end
163
+
164
+ # What the caller gets back for a completed batch: the handle itself. We do
165
+ # not download or parse the batch output - that is the caller's data and their
166
+ # provider SDK's job. Override with result_of: if you want something else.
167
+ def self.default_result(handle)
168
+ handle
169
+ end
170
+
171
+ # Best-effort cancel of a running batch. Never raises upward. Duck-typed
172
+ # against a handle.cancel method if the provider object carries one; otherwise
173
+ # a no-op. The deadline guard has already decided to fall back, so a cancel
174
+ # that fails to land only wastes provider spend on a batch nobody will read.
175
+ def self.default_cancel(handle)
176
+ handle.cancel if handle.respond_to?(:cancel)
177
+ end
178
+
179
+ # The custom_id of a provider result line, or nil. Duck-typed. Reads
180
+ # item["custom_id"] (or [:custom_id], or the attribute) - the field OpenAI and
181
+ # Anthropic both round-trip on a batch line. Nothing else is read.
182
+ def self.custom_id_of(item)
183
+ if item.respond_to?(:[])
184
+ return item["custom_id"] || item[:custom_id]
185
+ end
186
+ item.respond_to?(:custom_id) ? item.custom_id : nil
187
+ end
188
+
189
+ # Did this per-request line fail? Duck-typed against the provider shape. A
190
+ # batch output line carries an error (non-null) on failure, or a response with
191
+ # a non-2xx status_code. We read only those envelope fields - never the body.
192
+ def self.result_error?(item)
193
+ error =
194
+ if item.respond_to?(:[])
195
+ item["error"] || item[:error]
196
+ elsif item.respond_to?(:error)
197
+ item.error
198
+ end
199
+ return true if error
200
+
201
+ response =
202
+ if item.respond_to?(:[])
203
+ item["response"] || item[:response]
204
+ elsif item.respond_to?(:response)
205
+ item.response
206
+ end
207
+ if response.respond_to?(:[])
208
+ code = response["status_code"] || response[:status_code]
209
+ unless code.nil?
210
+ begin
211
+ n = Integer(code)
212
+ return !(n >= 200 && n < 300)
213
+ rescue ArgumentError, TypeError
214
+ return false
215
+ end
216
+ end
217
+ end
218
+ false
219
+ end
220
+
221
+ # One high-level batch job: deadline guard (#178), poll loop (#179), partial
222
+ # completion (#180).
223
+ #
224
+ # Created by Batchwatch::Client#batch. result blocks, polling with backoff and
225
+ # jitter until the batch is terminal or the deadline fires. For an async /
226
+ # worker context that does not want a blocked thread, poll_once drives the same
227
+ # state machine one step at a time.
228
+ #
229
+ # The clock / sleep / rng seams exist for deterministic tests: the backoff
230
+ # schedule and its jitter are pure functions of these, so a fake clock and a
231
+ # fake rng pin the exact poll count with no real sleeps. Production leaves them
232
+ # as the real time / random.
233
+ class BatchJob
234
+ def initialize(bw, model, deadline: nil, on_deadline: nil,
235
+ provider: "openai", poll: nil, cancel: nil, result_of: nil,
236
+ succeeded: nil, expired: nil,
237
+ poll_base: nil, poll_floor: nil,
238
+ poll_ceiling: POLL_CEILING_S, poll_backoff: POLL_BACKOFF,
239
+ poll_jitter: POLL_JITTER, use_p50_cadence: true,
240
+ poll_interval: nil, clock: nil, sleep: nil, rng: nil)
241
+ @bw = bw
242
+ @model = model
243
+ @provider = provider
244
+ @on_deadline = on_deadline
245
+ @deadline_s = Batchwatch.seconds(deadline)
246
+ @poll = poll || Batchwatch.method(:default_poll)
247
+ @cancel = cancel || Batchwatch.method(:default_cancel)
248
+ @result_of = result_of || Batchwatch.method(:default_result)
249
+ @succeeded = succeeded || Batchwatch.method(:default_succeeded)
250
+ @expired = expired || Batchwatch.method(:default_expired)
251
+
252
+ # poll_interval is #178's name for the (then-fixed) poll cadence. #179
253
+ # renamed it to poll_base (the FIRST interval before backoff) and split out
254
+ # an explicit poll_floor (the rate-limit guard). We honour the old name as
255
+ # an alias so #178's two-callable usage - and its tests, which set a tiny
256
+ # interval to make the deadline fire fast - keep working untouched (epic
257
+ # rule #4). When only poll_interval is given, a small value is also taken
258
+ # as the floor so the deadline-guard tests still poll as fast as they
259
+ # asked, not clamped up to the 5s default.
260
+ if poll_base.nil?
261
+ poll_base = poll_interval.nil? ? POLL_BASE_S : poll_interval
262
+ end
263
+ if poll_floor.nil?
264
+ poll_floor = poll_interval.nil? ? POLL_FLOOR_S : [poll_interval, POLL_FLOOR_S].min
265
+ end
266
+
267
+ # Poll cadence (#179). The floor and ceiling are the two guard rails; the
268
+ # base is stored raw because EVERY interval that reaches a sleep goes
269
+ # through clamp, which is the ONE place the floor and ceiling are applied
270
+ # (P7: one canonical site, so the rate-limit floor cannot be
271
+ # half-removed). A base below the floor therefore still polls at the floor.
272
+ @poll_floor = [0.001, poll_floor.to_f].max
273
+ @poll_ceiling = [@poll_floor, poll_ceiling.to_f].max
274
+ @poll_base = [0.0, poll_base.to_f].max
275
+ @poll_backoff = [1.0, poll_backoff.to_f].max
276
+ @poll_jitter = [[poll_jitter.to_f, 1.0].min, 0.0].max
277
+ @use_p50_cadence = use_p50_cadence
278
+
279
+ # Injectable seams for deterministic tests; real time/random otherwise.
280
+ @clock = clock || -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) }
281
+ @sleep = sleep || ->(s) { Kernel.sleep(s) }
282
+ @rand = rng || -> { Kernel.rand }
283
+
284
+ @handle = nil # the provider's batch object
285
+ @submitted_at = nil # monotonic clock at submit
286
+ @result = nil
287
+ @fell_back = false # did the deadline guard fire?
288
+ @expired_flag = false # did the batch hit the 24h expiry?
289
+ @resolved = false # has result run to completion?
290
+ @interval = nil # current backoff interval, lazily seeded
291
+ @poll_count = 0 # how many times we asked "are you done?"
292
+ @retries = nil # lazy per-job retry registry (#180)
293
+ end
294
+
295
+ attr_reader :handle
296
+
297
+ # --------------------------------------------------------------- lifecycle
298
+
299
+ # Run the caller's batch-create callable and remember the handle.
300
+ #
301
+ # +create+ is a zero-argument callable (a block, proc, or lambda) that
302
+ # returns the provider's batch object - typically
303
+ # -> { client.batches.create(...) }. We call it, store what it returns, and
304
+ # start the deadline clock. We never inspect the payload it built; the handle
305
+ # is opaque to us except for the questions the poll/result callables ask.
306
+ #
307
+ # Raising from create is the caller's own error and is left untouched - the
308
+ # batch never started, so there is nothing for us to guard.
309
+ def submit(create = nil, &block)
310
+ create ||= block
311
+ raise BatchJobError, "batchwatch: submit() needs a callable" if create.nil?
312
+
313
+ unless @handle.nil?
314
+ raise BatchJobError,
315
+ "batchwatch: this job was already submitted - create a new " \
316
+ "bw.batch(...) for a second batch"
317
+ end
318
+ @handle = create.call
319
+ @submitted_at = @clock.call
320
+ @handle
321
+ end
322
+
323
+ # Block until the batch finishes, or the deadline fires.
324
+ #
325
+ # Returns the batch result if it completed in time, otherwise the result of
326
+ # on_deadline. Idempotent: a second call returns the same result without
327
+ # re-waiting or re-reporting.
328
+ #
329
+ # Raises BatchJobError if called before submit.
330
+ def result
331
+ return @result if @resolved
332
+
333
+ if @handle.nil?
334
+ raise BatchJobError,
335
+ "batchwatch: result() before submit() - call " \
336
+ "job.submit(-> { client.batches.create(...) }) first"
337
+ end
338
+
339
+ completed_in_time = wait_loop
340
+ if completed_in_time && @succeeded.call(@handle)
341
+ @result = @result_of.call(@handle)
342
+ report_outcome(fell_back: false)
343
+ else
344
+ @result = fallback(timed_out: !completed_in_time)
345
+ end
346
+ @resolved = true
347
+ @result
348
+ end
349
+
350
+ # Take one non-blocking step of the poll loop and report the state.
351
+ #
352
+ # The non-blocking variant (#179): an async or worker context can drive the
353
+ # wait itself instead of handing us a thread. Call it, read the returned
354
+ # PollState, and - if it is RUNNING - sleep next_interval (which we have
355
+ # already advanced) before the next call. result is still the way to get the
356
+ # value once a step returns a terminal state; this only reports where the
357
+ # batch is.
358
+ #
359
+ # Unlike result this never sleeps and never falls back - it is a single
360
+ # question. The deadline is still honoured by result's own loop.
361
+ def poll_once
362
+ if @handle.nil?
363
+ raise BatchJobError,
364
+ "batchwatch: poll_once() before submit() - call " \
365
+ "job.submit(-> { client.batches.create(...) }) first"
366
+ end
367
+ @interval = seed_interval if @interval.nil?
368
+ state = poll_state
369
+ @interval = advance_interval(@interval) if state == PollState::RUNNING
370
+ state
371
+ end
372
+
373
+ # Seconds to sleep before the next poll_once, jitter applied. Already
374
+ # includes the rate-limit floor and the jitter, so an async driver can sleep
375
+ # exactly this and match the blocking loop's cadence.
376
+ def next_interval
377
+ @interval = seed_interval if @interval.nil?
378
+ sleep_for
379
+ end
380
+
381
+ # How many times we have asked the batch whether it is done. This is the
382
+ # number that gets someone rate-limited, so it is exposed for a caller - and
383
+ # a test - to assert a bound on.
384
+ def poll_count
385
+ @poll_count
386
+ end
387
+
388
+ # true once the batch was seen to reach the 24h expiry. A distinct terminal
389
+ # state, never folded into a timeout.
390
+ def expired
391
+ @expired_flag
392
+ end
393
+
394
+ # true once the deadline guard has fired and the fallback was used.
395
+ def fell_back
396
+ @fell_back
397
+ end
398
+
399
+ # ------------------------------------------------- partial completion (#180)
400
+
401
+ # Split the batch into landed / failed / expired, mapped by custom_id.
402
+ #
403
+ # A real batch is not binary: a job of 20,000 requests can come back with
404
+ # some landed, some failed per-request, and some never returned before the
405
+ # 24h expiry. This is where silent data loss lives - a naive "the job
406
+ # completed" reads all three as success. BatchResult separates them, with
407
+ # counts and the per-request ids, so the caller can act.
408
+ #
409
+ # +results+ is the per-request outcomes the caller downloaded from THEIR
410
+ # provider (we never fetch content). Each item is a mapping carrying a
411
+ # custom_id and either a success or an error - the shape the OpenAI /
412
+ # Anthropic batch output lines already have. +custom_ids+ is the full set the
413
+ # caller SUBMITTED; any id in it that has no result line is expired
414
+ # (outstanding at the 24h cutoff). Mapping is by custom_id, never by index:
415
+ # provider result ordering is not guaranteed.
416
+ #
417
+ # Pass neither and we read the whole-batch terminal status off the handle
418
+ # instead - a whole-batch expiry becomes an all-expired result, a whole-batch
419
+ # failure an all-failed one.
420
+ def split(results = nil, custom_ids = nil)
421
+ BatchResult.from_results(self, results, custom_ids)
422
+ end
423
+
424
+ # Resubmit ONLY the failed subset, once, with an idempotency key.
425
+ #
426
+ # +resubmit+ is a callable the caller hands us - resubmit.call(failed_ids) -
427
+ # that recreates a batch over just those ids. Same epic rule: we take the
428
+ # callable, never the payload. We call it with the list of failed custom_ids
429
+ # and remember the handle it returns as a fresh child job.
430
+ #
431
+ # Idempotent by construction: calling it twice submits ONE retry. Duplicating
432
+ # a 20,000-request job is an expensive failure, so the key is the sorted set
433
+ # of failed ids - a pure function of WHAT is being retried, not of when. A
434
+ # second call with the same failure set returns the same child without
435
+ # resubmitting; a genuinely different failure set is a different retry and
436
+ # does submit.
437
+ #
438
+ # Returns the child BatchJob, or nil when there is nothing to retry (an empty
439
+ # failure set is a no-op, not an empty batch).
440
+ def retry_failed(resubmit, result: nil)
441
+ res = result.nil? ? split : result
442
+ failed_ids = res.failed_ids.sort
443
+ return nil if failed_ids.empty?
444
+
445
+ @retries ||= {}
446
+ key = failed_ids
447
+ return @retries[key] if @retries.key?(key)
448
+
449
+ child = @bw.batch(@model, deadline: nil, on_deadline: @on_deadline,
450
+ provider: @provider)
451
+ child.submit(-> { resubmit.call(failed_ids.dup) })
452
+ @retries[key] = child
453
+ child
454
+ end
455
+
456
+ # ------------------------------------------------------- context-manager use
457
+
458
+ # A block form so the partial-completion cleanup has an obvious home; the
459
+ # base job has nothing to do on the way out. Yields self and returns the
460
+ # block's value. We do NOT force a result the caller never asked for.
461
+ def with
462
+ yield self
463
+ end
464
+
465
+ private
466
+
467
+ # ------------------------------------------------------------- the poll loop
468
+ #
469
+ # #179 owns the wait. The contract result leans on:
470
+ # wait_loop -> true if the batch reached a terminal status before the
471
+ # deadline (result then asks succeeded/expired what to
472
+ # do with it)
473
+ # false if the deadline arrived first
474
+ # side: it must NOT cancel, report, or fall back. Deciding what a false
475
+ # means is result's job.
476
+ def wait_loop
477
+ @interval = seed_interval
478
+ loop do
479
+ state = poll_state
480
+ return true if state != PollState::RUNNING
481
+ return false if past_deadline?
482
+
483
+ @sleep.call(sleep_for)
484
+ @interval = advance_interval(@interval)
485
+ end
486
+ end
487
+
488
+ # Ask the batch its state ONCE and classify it. Counts the poll.
489
+ def poll_state
490
+ @poll_count += 1
491
+ return PollState::RUNNING unless @poll.call(@handle)
492
+
493
+ # Terminal. Which kind?
494
+ if @expired.call(@handle)
495
+ @expired_flag = true
496
+ return PollState::EXPIRED
497
+ end
498
+ return PollState::DONE if @succeeded.call(@handle)
499
+
500
+ PollState::FAILED
501
+ end
502
+
503
+ # ---------------------------------------------------------- cadence + backoff
504
+
505
+ # The first poll interval: p50-informed when we can, fixed otherwise. We
506
+ # spend a fraction of the model's measured p50 before the first poll. The
507
+ # read fails open (measured_p50 swallows everything and returns nil), so an
508
+ # unreachable or silent batchwatch drops us to the fixed base with no
509
+ # interruption. Clamped into [floor, ceiling] so a wild p50 cannot push us
510
+ # past either guard.
511
+ def seed_interval
512
+ base = @poll_base
513
+ if @use_p50_cadence
514
+ p50 = measured_p50
515
+ base = p50 * POLL_P50_FRACTION if p50 && p50 > 0
516
+ end
517
+ clamp(base)
518
+ end
519
+
520
+ # Exponential backoff, clamped to the ceiling.
521
+ def advance_interval(interval)
522
+ clamp(interval * @poll_backoff)
523
+ end
524
+
525
+ # The ONE place the floor and ceiling are applied (P7).
526
+ def clamp(interval)
527
+ [@poll_floor, [@poll_ceiling, interval.to_f].min].max
528
+ end
529
+
530
+ # The model's measured p50 queue time in seconds, or nil. Reuses the
531
+ # client's existing read of /v1/wait (wait_now) - no new route, no new
532
+ # payload field. Fails open by construction: wait_now already swallows every
533
+ # error and returns nil when it cannot answer, and we defend once more here
534
+ # so a malformed number can never take the poll loop down. A nil means "poll
535
+ # on the fixed schedule", never "stop polling".
536
+ def measured_p50
537
+ info = @bw.wait_now(@model, provider: @provider, mode: "batch")
538
+ return nil unless info
539
+
540
+ %w[p50_s observed_wait_s planning_wait_s].each do |key|
541
+ value = info[key]
542
+ return value.to_f unless value.nil?
543
+ end
544
+ nil
545
+ rescue StandardError
546
+ nil
547
+ end
548
+
549
+ def past_deadline?
550
+ return false if @deadline_s.nil?
551
+
552
+ (@clock.call - @submitted_at) >= @deadline_s
553
+ end
554
+
555
+ # Sleep before the next poll: jittered interval, clamped to the deadline.
556
+ #
557
+ # Full jitter: the actual sleep is drawn uniformly from
558
+ # [max(floor, (1-jitter)*interval), interval]. interval is already floored
559
+ # (it comes from clamp), and we floor the window bottom too, so jitter can
560
+ # spread a fleet that all submitted at once without any client ever dipping
561
+ # under the rate-limit floor. Then we clamp to the deadline so a coarse
562
+ # interval never overshoots the guard.
563
+ def sleep_for
564
+ interval = @interval.nil? ? clamp(@poll_base) : @interval
565
+ low = [@poll_floor, interval * (1.0 - @poll_jitter)].max
566
+ delay = low + (interval - low) * @rand.call
567
+ return delay if @deadline_s.nil?
568
+
569
+ remaining = @deadline_s - (@clock.call - @submitted_at)
570
+ [0.0, [delay, remaining].min].max
571
+ end
572
+
573
+ # ------------------------------------------------------------- fallback
574
+
575
+ # Cancel the batch, run the sync fallback, report the outcome. Reached when
576
+ # the batch missed its deadline OR finished in a non-success terminal state
577
+ # (failed / expired). Either way the caller still needs an answer.
578
+ def fallback(timed_out:)
579
+ @fell_back = true
580
+ # Best-effort cancel first, so we are not paying for a batch nobody will
581
+ # read. A cancel failure is swallowed - the fallback matters more.
582
+ begin
583
+ @cancel.call(@handle)
584
+ rescue StandardError
585
+ nil
586
+ end
587
+
588
+ if @on_deadline.nil?
589
+ raise BatchJobError,
590
+ "batchwatch: the batch #{why_fallback(timed_out)} but no " \
591
+ "on_deadline fallback was given - pass " \
592
+ "on_deadline: -> { client.chat.completions.create(...) } " \
593
+ "to bw.batch(...)"
594
+ end
595
+
596
+ # Report BEFORE running the fallback: the outcome is already known (the
597
+ # batch would have missed), and the report goes out on a background thread,
598
+ # so it costs the caller's own call nothing.
599
+ report_outcome(fell_back: true)
600
+ @on_deadline.call
601
+ end
602
+
603
+ # Record the prediction outcome down the existing #101 accuracy path. A
604
+ # deadline fallback IS the server's "run_sync" case; a batch that completed
605
+ # in time is the "run_batch" hit. Reporting both sides is what makes the
606
+ # accuracy number honest. Failing to report must never affect the caller's
607
+ # job, so this rides the same fail-open contract as track().
608
+ def report_outcome(fell_back:)
609
+ @bw.record_job_outcome(@model, @provider, !fell_back, @deadline_s)
610
+ rescue StandardError
611
+ nil
612
+ end
613
+
614
+ def why_fallback(timed_out)
615
+ return "missed its deadline" if timed_out
616
+ return "expired (24h)" if @expired_flag
617
+
618
+ "did not succeed"
619
+ end
620
+ end
621
+
622
+ # The split of a batch into landed / failed / expired (#180).
623
+ #
624
+ # A real batch is three sets, not one: requests that landed, requests that
625
+ # failed per-request, and requests still outstanding when the 24h expiry hit.
626
+ # Reporting only "completed" loses the last two, which is exactly where silent
627
+ # data loss lives. This object carries the three sets, their counts, and the
628
+ # per-request ids, and - because provider ordering is not guaranteed - is built
629
+ # by mapping on custom_id, never by index.
630
+ #
631
+ # The measurement rule for a partial job (#180): an expiry is reported with
632
+ # status="expired" (NOT completed, which would pollute p90; NOT failed, which
633
+ # would lose the "queue was slow" signal), a per-request-failed job as
634
+ # completed with the landed count, and a clean job as completed. The client
635
+ # does not invent a fourth reading.
636
+ class BatchResult
637
+ attr_reader :job, :landed_ids, :failed_ids, :expired_ids, :results_by_id
638
+
639
+ def initialize(job, landed_ids:, failed_ids:, expired_ids:,
640
+ results_by_id: nil)
641
+ @job = job
642
+ @landed_ids = landed_ids.dup
643
+ @failed_ids = failed_ids.dup
644
+ @expired_ids = expired_ids.dup
645
+ # The caller's own result objects, keyed by custom_id, for the landed ones
646
+ # - so map-back is a hash lookup, never a positional zip.
647
+ @results_by_id = (results_by_id || {}).dup
648
+ end
649
+
650
+ # ---- counts, so a caller can branch without .length on three arrays -------
651
+
652
+ def landed
653
+ @landed_ids.length
654
+ end
655
+
656
+ def failed
657
+ @failed_ids.length
658
+ end
659
+
660
+ def expired
661
+ @expired_ids.length
662
+ end
663
+
664
+ def total
665
+ landed + failed + expired
666
+ end
667
+
668
+ # true only when every submitted request landed. No silent success: a job
669
+ # with anything failed or outstanding at expiry is NOT complete. This is the
670
+ # method a caller checks before treating the batch as done.
671
+ def complete?
672
+ total > 0 && failed.zero? && expired.zero?
673
+ end
674
+
675
+ # The caller's own landed result for one custom_id, or nil. The map-back the
676
+ # naive zip gets wrong: a lookup by id, so out-of-order provider results
677
+ # still reach the right caller object.
678
+ def result_for(custom_id)
679
+ @results_by_id[custom_id]
680
+ end
681
+
682
+ # Build the split from per-request outcomes, or from the handle status.
683
+ #
684
+ # See BatchJob#split for the argument contract. The per-request path
685
+ # classifies each result line by whether it carries an error, treats any
686
+ # submitted id with no line as expired, and keys the landed results by
687
+ # custom_id. The handle-status fallback reads the whole-batch terminal state
688
+ # for a provider that reports no lines.
689
+ def self.from_results(job, results = nil, custom_ids = nil)
690
+ return from_handle(job) if results.nil? && custom_ids.nil?
691
+
692
+ results = Array(results)
693
+ seen = {}
694
+ landed = []
695
+ failed = []
696
+ results.each do |item|
697
+ cid = Batchwatch.custom_id_of(item)
698
+ if cid.nil?
699
+ # A line we cannot map is not silently dropped into "landed" - that
700
+ # would be the zip-by-index mistake in another guise. It counts as
701
+ # failed so the caller sees the reconciliation gap.
702
+ failed << UNMAPPED_MARKER
703
+ next
704
+ end
705
+ seen[cid] = item
706
+ if Batchwatch.result_error?(item)
707
+ failed << cid
708
+ else
709
+ landed << cid
710
+ end
711
+ end
712
+
713
+ # Anything submitted but never returned is outstanding at expiry.
714
+ submitted = custom_ids.nil? ? seen.keys : Array(custom_ids)
715
+ expired = submitted.reject { |cid| seen.key?(cid) }
716
+
717
+ results_by_id = {}
718
+ landed.each { |cid| results_by_id[cid] = seen[cid] unless cid == UNMAPPED_MARKER }
719
+ new(job, landed_ids: landed.reject { |c| c == UNMAPPED_MARKER },
720
+ failed_ids: failed, expired_ids: expired,
721
+ results_by_id: results_by_id)
722
+ end
723
+
724
+ # Coarse split from the whole-batch terminal status. For a provider that
725
+ # gives us no per-request lines: an expired handle is all-expired, a
726
+ # non-success terminal handle all-failed, a success all-landed. There are no
727
+ # ids to list at this granularity, so the sets carry a single whole-batch
728
+ # marker each and the counts read 1/0/0-style.
729
+ def self.from_handle(job)
730
+ handle = job.handle
731
+ if Batchwatch.default_expired(handle) || job.expired
732
+ return new(job, landed_ids: [], failed_ids: [], expired_ids: [WHOLE_MARKER])
733
+ end
734
+ if Batchwatch.default_succeeded(handle)
735
+ return new(job, landed_ids: [WHOLE_MARKER], failed_ids: [], expired_ids: [])
736
+ end
737
+
738
+ new(job, landed_ids: [], failed_ids: [WHOLE_MARKER], expired_ids: [])
739
+ end
740
+ end
741
+ end
data/lib/batchwatch.rb CHANGED
@@ -20,3 +20,4 @@
20
20
  # end
21
21
  require_relative "batchwatch/spool"
22
22
  require_relative "batchwatch/client"
23
+ require_relative "batchwatch/job"
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: batchwatch
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.1
4
+ version: 0.2.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andreas Graae
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-08-29 00:00:00.000000000 Z
11
+ date: 2026-08-31 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: minitest
@@ -51,6 +51,7 @@ files:
51
51
  - README.md
52
52
  - lib/batchwatch.rb
53
53
  - lib/batchwatch/client.rb
54
+ - lib/batchwatch/job.rb
54
55
  - lib/batchwatch/spool.rb
55
56
  homepage: https://batchwatch.dev
56
57
  licenses: