stablemates-workhorse 0.11.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1686623ff5e225f372245830a169d9b248db6bd195ed3f446679e01098bc9610
4
- data.tar.gz: bde5f7e6517d2a7a3621b437237db776673156693b22fc1a46f42aa28305b563
3
+ metadata.gz: f684dcd8a1e590688046e5b175ba97db4c4b62320d78c12353c5bf2fdf294ca4
4
+ data.tar.gz: 748bf2d0d2cd17eb8717c1d5066f7add52c5c60895ffb5975038dac3803041ec
5
5
  SHA512:
6
- metadata.gz: eef44a6c8715cc1d6c0f7983fb42b3662332ad1302fd6d705e60eca9f2d4ed099d32fd53d1303a2ee1a0cdfa9ee08c3624c57516c967c22cb62f0366180bdf4b
7
- data.tar.gz: e9137fa4c8b8f6a5f20f800cb4abc482e6c750721758fb5432f407c229816b4a4186700e882eeb8859a8d21f8bcfa2f2be6be1ae734ef574a003227a8db82a94
6
+ metadata.gz: 7c298a44ca80299142012bf23d118876cb036c207974625dd02bd49b262cad60eefaef0ea455d22153c4a052c69dbdbcd8788d3c896bac0c8c9cdd237751239b
7
+ data.tar.gz: fffb3cbaa9f41f1f0efb59a6eff7f7fc4da4a840cf35b53def9a044ad3321cacbc380f0ddae14bf8209a789b61e8e16cfcddaccc106e723f2cd02171fa4b13d3
data/CHANGELOG.md CHANGED
@@ -3,6 +3,44 @@
3
3
  `stablemates-workhorse` gem versions and release notes live here. The gem carries the version the
4
4
  other SDKs carry, because every tag names one release of all of them.
5
5
 
6
+ ## 0.12.0 — 2026-11-07
7
+
8
+ The npm packages, Python distribution, Go module, Rust crate, Ruby gem, JVM artifacts, and .NET
9
+ packages release from one source commit.
10
+
11
+ Requires **schema v76**, Ruby **3.3** or newer, and PostgreSQL **15** or newer.
12
+
13
+ ### Upgrade steps
14
+
15
+ **Migrate the schema before starting updated processes.** Run `workhorse schema migrate` before any
16
+ process of this release starts, producers too. Migrations 0091 and 0092 move the schema from
17
+ version 89 to version **91**. They replace function bodies and change no stored rows. The SDK
18
+ compatibility floor stays at **76**. A 0.11 process keeps working on version 91, because a
19
+ migration inside a major line only adds.
20
+
21
+ ### A full-tier queue no longer makes PostgreSQL log an error at each tier probe
22
+
23
+ Before a tier probe, the worker now reads the tier of the queue from `queue_control` with one
24
+ `SELECT`. It sends the fast claim only to a fast-tier queue. Before, PostgreSQL rejected the probe
25
+ of each full-tier queue every 30 seconds and wrote that error to its log. A tier change still
26
+ reaches a running worker within 30 seconds. The change needs no migration and does not change the
27
+ SDK compatibility floor.
28
+
29
+ ### The dashboard task detail shows the task group of a member task
30
+
31
+ Migration 0091 (`0091-show-the-task-group-in-the-task-detail.sql`) moves the schema to version 90.
32
+ The `dashboard/v1` task detail document gains `group`, with the group ID, the member position, and
33
+ the group state of a member task, an additive change. The migration does not raise the SDK
34
+ compatibility floor.
35
+
36
+ ### The dashboard task estimate follows the current size of the task table
37
+
38
+ Migration 0092 (`0092-scale-the-dashboard-task-estimate-by-the-table-size.sql`) moves the schema to
39
+ version 91. `dashboard_task_estimate_v1` now scales the stored rows per page of the task table by
40
+ its current number of pages. After a bulk enqueue, the dashboard sidebar counts and the Queues page
41
+ use their estimates before autoanalyze runs. The migration does not raise the SDK compatibility
42
+ floor.
43
+
6
44
  ## 0.11.0 — 2026-10-28
7
45
 
8
46
  The npm packages, Python distribution, Go module, Rust crate, Ruby gem, JVM artifacts, and .NET
data/README.md CHANGED
@@ -30,7 +30,7 @@ bundle add connection_pool
30
30
  Install the schema once, as a deployment step. The application never installs or migrates it.
31
31
 
32
32
  ```bash
33
- npx --package @stablemates/workhorse@0.11.0 workhorse schema install
33
+ npx --package @stablemates/workhorse@0.12.0 workhorse schema install
34
34
  ```
35
35
 
36
36
  The machine that runs that deployment step needs Node.js 22 or newer. The application itself needs
@@ -11,7 +11,7 @@ module Stablemates
11
11
  MINIMUM_PROTOCOL_VERSION = 5
12
12
  MAXIMUM_PROTOCOL_VERSION = 5
13
13
  MINIMUM_SCHEMA_VERSION = 76
14
- MAXIMUM_SCHEMA_VERSION = 89
14
+ MAXIMUM_SCHEMA_VERSION = 91
15
15
  # PostgreSQL's atomic enqueue batch limit.
16
16
  MAX_ENQUEUE_BATCH_SIZE = 1000
17
17
  DEFAULT_TASK_VALUE_MAX_BYTES = 1048576
@@ -347,6 +347,9 @@ module Stablemates
347
347
  # `queue_control` (internal)
348
348
  QUEUE_CONTROL = 'SELECT queue_name, paused, tier, record_attempts, record_claims FROM workhorse.queue_control'
349
349
 
350
+ # `queue_tier` (internal)
351
+ QUEUE_TIER = 'SELECT COALESCE((SELECT tier FROM workhorse.queue_control WHERE queue_name = $1::text), \'full\') AS tier'
352
+
350
353
  # `schedule_definition` (internal)
351
354
  SCHEDULE_DEFINITION = 'SELECT DISTINCT namespace FROM workhorse.schedule_definition ORDER BY namespace'
352
355
 
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Stablemates
4
4
  module Workhorse
5
- VERSION = "0.11.0"
5
+ VERSION = "0.12.0"
6
6
  end
7
7
  end
@@ -44,6 +44,8 @@ module Stablemates
44
44
  # a stop statement of the previous run, checks for a stop again.
45
45
  STARTUP_STOP_CHECK = 0.05
46
46
  COMPLETION_BATCH_LIMIT = 100
47
+ # How long, in seconds, a worker claims a full-tier queue through claim_many_v1 before it reads
48
+ # the tier again. A queue can move to the fast tier only while it holds no live tasks (ADR 0077).
47
49
  TIER_PROBE_INTERVAL = 30
48
50
  # Maintenance writes shared due times and retention cutoffs, so the database clock chooses its
49
51
  # time. PostgreSQL resolves this timestamptz input at the start of the autocommit statement.
@@ -961,12 +963,15 @@ module Stablemates
961
963
  end
962
964
  end
963
965
 
966
+ # The lease of a claimed task starts at its claim statement, not at the tier read before it, so
967
+ # a slow read cannot make a task look expired as soon as the worker leases it.
964
968
  def claim(queue, limit, fast_limit, gate)
965
969
  started_at = monotonic
966
970
  Telemetry.span("workhorse.claim", {"workhorse.queue.name" => queue}) do |span|
967
- rows = claim_queue(queue, limit, started_at, fast_limit, gate)
971
+ rows, leased_at = claim_queue(queue, limit, started_at, fast_limit, gate)
968
972
  if rows.nil?
969
973
  Telemetry.set_attribute(span, "workhorse.queue.tier", "full")
974
+ leased_at = monotonic
970
975
  rows = @statement_watch.watch("claim") do
971
976
  permitted(gate) do |build|
972
977
  @executor.rows(SqlCatalogue::CLAIM_MANY_V1, build.call { [queue, @worker_id, limit.to_s, @lease_ms.to_s] })
@@ -977,7 +982,7 @@ module Stablemates
977
982
  tasks = rows.map { |row| ClaimedTask.from_row(row, queue) }
978
983
  record_claimed(queue, started_at, tasks)
979
984
  Telemetry.task_span_attributes(tasks.first).each { |key, value| Telemetry.set_attribute(span, key, value) } if tasks.any?
980
- tasks.map { |task| [task, started_at] }
985
+ tasks.map { |task| [task, leased_at] }
981
986
  end
982
987
  end
983
988
 
@@ -1014,14 +1019,26 @@ module Stablemates
1014
1019
  end
1015
1020
 
1016
1021
  # Claims through the fast tier, or returns nil when the queue is on the full tier. The fast
1017
- # claim is complete_many_and_claim_v1 with no completions. A full-tier queue rejects it, and
1018
- # the worker then claims that queue through claim_many_v1 until the next probe.
1019
- def claim_queue(queue, limit, sent_at, fast_limit, gate)
1020
- return nil if @state_lock.synchronize { @full_tier_until.fetch(queue, -Float::INFINITY) > sent_at }
1021
-
1022
+ # claim is complete_many_and_claim_v1 with no completions. Before the worker sends a fast claim
1023
+ # to a queue of unknown tier, it reads the tier. A full-tier answer makes the worker claim that
1024
+ # queue through claim_many_v1 until the next probe. Thus a full-tier queue never receives the
1025
+ # fast claim, which PostgreSQL rejects with an error that it writes to its log. A queue that
1026
+ # moved to the full tier after the read still rejects the fast claim, and the worker then marks
1027
+ # it full-tier. Returns the claimed rows and the monotonic time the claim was sent.
1028
+ # +probed_at+ times the probe interval. The lease starts at the claim, after the tier read.
1029
+ def claim_queue(queue, limit, probed_at, fast_limit, gate)
1030
+ full_until, known_fast = @state_lock.synchronize do
1031
+ [@full_tier_until.fetch(queue, -Float::INFINITY), @fast_tier_queues.include?(queue)]
1032
+ end
1033
+ return nil if full_until > probed_at
1034
+ unless known_fast || reads_fast_tier?(queue, gate)
1035
+ mark_full_tier(queue, probed_at)
1036
+ return nil
1037
+ end
1022
1038
  fast_limit = [limit, fast_limit].min
1023
- return [] if fast_limit <= 0
1039
+ return [[], monotonic] if fast_limit <= 0
1024
1040
 
1041
+ sent_at = monotonic
1025
1042
  begin
1026
1043
  rows = @statement_watch.watch("claim") do
1027
1044
  permitted(gate) do |build|
@@ -1030,7 +1047,7 @@ module Stablemates
1030
1047
  end
1031
1048
  end
1032
1049
  rescue FastTierUnsupportedError
1033
- mark_full_tier(queue, sent_at)
1050
+ mark_full_tier(queue, probed_at)
1034
1051
  return nil
1035
1052
  end
1036
1053
  # A claim that finds nothing still returns one row, with every claim column null.
@@ -1040,10 +1057,18 @@ module Stablemates
1040
1057
  @fast_tier_queues.add(queue)
1041
1058
  claimed.each { |row| @fast_task_ids.add(row["task_id"]) }
1042
1059
  end
1043
- claimed
1060
+ [claimed, sent_at]
1061
+ end
1062
+
1063
+ # Reads the tier of +queue+ through the claim's gate. A queue with no control row is full-tier.
1064
+ def reads_fast_tier?(queue, gate)
1065
+ rows = @statement_watch.watch("claim") do
1066
+ permitted(gate) { |build| @executor.rows(SqlCatalogue::QUEUE_TIER, build.call { [queue] }) }
1067
+ end
1068
+ rows.first&.fetch("tier") == "fast"
1044
1069
  end
1045
1070
 
1046
- # Claims a queue that rejected a fast-tier statement through claim_many_v1 until the probe.
1071
+ # Claims a full-tier queue through claim_many_v1 until the probe.
1047
1072
  def mark_full_tier(queue, sent_at)
1048
1073
  @state_lock.synchronize do
1049
1074
  @full_tier_until[queue] = sent_at + TIER_PROBE_INTERVAL
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: stablemates-workhorse
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.11.0
4
+ version: 0.12.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Stablemates