solid_queue_autoscaler 1.0.11 → 1.0.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: cf794daeb74474c136c8aec706793bf74617dcb610abf42df89ddb4fefd99274
4
- data.tar.gz: 96ec9ad6993871c7773ff524c5d71ff3e919ff89aacba5587d5dee63aa277f5d
3
+ metadata.gz: bafb0cda485024f43bd0e73291bcb0bbbc7275e3f897d4a2acafbd6496140b98
4
+ data.tar.gz: 4b443c68ca28df2b3efd62a4cd7ae30f6cf53e978c941a1e60aa7ebd0c7befb0
5
5
  SHA512:
6
- metadata.gz: 0b8dd105d028035aee534300ee1d91af9193faefe4e27c8ba3d72c98b3c401cdc52a04894bdeef0b14ebdd7a42f120e563bb378509f2a9ac046a18554c7079d0
7
- data.tar.gz: 9980ac5a53affb82b264cd9e7c9bacb388d26a5ecc0116fdd1e22ce527c66ec63bc51d5fe47a492df1941df1b4d0052cfed721e7e4b62566a5c70ab885852f9c
6
+ metadata.gz: 943bd220740694c827afc4c3aede77cac6da3b419eda051c686b5fb07cfbb69da28b6b8d5882fcf61122bfc6991352e52b9dab2b7f05982333324efbce20205f
7
+ data.tar.gz: a201785b1ecc766f744a7d705f64dd1e7afaa611e004f3fb9f003a28a05a397fc412ec276a17b99c4bbad0cf89ed476884f510986e3f95e9f9385252d7e15d31
data/CHANGELOG.md CHANGED
@@ -7,6 +7,61 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [1.0.15] - 2025-01-30
11
+
12
+ ### Fixed
13
+ - **Fixed Heroku adapter 404 error when querying scaled-to-zero dynos** - When a dyno type is scaled to 0 and removed from Heroku's formation, the API returns 404. The adapter now handles this gracefully:
14
+ - `current_workers` returns 0 instead of raising an error when formation doesn't exist
15
+ - `scale` falls back to `batch_update` API to create the formation when `update` returns 404
16
+ - Added `create_formation` private method using Heroku's batch_update endpoint
17
+ - This enables full scale-to-zero support with `min_workers = 0`
18
+
19
+ ## [1.0.14] - 2025-01-18
20
+
21
+ ### Added
22
+ - **SQLite and MySQL support for advisory locks** - AdvisoryLock now supports multiple database adapters:
23
+ - PostgreSQL: Uses native `pg_try_advisory_lock/pg_advisory_unlock`
24
+ - MySQL/Trilogy: Uses `GET_LOCK/RELEASE_LOCK`
25
+ - SQLite: Uses table-based locking with auto-created locks table
26
+ - Other databases: Falls back to table-based locking
27
+ - Automatic adapter detection via `connection.adapter_name`
28
+ - Stale lock cleanup (locks older than 5 minutes are removed)
29
+ - Lock ownership tracking (`hostname:pid:thread_id`)
30
+
31
+ - **Comprehensive configuration tests** - Added 100+ tests across Rails and Sinatra dummy apps:
32
+ - Tests for ALL configuration options (job_queue, job_priority, scaling thresholds, cooldowns, etc.)
33
+ - Decision engine threshold tests verifying scaling logic
34
+ - End-to-end tests with mocked Heroku API verifying full scaling workflow
35
+ - Queue name and priority regression tests (prevents jobs going to wrong queue)
36
+
37
+ - **GitHub Actions integration test workflow** - New CI job that runs dummy app tests:
38
+ - Runs Rails dummy app tests (62 tests)
39
+ - Runs Sinatra dummy app tests (58 tests)
40
+ - Ensures queue name, priority, and E2E scaling tests pass before release
41
+
42
+ - **Release workflow now requires CI to pass** - Updated release.yml to use `workflow_run` trigger:
43
+ - Release only runs after CI workflow completes successfully
44
+ - All unit tests, integration tests, and linting must pass before publishing
45
+
46
+ ### Fixed
47
+ - **Fixed test pollution in autoscale_job_spec** - Changed from using RSpec's `described_class` (which caches class references) to dynamic constant lookup, preventing stale class reference issues when tests reload the AutoscaleJob class
48
+
49
+ ## [1.0.13] - 2025-01-17
50
+
51
+ ### Fixed
52
+ - **Fixed AutoscaleJob queue_name type mismatch** - Queue name is now converted to string when set via `apply_job_settings!`
53
+ - ActiveJob internally uses strings for queue names, but the configuration uses symbols
54
+ - This caused jobs to have symbol queue names (`:autoscaler`) instead of string (`"autoscaler"`)
55
+ - Now `apply_job_settings!` calls `.to_s` on the job_queue to ensure consistent string format
56
+
57
+ ## [1.0.12] - 2025-01-17
58
+
59
+ ### Fixed
60
+ - **Fixed AutoscaleJob being enqueued to "default" queue** - Added `queue_as :autoscaler` to the job class
61
+ - The issue was that SolidQueue recurring jobs capture the queue name during initialization, BEFORE Rails `after_initialize` hooks run
62
+ - Without a static `queue_as` in the class, jobs defaulted to the "default" queue
63
+ - The `apply_job_settings!` method can still override this via configuration, but the default must be set in the class for SolidQueue recurring to work correctly
64
+
10
65
  ## [1.0.11] - 2025-01-17
11
66
 
12
67
  ### Fixed
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Migration for SolidQueueAutoscaler locks table.
4
+ # This table is used for advisory locking on databases that don't support
5
+ # native advisory locks (SQLite, etc.).
6
+ #
7
+ # NOTE: This migration is OPTIONAL. The locks table is automatically created
8
+ # when first needed. Only use this migration if you prefer to manage the
9
+ # table schema explicitly.
10
+ #
11
+ # For multi-database setups (SolidQueue in separate database):
12
+ # This migration should be placed in db/queue_migrate/ (or your queue DB's migration path)
13
+ # Run with: rails db:migrate:queue
14
+ #
15
+ # For single-database setups:
16
+ # Place in db/migrate/ and run: rails db:migrate
17
+ #
18
+ class CreateSolidQueueAutoscalerLocks < ActiveRecord::Migration<%= migration_version %>
19
+ def change
20
+ create_table :solid_queue_autoscaler_locks, id: false do |t|
21
+ t.string :lock_key, null: false, primary_key: true
22
+ t.integer :lock_id, null: false
23
+ t.datetime :locked_at, null: false
24
+ t.string :locked_by, null: false
25
+ end
26
+
27
+ # Index for cleanup of stale locks
28
+ add_index :solid_queue_autoscaler_locks, :locked_at
29
+ end
30
+ end
@@ -32,6 +32,10 @@ module SolidQueueAutoscaler
32
32
  # end
33
33
  # end
34
34
  class Base
35
+ # Default retry configuration for transient network errors
36
+ DEFAULT_MAX_RETRIES = 3
37
+ DEFAULT_RETRY_DELAYS = [1, 2, 4].freeze # Exponential backoff in seconds
38
+
35
39
  # @param config [Configuration] the autoscaler configuration
36
40
  def initialize(config:)
37
41
  @config = config
@@ -97,6 +101,32 @@ module SolidQueueAutoscaler
97
101
  def log_dry_run(message)
98
102
  logger.info("[DRY RUN] #{message}")
99
103
  end
104
+
105
+ # Executes a block with retry logic for transient errors.
106
+ # Uses exponential backoff with configurable delays.
107
+ #
108
+ # @param error_classes [Array<Class>] Exception classes that should trigger a retry
109
+ # @param max_retries [Integer] Maximum number of retry attempts (default: 3)
110
+ # @param delays [Array<Integer>] Delay in seconds for each retry (default: [1, 2, 4])
111
+ # @param retryable_check [Proc, nil] Optional proc to determine if a specific error should be retried
112
+ # @yield The block to execute with retry logic
113
+ # @return [Object] The result of the block
114
+ def with_retry(error_classes, max_retries: DEFAULT_MAX_RETRIES, delays: DEFAULT_RETRY_DELAYS, retryable_check: nil)
115
+ attempts = 0
116
+ begin
117
+ attempts += 1
118
+ yield
119
+ rescue *error_classes => e
120
+ should_retry = retryable_check ? retryable_check.call(e) : true
121
+ if attempts < max_retries && should_retry
122
+ delay = delays[attempts - 1] || delays.last
123
+ logger&.warn("[Autoscaler] #{name} API error (attempt #{attempts}/#{max_retries}), retrying in #{delay}s: #{e.message}")
124
+ sleep(delay)
125
+ retry
126
+ end
127
+ raise
128
+ end
129
+ end
100
130
  end
101
131
  end
102
132
  end
@@ -20,10 +20,6 @@ module SolidQueueAutoscaler
20
20
  # config.process_type = 'worker'
21
21
  # end
22
22
  class Heroku < Base
23
- # Retry configuration for transient network errors
24
- MAX_RETRIES = 3
25
- RETRY_DELAYS = [1, 2, 4].freeze # Exponential backoff in seconds
26
-
27
23
  # Errors that are safe to retry (transient network issues)
28
24
  RETRYABLE_ERRORS = [
29
25
  Excon::Error::Timeout,
@@ -32,11 +28,18 @@ module SolidQueueAutoscaler
32
28
  ].freeze
33
29
 
34
30
  def current_workers
35
- with_retry do
31
+ with_retry(RETRYABLE_ERRORS, retryable_check: method(:retryable_error?)) do
36
32
  formation = client.formation.info(app_name, process_type)
37
33
  formation['quantity']
38
34
  end
39
35
  rescue Excon::Error => e
36
+ # Handle 404 gracefully - formation doesn't exist means 0 workers
37
+ # This happens when a dyno type is scaled to 0 and removed from formation
38
+ if e.respond_to?(:response) && e.response&.status == 404
39
+ logger&.debug("[Autoscaler] Formation '#{process_type}' not found, treating as 0 workers")
40
+ return 0
41
+ end
42
+
40
43
  raise HerokuAPIError.new(
41
44
  "Failed to get formation info: #{e.message}",
42
45
  status_code: e.respond_to?(:response) ? e.response&.status : nil,
@@ -50,11 +53,17 @@ module SolidQueueAutoscaler
50
53
  return quantity
51
54
  end
52
55
 
53
- with_retry do
56
+ with_retry(RETRYABLE_ERRORS, retryable_check: method(:retryable_error?)) do
54
57
  client.formation.update(app_name, process_type, { quantity: quantity })
55
58
  end
56
59
  quantity
57
60
  rescue Excon::Error => e
61
+ # Handle 404 by trying to create the formation via batch_update
62
+ # This happens when scaling up a dyno type that was previously scaled to 0
63
+ if e.respond_to?(:response) && e.response&.status == 404
64
+ return create_formation(quantity)
65
+ end
66
+
58
67
  raise HerokuAPIError.new(
59
68
  "Failed to scale #{process_type} to #{quantity}: #{e.message}",
60
69
  status_code: e.respond_to?(:response) ? e.response&.status : nil,
@@ -88,22 +97,29 @@ module SolidQueueAutoscaler
88
97
 
89
98
  private
90
99
 
91
- # Executes a block with retry logic for transient network errors.
92
- # Uses exponential backoff: 1s, 2s, 4s delays between retries.
93
- def with_retry
94
- attempts = 0
95
- begin
96
- attempts += 1
97
- yield
98
- rescue *RETRYABLE_ERRORS => e
99
- if attempts < MAX_RETRIES && retryable_error?(e)
100
- delay = RETRY_DELAYS[attempts - 1] || RETRY_DELAYS.last
101
- logger&.warn("[Autoscaler] Heroku API error (attempt #{attempts}/#{MAX_RETRIES}), retrying in #{delay}s: #{e.message}")
102
- sleep(delay)
103
- retry
104
- end
105
- raise
100
+ # Creates a formation that doesn't exist using batch_update.
101
+ # This is needed when scaling up a dyno type that was previously scaled to 0.
102
+ #
103
+ # @param quantity [Integer] desired worker count
104
+ # @return [Integer] the new worker count
105
+ # @raise [HerokuAPIError] if the API call fails
106
+ def create_formation(quantity)
107
+ logger&.info("[Autoscaler] Formation '#{process_type}' not found, creating with quantity #{quantity}")
108
+
109
+ with_retry(RETRYABLE_ERRORS, retryable_check: method(:retryable_error?)) do
110
+ client.formation.batch_update(app_name, {
111
+ updates: [
112
+ { type: process_type, quantity: quantity }
113
+ ]
114
+ })
106
115
  end
116
+ quantity
117
+ rescue Excon::Error => e
118
+ raise HerokuAPIError.new(
119
+ "Failed to create formation #{process_type} with quantity #{quantity}: #{e.message}",
120
+ status_code: e.respond_to?(:response) ? e.response&.status : nil,
121
+ response_body: e.respond_to?(:response) ? e.response&.body : nil
122
+ )
107
123
  end
108
124
 
109
125
  # Determines if an error should be retried.
@@ -1,5 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require 'net/http'
4
+
3
5
  module SolidQueueAutoscaler
4
6
  module Adapters
5
7
  # Kubernetes adapter for scaling Deployment replicas.
@@ -30,15 +32,21 @@ module SolidQueueAutoscaler
30
32
  # Kubernetes API path for apps/v1 group
31
33
  APPS_API_VERSION = 'apis/apps/v1'
32
34
 
33
- # Retry configuration for transient network errors
34
- MAX_RETRIES = 3
35
- RETRY_DELAYS = [1, 2, 4].freeze # Exponential backoff in seconds
36
-
37
35
  # Default timeout for Kubernetes API calls (seconds)
38
36
  DEFAULT_TIMEOUT = 30
39
37
 
38
+ # Errors that are safe to retry (transient network issues)
39
+ RETRYABLE_ERRORS = [
40
+ Errno::ECONNREFUSED,
41
+ Errno::ETIMEDOUT,
42
+ Errno::ECONNRESET,
43
+ Net::OpenTimeout,
44
+ Net::ReadTimeout,
45
+ SocketError
46
+ ].freeze
47
+
40
48
  def current_workers
41
- with_retry do
49
+ with_retry(RETRYABLE_ERRORS) do
42
50
  deployment = apps_client.get_deployment(deployment_name, namespace)
43
51
  deployment.spec.replicas
44
52
  end
@@ -52,7 +60,7 @@ module SolidQueueAutoscaler
52
60
  return quantity
53
61
  end
54
62
 
55
- with_retry do
63
+ with_retry(RETRYABLE_ERRORS) do
56
64
  patch_body = { spec: { replicas: quantity } }
57
65
  apps_client.patch_deployment(deployment_name, patch_body, namespace)
58
66
  end
@@ -75,25 +83,6 @@ module SolidQueueAutoscaler
75
83
 
76
84
  private
77
85
 
78
- # Executes a block with retry logic for transient network errors.
79
- # Uses exponential backoff: 1s, 2s, 4s delays between retries.
80
- def with_retry
81
- attempts = 0
82
- begin
83
- attempts += 1
84
- yield
85
- rescue Errno::ECONNREFUSED, Errno::ETIMEDOUT, Errno::ECONNRESET,
86
- Net::OpenTimeout, Net::ReadTimeout, SocketError => e
87
- if attempts < MAX_RETRIES
88
- delay = RETRY_DELAYS[attempts - 1] || RETRY_DELAYS.last
89
- logger&.warn("[Autoscaler] Kubernetes API error (attempt #{attempts}/#{MAX_RETRIES}), retrying in #{delay}s: #{e.message}")
90
- sleep(delay)
91
- retry
92
- end
93
- raise
94
- end
95
- end
96
-
97
86
  def apps_client
98
87
  @apps_client ||= build_apps_client
99
88
  end
@@ -1,12 +1,14 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require 'zlib'
4
+ require 'socket'
4
5
 
5
6
  module SolidQueueAutoscaler
6
- # PostgreSQL advisory lock wrapper for singleton enforcement.
7
+ # Advisory lock wrapper for singleton enforcement.
8
+ # Supports both PostgreSQL (native advisory locks) and SQLite (table-based locks).
7
9
  #
8
- # IMPORTANT: PgBouncer Compatibility Warning
9
- # ==========================================
10
+ # IMPORTANT: PgBouncer Compatibility Warning (PostgreSQL only)
11
+ # ============================================================
10
12
  # PostgreSQL advisory locks are connection-scoped (session-level locks).
11
13
  # If you're using PgBouncer in transaction pooling mode, advisory locks
12
14
  # will NOT work correctly because:
@@ -24,6 +26,10 @@ module SolidQueueAutoscaler
24
26
  # lock acquisition always failing, PgBouncer is likely the cause.
25
27
  #
26
28
  class AdvisoryLock
29
+ LOCKS_TABLE_NAME = 'solid_queue_autoscaler_locks'
30
+ # Stale lock timeout - locks older than this are considered abandoned (5 minutes)
31
+ STALE_LOCK_TIMEOUT_SECONDS = 300
32
+
27
33
  attr_reader :lock_key, :timeout
28
34
 
29
35
  def initialize(lock_key: nil, timeout: nil, config: nil)
@@ -31,6 +37,7 @@ module SolidQueueAutoscaler
31
37
  @lock_key = lock_key || @config.lock_key
32
38
  @timeout = timeout || @config.lock_timeout_seconds
33
39
  @lock_acquired = false
40
+ @strategy = nil
34
41
  end
35
42
 
36
43
  def with_lock
@@ -43,20 +50,14 @@ module SolidQueueAutoscaler
43
50
  def try_lock
44
51
  return false if @lock_acquired
45
52
 
46
- result = connection.select_value(
47
- "SELECT pg_try_advisory_lock(#{lock_id})"
48
- )
49
- @lock_acquired = [true, 't'].include?(result)
53
+ @lock_acquired = lock_strategy.try_lock
50
54
  @lock_acquired
51
55
  end
52
56
 
53
57
  def acquire!
54
58
  return true if @lock_acquired
55
59
 
56
- result = connection.select_value(
57
- "SELECT pg_try_advisory_lock(#{lock_id})"
58
- )
59
- @lock_acquired = [true, 't'].include?(result)
60
+ @lock_acquired = lock_strategy.try_lock
60
61
 
61
62
  raise LockError, "Could not acquire advisory lock '#{lock_key}' (id: #{lock_id})" unless @lock_acquired
62
63
 
@@ -66,7 +67,7 @@ module SolidQueueAutoscaler
66
67
  def release
67
68
  return false unless @lock_acquired
68
69
 
69
- connection.execute("SELECT pg_advisory_unlock(#{lock_id})")
70
+ lock_strategy.release
70
71
  @lock_acquired = false
71
72
  true
72
73
  end
@@ -87,5 +88,160 @@ module SolidQueueAutoscaler
87
88
  hash & 0x7FFFFFFF
88
89
  end
89
90
  end
91
+
92
+ def lock_strategy
93
+ @strategy ||= create_lock_strategy
94
+ end
95
+
96
+ def create_lock_strategy
97
+ adapter_name = connection.adapter_name.downcase
98
+
99
+ case adapter_name
100
+ when /postgresql/, /postgis/
101
+ PostgreSQLLockStrategy.new(connection: connection, lock_id: lock_id, lock_key: lock_key)
102
+ when /sqlite/
103
+ SQLiteLockStrategy.new(connection: connection, lock_id: lock_id, lock_key: lock_key)
104
+ when /mysql/, /trilogy/
105
+ MySQLLockStrategy.new(connection: connection, lock_id: lock_id, lock_key: lock_key)
106
+ else
107
+ # Fall back to table-based locking for unknown adapters
108
+ TableBasedLockStrategy.new(connection: connection, lock_id: lock_id, lock_key: lock_key)
109
+ end
110
+ end
111
+
112
+ # Base class for lock strategies
113
+ class BaseLockStrategy
114
+ def initialize(connection:, lock_id:, lock_key:)
115
+ @connection = connection
116
+ @lock_id = lock_id
117
+ @lock_key = lock_key
118
+ end
119
+
120
+ def try_lock
121
+ raise NotImplementedError, "#{self.class} must implement #try_lock"
122
+ end
123
+
124
+ def release
125
+ raise NotImplementedError, "#{self.class} must implement #release"
126
+ end
127
+
128
+ protected
129
+
130
+ attr_reader :connection, :lock_id, :lock_key
131
+ end
132
+
133
+ # PostgreSQL native advisory locks
134
+ class PostgreSQLLockStrategy < BaseLockStrategy
135
+ def try_lock
136
+ result = connection.select_value(
137
+ "SELECT pg_try_advisory_lock(#{lock_id})"
138
+ )
139
+ [true, 't'].include?(result)
140
+ end
141
+
142
+ def release
143
+ connection.execute("SELECT pg_advisory_unlock(#{lock_id})")
144
+ true
145
+ end
146
+ end
147
+
148
+ # MySQL named locks (GET_LOCK/RELEASE_LOCK)
149
+ class MySQLLockStrategy < BaseLockStrategy
150
+ def try_lock
151
+ # MySQL GET_LOCK returns 1 on success, 0 if timeout, NULL on error
152
+ result = connection.select_value(
153
+ "SELECT GET_LOCK(#{connection.quote(lock_key)}, 0)"
154
+ )
155
+ result == 1
156
+ end
157
+
158
+ def release
159
+ connection.execute("SELECT RELEASE_LOCK(#{connection.quote(lock_key)})")
160
+ true
161
+ end
162
+ end
163
+
164
+ # Table-based locking for databases without native advisory lock support
165
+ # Uses a simple locks table with INSERT/DELETE for lock management
166
+ class TableBasedLockStrategy < BaseLockStrategy
167
+ def try_lock
168
+ ensure_locks_table_exists!
169
+ cleanup_stale_locks!
170
+
171
+ # Try to insert a lock record
172
+ begin
173
+ connection.execute(<<~SQL)
174
+ INSERT INTO #{quoted_table_name} (lock_key, lock_id, locked_at, locked_by)
175
+ VALUES (#{connection.quote(lock_key)}, #{lock_id}, #{connection.quote(Time.now.utc.iso8601)}, #{connection.quote(lock_owner)})
176
+ SQL
177
+ true
178
+ rescue ActiveRecord::RecordNotUnique, ActiveRecord::StatementInvalid => e
179
+ # Lock already held by another process
180
+ # StatementInvalid catches SQLite's UNIQUE constraint violation
181
+ return false if e.message.include?('UNIQUE') || e.message.include?('duplicate')
182
+
183
+ raise
184
+ end
185
+ end
186
+
187
+ def release
188
+ return true unless table_exists?
189
+
190
+ connection.execute(<<~SQL)
191
+ DELETE FROM #{quoted_table_name}
192
+ WHERE lock_key = #{connection.quote(lock_key)}
193
+ AND locked_by = #{connection.quote(lock_owner)}
194
+ SQL
195
+ true
196
+ end
197
+
198
+ private
199
+
200
+ def ensure_locks_table_exists!
201
+ return if table_exists?
202
+
203
+ create_locks_table!
204
+ end
205
+
206
+ def table_exists?
207
+ @table_exists ||= connection.table_exists?(LOCKS_TABLE_NAME)
208
+ end
209
+
210
+ def create_locks_table!
211
+ connection.execute(<<~SQL)
212
+ CREATE TABLE IF NOT EXISTS #{quoted_table_name} (
213
+ lock_key VARCHAR(255) NOT NULL PRIMARY KEY,
214
+ lock_id INTEGER NOT NULL,
215
+ locked_at DATETIME NOT NULL,
216
+ locked_by VARCHAR(255) NOT NULL
217
+ )
218
+ SQL
219
+ @table_exists = true
220
+ end
221
+
222
+ def cleanup_stale_locks!
223
+ # Remove locks older than STALE_LOCK_TIMEOUT_SECONDS
224
+ stale_threshold = (Time.now.utc - STALE_LOCK_TIMEOUT_SECONDS).iso8601
225
+ connection.execute(<<~SQL)
226
+ DELETE FROM #{quoted_table_name}
227
+ WHERE locked_at < #{connection.quote(stale_threshold)}
228
+ SQL
229
+ end
230
+
231
+ def quoted_table_name
232
+ connection.quote_table_name(LOCKS_TABLE_NAME)
233
+ end
234
+
235
+ def lock_owner
236
+ # Unique identifier for this process/thread
237
+ @lock_owner ||= "#{Socket.gethostname}:#{Process.pid}:#{Thread.current.object_id}"
238
+ end
239
+ end
240
+
241
+ # SQLite table-based locking (SQLite doesn't have advisory locks)
242
+ # Defined after TableBasedLockStrategy since it inherits from it
243
+ class SQLiteLockStrategy < TableBasedLockStrategy
244
+ # Inherits all behavior from TableBasedLockStrategy
245
+ end
90
246
  end
91
247
  end
@@ -2,15 +2,17 @@
2
2
 
3
3
  module SolidQueueAutoscaler
4
4
  class AutoscaleJob < ActiveJob::Base
5
- # IMPORTANT: Use a static queue name so SolidQueue recurring jobs work correctly.
6
- # When using SolidQueue recurring.yml without specifying queue:, SolidQueue
7
- # checks the job class's queue_name attribute. A dynamic queue_as block
8
- # returns a Proc that isn't evaluated by recurring jobs, causing jobs to
9
- # go to 'default' queue instead.
5
+ # Default queue - this MUST be set here (not dynamically) because SolidQueue
6
+ # recurring jobs capture the queue name during initialization, BEFORE
7
+ # Rails after_initialize hooks run.
10
8
  #
11
- # To use a custom queue:
12
- # 1. Set queue: in your recurring.yml (recommended)
13
- # 2. Or use AutoscaleJob.set(queue: :my_queue).perform_later
9
+ # The apply_job_settings! method can override this after Rails initializers
10
+ # run, but the default must be set here for SolidQueue recurring to work.
11
+ #
12
+ # You can customize the queue via:
13
+ # config.job_queue = :my_queue
14
+ #
15
+ # For SolidQueue recurring.yml, you can also set queue: directly in the YAML.
14
16
  queue_as :autoscaler
15
17
 
16
18
  discard_on ConfigurationError
@@ -11,6 +11,11 @@ module SolidQueueAutoscaler
11
11
  # Configuration happens via initializer, nothing to do here
12
12
  end
13
13
 
14
+ # After all initializers have run, apply job settings from configuration
15
+ config.after_initialize do
16
+ SolidQueueAutoscaler.apply_job_settings!
17
+ end
18
+
14
19
  rake_tasks do
15
20
  namespace :solid_queue_autoscaler do
16
21
  desc 'Run the autoscaler once for a specific worker (default: :default). Use WORKER=name'
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SolidQueueAutoscaler
4
- VERSION = '1.0.11'
4
+ VERSION = '1.0.15'
5
5
  end
@@ -100,6 +100,33 @@ module SolidQueueAutoscaler
100
100
  end
101
101
  end
102
102
 
103
+ # Apply job settings (queue, priority) from configuration to AutoscaleJob.
104
+ # Called automatically after Rails initializers run via the railtie.
105
+ # Uses the first configured worker's job_queue/job_priority settings.
106
+ def apply_job_settings!
107
+ return unless defined?(AutoscaleJob)
108
+ return if configurations.empty?
109
+
110
+ # Use the first configured worker's settings
111
+ first_config = configurations.values.first
112
+ job_queue = first_config&.job_queue || :autoscaler
113
+ job_priority = first_config&.job_priority
114
+
115
+ # Set the queue_name class attribute directly (not via queue_as block)
116
+ # This ensures SolidQueue recurring jobs pick up the correct queue
117
+ # Convert to string since ActiveJob internally uses strings for queue names
118
+ AutoscaleJob.queue_name = job_queue.to_s
119
+
120
+ # Set priority if configured
121
+ if job_priority && AutoscaleJob.respond_to?(:priority=)
122
+ AutoscaleJob.priority = job_priority
123
+ end
124
+
125
+ first_config&.logger&.debug(
126
+ "[SolidQueueAutoscaler] AutoscaleJob configured: queue=#{job_queue}, priority=#{job_priority || 'default'}"
127
+ )
128
+ end
129
+
103
130
  # Verify the installation is complete and working.
104
131
  # Prints a human-friendly report (when verbose: true) and returns a VerificationResult.
105
132
  #
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: solid_queue_autoscaler
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.11
4
+ version: 1.0.15
5
5
  platform: ruby
6
6
  authors:
7
7
  - reillyse
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2026-01-17 00:00:00.000000000 Z
11
+ date: 2026-01-30 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activerecord
@@ -122,6 +122,20 @@ dependencies:
122
122
  - - "~>"
123
123
  - !ruby/object:Gem::Version
124
124
  version: '3.18'
125
+ - !ruby/object:Gem::Dependency
126
+ name: sqlite3
127
+ requirement: !ruby/object:Gem::Requirement
128
+ requirements:
129
+ - - ">="
130
+ - !ruby/object:Gem::Version
131
+ version: '0'
132
+ type: :development
133
+ prerelease: false
134
+ version_requirements: !ruby/object:Gem::Requirement
135
+ requirements:
136
+ - - ">="
137
+ - !ruby/object:Gem::Version
138
+ version: '0'
125
139
  description: A control plane for Solid Queue on Heroku that automatically scales worker
126
140
  dynos based on queue depth, job latency, and throughput. Uses PostgreSQL advisory
127
141
  locks for singleton behavior and the Heroku Platform API for scaling.
@@ -143,6 +157,7 @@ files:
143
157
  - lib/generators/solid_queue_autoscaler/migration_generator.rb
144
158
  - lib/generators/solid_queue_autoscaler/templates/README
145
159
  - lib/generators/solid_queue_autoscaler/templates/create_solid_queue_autoscaler_events.rb.erb
160
+ - lib/generators/solid_queue_autoscaler/templates/create_solid_queue_autoscaler_locks.rb.erb
146
161
  - lib/generators/solid_queue_autoscaler/templates/create_solid_queue_autoscaler_state.rb.erb
147
162
  - lib/generators/solid_queue_autoscaler/templates/initializer.rb
148
163
  - lib/solid_queue_autoscaler.rb