solid_queue_autoscaler 1.0.15 → 1.0.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: bafb0cda485024f43bd0e73291bcb0bbbc7275e3f897d4a2acafbd6496140b98
4
- data.tar.gz: 4b443c68ca28df2b3efd62a4cd7ae30f6cf53e978c941a1e60aa7ebd0c7befb0
3
+ metadata.gz: 0d2ec8d0897f2312d05ccc10d075e5b90bd7c31e5e699fc34bfff13ba5be513b
4
+ data.tar.gz: b6e26a9f33e0c86f8809c4c52f04059ed62b4f1c3097ef6421a25cafb5eeed72
5
5
  SHA512:
6
- metadata.gz: 943bd220740694c827afc4c3aede77cac6da3b419eda051c686b5fb07cfbb69da28b6b8d5882fcf61122bfc6991352e52b9dab2b7f05982333324efbce20205f
7
- data.tar.gz: a201785b1ecc766f744a7d705f64dd1e7afaa611e004f3fb9f003a28a05a397fc412ec276a17b99c4bbad0cf89ed476884f510986e3f95e9f9385252d7e15d31
6
+ metadata.gz: 4d9cd4937e412fded6a640c9044dfa846b7006848647a3ffa2c78a70fe3f0b03bf1b5f34dc0555f0e5489e0d3c4d23fef575c4e7f16ff421ad2385355ea6919f
7
+ data.tar.gz: 1ed169508ba540f4ec3dcfaf6bcea5d04812f4385c4231f0033a98d10b891d3722f2b0d46184d4bd7ffea9b860b6e70f5b3b07856533104ae06e3d31d09e2be7
data/CHANGELOG.md CHANGED
@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [1.0.16] - 2025-01-30
11
+
12
+ ### Added
13
+ - **Comprehensive scale-to-zero documentation** - Added dedicated "Scale to Zero" section in README:
14
+ - Explains how `min_workers = 0` works with Heroku formation behavior
15
+ - Documents the v1.0.15 fix for graceful 404 handling
16
+ - Includes configuration examples and cold-start latency considerations
17
+ - Guidance on where to run the autoscaler (web dyno vs workers)
18
+ - Updated Features list and linked Cost-Optimized example
19
+
10
20
  ## [1.0.15] - 2025-01-30
11
21
 
12
22
  ### Fixed
data/README.md CHANGED
@@ -10,12 +10,98 @@ A control plane for [Solid Queue](https://github.com/rails/solid_queue) that aut
10
10
  - **Metrics-based scaling**: Scales based on queue depth, job latency, and throughput
11
11
  - **Multiple scaling strategies**: Fixed increment or proportional scaling based on load
12
12
  - **Multi-worker support**: Configure and scale different worker types independently
13
+ - **Scale to zero**: Full support for `min_workers = 0` to eliminate costs during idle periods
13
14
  - **Platform adapters**: Native support for Heroku and Kubernetes
14
15
  - **Singleton execution**: Uses PostgreSQL advisory locks to ensure only one autoscaler runs at a time
15
16
  - **Safety features**: Cooldowns, min/max limits, dry-run mode
16
17
  - **Rails integration**: Configuration via initializer, Railtie with rake tasks
17
18
  - **Flexible execution**: Run as a recurring Solid Queue job or standalone
18
19
 
20
+ ## Scale to Zero
21
+
22
+ The autoscaler fully supports scaling workers to zero (`min_workers = 0`), allowing you to eliminate worker costs during idle periods.
23
+
24
+ ### How It Works
25
+
26
+ When you configure `min_workers = 0` and the queue becomes idle, the autoscaler will scale your workers down to zero. This is ideal for:
27
+
28
+ - **Development/staging environments** with sporadic usage
29
+ - **Batch processing workers** that only run when jobs are queued
30
+ - **Cost-sensitive applications** with predictable idle periods
31
+
32
+ ### Heroku Formation Behavior
33
+
34
+ On Heroku, when a dyno type is scaled to 0, it gets **removed from the formation entirely**. This means:
35
+
36
+ 1. `heroku ps:scale worker=0` removes the `worker` formation
37
+ 2. Subsequent API calls to get formation info return **404 Not Found**
38
+ 3. When scaling back up, the formation must be **recreated**
39
+
40
+ As of **v1.0.15**, the autoscaler handles this gracefully:
41
+
42
+ - When querying a non-existent formation, it returns `0` workers (instead of raising an error)
43
+ - When scaling up a non-existent formation, it automatically creates it using Heroku's batch update API
44
+ - This enables seamless scale-to-zero → scale-up workflows
45
+
46
+ ### Configuration Example
47
+
48
+ ```ruby
49
+ SolidQueueAutoscaler.configure(:batch_worker) do |config|
50
+ config.adapter = :heroku
51
+ config.heroku_api_key = ENV['HEROKU_API_KEY']
52
+ config.heroku_app_name = ENV['HEROKU_APP_NAME']
53
+ config.process_type = 'batch_worker'
54
+
55
+ # Enable scale-to-zero
56
+ config.min_workers = 0
57
+ config.max_workers = 5
58
+
59
+ # Scale up immediately when any job is queued
60
+ config.scale_up_queue_depth = 1
61
+ config.scale_up_latency_seconds = 60
62
+
63
+ # Scale down when completely idle
64
+ config.scale_down_queue_depth = 0
65
+ config.scale_down_latency_seconds = 10
66
+
67
+ # Longer scale-down cooldown to avoid premature scaling to zero
68
+ config.scale_up_cooldown_seconds = 30
69
+ config.scale_down_cooldown_seconds = 300 # 5 minutes
70
+ end
71
+ ```
72
+
73
+ ### Important Considerations
74
+
75
+ **Cold-start latency**: When workers are at zero and a job is enqueued, there will be latency before the job is processed:
76
+ 1. The autoscaler job must run (depends on your `schedule` interval)
77
+ 2. The autoscaler must scale up workers
78
+ 3. Heroku must provision and start the dyno (~10-30 seconds)
79
+ 4. The worker must boot and start processing
80
+
81
+ Total cold-start time is typically **30-90 seconds** depending on your configuration and dyno startup time.
82
+
83
+ **Where to run the autoscaler**: The autoscaler job **must run on a process that's always running** (like your web dyno), NOT on the workers being scaled. If the autoscaler runs on workers and those workers scale to zero, there's nothing to scale them back up!
84
+
85
+ ```yaml
86
+ # config/recurring.yml - runs on whatever process runs the dispatcher
87
+ autoscaler_batch:
88
+ class: SolidQueueAutoscaler::AutoscaleJob
89
+ queue: autoscaler
90
+ schedule: every 30 seconds
91
+ args: [:batch_worker]
92
+ ```
93
+
94
+ **Procfile setup**: Ensure your web dyno runs the Solid Queue dispatcher (or use a dedicated always-on dyno):
95
+
96
+ ```
97
+ # Procfile
98
+ web: bundle exec puma -C config/puma.rb
99
+ worker: bundle exec rake solid_queue:start
100
+ batch_worker: bundle exec rake solid_queue:start
101
+ ```
102
+
103
+ Alternatively, run the dispatcher in a thread within your web process using `solid_queue.yml` configuration.
104
+
19
105
  ## Installation
20
106
 
21
107
  Add to your Gemfile:
@@ -308,7 +394,7 @@ autoscaler:
308
394
 
309
395
  ### Cost-Optimized Setup (Scale to Zero)
310
396
 
311
- For apps with sporadic workloads where you want to minimize costs during idle periods:
397
+ For apps with sporadic workloads where you want to minimize costs during idle periods. See the [Scale to Zero](#scale-to-zero) section for full details on how this works.
312
398
 
313
399
  ```ruby
314
400
  SolidQueueAutoscaler.configure do |config|
@@ -337,7 +423,7 @@ SolidQueueAutoscaler.configure do |config|
337
423
  end
338
424
  ```
339
425
 
340
- **⚠️ Note:** With `min_workers = 0`, there's cold-start latency when the first job arrives. The autoscaler must run on a web dyno or separate process, not on the workers themselves.
426
+ **⚠️ Note:** With `min_workers = 0`, there's cold-start latency (~30-90s) when the first job arrives. The autoscaler must run on a web dyno or separate always-on process, not on the workers themselves. See [Scale to Zero](#scale-to-zero) for details.
341
427
 
342
428
  ---
343
429
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SolidQueueAutoscaler
4
- VERSION = '1.0.15'
4
+ VERSION = '1.0.16'
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: solid_queue_autoscaler
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.15
4
+ version: 1.0.16
5
5
  platform: ruby
6
6
  authors:
7
7
  - reillyse
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2026-01-30 00:00:00.000000000 Z
11
+ date: 2026-01-31 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activerecord