async-background 0.6.0 → 0.6.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b7e4015afbfcd9ee8291c28caa34e69de838f283fd0d6374a81cc0d53a87b90d
4
- data.tar.gz: b0333ec4f4626895e01c2b77da17abb569c912e6449def849fa8e0918fb06a58
3
+ metadata.gz: f1890dfac1632afedbbb20c367d4d770c3037f6201bb6fac77396f07481df477
4
+ data.tar.gz: 1b435f70b2fd290bd569c0c081d19ae94250a181367ada7d3b4d157c213fd5b8
5
5
  SHA512:
6
- metadata.gz: 245fa669ebf1573e37770eb16f86904921cbe0f0effb40099c613bda9732e4e8732854f24cdf84421efa877d69e25271544bc279bf5e35ff3c7e364cd8689252
7
- data.tar.gz: aff2eeba3e4793b4d5e686fb5bf3f27c185f61492aee837d7100a9dde35c3d97cf904c0c037c4dacd1655dd0da49b9f556d5003620e0e0f2c256735539805f29
6
+ metadata.gz: 0e9c2398241b48d0cc8fe66ed0b227cca3c93a5969e7f753c0a8dbb08a41462626d73709ae9eece1a254f8567f4a9044958e96a2658a77ab8f6b11bbdb6a1c14
7
+ data.tar.gz: 702016f46a3bbaa255735defcc2746baafe66a26400d2ad9525b80d89848970143998a706ceb4644519433cfcd558d9936c0caae4f968e335a2f8b8496afdec1
data/CHANGELOG.md CHANGED
@@ -1,5 +1,41 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.6.2
4
+
5
+ ### Features
6
+ - **Configurable timeout for queue jobs** — queue jobs previously used a hardcoded 30-second timeout (`DEFAULT_TIMEOUT`). Now configurable via `options` hash at two levels:
7
+ ```ruby
8
+ # Class-level default
9
+ class HeavyImportJob
10
+ include Async::Background::Job
11
+ options timeout: 600
12
+
13
+ def perform(user_id) = # ...
14
+ end
15
+
16
+ # Call-site override (wins over class-level)
17
+ HeavyImportJob.perform_async(user_id, options: { timeout: 120 })
18
+ ```
19
+ Priority: call-site `options:` → class-level `options` → `DEFAULT_TIMEOUT` (30s). Options are merged at enqueue time so the runner simply reads the final value from the payload
20
+ - **`options:` hash across the entire enqueue chain** — single extensible contract from `perform_async` through `Client` down to `Store`. Currently supports `:timeout`, designed to accommodate future keys (e.g. `:retry`) without API changes
21
+ - **`Job::Options` schema via `Data.define`** — declares known option keys with types and defaults. Unknown keys raise `ArgumentError`, invalid types raise `TypeError`. No manual validation code
22
+ - **`options TEXT` column in SQLite** — stores the merged options hash as JSON. Extensible without schema changes when new options are added
23
+
24
+ ### Improvements
25
+ - **Queue timeout logged on failure** — `run_queue_job` error log now includes actual timeout value: `"timed out after 120s"` instead of generic `"timed out"`
26
+ - **Idempotent schema migration** — existing databases get `ALTER TABLE jobs ADD COLUMN options TEXT` on first connection, wrapped in `rescue nil` for safe re-runs. New databases include the column in `CREATE TABLE`
27
+
28
+ ## 0.6.1
29
+
30
+ ### Bug Fixes
31
+ - **Runner: cron jobs busy-loop on overlap skip** — when a scheduled run was skipped because the previous one was still active, the entry was re-pushed to the heap without calling `reschedule`. For cron jobs (where `interval` is `nil`), this meant `next_run_at` was never advanced to the next cron tick, causing the entry to be picked up again immediately on the next loop iteration. Skip branch now calls `entry.reschedule(monotonic_now)` like the normal path
32
+ - **Store: prepared statement not reset on fetch error** — `@fetch_stmt.reset!` was called after `execute` returned, so an exception inside `execute` left the statement in a dirty state and the next `fetch` could fail. Wrapped in `begin/ensure` to guarantee reset
33
+
34
+ ### Improvements
35
+ - **SocketNotifier: non-blocking enqueue with ring fallback** — `notify_all` no longer connects to all N worker sockets on every enqueue. `UNIXSocket.new` is a blocking, non-fiber-aware syscall, and notifying every worker blocked the Falcon reactor for N `connect()` calls on the hot HTTP enqueue path. Now wakes a single worker chosen by random offset, falling back through the ring only if the chosen worker is dead (`ECONNREFUSED` etc.). Happy path: 1 connect. Worst case (all workers down): N connects — same as before, but only when actually needed. Safe because the queue is shared in SQLite, not sharded per worker
36
+ - **SocketNotifier: cleaned up `UNAVAILABLE` error list** — removed `IO::WaitWritable` and `Errno::EAGAIN`. They implied "socket buffer full", but `write_nonblock` of a single byte to a freshly-opened connection cannot fill the kernel buffer. Listing them only misled readers
37
+ - **Store: partial index for pending lookup** — replaced `idx_jobs_status_run_at_id(status, run_at, id)` with partial index `idx_jobs_pending(run_at, id) WHERE status = 'pending'`. Smaller on disk, cheaper to update, and matches the only query that uses it (`fetch`). `done`/`failed`/`running` rows no longer occupy index pages
38
+
3
39
  ## 0.6.0
4
40
 
5
41
  ### Breaking Changes
data/README.md CHANGED
@@ -1,76 +1,74 @@
1
1
  # Async::Background
2
2
 
3
- A lightweight, production-grade cron/interval scheduler for Ruby's [Async](https://github.com/socketry/async) ecosystem. Designed for [Falcon](https://github.com/socketry/falcon) but works with any Async-based application.
3
+ A lightweight cron, interval, and job-queue scheduler for Ruby's [Async](https://github.com/socketry/async) ecosystem. Built for [Falcon](https://github.com/socketry/falcon), works with any Async app.
4
4
 
5
- ## What It Does
6
-
7
- - **Cron & interval scheduling**single event loop + min-heap, scales to hundreds of jobs
8
- - **Dynamic job queue** — enqueue jobs at runtime via SQLite, pick up by background workers
9
- - **Delayed jobs** schedule jobs for future execution with `perform_in` / `perform_at` (Sidekiq-like API)
10
- - **Multi-process safe** — deterministic worker sharding via `Zlib.crc32`, no duplicate execution
11
- - **Skip overlapping** — if a job is still running when its next tick arrives, the tick is skipped
12
- - **Timeout protection** — per-job configurable timeout via `Async::Task#with_timeout`
13
- - **Startup jitter** — random delay to prevent thundering herd after restart
14
- - **Optional metrics** — shared memory performance tracking with `async-utilization`
5
+ - **Cron & interval scheduling** on a single event loop with a min-heap
6
+ - **Dynamic job queue** backed by SQLite, with delayed jobs (`perform_in` / `perform_at`)
7
+ - **Cross-process wake-ups** over Unix domain sockets web workers can enqueue and instantly wake background workers
8
+ - **Multi-process safe** — deterministic worker sharding, no duplicate execution
9
+ - **Per-job timeouts**, skip-on-overlap, startup jitter, optional metrics
15
10
 
16
11
  ## Requirements
17
12
 
18
- - **Ruby >= 3.3** — Fiber Scheduler production-ready ([why?](#why-ruby-33))
19
- - **Async ~> 2.0** Fiber Scheduler-based concurrency
20
- - **Fugit ~> 1.0** cron expression parsing
13
+ - Ruby >= 3.3
14
+ - `async ~> 2.0`, `fugit ~> 1.0`
15
+ - `sqlite3 ~> 2.0` (optional, for the job queue)
16
+ - `async-utilization ~> 0.3` (optional, for metrics)
21
17
 
22
- ## Installation
18
+ ## Install
23
19
 
24
20
  ```ruby
25
21
  # Gemfile
26
22
  gem "async-background"
27
-
28
- # Optional
29
- gem "sqlite3", "~> 2.0" # for dynamic job queue
30
- gem "async-utilization", "~> 0.3" # for metrics
23
+ gem "sqlite3", "~> 2.0" # optional
24
+ gem "async-utilization", "~> 0.3" # optional
31
25
  ```
32
26
 
33
27
  ## ➡️ [Get Started](docs/GET_STARTED.md)
34
28
 
35
- Step-by-step setup guide: schedule config, Falcon integration, Docker, dynamic queue, delayed jobs.
29
+ Full setup walkthrough: schedule config, Falcon integration, Docker, queue, delayed jobs.
36
30
 
37
31
  ---
38
32
 
39
- ## Quick Example: Job Module
40
-
41
- Include `Async::Background::Job` for a Sidekiq-like interface:
33
+ ## Quick Look
42
34
 
43
35
  ```ruby
44
36
  class SendEmailJob
45
37
  include Async::Background::Job
46
38
 
47
39
  def perform(user_id, template)
48
- user = User.find(user_id)
49
- Mailer.send(user, template)
40
+ Mailer.send(User.find(user_id), template)
50
41
  end
51
42
  end
52
43
 
53
- # Immediate execution in the queue
54
44
  SendEmailJob.perform_async(user_id, "welcome")
55
-
56
- # Execute after 5 minutes
57
45
  SendEmailJob.perform_in(300, user_id, "reminder")
58
-
59
- # Execute at a specific time
60
- SendEmailJob.perform_at(Time.new(2026, 4, 1, 9, 0, 0), user_id, "scheduled")
46
+ SendEmailJob.perform_at(Time.new(2026, 4, 1, 9), user_id, "scheduled")
61
47
  ```
62
48
 
63
- Or use the lower-level API directly:
49
+ Schedule recurring jobs in `config/schedule.yml`:
64
50
 
65
- ```ruby
66
- Async::Background::Queue.enqueue(SendEmailJob, user_id, "welcome")
67
- Async::Background::Queue.enqueue_in(300, SendEmailJob, user_id, "reminder")
68
- Async::Background::Queue.enqueue_at(Time.new(2026, 4, 1, 9, 0, 0), SendEmailJob, user_id, "scheduled")
51
+ ```yaml
52
+ sync_products:
53
+ class: SyncProductsJob
54
+ every: 60
55
+
56
+ daily_report:
57
+ class: DailyReportJob
58
+ cron: "0 3 * * *"
59
+ timeout: 120
69
60
  ```
70
61
 
62
+ | Key | Description |
63
+ |---|---|
64
+ | `class` | Job class — must include `Async::Background::Job` |
65
+ | `every` / `cron` | One of: interval in seconds, or cron expression |
66
+ | `timeout` | Max execution time in seconds (default: 30) |
67
+ | `worker` | Pin to a specific worker. Default: `crc32(name) % total_workers` |
68
+
71
69
  ---
72
70
 
73
- ## ⚠️ Important Notes
71
+ ## Gotchas
74
72
 
75
73
  ### Docker: SQLite requires a named volume
76
74
 
@@ -78,18 +76,20 @@ The SQLite database **must not** live on Docker's `overlay2` filesystem. The `ov
78
76
 
79
77
  ```yaml
80
78
  # docker-compose.yml
81
- volumes:
82
- - queue-data:/app/tmp/queue # ← named volume, NOT overlay2
79
+ services:
80
+ app:
81
+ volumes:
82
+ - queue-data:/app/tmp/queue # ← named volume, NOT overlay2
83
83
 
84
84
  volumes:
85
85
  queue-data:
86
86
  ```
87
87
 
88
- Without this, you will get database crashes in multi-process mode. See [Get Started → Step 3](docs/GET_STARTED.md#step-3-docker) for details.
88
+ Without this, you will get database crashes in multi-process mode. See [Get Started → Step 3](docs/GET_STARTED.md#step-3-docker) for details. If you can't use a named volume, pass `queue_mmap: false` to disable mmap entirely.
89
89
 
90
- ### Fork safety
90
+ ### Other gotchas
91
91
 
92
- SQLite connections **must not** cross `fork()` boundaries. Always open connections **after** fork (inside `container.run` block), never before. The gem handles this internally via lazy `ensure_connection`, but if you create a `Queue::Store` manually for schema setup, close it before fork:
92
+ **Don't share SQLite connections across `fork()`.** The gem opens connections lazily after fork, but if you create a `Queue::Store` manually for schema setup, close it before forking:
93
93
 
94
94
  ```ruby
95
95
  store = Async::Background::Queue::Store.new(path: db_path)
@@ -97,96 +97,49 @@ store.ensure_database!
97
97
  store.close # ← before fork
98
98
  ```
99
99
 
100
- ### Clock handling
101
-
102
- The `Clock` module provides shared time helpers used across the codebase:
103
-
104
- - **`monotonic_now`** (`CLOCK_MONOTONIC`) — for in-process intervals and durations, immune to NTP drift / wall-clock jumps
105
- - **`realtime_now`** (`CLOCK_REALTIME`) — for persisted timestamps (SQLite `run_at`, `created_at`, `locked_at`)
106
-
107
- Interval jobs use monotonic clock. Cron jobs use `Time.now` because "every day at 3am" must respect real time. These are different clocks by design.
100
+ **Two clocks, on purpose.** Interval jobs use `CLOCK_MONOTONIC` (immune to NTP drift). Cron jobs use wall-clock time, because "every day at 3am" needs to mean 3am.
108
101
 
109
102
  ---
110
103
 
111
- ## Architecture
104
+ ## How it works
112
105
 
113
106
  ```
114
- schedule.yml
115
-
116
-
117
- build_heap ← parse config, validate, assign workers
118
-
119
-
120
- MinHeap<Entry> ← O(log N) push/pop, sorted by next_run_at
121
-
122
-
123
- 1 scheduler loop ← single Async task, sleeps until next entry
124
-
125
-
126
- Semaphore ← limits concurrent job execution
127
-
128
-
129
- run_job ← timeout, logging, error handling
107
+ schedule.yml ─► build_heap ─► MinHeap<Entry> ─► scheduler loop ─► Semaphore ─► run_job
130
108
  ```
131
109
 
132
- ### Queue Architecture
110
+ A single Async task sleeps until the next entry is due, then dispatches it under a semaphore that caps concurrency. Overlapping ticks are skipped and rescheduled.
111
+
112
+ The dynamic queue runs alongside it:
133
113
 
134
114
  ```
135
- Producer (web/console) Consumer (background worker)
136
-
137
-
138
- Queue::Client Queue::Store.fetch
139
- (WHERE run_at <= now)
140
- ├─ push(class, args, run_at)
141
- ├─ push_in(delay, class, args)
142
- └─ push_at(time, class, args) run_queue_job
143
-
144
- ▼ ▼
145
- Queue::Store ──── SQLite ──── Queue::Notifier
146
- (INSERT job) (jobs table) (IO.pipe wakeup)
115
+ Producer (web/console) Consumer (background worker)
116
+
117
+
118
+ Queue::Client Queue::Store#fetch
119
+ push / push_in / push_at (run_at <= now)
120
+
121
+
122
+ Queue::Store ──── SQLite (jobs) ──── SocketWaker
123
+
124
+ └───────► SocketNotifier ───────────────┘
125
+ (UNIX socket wake-up, ~80µs)
147
126
  ```
148
127
 
149
- ## Schedule Config
150
-
151
- | Key | Required | Description |
152
- |---|---|---|
153
- | `class` | yes | Must include `Async::Background::Job` |
154
- | `every` | one of | Interval in seconds between runs |
155
- | `cron` | one of | Cron expression (parsed by Fugit) |
156
- | `timeout` | no | Max execution time in seconds (default: 30) |
157
- | `worker` | no | Pin to specific worker index. If omitted — `crc32(name) % total_workers` |
158
-
159
- ## SQLite Pragmas
160
-
161
- | Pragma | Value | Why |
162
- |---|---|---|
163
- | `journal_mode` | WAL | Concurrent reads during writes |
164
- | `synchronous` | NORMAL | Safe with WAL, lower fsync overhead |
165
- | `mmap_size` | 256 MB | Fast reads ([requires proper filesystem](#docker-sqlite-requires-a-named-volume)) |
166
- | `cache_size` | 16000 pages | ~64 MB page cache |
167
- | `busy_timeout` | 5000 ms | Wait instead of failing on lock contention |
128
+ Jobs are persisted in SQLite, so a missed wake-up is never a lost job — workers also poll every 5 seconds as a safety net.
168
129
 
169
130
  ## Metrics
170
131
 
171
- When `async-utilization` gem is available, metrics are collected in shared memory (`/tmp/async-background.shm`) with lock-free updates per worker.
132
+ With `async-utilization` installed, per-worker stats land in shared memory at `/tmp/async-background.shm` with lock-free updates.
172
133
 
173
134
  ```ruby
174
135
  runner.metrics.values
175
136
  # => { total_runs: 142, total_successes: 140, total_failures: 2,
176
- # total_timeouts: 0, total_skips: 5, active_jobs: 1,
177
- # last_run_at: 1774445243, last_duration_ms: 1250 }
137
+ # total_timeouts: 0, total_skips: 5, active_jobs: 1, ... }
178
138
 
179
- # Read all workers at once (no server needed)
180
139
  Async::Background::Metrics.read_all(total_workers: 2)
181
140
  ```
182
141
 
183
- Without the gem metrics are silently disabled, zero overhead.
184
-
185
- ## Why Ruby 3.3?
186
-
187
- - Ruby 3.0 introduced Fiber Scheduler but had critical bugs
188
- - Ruby 3.2 is the first production-ready release (per Samuel Williams)
189
- - `io-event >= 1.14` (pulled by latest `async`) requires Ruby `>= 3.3`
142
+ Without the gem, metrics are silently disabled zero overhead.
190
143
 
191
144
  ## License
192
145
 
@@ -3,6 +3,12 @@
3
3
  module Async
4
4
  module Background
5
5
  module Job
6
+ DEFAULT_TIMEOUT = 120
7
+
8
+ Options = Data.define(:timeout) do
9
+ def initialize(timeout: DEFAULT_TIMEOUT) = super(timeout: Integer(timeout))
10
+ end
11
+
6
12
  def self.included(base)
7
13
  base.extend(ClassMethods)
8
14
  end
@@ -12,17 +18,23 @@ module Async
12
18
  new.perform(*args)
13
19
  end
14
20
 
15
- def perform_async(*args)
16
- Async::Background::Queue.enqueue(self, *args)
21
+ def perform_async(*args, options: {})
22
+ Async::Background::Queue.enqueue(self, *args, options: options)
17
23
  end
18
24
 
19
- def perform_in(delay, *args)
20
- Async::Background::Queue.enqueue_in(delay, self, *args)
25
+ def perform_in(delay, *args, options: {})
26
+ Async::Background::Queue.enqueue_in(delay, self, *args, options: options)
21
27
  end
22
28
 
23
- def perform_at(time, *args)
24
- Async::Background::Queue.enqueue_at(time, self, *args)
29
+ def perform_at(time, *args, options: {})
30
+ Async::Background::Queue.enqueue_at(time, self, *args, options: options)
25
31
  end
32
+
33
+ def options(**values)
34
+ @options = Options.new(**values).to_h
35
+ end
36
+
37
+ def resolve_options = @options || {}
26
38
  end
27
39
 
28
40
  def perform(*args)
@@ -5,6 +5,8 @@ require_relative '../clock'
5
5
  module Async
6
6
  module Background
7
7
  module Queue
8
+ EMPTY_OPTIONS = {}.freeze
9
+
8
10
  class Client
9
11
  include Clock
10
12
 
@@ -13,43 +15,53 @@ module Async
13
15
  @notifier = notifier
14
16
  end
15
17
 
16
- def push(class_name, args = [], run_at = nil)
17
- id = @store.enqueue(class_name, args, run_at)
18
+ def push(class_name, args = [], run_at = nil, options: {})
19
+ id = @store.enqueue(class_name, args, run_at, options: options)
18
20
  @notifier&.notify_all
19
21
  id
20
22
  end
21
23
 
22
- def push_in(delay, class_name, args = [])
24
+ def push_in(delay, class_name, args = [], options: {})
23
25
  run_at = realtime_now + delay.to_f
24
- push(class_name, args, run_at)
26
+ push(class_name, args, run_at, options: options)
25
27
  end
26
28
 
27
- def push_at(time, class_name, args = [])
29
+ def push_at(time, class_name, args = [], options: {})
28
30
  run_at = time.respond_to?(:to_f) ? time.to_f : time
29
- push(class_name, args, run_at)
31
+ push(class_name, args, run_at, options: options)
30
32
  end
31
33
  end
32
34
 
33
35
  class << self
34
36
  attr_accessor :default_client
35
37
 
36
- def enqueue(job_class, *args)
38
+ def enqueue(job_class, *args, options: {})
37
39
  ensure_configured!
38
- default_client.push(resolve_class_name(job_class), args)
40
+ merged = build_options(job_class, options)
41
+ default_client.push(resolve_class_name(job_class), args, nil, options: merged)
39
42
  end
40
43
 
41
- def enqueue_in(delay, job_class, *args)
44
+ def enqueue_in(delay, job_class, *args, options: {})
42
45
  ensure_configured!
43
- default_client.push_in(delay, resolve_class_name(job_class), args)
46
+ merged = build_options(job_class, options)
47
+ default_client.push_in(delay, resolve_class_name(job_class), args, options: merged)
44
48
  end
45
49
 
46
- def enqueue_at(time, job_class, *args)
50
+ def enqueue_at(time, job_class, *args, options: {})
47
51
  ensure_configured!
48
- default_client.push_at(time, resolve_class_name(job_class), args)
52
+ merged = build_options(job_class, options)
53
+ default_client.push_at(time, resolve_class_name(job_class), args, options: merged)
49
54
  end
50
55
 
51
56
  private
52
57
 
58
+ def build_options(job_class, call_site_options)
59
+ merged = resolve_options(job_class).merge!(call_site_options.compact)
60
+ return EMPTY_OPTIONS if merged.empty?
61
+
62
+ Job::Options.new(**merged).to_h
63
+ end
64
+
53
65
  def ensure_configured!
54
66
  raise "Async::Background::Queue not configured" unless default_client
55
67
  end
@@ -60,6 +72,12 @@ module Async
60
72
 
61
73
  raise ArgumentError, "#{job_class} must include Async::Background::Job"
62
74
  end
75
+
76
+ def resolve_options(job_class)
77
+ return {} unless job_class.respond_to?(:resolve_options)
78
+
79
+ job_class.resolve_options.dup
80
+ end
63
81
  end
64
82
  end
65
83
  end
@@ -6,13 +6,11 @@ module Async
6
6
  module Background
7
7
  module Queue
8
8
  class SocketNotifier
9
- # Errors that indicate a worker is unavailable - silently skip
9
+ # Errors that indicate a worker is unavailable - silently skip and try the next.
10
10
  UNAVAILABLE = [
11
11
  Errno::ENOENT, # Socket file doesn't exist (worker hasn't started yet)
12
12
  Errno::ECONNREFUSED, # File exists but no one listening (worker died)
13
13
  Errno::EPIPE, # Connection broken during write
14
- Errno::EAGAIN, # Socket buffer full - wake-up already queued
15
- IO::WaitWritable, # Same as EAGAIN on some platforms
16
14
  Errno::ECONNRESET # Connection reset by peer
17
15
  ].freeze
18
16
 
@@ -22,8 +20,12 @@ module Async
22
20
  end
23
21
 
24
22
  def notify_all
25
- (1..@total_workers).each do |worker_index|
26
- notify_one(worker_index)
23
+ return if @total_workers <= 0
24
+
25
+ start = rand(@total_workers)
26
+ @total_workers.times do |i|
27
+ worker_index = ((start + i) % @total_workers) + 1
28
+ return if notify_one(worker_index)
27
29
  end
28
30
  end
29
31
 
@@ -37,14 +39,12 @@ module Async
37
39
  ensure
38
40
  sock.close rescue nil
39
41
  end
42
+ true
40
43
  rescue *UNAVAILABLE
41
- # Worker is unavailable - not a problem.
42
- # The job is already in the database. The worker will:
43
- # - Pick it up on next poll (within QUEUE_POLL_INTERVAL seconds), or
44
- # - Pick it up when it starts/restarts via normal fetch loop
44
+ false
45
45
  rescue => e
46
- # Unexpected error - log but don't crash the enqueue operation
47
46
  Console.logger.warn(self) { "SocketNotifier#notify_one(#{worker_index}) failed: #{e.class} #{e.message}" } rescue nil
47
+ false
48
48
  end
49
49
 
50
50
  def socket_path(worker_index)
@@ -15,13 +15,14 @@ module Async
15
15
  id INTEGER PRIMARY KEY,
16
16
  class_name TEXT NOT NULL,
17
17
  args TEXT NOT NULL DEFAULT '[]',
18
+ options TEXT,
18
19
  status TEXT NOT NULL DEFAULT 'pending',
19
20
  created_at REAL NOT NULL,
20
21
  run_at REAL NOT NULL,
21
22
  locked_by INTEGER,
22
23
  locked_at REAL
23
24
  );
24
- CREATE INDEX IF NOT EXISTS idx_jobs_status_run_at_id ON jobs(status, run_at, id);
25
+ CREATE INDEX IF NOT EXISTS idx_jobs_pending ON jobs(run_at, id) WHERE status = 'pending';
25
26
  SQL
26
27
 
27
28
  MMAP_SIZE = 268_435_456
@@ -45,6 +46,7 @@ module Async
45
46
  def initialize(path: self.class.default_path, mmap: true)
46
47
  @path = path
47
48
  @mmap = mmap
49
+ @pragma_sql = PRAGMAS.call(mmap ? MMAP_SIZE : 0).freeze
48
50
  @db = nil
49
51
  @schema_checked = false
50
52
  @last_cleanup_at = nil
@@ -54,17 +56,18 @@ module Async
54
56
  require_sqlite3
55
57
  db = SQLite3::Database.new(@path)
56
58
  db.execute('PRAGMA busy_timeout = 5000')
57
- db.execute_batch(PRAGMAS.call(@mmap ? MMAP_SIZE : 0))
59
+ db.execute_batch(@pragma_sql)
58
60
  db.execute_batch(SCHEMA)
59
61
  db.execute("PRAGMA wal_checkpoint(TRUNCATE)")
60
62
  db.close
61
63
  @schema_checked = true
62
64
  end
63
65
 
64
- def enqueue(class_name, args = [], run_at = nil)
66
+ def enqueue(class_name, args = [], run_at = nil, options: {})
65
67
  ensure_connection
66
68
  run_at ||= realtime_now
67
- @enqueue_stmt.execute(class_name, JSON.generate(args), realtime_now, run_at)
69
+ options_json = options.empty? ? nil : JSON.generate(options)
70
+ @enqueue_stmt.execute(class_name, JSON.generate(args), options_json, realtime_now, run_at)
68
71
  @db.last_insert_row_id
69
72
  end
70
73
 
@@ -72,14 +75,20 @@ module Async
72
75
  ensure_connection
73
76
  now = realtime_now
74
77
  @db.execute("BEGIN IMMEDIATE")
75
- results = @fetch_stmt.execute(worker_id, now, now)
76
- row = results.first
77
- @fetch_stmt.reset!
78
+
79
+ begin
80
+ results = @fetch_stmt.execute(worker_id, now, now)
81
+ row = results.first
82
+ ensure
83
+ @fetch_stmt.reset! rescue nil
84
+ end
85
+
78
86
  @db.execute("COMMIT")
79
87
  return unless row
80
88
 
81
89
  maybe_cleanup
82
- { id: row[0], class_name: row[1], args: JSON.parse(row[2]) }
90
+ options = row[3] ? JSON.parse(row[3], symbolize_names: true) : {}
91
+ { id: row[0], class_name: row[1], args: JSON.parse(row[2]), options: options }
83
92
  rescue
84
93
  @db.execute("ROLLBACK") rescue nil
85
94
  raise
@@ -130,10 +139,11 @@ module Async
130
139
  finalize_statements
131
140
  @db = SQLite3::Database.new(@path)
132
141
  @db.execute('PRAGMA busy_timeout = 5000')
133
- @db.execute_batch(PRAGMAS.call(@mmap ? MMAP_SIZE : 0))
142
+ @db.execute_batch(@pragma_sql)
134
143
 
135
144
  unless @schema_checked
136
145
  @db.execute_batch(SCHEMA)
146
+ @db.execute("ALTER TABLE jobs ADD COLUMN options TEXT") rescue nil
137
147
  @schema_checked = true
138
148
  end
139
149
 
@@ -143,7 +153,7 @@ module Async
143
153
 
144
154
  def prepare_statements
145
155
  @enqueue_stmt = @db.prepare(
146
- "INSERT INTO jobs (class_name, args, created_at, run_at) VALUES (?, ?, ?, ?)"
156
+ "INSERT INTO jobs (class_name, args, options, created_at, run_at) VALUES (?, ?, ?, ?, ?)"
147
157
  )
148
158
 
149
159
  @fetch_stmt = @db.prepare(<<~SQL)
@@ -155,7 +165,7 @@ module Async
155
165
  ORDER BY run_at, id
156
166
  LIMIT 1
157
167
  )
158
- RETURNING id, class_name, args
168
+ RETURNING id, class_name, args, options
159
169
  SQL
160
170
 
161
171
  @complete_stmt = @db.prepare("UPDATE jobs SET status = 'done' WHERE id = ?")
@@ -170,11 +180,11 @@ module Async
170
180
  end
171
181
 
172
182
  def finalize_statements
173
- %i[enqueue_stmt fetch_stmt complete_stmt fail_stmt requeue_stmt cleanup_stmt].each do |name|
174
- stmt = instance_variable_get(:"@#{name}")
175
- stmt&.close rescue nil
176
- instance_variable_set(:"@#{name}", nil)
183
+ [@enqueue_stmt, @fetch_stmt, @complete_stmt, @fail_stmt, @requeue_stmt, @cleanup_stmt].each do |s|
184
+ s&.close rescue next
177
185
  end
186
+
187
+ @enqueue_stmt = @fetch_stmt = @complete_stmt = @fail_stmt = @requeue_stmt = @cleanup_stmt = nil
178
188
  end
179
189
 
180
190
  def maybe_cleanup
@@ -188,7 +198,6 @@ module Async
188
198
  @db.execute("PRAGMA incremental_vacuum")
189
199
  end
190
200
  end
191
-
192
201
  end
193
202
  end
194
203
  end
@@ -114,11 +114,13 @@ module Async
114
114
  class_name = job[:class_name]
115
115
  args = job[:args]
116
116
  klass = resolve_job_class(class_name)
117
+ opts = Job::Options.new(**job[:options])
118
+ timeout = opts.timeout
117
119
 
118
120
  metrics.job_started(nil)
119
121
  t = monotonic_now
120
122
 
121
- job_task.with_timeout(DEFAULT_TIMEOUT) { klass.perform_now(*args) }
123
+ job_task.with_timeout(timeout) { klass.perform_now(*args) }
122
124
 
123
125
  duration = monotonic_now - t
124
126
  metrics.job_finished(nil, duration)
@@ -130,7 +132,7 @@ module Async
130
132
  rescue ::Async::TimeoutError
131
133
  metrics.job_timed_out(nil)
132
134
  @queue_store.fail(job[:id])
133
- logger.error('Async::Background') { "queue(#{class_name}): timed out" }
135
+ logger.error('Async::Background') { "queue(#{class_name}): timed out after #{timeout}s" }
134
136
  rescue => e
135
137
  metrics.job_failed(nil, e)
136
138
  @queue_store.fail(job[:id])
@@ -170,13 +172,16 @@ module Async
170
172
  if entry.running
171
173
  logger.warn('Async::Background') { "#{entry.name}: skipped, previous run still active" }
172
174
  metrics.job_skipped(entry)
173
- else
174
- entry.running = true
175
- semaphore.async do |job_task|
176
- run_job(job_task, entry)
177
- ensure
178
- entry.running = false
179
- end
175
+ entry.reschedule(monotonic_now)
176
+ heap.replace_top(entry)
177
+ next
178
+ end
179
+
180
+ entry.running = true
181
+ semaphore.async do |job_task|
182
+ run_job(job_task, entry)
183
+ ensure
184
+ entry.running = false
180
185
  end
181
186
 
182
187
  entry.reschedule(monotonic_now)
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Async
4
4
  module Background
5
- VERSION = '0.6.0'
5
+ VERSION = '0.6.2'
6
6
  end
7
7
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: async-background
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.0
4
+ version: 0.6.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Roman Hajdarov
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-04-07 00:00:00.000000000 Z
11
+ date: 2026-04-16 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: async