async-background 0.4.5 → 0.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c9a1ac97495dc7d6a39adf802a847b62e3601df0fe0ef4aa70dbcc248a2f77b8
4
- data.tar.gz: 74bab717411caf3e8a7b469666e9df5f8f59d540ebd98446c5e44e167cacd3c9
3
+ metadata.gz: 26e459d9e86a150ff8b3126ce8899ac9beb60355e3313d5510c1c0c6dec0bfd7
4
+ data.tar.gz: 93a52a9139f902e1acee7b58ea98dceadba2219537329b785f3a103009f72401
5
5
  SHA512:
6
- metadata.gz: fe41a129f66a72c51016397b866ee9ff31b973d5e92ee6af1f61c05ce87f8f85f9824a68499caefcf950cdb33186c74749817e0ae8b89a483ede580d412e13fc
7
- data.tar.gz: 92208ce41c49095cdf652b798ed892e4db5ae12c504cfe94c45bf124054256a94906fba14f68365f938a6d7289c76ca79d7ae7f6fac53bb76d8097d463db684f
6
+ metadata.gz: 3dcfcc9089b2bb8b5b3fb01341039fa1dea32896f6e3845f137bd9e307b366b2716208d4ecb533b266cf31c3b0466535817d7dc9d6c1a0876e113519417a0512
7
+ data.tar.gz: ece53f421338c16a02548754a6c1a2e8a62988659a83f4575e8ad23d01fd987958e6d3fb94360607444ff34a4dc0fb8844fdfdbca9228c2c8af09b1aec486b36
data/CHANGELOG.md ADDED
@@ -0,0 +1,160 @@
1
+ # Changelog
2
+
3
+ ## 0.5.1
4
+
5
+ ### Testing Infrastructure
6
+ - **Comprehensive CI setup** — full Docker-based integration testing environment with `Dockerfile.ci`, `docker-compose.ci.yml`, and `Gemfile.ci`
7
+ - **End-to-end scenario testing** — new `ci/scenario_test.rb` validates real-world scenarios with forked workers:
8
+ - Normal execution of fast/slow/failing jobs across multiple workers
9
+ - Crash recovery after SIGKILL with automatic job pickup by remaining workers
10
+ - No duplicate execution guarantees under worker crashes
11
+ - Proper job distribution validation across worker pool
12
+ - **Test fixtures** — dedicated `ci/fixtures/jobs.rb` and `ci/fixtures/schedule.yml` for scenario testing
13
+
14
+ ### Bug Fixes
15
+ - **SQLite busy timeout** — added `PRAGMA busy_timeout = 5000` to `Queue::Store` to prevent `SQLITE_BUSY` errors under concurrent multi-process database access
16
+ - **Enhanced Queue::Notifier error handling** — restructured IO error handling with clearer categorization:
17
+ - `WRITE_DROPPED` for write failures (`IO::WaitWritable`, `Errno::EAGAIN`, `IOError`, `Errno::EPIPE`) — all non-fatal as job is already in store
18
+ - `READ_EXHAUSTED` for read exhaustion (`IO::WaitReadable`, `EOFError`, `IOError`) — normal drain completion
19
+ - Added explanatory comments for each error type and handling strategy
20
+
21
+ ## 0.5.0
22
+
23
+ ### Features
24
+ - **Delayed jobs** — full support for scheduling jobs in the future
25
+ - `Queue::Client#push_in(delay, class_name, args)` — enqueue with delay in seconds
26
+ - `Queue::Client#push_at(time, class_name, args)` — enqueue at a specific time
27
+ - `Queue.enqueue_in(delay, job_class, *args)` — class-level delayed enqueue
28
+ - `Queue.enqueue_at(time, job_class, *args)` — class-level scheduled enqueue
29
+ - New `run_at` column in SQLite `jobs` table — jobs are only fetched when `run_at <= now`
30
+ - **Job module** — Sidekiq-like `include Async::Background::Job` interface
31
+ - `perform_async(*args)` — immediate queue execution
32
+ - `perform_in(delay, *args)` — delayed execution after N seconds
33
+ - `perform_at(time, *args)` — scheduled execution at a specific time
34
+ - Instance-level `#perform` with class-level `perform_now` delegation
35
+ - **Clock module** — shared `monotonic_now` / `realtime_now` helpers extracted into `Async::Background::Clock`, included by `Runner`, `Queue::Store`, and `Queue::Client`
36
+
37
+ ### Bug Fixes
38
+ - **Runner: incorrect task in `with_timeout`** — `semaphore.async { |job_task| ... }` now correctly receives the child task instead of capturing the parent `task` from the outer scope. Previously, `with_timeout` was applied to the parent task, which could cancel unrelated work
39
+
40
+ ### Improvements
41
+ - **Job API: `#perform` instead of `#perform_now`** — job classes now define `#perform` instance method. The class-level `perform_now` creates instance and calls `#perform`, aligning with ActiveJob / Sidekiq conventions
42
+ - Updated error messages: validation failures now suggest `must include Async::Background::Job` instead of `must implement .perform_now`
43
+ - `Queue::Client` — extracted private `ensure_configured!` and `resolve_class_name` methods for cleaner validation and class name resolution logic
44
+ - `Queue::Notifier` — extracted `IO_ERRORS` constant (`IO::WaitReadable`, `EOFError`, `IOError`) for cleaner `rescue` in `drain`
45
+ - `Queue::Store` — replaced index `idx_jobs_status_id(status, id)` with `idx_jobs_status_run_at_id(status, run_at, id)` for efficient delayed job lookups
46
+ - `Queue::Store` — `fetch` SQL now uses `WHERE status = 'pending' AND run_at <= ?` with `ORDER BY run_at, id` to process jobs in scheduled order
47
+ - Removed duplicated `monotonic_now` / `realtime_now` from `Runner` and `Store` — now provided by `Clock` module
48
+ - Updated documentation: README (Job module examples, Queue architecture diagram, Clock section), GET_STARTED (delayed jobs guide, Job module usage, minimal queue-only example)
49
+
50
+ ## 0.4.5
51
+
52
+ ### Breaking Changes
53
+ - `PRAGMAS` is now a frozen lambda `PRAGMAS.call(mmap_size)` instead of a static string — if you referenced this constant directly, update your code
54
+
55
+ ### Features
56
+ - New `queue_mmap:` parameter on `Runner` (default: `true`) — allows disabling SQLite mmap for environments where it's unsafe (Docker overlay2)
57
+ - New `mmap:` parameter on `Queue::Store` (default: `true`) — controls `PRAGMA mmap_size` (256 MB when enabled, 0 when disabled)
58
+ - Public `attr_reader :queue_store` on `Runner` — eliminates need for `instance_variable_get` when sharing Store with Client
59
+
60
+ ### Bug Fixes
61
+ - **CRITICAL: fetch race condition** — wrapped `UPDATE ... RETURNING` in `BEGIN IMMEDIATE` transaction to prevent two workers from picking up the same job simultaneously
62
+ - **CRITICAL: mmap + Docker overlay2** — `overlay2` filesystem does not guarantee `write()`/`mmap()` coherence, causing SQLite WAL corruption under concurrent multi-process access. mmap is now configurable via `queue_mmap: false` instead of being hardcoded. Documented proper Docker setup with named volumes in `docs/GET_STARTED.md`
63
+ - **`PRAGMA optimize` on shutdown** — wrapped in `rescue nil` to prevent `SQLite3::BusyException` when another process holds the write lock during graceful shutdown
64
+ - **`PRAGMA incremental_vacuum` was a no-op** — added `PRAGMA auto_vacuum = INCREMENTAL` to schema. Without it, `incremental_vacuum` does nothing. Note: only takes effect on newly created databases; existing databases require a one-time `VACUUM`
65
+
66
+ ### Improvements
67
+ - Replaced index `idx_jobs_status(status)` with composite `idx_jobs_status_id(status, id)` — eliminates sort step in `fetch` query (`ORDER BY id LIMIT 1` is now a direct B-tree lookup)
68
+ - Fixed `finalize_statements` — changed `%i[@enqueue_stmt ...]` to `%i[enqueue_stmt ...]` with `:"@#{name}"` interpolation for idiomatic `instance_variable_get`/`set` usage
69
+ - Added documentation: `README.md` (concise, with warning markers) and `docs/GET_STARTED.md` (step-by-step guide covering schedule config, Falcon integration, Docker setup, dynamic queue)
70
+
71
+ ## 0.4.0
72
+
73
+ ### Features
74
+ - **Dynamic job queue** — enqueue jobs at runtime from any process (web, console, rake) with automatic execution by background workers
75
+ - `Queue::Store` — SQLite-backed persistent storage with WAL mode, prepared statements, and optimized pragmas
76
+ - `Queue::Notifier` — `IO.pipe`-based zero-cost wakeup between producer and consumer processes (no polling)
77
+ - `Queue::Client` — public API: `Async::Background::Queue.enqueue(JobClass, *args)`
78
+ - Automatic recovery of stale `running` jobs on worker restart
79
+ - Periodic cleanup of completed jobs (piggyback on fetch, every 5 minutes)
80
+ - `PRAGMA incremental_vacuum` when cleanup removes 100+ rows
81
+ - Worker isolation via `ISOLATION_FORKS` env variable — exclude specific workers from queue processing
82
+ - Custom database path via `queue_db_path` parameter
83
+ - Requires optional `sqlite3` gem (`~> 2.0`) — not included by default, must be added to Gemfile explicitly
84
+ - New Runner parameters: `queue_notifier:` and `queue_db_path:`
85
+
86
+ ### Improvements
87
+ - Unified `monotonic_now` usage across `run_job` and `run_queue_job` (was using direct `Process.clock_gettime` call in `run_job`)
88
+ - `Queue::Notifier#drain` — moved `rescue` inside the loop to avoid stack unwinding on each drain cycle
89
+
90
+ ## 0.3.0
91
+
92
+ ### Features
93
+ - Added optional metrics collection system using shared memory
94
+ - New `Metrics` class with worker-specific performance tracking
95
+ - Public API: `runner.metrics.enabled?`, `runner.metrics.values`, `Metrics.read_all()`
96
+ - Tracks total runs, successes, failures, timeouts, skips, active jobs, and execution times
97
+ - Requires optional `async-utilization` gem dependency
98
+ - Metrics stored in `/tmp/async-background.shm` with lock-free updates per worker
99
+
100
+ ## 0.2.6
101
+
102
+ ### Improvements
103
+ - Micro-optimization in `wait_with_shutdown` method: use passed `task` parameter instead of `Async::Task.current` for better consistency and slight performance improvement
104
+
105
+ ## 0.2.5
106
+
107
+ ### Features
108
+ - Added graceful shutdown via signal handlers for SIGINT and SIGTERM
109
+ - Enhanced process lifecycle management with proper signal handling using `Signal.trap` and IO.pipe for async communication
110
+ - Improved robustness for production deployments and container orchestration
111
+ - Updated dependencies to work with latest Async 2.x API (removed deprecated `:parent` parameter usage)
112
+
113
+ ## 0.2.4
114
+
115
+ ### Improvements
116
+ - Removed hardcoded version warning from main module (was checking against fixed list: 0.1.0, 0.2.2, 0.2.3). Use semantic versioning with pre-release suffixes for unstable versions (e.g., 0.3.0.alpha1) instead
117
+ - Removed hardcoded stable versions list from gemspec description — metadata should describe functionality, not versioning
118
+ - Changed `while true` to idiomatic `loop do` in run method
119
+ - Added `Gemfile.lock` to .gitignore (gems should not commit lockfile)
120
+ - Updated README: clarified that job class must respond to `.perform_now` class method (removed confusing mention of instance `#perform`)
121
+
122
+ ## 0.2.2
123
+
124
+ ### Bug Fixes
125
+ - **CRITICAL**: Removed logger parameter from Runner initialize (was unused). Fixed initialization to use Console.logger directly which now properly initializes in forked processes with correct context
126
+
127
+ ## 0.2.1
128
+
129
+ ### Bug Fixes
130
+ - **CRITICAL**: Added missing `require 'console'` in main module. Logger was nil because Console gem was not imported, causing `undefined method 'info' for nil` errors on worker initialization
131
+
132
+ ## 0.2.0
133
+
134
+ ### Bug Fixes
135
+ - **CRITICAL**: Removed hidden ActiveSupport dependency. Replaced `safe_constantize` with `Object.const_get` + `NameError` handling
136
+ - **CRITICAL**: Fixed validator mismatch: now validates `.perform_now` (class method) instead of `.perform` (instance method)
137
+ - **CRITICAL**: Fixed race condition where entry could disappear from heap during execution. `reschedule` and `heap.push` now always execute after job processing
138
+ - Added full exception backtrace to error logs for production debugging
139
+ - Improved YAML security by removing `Symbol` from `permitted_classes`
140
+ - Removed Mutex from graceful shutdown (anti-pattern in Async). Boolean assignment is atomic in MRI
141
+
142
+ ### Features
143
+ - Added optional `logger` parameter to Runner constructor for custom loggers (Rails.logger, etc.)
144
+ - Added `stop()` method for graceful shutdown
145
+ - Added `running?()` method to check scheduler status
146
+
147
+ ### Breaking Changes
148
+ - Job class validation now checks for `.perform_now` class method (was checking for `.perform` instance method)
149
+
150
+ ## 0.1.0
151
+
152
+ - Initial release
153
+ - Single event loop with min-heap timer (O(log N) scheduling)
154
+ - Skip overlapping execution
155
+ - Startup jitter to prevent thundering herd
156
+ - Monotonic clock for interval jobs, wall clock for cron jobs
157
+ - Deterministic worker sharding via Zlib.crc32
158
+ - Semaphore-based concurrency control
159
+ - Per-job timeout protection
160
+ - Structured logging via Console
data/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Haydarov Roman
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,193 @@
1
+ # Async::Background
2
+
3
+ A lightweight, production-grade cron/interval scheduler for Ruby's [Async](https://github.com/socketry/async) ecosystem. Designed for [Falcon](https://github.com/socketry/falcon) but works with any Async-based application.
4
+
5
+ ## What It Does
6
+
7
+ - **Cron & interval scheduling** — single event loop + min-heap, scales to hundreds of jobs
8
+ - **Dynamic job queue** — enqueue jobs at runtime via SQLite, pick up by background workers
9
+ - **Delayed jobs** — schedule jobs for future execution with `perform_in` / `perform_at` (Sidekiq-like API)
10
+ - **Multi-process safe** — deterministic worker sharding via `Zlib.crc32`, no duplicate execution
11
+ - **Skip overlapping** — if a job is still running when its next tick arrives, the tick is skipped
12
+ - **Timeout protection** — per-job configurable timeout via `Async::Task#with_timeout`
13
+ - **Startup jitter** — random delay to prevent thundering herd after restart
14
+ - **Optional metrics** — shared memory performance tracking with `async-utilization`
15
+
16
+ ## Requirements
17
+
18
+ - **Ruby >= 3.3** — Fiber Scheduler production-ready ([why?](#why-ruby-33))
19
+ - **Async ~> 2.0** — Fiber Scheduler-based concurrency
20
+ - **Fugit ~> 1.0** — cron expression parsing
21
+
22
+ ## Installation
23
+
24
+ ```ruby
25
+ # Gemfile
26
+ gem "async-background"
27
+
28
+ # Optional
29
+ gem "sqlite3", "~> 2.0" # for dynamic job queue
30
+ gem "async-utilization", "~> 0.3" # for metrics
31
+ ```
32
+
33
+ ## ➡️ [Get Started](docs/GET_STARTED.md)
34
+
35
+ Step-by-step setup guide: schedule config, Falcon integration, Docker, dynamic queue, delayed jobs.
36
+
37
+ ---
38
+
39
+ ## Quick Example: Job Module
40
+
41
+ Include `Async::Background::Job` for a Sidekiq-like interface:
42
+
43
+ ```ruby
44
+ class SendEmailJob
45
+ include Async::Background::Job
46
+
47
+ def perform(user_id, template)
48
+ user = User.find(user_id)
49
+ Mailer.send(user, template)
50
+ end
51
+ end
52
+
53
+ # Immediate execution in the queue
54
+ SendEmailJob.perform_async(user_id, "welcome")
55
+
56
+ # Execute after 5 minutes
57
+ SendEmailJob.perform_in(300, user_id, "reminder")
58
+
59
+ # Execute at a specific time
60
+ SendEmailJob.perform_at(Time.new(2026, 4, 1, 9, 0, 0), user_id, "scheduled")
61
+ ```
62
+
63
+ Or use the lower-level API directly:
64
+
65
+ ```ruby
66
+ Async::Background::Queue.enqueue(SendEmailJob, user_id, "welcome")
67
+ Async::Background::Queue.enqueue_in(300, SendEmailJob, user_id, "reminder")
68
+ Async::Background::Queue.enqueue_at(Time.new(2026, 4, 1, 9, 0, 0), SendEmailJob, user_id, "scheduled")
69
+ ```
70
+
71
+ ---
72
+
73
+ ## ⚠️ Important Notes
74
+
75
+ ### Docker: SQLite requires a named volume
76
+
77
+ The SQLite database **must not** live on Docker's `overlay2` filesystem. The `overlay2` driver breaks coherence between `write()` and `mmap()`, which corrupts SQLite WAL under concurrent access.
78
+
79
+ ```yaml
80
+ # docker-compose.yml
81
+ volumes:
82
+ - queue-data:/app/tmp/queue # ← named volume, NOT overlay2
83
+
84
+ volumes:
85
+ queue-data:
86
+ ```
87
+
88
+ Without this, you will get database crashes in multi-process mode. See [Get Started → Step 3](docs/GET_STARTED.md#step-3-docker) for details.
89
+
90
+ ### Fork safety
91
+
92
+ SQLite connections **must not** cross `fork()` boundaries. Always open connections **after** fork (inside `container.run` block), never before. The gem handles this internally via lazy `ensure_connection`, but if you create a `Queue::Store` manually for schema setup, close it before fork:
93
+
94
+ ```ruby
95
+ store = Async::Background::Queue::Store.new(path: db_path)
96
+ store.ensure_database!
97
+ store.close # ← before fork
98
+ ```
99
+
100
+ ### Clock handling
101
+
102
+ The `Clock` module provides shared time helpers used across the codebase:
103
+
104
+ - **`monotonic_now`** (`CLOCK_MONOTONIC`) — for in-process intervals and durations, immune to NTP drift / wall-clock jumps
105
+ - **`realtime_now`** (`CLOCK_REALTIME`) — for persisted timestamps (SQLite `run_at`, `created_at`, `locked_at`)
106
+
107
+ Interval jobs use monotonic clock. Cron jobs use `Time.now` because "every day at 3am" must respect real time. These are different clocks by design.
108
+
109
+ ---
110
+
111
+ ## Architecture
112
+
113
+ ```
114
+ schedule.yml
115
+
116
+
117
+ build_heap ← parse config, validate, assign workers
118
+
119
+
120
+ MinHeap<Entry> ← O(log N) push/pop, sorted by next_run_at
121
+
122
+
123
+ 1 scheduler loop ← single Async task, sleeps until next entry
124
+
125
+
126
+ Semaphore ← limits concurrent job execution
127
+
128
+
129
+ run_job ← timeout, logging, error handling
130
+ ```
131
+
132
+ ### Queue Architecture
133
+
134
+ ```
135
+ Producer (web/console) Consumer (background worker)
136
+ │ │
137
+ ▼ ▼
138
+ Queue::Client Queue::Store.fetch
139
+ │ (WHERE run_at <= now)
140
+ ├─ push(class, args, run_at) │
141
+ ├─ push_in(delay, class, args) ▼
142
+ └─ push_at(time, class, args) run_queue_job
143
+ │ │
144
+ ▼ ▼
145
+ Queue::Store ──── SQLite ──── Queue::Notifier
146
+ (INSERT job) (jobs table) (IO.pipe wakeup)
147
+ ```
148
+
149
+ ## Schedule Config
150
+
151
+ | Key | Required | Description |
152
+ |---|---|---|
153
+ | `class` | yes | Must include `Async::Background::Job` |
154
+ | `every` | one of | Interval in seconds between runs |
155
+ | `cron` | one of | Cron expression (parsed by Fugit) |
156
+ | `timeout` | no | Max execution time in seconds (default: 30) |
157
+ | `worker` | no | Pin to specific worker index. If omitted — `crc32(name) % total_workers` |
158
+
159
+ ## SQLite Pragmas
160
+
161
+ | Pragma | Value | Why |
162
+ |---|---|---|
163
+ | `journal_mode` | WAL | Concurrent reads during writes |
164
+ | `synchronous` | NORMAL | Safe with WAL, lower fsync overhead |
165
+ | `mmap_size` | 256 MB | Fast reads ([requires proper filesystem](#docker-sqlite-requires-a-named-volume)) |
166
+ | `cache_size` | 16000 pages | ~64 MB page cache |
167
+ | `busy_timeout` | 5000 ms | Wait instead of failing on lock contention |
168
+
169
+ ## Metrics
170
+
171
+ When `async-utilization` gem is available, metrics are collected in shared memory (`/tmp/async-background.shm`) with lock-free updates per worker.
172
+
173
+ ```ruby
174
+ runner.metrics.values
175
+ # => { total_runs: 142, total_successes: 140, total_failures: 2,
176
+ # total_timeouts: 0, total_skips: 5, active_jobs: 1,
177
+ # last_run_at: 1774445243, last_duration_ms: 1250 }
178
+
179
+ # Read all workers at once (no server needed)
180
+ Async::Background::Metrics.read_all(total_workers: 2)
181
+ ```
182
+
183
+ Without the gem — metrics are silently disabled, zero overhead.
184
+
185
+ ## Why Ruby 3.3?
186
+
187
+ - Ruby 3.0 introduced Fiber Scheduler but had critical bugs
188
+ - Ruby 3.2 is the first production-ready release (per Samuel Williams)
189
+ - `io-event >= 1.14` (pulled by latest `async`) requires Ruby `>= 3.3`
190
+
191
+ ## License
192
+
193
+ MIT
@@ -0,0 +1,43 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'lib/async/background/version'
4
+
5
+ Gem::Specification.new do |spec|
6
+ spec.name = 'async-background'
7
+ spec.version = Async::Background::VERSION
8
+ spec.authors = ['Roman Hajdarov']
9
+ spec.email = ['romnhajdarov@gmail.com']
10
+
11
+ spec.summary = 'Lightweight heap-based cron/interval scheduler for Async.'
12
+ spec.description = 'A production-grade lightweight scheduler built on top of Async. ' \
13
+ 'Single event loop with min-heap timer, skip-overlapping execution, ' \
14
+ 'jitter, monotonic clock intervals, semaphore concurrency control, ' \
15
+ 'and deterministic worker sharding. Designed for Falcon but works ' \
16
+ 'with any Async-based application.'
17
+
18
+ spec.homepage = 'https://github.com/roman-haidarov/async-background'
19
+ spec.license = 'MIT'
20
+
21
+ spec.metadata = {
22
+ 'source_code_uri' => 'https://github.com/roman-haidarov/async-background',
23
+ 'changelog_uri' => 'https://github.com/roman-haidarov/async-background/blob/main/CHANGELOG.md',
24
+ 'bug_tracker_uri' => 'https://github.com/roman-haidarov/async-background/issues'
25
+ }
26
+
27
+ spec.files = Dir.glob('lib/**/*', base: __dir__) +
28
+ %w[README.md CHANGELOG.md LICENSE async-background.gemspec]
29
+ spec.require_paths = ['lib']
30
+
31
+ spec.required_ruby_version = '>= 3.3'
32
+
33
+ spec.add_dependency 'async', '~> 2.0'
34
+ spec.add_dependency 'console', '~> 1.0'
35
+ spec.add_dependency 'fugit', '~> 1.0'
36
+
37
+ # Optional: add to your own Gemfile if you need these features
38
+ # gem 'sqlite3', '~> 2.0' # dynamic job queue
39
+ # gem 'async-utilization', '~> 0.3' # shared-memory worker metrics
40
+
41
+ spec.add_development_dependency 'rake', '~> 13.0'
42
+ spec.add_development_dependency 'rspec', '~> 3.12'
43
+ end
@@ -0,0 +1,25 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Async
4
+ module Background
5
+ # Shared clock helpers used across Runner, Queue::Store, and Queue::Client.
6
+ #
7
+ # monotonic_now — CLOCK_MONOTONIC, for in-process intervals and durations
8
+ # (immune to NTP drift / wall-clock jumps)
9
+ #
10
+ # realtime_now — CLOCK_REALTIME, for persisted timestamps (SQLite run_at,
11
+ # created_at, locked_at) and human-readable metrics
12
+ #
13
+ module Clock
14
+ private
15
+
16
+ def monotonic_now
17
+ Process.clock_gettime(Process::CLOCK_MONOTONIC)
18
+ end
19
+
20
+ def realtime_now
21
+ Process.clock_gettime(Process::CLOCK_REALTIME)
22
+ end
23
+ end
24
+ end
25
+ end
@@ -3,8 +3,6 @@
3
3
  module Async
4
4
  module Background
5
5
  class Entry
6
- MIN_SLEEP_TIME = 0.1
7
-
8
6
  attr_reader :name, :job_class, :interval, :cron, :timeout
9
7
  attr_accessor :next_run_at, :running
10
8
 
@@ -25,7 +23,7 @@ module Async
25
23
  else
26
24
  now_wall = Time.now
27
25
  wait = cron.next_time(now_wall).to_f - now_wall.to_f
28
- @next_run_at = monotonic_now + [wait, MIN_SLEEP_TIME].max
26
+ @next_run_at = monotonic_now + [wait, Async::Background::MIN_SLEEP_TIME].max
29
27
  end
30
28
  end
31
29
  end
@@ -0,0 +1,33 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Async
4
+ module Background
5
+ module Job
6
+ def self.included(base)
7
+ base.extend(ClassMethods)
8
+ end
9
+
10
+ module ClassMethods
11
+ def perform_now(*args)
12
+ new.perform(*args)
13
+ end
14
+
15
+ def perform_async(*args)
16
+ Async::Background::Queue.enqueue(self, *args)
17
+ end
18
+
19
+ def perform_in(delay, *args)
20
+ Async::Background::Queue.enqueue_in(delay, self, *args)
21
+ end
22
+
23
+ def perform_at(time, *args)
24
+ Async::Background::Queue.enqueue_at(time, self, *args)
25
+ end
26
+ end
27
+
28
+ def perform(*args)
29
+ raise NotImplementedError, "#{self.class} must implement #perform"
30
+ end
31
+ end
32
+ end
33
+ end
@@ -1,38 +1,64 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative '../clock'
4
+
3
5
  module Async
4
6
  module Background
5
7
  module Queue
6
- # Usage:
7
- # Async::Background::Queue.enqueue(SendEmailJob, user_id, "welcome")
8
- #
9
8
  class Client
9
+ include Clock
10
+
10
11
  def initialize(store:, notifier: nil)
11
12
  @store = store
12
13
  @notifier = notifier
13
14
  end
14
15
 
15
- def push(class_name, args = [])
16
- id = @store.enqueue(class_name, args)
16
+ def push(class_name, args = [], run_at = nil)
17
+ id = @store.enqueue(class_name, args, run_at)
17
18
  @notifier&.notify
18
19
  id
19
20
  end
21
+
22
+ def push_in(delay, class_name, args = [])
23
+ run_at = realtime_now + delay.to_f
24
+ push(class_name, args, run_at)
25
+ end
26
+
27
+ def push_at(time, class_name, args = [])
28
+ run_at = time.respond_to?(:to_f) ? time.to_f : time
29
+ push(class_name, args, run_at)
30
+ end
20
31
  end
21
32
 
22
33
  class << self
23
34
  attr_accessor :default_client
24
35
 
25
36
  def enqueue(job_class, *args)
37
+ ensure_configured!
38
+ default_client.push(resolve_class_name(job_class), args)
39
+ end
40
+
41
+ def enqueue_in(delay, job_class, *args)
42
+ ensure_configured!
43
+ default_client.push_in(delay, resolve_class_name(job_class), args)
44
+ end
45
+
46
+ def enqueue_at(time, job_class, *args)
47
+ ensure_configured!
48
+ default_client.push_at(time, resolve_class_name(job_class), args)
49
+ end
50
+
51
+ private
52
+
53
+ def ensure_configured!
26
54
  raise "Async::Background::Queue not configured" unless default_client
55
+ end
27
56
 
28
- if job_class.is_a?(String)
29
- class_name = job_class
30
- else
31
- raise ArgumentError, "#{job_class} must implement .perform_now" unless job_class.respond_to?(:perform_now)
32
- class_name = job_class.name
33
- end
57
+ def resolve_class_name(job_class)
58
+ return job_class if job_class.is_a?(String)
59
+ return job_class.name if job_class.respond_to?(:perform_now)
34
60
 
35
- default_client.push(class_name, args)
61
+ raise ArgumentError, "#{job_class} must include Async::Background::Job"
36
62
  end
37
63
  end
38
64
  end
@@ -4,6 +4,20 @@ module Async
4
4
  module Background
5
5
  module Queue
6
6
  class Notifier
7
+ # Error groups
8
+ WRITE_DROPPED = [
9
+ IO::WaitWritable, # buffer full — consumer is behind, skip
10
+ Errno::EAGAIN, # same as above on some platforms
11
+ IOError, # our own writer end has been closed
12
+ Errno::EPIPE # the reader end is gone (consumer crashed) — skip
13
+ ].freeze
14
+
15
+ READ_EXHAUSTED = [
16
+ IO::WaitReadable, # nothing left in the buffer — normal exit
17
+ EOFError, # writer end closed — no more data ever
18
+ IOError # our own reader end has been closed
19
+ ].freeze
20
+
7
21
  attr_reader :reader, :writer
8
22
 
9
23
  def initialize
@@ -14,10 +28,10 @@ module Async
14
28
 
15
29
  def notify
16
30
  @writer.write_nonblock("\x01")
17
- rescue IO::WaitWritable, Errno::EAGAIN
18
- # pipe buffer full consumer is already behind, skip
19
- rescue IOError
20
- # pipe closed
31
+ rescue *WRITE_DROPPED
32
+ # All write failures are non-fatal: the job is already in the
33
+ # store, and missing one wake-up only delays pickup by at most
34
+ # one poll interval.
21
35
  end
22
36
 
23
37
  def wait(timeout: nil)
@@ -51,7 +65,7 @@ module Async
51
65
  def drain
52
66
  loop do
53
67
  @reader.read_nonblock(256)
54
- rescue IO::WaitReadable, EOFError
68
+ rescue *READ_EXHAUSTED
55
69
  break
56
70
  end
57
71
  nil
@@ -1,11 +1,14 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require 'json'
4
+ require_relative '../clock'
4
5
 
5
6
  module Async
6
7
  module Background
7
8
  module Queue
8
9
  class Store
10
+ include Clock
11
+
9
12
  SCHEMA = <<~SQL
10
13
  PRAGMA auto_vacuum = INCREMENTAL;
11
14
  CREATE TABLE IF NOT EXISTS jobs (
@@ -14,10 +17,11 @@ module Async
14
17
  args TEXT NOT NULL DEFAULT '[]',
15
18
  status TEXT NOT NULL DEFAULT 'pending',
16
19
  created_at REAL NOT NULL,
20
+ run_at REAL NOT NULL,
17
21
  locked_by INTEGER,
18
22
  locked_at REAL
19
23
  );
20
- CREATE INDEX IF NOT EXISTS idx_jobs_status_id ON jobs(status, id);
24
+ CREATE INDEX IF NOT EXISTS idx_jobs_status_run_at_id ON jobs(status, run_at, id);
21
25
  SQL
22
26
 
23
27
  MMAP_SIZE = 268_435_456
@@ -49,6 +53,7 @@ module Async
49
53
  def ensure_database!
50
54
  require_sqlite3
51
55
  db = SQLite3::Database.new(@path)
56
+ db.execute('PRAGMA busy_timeout = 5000')
52
57
  db.execute_batch(PRAGMAS.call(@mmap ? MMAP_SIZE : 0))
53
58
  db.execute_batch(SCHEMA)
54
59
  db.execute("PRAGMA wal_checkpoint(TRUNCATE)")
@@ -56,16 +61,20 @@ module Async
56
61
  @schema_checked = true
57
62
  end
58
63
 
59
- def enqueue(class_name, args = [])
64
+ def enqueue(class_name, args = [], run_at = nil)
60
65
  ensure_connection
61
- @enqueue_stmt.execute(class_name, JSON.generate(args), realtime_now)
66
+ run_at ||= realtime_now
67
+ @enqueue_stmt.execute(class_name, JSON.generate(args), realtime_now, run_at)
62
68
  @db.last_insert_row_id
63
69
  end
64
70
 
65
71
  def fetch(worker_id)
66
72
  ensure_connection
73
+ now = realtime_now
67
74
  @db.execute("BEGIN IMMEDIATE")
68
- row = @fetch_stmt.execute(worker_id, realtime_now).first
75
+ results = @fetch_stmt.execute(worker_id, now, now)
76
+ row = results.first
77
+ @fetch_stmt.reset!
69
78
  @db.execute("COMMIT")
70
79
  return unless row
71
80
 
@@ -75,7 +84,6 @@ module Async
75
84
  @db.execute("ROLLBACK") rescue nil
76
85
  raise
77
86
  end
78
-
79
87
  def complete(job_id)
80
88
  ensure_connection
81
89
  @complete_stmt.execute(job_id)
@@ -121,6 +129,7 @@ module Async
121
129
  require_sqlite3
122
130
  finalize_statements
123
131
  @db = SQLite3::Database.new(@path)
132
+ @db.execute('PRAGMA busy_timeout = 5000')
124
133
  @db.execute_batch(PRAGMAS.call(@mmap ? MMAP_SIZE : 0))
125
134
 
126
135
  unless @schema_checked
@@ -134,7 +143,7 @@ module Async
134
143
 
135
144
  def prepare_statements
136
145
  @enqueue_stmt = @db.prepare(
137
- "INSERT INTO jobs (class_name, args, created_at) VALUES (?, ?, ?)"
146
+ "INSERT INTO jobs (class_name, args, created_at, run_at) VALUES (?, ?, ?, ?)"
138
147
  )
139
148
 
140
149
  @fetch_stmt = @db.prepare(<<~SQL)
@@ -142,8 +151,8 @@ module Async
142
151
  SET status = 'running', locked_by = ?, locked_at = ?
143
152
  WHERE id = (
144
153
  SELECT id FROM jobs
145
- WHERE status = 'pending'
146
- ORDER BY id
154
+ WHERE status = 'pending' AND run_at <= ?
155
+ ORDER BY run_at, id
147
156
  LIMIT 1
148
157
  )
149
158
  RETURNING id, class_name, args
@@ -180,13 +189,6 @@ module Async
180
189
  end
181
190
  end
182
191
 
183
- def realtime_now
184
- Process.clock_gettime(Process::CLOCK_REALTIME)
185
- end
186
-
187
- def monotonic_now
188
- Process.clock_gettime(Process::CLOCK_MONOTONIC)
189
- end
190
192
  end
191
193
  end
192
194
  end
@@ -7,12 +7,14 @@ module Async
7
7
  module Background
8
8
  class ConfigError < StandardError; end
9
9
 
10
- DEFAULT_TIMEOUT = 30
11
- MIN_SLEEP_TIME = 0.1
12
- MAX_JITTER = 5
10
+ DEFAULT_TIMEOUT = 30
11
+ MIN_SLEEP_TIME = 0.1
12
+ MAX_JITTER = 5
13
13
  QUEUE_POLL_INTERVAL = 5
14
14
 
15
15
  class Runner
16
+ include Clock
17
+
16
18
  attr_reader :logger, :semaphore, :heap, :worker_index, :total_workers, :shutdown, :metrics, :queue_store
17
19
 
18
20
  def initialize(
@@ -53,7 +55,7 @@ module Async
53
55
  @running = false
54
56
  logger.info { "Async::Background: stopping gracefully" }
55
57
  shutdown.signal
56
- @queue_notifier&.notify # unblock queue listener from @reader.wait_readable
58
+ @queue_notifier&.notify
57
59
  end
58
60
 
59
61
  def running?
@@ -96,13 +98,13 @@ module Async
96
98
  job = @queue_store.fetch(worker_index)
97
99
  break unless job
98
100
 
99
- semaphore.async { run_queue_job(task, job) }
101
+ semaphore.async { |job_task| run_queue_job(job_task, job) }
100
102
  end
101
103
  end
102
104
  end
103
105
  end
104
106
 
105
- def run_queue_job(task, job)
107
+ def run_queue_job(job_task, job)
106
108
  class_name = job[:class_name]
107
109
  args = job[:args]
108
110
  klass = resolve_job_class(class_name)
@@ -110,7 +112,7 @@ module Async
110
112
  metrics.job_started(nil)
111
113
  t = monotonic_now
112
114
 
113
- task.with_timeout(DEFAULT_TIMEOUT) { klass.perform_now(*args) }
115
+ job_task.with_timeout(DEFAULT_TIMEOUT) { klass.perform_now(*args) }
114
116
 
115
117
  duration = monotonic_now - t
116
118
  metrics.job_finished(nil, duration)
@@ -140,7 +142,7 @@ module Async
140
142
  mod.const_get(name, false)
141
143
  end
142
144
 
143
- raise ConfigError, "#{class_name} must implement .perform_now" unless klass.respond_to?(:perform_now)
145
+ raise ConfigError, "#{class_name} must include Async::Background::Job" unless klass.respond_to?(:perform_now)
144
146
 
145
147
  klass
146
148
  end
@@ -164,8 +166,8 @@ module Async
164
166
  metrics.job_skipped(entry)
165
167
  else
166
168
  entry.running = true
167
- semaphore.async do
168
- run_job(task, entry)
169
+ semaphore.async do |job_task|
170
+ run_job(job_task, entry)
169
171
  ensure
170
172
  entry.running = false
171
173
  end
@@ -252,7 +254,7 @@ module Async
252
254
  raise ConfigError, "[#{name}] unknown class: #{class_name}"
253
255
  end
254
256
 
255
- raise ConfigError, "[#{name}] #{class_name} must implement .perform_now" unless job_class.respond_to?(:perform_now)
257
+ raise ConfigError, "[#{name}] #{class_name} must include Async::Background::Job" unless job_class.respond_to?(:perform_now)
256
258
 
257
259
  interval = config['every']&.then { |v|
258
260
  int = v.to_i
@@ -274,14 +276,10 @@ module Async
274
276
  }
275
277
  end
276
278
 
277
- def monotonic_now
278
- Process.clock_gettime(Process::CLOCK_MONOTONIC)
279
- end
280
-
281
- def run_job(task, entry)
279
+ def run_job(job_task, entry)
282
280
  metrics.job_started(entry)
283
281
  t = monotonic_now
284
- task.with_timeout(entry.timeout) { entry.job_class.perform_now }
282
+ job_task.with_timeout(entry.timeout) { entry.job_class.perform_now }
285
283
 
286
284
  duration = monotonic_now - t
287
285
  metrics.job_finished(entry, duration)
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Async
4
4
  module Background
5
- VERSION = '0.4.5'
5
+ VERSION = '0.5.1'
6
6
  end
7
7
  end
@@ -7,7 +7,11 @@ require 'fugit'
7
7
  require 'tmpdir'
8
8
 
9
9
  require_relative 'background/version'
10
+ require_relative 'background/clock'
10
11
  require_relative 'background/min_heap'
11
12
  require_relative 'background/entry'
12
13
  require_relative 'background/metrics'
13
14
  require_relative 'background/runner'
15
+ require_relative 'background/queue/store'
16
+ require_relative 'background/queue/client'
17
+ require_relative 'background/job'
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: async-background
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.5
4
+ version: 0.5.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Roman Hajdarov
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-03-29 00:00:00.000000000 Z
11
+ date: 2026-04-07 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: async
@@ -90,8 +90,14 @@ executables: []
90
90
  extensions: []
91
91
  extra_rdoc_files: []
92
92
  files:
93
+ - CHANGELOG.md
94
+ - LICENSE
95
+ - README.md
96
+ - async-background.gemspec
93
97
  - lib/async/background.rb
98
+ - lib/async/background/clock.rb
94
99
  - lib/async/background/entry.rb
100
+ - lib/async/background/job.rb
95
101
  - lib/async/background/metrics.rb
96
102
  - lib/async/background/min_heap.rb
97
103
  - lib/async/background/queue/client.rb