async-background 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +197 -0
- data/LICENSE +21 -0
- data/README.md +193 -0
- data/async-background.gemspec +43 -0
- data/lib/async/background/queue/client.rb +1 -1
- data/lib/async/background/queue/notifier.rb +20 -6
- data/lib/async/background/queue/socket_notifier.rb +56 -0
- data/lib/async/background/queue/socket_waker.rb +118 -0
- data/lib/async/background/queue/store.rb +2 -0
- data/lib/async/background/runner.rb +17 -11
- data/lib/async/background/version.rb +1 -1
- data/lib/async/background.rb +1 -0
- metadata +8 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b7e4015afbfcd9ee8291c28caa34e69de838f283fd0d6374a81cc0d53a87b90d
|
|
4
|
+
data.tar.gz: b0333ec4f4626895e01c2b77da17abb569c912e6449def849fa8e0918fb06a58
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 245fa669ebf1573e37770eb16f86904921cbe0f0effb40099c613bda9732e4e8732854f24cdf84421efa877d69e25271544bc279bf5e35ff3c7e364cd8689252
|
|
7
|
+
data.tar.gz: aff2eeba3e4793b4d5e686fb5bf3f27c185f61492aee837d7100a9dde35c3d97cf904c0c037c4dacd1655dd0da49b9f556d5003620e0e0f2c256735539805f29
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.6.0
|
|
4
|
+
|
|
5
|
+
### Breaking Changes
|
|
6
|
+
- **Queue notification system completely rewritten** — replaced pipe-based `Notifier` with Unix domain socket-based architecture
|
|
7
|
+
- `Runner` now takes `queue_socket_dir:` parameter instead of `queue_notifier:`
|
|
8
|
+
- Removed `Notifier#for_producer!` and `Notifier#for_consumer!` — no longer needed
|
|
9
|
+
- `Client#push` now calls `notifier.notify_all` instead of `notifier.notify`
|
|
10
|
+
|
|
11
|
+
### Features
|
|
12
|
+
- **Unix domain socket-based notifications** — solves all cross-process notification problems
|
|
13
|
+
- New `SocketWaker` class (consumer-side) — each worker listens on its own Unix socket (`/tmp/queue/sockets/async_bg_worker_N.sock`)
|
|
14
|
+
- New `SocketNotifier` class (producer-side) — connects to all worker sockets to broadcast wake-ups
|
|
15
|
+
- **Cross-process wake-up now works correctly** — web workers → background workers, background workers → background workers
|
|
16
|
+
- **Fork-safe by design** — no shared file descriptors, each process creates its own socket after fork
|
|
17
|
+
- **Resilient to restarts** — stale socket cleanup on worker startup, graceful degradation if worker unavailable
|
|
18
|
+
- **Sub-100µs latency** — typical wake-up time 30-80µs vs previous 5-second polling fallback
|
|
19
|
+
|
|
20
|
+
### Bug Fixes
|
|
21
|
+
- **CRITICAL: Notifier bug in recommended setup** — the old pipe-based `Notifier` was fundamentally broken in multi-fork scenarios:
|
|
22
|
+
- `for_consumer!` closed the writer end in each child process, making `Client#push → notify` fail silently with `IOError`
|
|
23
|
+
- All writes were caught by `WRITE_DROPPED` rescue block, causing jobs to use 5-second polling instead of instant wake-up
|
|
24
|
+
- Web workers had no way to notify background workers (no shared pipe after fork)
|
|
25
|
+
- The bug was masked by `WRITE_DROPPED` silently catching `IOError` — appeared to work but degraded to polling
|
|
26
|
+
- **Socket cleanup race conditions** — `SocketWaker#cleanup_stale_socket` now validates if socket is truly stale by attempting connection
|
|
27
|
+
|
|
28
|
+
### Improvements
|
|
29
|
+
- Updated `docs/GET_STARTED.md` with new socket-based setup for Falcon
|
|
30
|
+
- Added section on web worker → background worker job enqueuing with full example
|
|
31
|
+
- Changed environment variable from `QUEUE_SOCKET_PATH` to `QUEUE_SOCKET_DIR` (directory instead of single socket path)
|
|
32
|
+
- Better error handling in `SocketWaker` and `SocketNotifier` with comprehensive `UNAVAILABLE` error list
|
|
33
|
+
- Integrated with `Async::Notification` for local wake-ups (shutdown signals)
|
|
34
|
+
|
|
35
|
+
### Technical Details
|
|
36
|
+
- **Why sockets over pipes?** Pipes require shared FDs across fork boundaries. The recommended Falcon setup calls `for_consumer!` in each child, which closes the writer, breaking the notification chain. Sockets use filesystem paths — any process can connect without inherited FDs.
|
|
37
|
+
- **Performance impact:** Adding ~80µs per enqueue for 8 workers (8 socket connections) vs ~100µs for SQLite transaction = negligible overhead
|
|
38
|
+
- **Graceful degradation:** If worker socket unavailable (`ENOENT`, `ECONNREFUSED`), producer silently skips — job still in database, will be picked up on next poll (5s max delay)
|
|
39
|
+
|
|
40
|
+
## 0.5.1
|
|
41
|
+
|
|
42
|
+
### Testing Infrastructure
|
|
43
|
+
- **Comprehensive CI setup** — full Docker-based integration testing environment with `Dockerfile.ci`, `docker-compose.ci.yml`, and `Gemfile.ci`
|
|
44
|
+
- **End-to-end scenario testing** — new `ci/scenario_test.rb` validates real-world scenarios with forked workers:
|
|
45
|
+
- Normal execution of fast/slow/failing jobs across multiple workers
|
|
46
|
+
- Crash recovery after SIGKILL with automatic job pickup by remaining workers
|
|
47
|
+
- No duplicate execution guarantees under worker crashes
|
|
48
|
+
- Proper job distribution validation across worker pool
|
|
49
|
+
- **Test fixtures** — dedicated `ci/fixtures/jobs.rb` and `ci/fixtures/schedule.yml` for scenario testing
|
|
50
|
+
|
|
51
|
+
### Bug Fixes
|
|
52
|
+
- **SQLite busy timeout** — added `PRAGMA busy_timeout = 5000` to `Queue::Store` to prevent `SQLITE_BUSY` errors under concurrent multi-process database access
|
|
53
|
+
- **Enhanced Queue::Notifier error handling** — restructured IO error handling with clearer categorization:
|
|
54
|
+
- `WRITE_DROPPED` for write failures (`IO::WaitWritable`, `Errno::EAGAIN`, `IOError`, `Errno::EPIPE`) — all non-fatal as job is already in store
|
|
55
|
+
- `READ_EXHAUSTED` for read exhaustion (`IO::WaitReadable`, `EOFError`, `IOError`) — normal drain completion
|
|
56
|
+
- Added explanatory comments for each error type and handling strategy
|
|
57
|
+
|
|
58
|
+
## 0.5.0
|
|
59
|
+
|
|
60
|
+
### Features
|
|
61
|
+
- **Delayed jobs** — full support for scheduling jobs in the future
|
|
62
|
+
- `Queue::Client#push_in(delay, class_name, args)` — enqueue with delay in seconds
|
|
63
|
+
- `Queue::Client#push_at(time, class_name, args)` — enqueue at a specific time
|
|
64
|
+
- `Queue.enqueue_in(delay, job_class, *args)` — class-level delayed enqueue
|
|
65
|
+
- `Queue.enqueue_at(time, job_class, *args)` — class-level scheduled enqueue
|
|
66
|
+
- New `run_at` column in SQLite `jobs` table — jobs are only fetched when `run_at <= now`
|
|
67
|
+
- **Job module** — Sidekiq-like `include Async::Background::Job` interface
|
|
68
|
+
- `perform_async(*args)` — immediate queue execution
|
|
69
|
+
- `perform_in(delay, *args)` — delayed execution after N seconds
|
|
70
|
+
- `perform_at(time, *args)` — scheduled execution at a specific time
|
|
71
|
+
- Instance-level `#perform` with class-level `perform_now` delegation
|
|
72
|
+
- **Clock module** — shared `monotonic_now` / `realtime_now` helpers extracted into `Async::Background::Clock`, included by `Runner`, `Queue::Store`, and `Queue::Client`
|
|
73
|
+
|
|
74
|
+
### Bug Fixes
|
|
75
|
+
- **Runner: incorrect task in `with_timeout`** — `semaphore.async { |job_task| ... }` now correctly receives the child task instead of capturing the parent `task` from the outer scope. Previously, `with_timeout` was applied to the parent task, which could cancel unrelated work
|
|
76
|
+
|
|
77
|
+
### Improvements
|
|
78
|
+
- **Job API: `#perform` instead of `#perform_now`** — job classes now define `#perform` instance method. The class-level `perform_now` creates instance and calls `#perform`, aligning with ActiveJob / Sidekiq conventions
|
|
79
|
+
- Updated error messages: validation failures now suggest `must include Async::Background::Job` instead of `must implement .perform_now`
|
|
80
|
+
- `Queue::Client` — extracted private `ensure_configured!` and `resolve_class_name` methods for cleaner validation and class name resolution logic
|
|
81
|
+
- `Queue::Notifier` — extracted `IO_ERRORS` constant (`IO::WaitReadable`, `EOFError`, `IOError`) for cleaner `rescue` in `drain`
|
|
82
|
+
- `Queue::Store` — replaced index `idx_jobs_status_id(status, id)` with `idx_jobs_status_run_at_id(status, run_at, id)` for efficient delayed job lookups
|
|
83
|
+
- `Queue::Store` — `fetch` SQL now uses `WHERE status = 'pending' AND run_at <= ?` with `ORDER BY run_at, id` to process jobs in scheduled order
|
|
84
|
+
- Removed duplicated `monotonic_now` / `realtime_now` from `Runner` and `Store` — now provided by `Clock` module
|
|
85
|
+
- Updated documentation: README (Job module examples, Queue architecture diagram, Clock section), GET_STARTED (delayed jobs guide, Job module usage, minimal queue-only example)
|
|
86
|
+
|
|
87
|
+
## 0.4.5
|
|
88
|
+
|
|
89
|
+
### Breaking Changes
|
|
90
|
+
- `PRAGMAS` is now a frozen lambda `PRAGMAS.call(mmap_size)` instead of a static string — if you referenced this constant directly, update your code
|
|
91
|
+
|
|
92
|
+
### Features
|
|
93
|
+
- New `queue_mmap:` parameter on `Runner` (default: `true`) — allows disabling SQLite mmap for environments where it's unsafe (Docker overlay2)
|
|
94
|
+
- New `mmap:` parameter on `Queue::Store` (default: `true`) — controls `PRAGMA mmap_size` (256 MB when enabled, 0 when disabled)
|
|
95
|
+
- Public `attr_reader :queue_store` on `Runner` — eliminates need for `instance_variable_get` when sharing Store with Client
|
|
96
|
+
|
|
97
|
+
### Bug Fixes
|
|
98
|
+
- **CRITICAL: fetch race condition** — wrapped `UPDATE ... RETURNING` in `BEGIN IMMEDIATE` transaction to prevent two workers from picking up the same job simultaneously
|
|
99
|
+
- **CRITICAL: mmap + Docker overlay2** — `overlay2` filesystem does not guarantee `write()`/`mmap()` coherence, causing SQLite WAL corruption under concurrent multi-process access. mmap is now configurable via `queue_mmap: false` instead of being hardcoded. Documented proper Docker setup with named volumes in `docs/GET_STARTED.md`
|
|
100
|
+
- **`PRAGMA optimize` on shutdown** — wrapped in `rescue nil` to prevent `SQLite3::BusyException` when another process holds the write lock during graceful shutdown
|
|
101
|
+
- **`PRAGMA incremental_vacuum` was a no-op** — added `PRAGMA auto_vacuum = INCREMENTAL` to schema. Without it, `incremental_vacuum` does nothing. Note: only takes effect on newly created databases; existing databases require a one-time `VACUUM`
|
|
102
|
+
|
|
103
|
+
### Improvements
|
|
104
|
+
- Replaced index `idx_jobs_status(status)` with composite `idx_jobs_status_id(status, id)` — eliminates sort step in `fetch` query (`ORDER BY id LIMIT 1` is now a direct B-tree lookup)
|
|
105
|
+
- Fixed `finalize_statements` — changed `%i[@enqueue_stmt ...]` to `%i[enqueue_stmt ...]` with `:"@#{name}"` interpolation for idiomatic `instance_variable_get`/`set` usage
|
|
106
|
+
- Added documentation: `README.md` (concise, with warning markers) and `docs/GET_STARTED.md` (step-by-step guide covering schedule config, Falcon integration, Docker setup, dynamic queue)
|
|
107
|
+
|
|
108
|
+
## 0.4.0
|
|
109
|
+
|
|
110
|
+
### Features
|
|
111
|
+
- **Dynamic job queue** — enqueue jobs at runtime from any process (web, console, rake) with automatic execution by background workers
|
|
112
|
+
- `Queue::Store` — SQLite-backed persistent storage with WAL mode, prepared statements, and optimized pragmas
|
|
113
|
+
- `Queue::Notifier` — `IO.pipe`-based zero-cost wakeup between producer and consumer processes (no polling)
|
|
114
|
+
- `Queue::Client` — public API: `Async::Background::Queue.enqueue(JobClass, *args)`
|
|
115
|
+
- Automatic recovery of stale `running` jobs on worker restart
|
|
116
|
+
- Periodic cleanup of completed jobs (piggyback on fetch, every 5 minutes)
|
|
117
|
+
- `PRAGMA incremental_vacuum` when cleanup removes 100+ rows
|
|
118
|
+
- Worker isolation via `ISOLATION_FORKS` env variable — exclude specific workers from queue processing
|
|
119
|
+
- Custom database path via `queue_db_path` parameter
|
|
120
|
+
- Requires optional `sqlite3` gem (`~> 2.0`) — not included by default, must be added to Gemfile explicitly
|
|
121
|
+
- New Runner parameters: `queue_notifier:` and `queue_db_path:`
|
|
122
|
+
|
|
123
|
+
### Improvements
|
|
124
|
+
- Unified `monotonic_now` usage across `run_job` and `run_queue_job` (was using direct `Process.clock_gettime` call in `run_job`)
|
|
125
|
+
- `Queue::Notifier#drain` — moved `rescue` inside the loop to avoid stack unwinding on each drain cycle
|
|
126
|
+
|
|
127
|
+
## 0.3.0
|
|
128
|
+
|
|
129
|
+
### Features
|
|
130
|
+
- Added optional metrics collection system using shared memory
|
|
131
|
+
- New `Metrics` class with worker-specific performance tracking
|
|
132
|
+
- Public API: `runner.metrics.enabled?`, `runner.metrics.values`, `Metrics.read_all()`
|
|
133
|
+
- Tracks total runs, successes, failures, timeouts, skips, active jobs, and execution times
|
|
134
|
+
- Requires optional `async-utilization` gem dependency
|
|
135
|
+
- Metrics stored in `/tmp/async-background.shm` with lock-free updates per worker
|
|
136
|
+
|
|
137
|
+
## 0.2.6
|
|
138
|
+
|
|
139
|
+
### Improvements
|
|
140
|
+
- Micro-optimization in `wait_with_shutdown` method: use passed `task` parameter instead of `Async::Task.current` for better consistency and slight performance improvement
|
|
141
|
+
|
|
142
|
+
## 0.2.5
|
|
143
|
+
|
|
144
|
+
### Features
|
|
145
|
+
- Added graceful shutdown via signal handlers for SIGINT and SIGTERM
|
|
146
|
+
- Enhanced process lifecycle management with proper signal handling using `Signal.trap` and IO.pipe for async communication
|
|
147
|
+
- Improved robustness for production deployments and container orchestration
|
|
148
|
+
- Updated dependencies to work with latest Async 2.x API (removed deprecated `:parent` parameter usage)
|
|
149
|
+
|
|
150
|
+
## 0.2.4
|
|
151
|
+
|
|
152
|
+
### Improvements
|
|
153
|
+
- Removed hardcoded version warning from main module (was checking against fixed list: 0.1.0, 0.2.2, 0.2.3). Use semantic versioning with pre-release suffixes for unstable versions (e.g., 0.3.0.alpha1) instead
|
|
154
|
+
- Removed hardcoded stable versions list from gemspec description — metadata should describe functionality, not versioning
|
|
155
|
+
- Changed `while true` to idiomatic `loop do` in run method
|
|
156
|
+
- Added `Gemfile.lock` to .gitignore (gems should not commit lockfile)
|
|
157
|
+
- Updated README: clarified that job class must respond to `.perform_now` class method (removed confusing mention of instance `#perform`)
|
|
158
|
+
|
|
159
|
+
## 0.2.2
|
|
160
|
+
|
|
161
|
+
### Bug Fixes
|
|
162
|
+
- **CRITICAL**: Removed logger parameter from Runner initialize (was unused). Fixed initialization to use Console.logger directly which now properly initializes in forked processes with correct context
|
|
163
|
+
|
|
164
|
+
## 0.2.1
|
|
165
|
+
|
|
166
|
+
### Bug Fixes
|
|
167
|
+
- **CRITICAL**: Added missing `require 'console'` in main module. Logger was nil because Console gem was not imported, causing `undefined method 'info' for nil` errors on worker initialization
|
|
168
|
+
|
|
169
|
+
## 0.2.0
|
|
170
|
+
|
|
171
|
+
### Bug Fixes
|
|
172
|
+
- **CRITICAL**: Removed hidden ActiveSupport dependency. Replaced `safe_constantize` with `Object.const_get` + `NameError` handling
|
|
173
|
+
- **CRITICAL**: Fixed validator mismatch: now validates `.perform_now` (class method) instead of `.perform` (instance method)
|
|
174
|
+
- **CRITICAL**: Fixed race condition where entry could disappear from heap during execution. `reschedule` and `heap.push` now always execute after job processing
|
|
175
|
+
- Added full exception backtrace to error logs for production debugging
|
|
176
|
+
- Improved YAML security by removing `Symbol` from `permitted_classes`
|
|
177
|
+
- Removed Mutex from graceful shutdown (anti-pattern in Async). Boolean assignment is atomic in MRI
|
|
178
|
+
|
|
179
|
+
### Features
|
|
180
|
+
- Added optional `logger` parameter to Runner constructor for custom loggers (Rails.logger, etc.)
|
|
181
|
+
- Added `stop()` method for graceful shutdown
|
|
182
|
+
- Added `running?()` method to check scheduler status
|
|
183
|
+
|
|
184
|
+
### Breaking Changes
|
|
185
|
+
- Job class validation now checks for `.perform_now` class method (was checking for `.perform` instance method)
|
|
186
|
+
|
|
187
|
+
## 0.1.0
|
|
188
|
+
|
|
189
|
+
- Initial release
|
|
190
|
+
- Single event loop with min-heap timer (O(log N) scheduling)
|
|
191
|
+
- Skip overlapping execution
|
|
192
|
+
- Startup jitter to prevent thundering herd
|
|
193
|
+
- Monotonic clock for interval jobs, wall clock for cron jobs
|
|
194
|
+
- Deterministic worker sharding via Zlib.crc32
|
|
195
|
+
- Semaphore-based concurrency control
|
|
196
|
+
- Per-job timeout protection
|
|
197
|
+
- Structured logging via Console
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Haydarov Roman
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,193 @@
|
|
|
1
|
+
# Async::Background
|
|
2
|
+
|
|
3
|
+
A lightweight, production-grade cron/interval scheduler for Ruby's [Async](https://github.com/socketry/async) ecosystem. Designed for [Falcon](https://github.com/socketry/falcon) but works with any Async-based application.
|
|
4
|
+
|
|
5
|
+
## What It Does
|
|
6
|
+
|
|
7
|
+
- **Cron & interval scheduling** — single event loop + min-heap, scales to hundreds of jobs
|
|
8
|
+
- **Dynamic job queue** — enqueue jobs at runtime via SQLite, pick up by background workers
|
|
9
|
+
- **Delayed jobs** — schedule jobs for future execution with `perform_in` / `perform_at` (Sidekiq-like API)
|
|
10
|
+
- **Multi-process safe** — deterministic worker sharding via `Zlib.crc32`, no duplicate execution
|
|
11
|
+
- **Skip overlapping** — if a job is still running when its next tick arrives, the tick is skipped
|
|
12
|
+
- **Timeout protection** — per-job configurable timeout via `Async::Task#with_timeout`
|
|
13
|
+
- **Startup jitter** — random delay to prevent thundering herd after restart
|
|
14
|
+
- **Optional metrics** — shared memory performance tracking with `async-utilization`
|
|
15
|
+
|
|
16
|
+
## Requirements
|
|
17
|
+
|
|
18
|
+
- **Ruby >= 3.3** — Fiber Scheduler production-ready ([why?](#why-ruby-33))
|
|
19
|
+
- **Async ~> 2.0** — Fiber Scheduler-based concurrency
|
|
20
|
+
- **Fugit ~> 1.0** — cron expression parsing
|
|
21
|
+
|
|
22
|
+
## Installation
|
|
23
|
+
|
|
24
|
+
```ruby
|
|
25
|
+
# Gemfile
|
|
26
|
+
gem "async-background"
|
|
27
|
+
|
|
28
|
+
# Optional
|
|
29
|
+
gem "sqlite3", "~> 2.0" # for dynamic job queue
|
|
30
|
+
gem "async-utilization", "~> 0.3" # for metrics
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## ➡️ [Get Started](docs/GET_STARTED.md)
|
|
34
|
+
|
|
35
|
+
Step-by-step setup guide: schedule config, Falcon integration, Docker, dynamic queue, delayed jobs.
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## Quick Example: Job Module
|
|
40
|
+
|
|
41
|
+
Include `Async::Background::Job` for a Sidekiq-like interface:
|
|
42
|
+
|
|
43
|
+
```ruby
|
|
44
|
+
class SendEmailJob
|
|
45
|
+
include Async::Background::Job
|
|
46
|
+
|
|
47
|
+
def perform(user_id, template)
|
|
48
|
+
user = User.find(user_id)
|
|
49
|
+
Mailer.send(user, template)
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
# Immediate execution in the queue
|
|
54
|
+
SendEmailJob.perform_async(user_id, "welcome")
|
|
55
|
+
|
|
56
|
+
# Execute after 5 minutes
|
|
57
|
+
SendEmailJob.perform_in(300, user_id, "reminder")
|
|
58
|
+
|
|
59
|
+
# Execute at a specific time
|
|
60
|
+
SendEmailJob.perform_at(Time.new(2026, 4, 1, 9, 0, 0), user_id, "scheduled")
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Or use the lower-level API directly:
|
|
64
|
+
|
|
65
|
+
```ruby
|
|
66
|
+
Async::Background::Queue.enqueue(SendEmailJob, user_id, "welcome")
|
|
67
|
+
Async::Background::Queue.enqueue_in(300, SendEmailJob, user_id, "reminder")
|
|
68
|
+
Async::Background::Queue.enqueue_at(Time.new(2026, 4, 1, 9, 0, 0), SendEmailJob, user_id, "scheduled")
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## ⚠️ Important Notes
|
|
74
|
+
|
|
75
|
+
### Docker: SQLite requires a named volume
|
|
76
|
+
|
|
77
|
+
The SQLite database **must not** live on Docker's `overlay2` filesystem. The `overlay2` driver breaks coherence between `write()` and `mmap()`, which corrupts SQLite WAL under concurrent access.
|
|
78
|
+
|
|
79
|
+
```yaml
|
|
80
|
+
# docker-compose.yml
|
|
81
|
+
volumes:
|
|
82
|
+
- queue-data:/app/tmp/queue # ← named volume, NOT overlay2
|
|
83
|
+
|
|
84
|
+
volumes:
|
|
85
|
+
queue-data:
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Without this, you will get database crashes in multi-process mode. See [Get Started → Step 3](docs/GET_STARTED.md#step-3-docker) for details.
|
|
89
|
+
|
|
90
|
+
### Fork safety
|
|
91
|
+
|
|
92
|
+
SQLite connections **must not** cross `fork()` boundaries. Always open connections **after** fork (inside `container.run` block), never before. The gem handles this internally via lazy `ensure_connection`, but if you create a `Queue::Store` manually for schema setup, close it before fork:
|
|
93
|
+
|
|
94
|
+
```ruby
|
|
95
|
+
store = Async::Background::Queue::Store.new(path: db_path)
|
|
96
|
+
store.ensure_database!
|
|
97
|
+
store.close # ← before fork
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
### Clock handling
|
|
101
|
+
|
|
102
|
+
The `Clock` module provides shared time helpers used across the codebase:
|
|
103
|
+
|
|
104
|
+
- **`monotonic_now`** (`CLOCK_MONOTONIC`) — for in-process intervals and durations, immune to NTP drift / wall-clock jumps
|
|
105
|
+
- **`realtime_now`** (`CLOCK_REALTIME`) — for persisted timestamps (SQLite `run_at`, `created_at`, `locked_at`)
|
|
106
|
+
|
|
107
|
+
Interval jobs use monotonic clock. Cron jobs use `Time.now` because "every day at 3am" must respect real time. These are different clocks by design.
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## Architecture
|
|
112
|
+
|
|
113
|
+
```
|
|
114
|
+
schedule.yml
|
|
115
|
+
│
|
|
116
|
+
▼
|
|
117
|
+
build_heap ← parse config, validate, assign workers
|
|
118
|
+
│
|
|
119
|
+
▼
|
|
120
|
+
MinHeap<Entry> ← O(log N) push/pop, sorted by next_run_at
|
|
121
|
+
│
|
|
122
|
+
▼
|
|
123
|
+
1 scheduler loop ← single Async task, sleeps until next entry
|
|
124
|
+
│
|
|
125
|
+
▼
|
|
126
|
+
Semaphore ← limits concurrent job execution
|
|
127
|
+
│
|
|
128
|
+
▼
|
|
129
|
+
run_job ← timeout, logging, error handling
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
### Queue Architecture
|
|
133
|
+
|
|
134
|
+
```
|
|
135
|
+
Producer (web/console) Consumer (background worker)
|
|
136
|
+
│ │
|
|
137
|
+
▼ ▼
|
|
138
|
+
Queue::Client Queue::Store.fetch
|
|
139
|
+
│ (WHERE run_at <= now)
|
|
140
|
+
├─ push(class, args, run_at) │
|
|
141
|
+
├─ push_in(delay, class, args) ▼
|
|
142
|
+
└─ push_at(time, class, args) run_queue_job
|
|
143
|
+
│ │
|
|
144
|
+
▼ ▼
|
|
145
|
+
Queue::Store ──── SQLite ──── Queue::Notifier
|
|
146
|
+
(INSERT job) (jobs table) (IO.pipe wakeup)
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
## Schedule Config
|
|
150
|
+
|
|
151
|
+
| Key | Required | Description |
|
|
152
|
+
|---|---|---|
|
|
153
|
+
| `class` | yes | Must include `Async::Background::Job` |
|
|
154
|
+
| `every` | one of | Interval in seconds between runs |
|
|
155
|
+
| `cron` | one of | Cron expression (parsed by Fugit) |
|
|
156
|
+
| `timeout` | no | Max execution time in seconds (default: 30) |
|
|
157
|
+
| `worker` | no | Pin to specific worker index. If omitted — `crc32(name) % total_workers` |
|
|
158
|
+
|
|
159
|
+
## SQLite Pragmas
|
|
160
|
+
|
|
161
|
+
| Pragma | Value | Why |
|
|
162
|
+
|---|---|---|
|
|
163
|
+
| `journal_mode` | WAL | Concurrent reads during writes |
|
|
164
|
+
| `synchronous` | NORMAL | Safe with WAL, lower fsync overhead |
|
|
165
|
+
| `mmap_size` | 256 MB | Fast reads ([requires proper filesystem](#docker-sqlite-requires-a-named-volume)) |
|
|
166
|
+
| `cache_size` | 16000 pages | ~64 MB page cache |
|
|
167
|
+
| `busy_timeout` | 5000 ms | Wait instead of failing on lock contention |
|
|
168
|
+
|
|
169
|
+
## Metrics
|
|
170
|
+
|
|
171
|
+
When `async-utilization` gem is available, metrics are collected in shared memory (`/tmp/async-background.shm`) with lock-free updates per worker.
|
|
172
|
+
|
|
173
|
+
```ruby
|
|
174
|
+
runner.metrics.values
|
|
175
|
+
# => { total_runs: 142, total_successes: 140, total_failures: 2,
|
|
176
|
+
# total_timeouts: 0, total_skips: 5, active_jobs: 1,
|
|
177
|
+
# last_run_at: 1774445243, last_duration_ms: 1250 }
|
|
178
|
+
|
|
179
|
+
# Read all workers at once (no server needed)
|
|
180
|
+
Async::Background::Metrics.read_all(total_workers: 2)
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
Without the gem — metrics are silently disabled, zero overhead.
|
|
184
|
+
|
|
185
|
+
## Why Ruby 3.3?
|
|
186
|
+
|
|
187
|
+
- Ruby 3.0 introduced Fiber Scheduler but had critical bugs
|
|
188
|
+
- Ruby 3.2 is the first production-ready release (per Samuel Williams)
|
|
189
|
+
- `io-event >= 1.14` (pulled by latest `async`) requires Ruby `>= 3.3`
|
|
190
|
+
|
|
191
|
+
## License
|
|
192
|
+
|
|
193
|
+
MIT
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative 'lib/async/background/version'
|
|
4
|
+
|
|
5
|
+
Gem::Specification.new do |spec|
|
|
6
|
+
spec.name = 'async-background'
|
|
7
|
+
spec.version = Async::Background::VERSION
|
|
8
|
+
spec.authors = ['Roman Hajdarov']
|
|
9
|
+
spec.email = ['romnhajdarov@gmail.com']
|
|
10
|
+
|
|
11
|
+
spec.summary = 'Lightweight heap-based cron/interval scheduler for Async.'
|
|
12
|
+
spec.description = 'A production-grade lightweight scheduler built on top of Async. ' \
|
|
13
|
+
'Single event loop with min-heap timer, skip-overlapping execution, ' \
|
|
14
|
+
'jitter, monotonic clock intervals, semaphore concurrency control, ' \
|
|
15
|
+
'and deterministic worker sharding. Designed for Falcon but works ' \
|
|
16
|
+
'with any Async-based application.'
|
|
17
|
+
|
|
18
|
+
spec.homepage = 'https://github.com/roman-haidarov/async-background'
|
|
19
|
+
spec.license = 'MIT'
|
|
20
|
+
|
|
21
|
+
spec.metadata = {
|
|
22
|
+
'source_code_uri' => 'https://github.com/roman-haidarov/async-background',
|
|
23
|
+
'changelog_uri' => 'https://github.com/roman-haidarov/async-background/blob/main/CHANGELOG.md',
|
|
24
|
+
'bug_tracker_uri' => 'https://github.com/roman-haidarov/async-background/issues'
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
spec.files = Dir.glob('lib/**/*', base: __dir__) +
|
|
28
|
+
%w[README.md CHANGELOG.md LICENSE async-background.gemspec]
|
|
29
|
+
spec.require_paths = ['lib']
|
|
30
|
+
|
|
31
|
+
spec.required_ruby_version = '>= 3.3'
|
|
32
|
+
|
|
33
|
+
spec.add_dependency 'async', '~> 2.0'
|
|
34
|
+
spec.add_dependency 'console', '~> 1.0'
|
|
35
|
+
spec.add_dependency 'fugit', '~> 1.0'
|
|
36
|
+
|
|
37
|
+
# Optional: add to your own Gemfile if you need these features
|
|
38
|
+
# gem 'sqlite3', '~> 2.0' # dynamic job queue
|
|
39
|
+
# gem 'async-utilization', '~> 0.3' # shared-memory worker metrics
|
|
40
|
+
|
|
41
|
+
spec.add_development_dependency 'rake', '~> 13.0'
|
|
42
|
+
spec.add_development_dependency 'rspec', '~> 3.12'
|
|
43
|
+
end
|
|
@@ -4,7 +4,19 @@ module Async
|
|
|
4
4
|
module Background
|
|
5
5
|
module Queue
|
|
6
6
|
class Notifier
|
|
7
|
-
|
|
7
|
+
# Error groups
|
|
8
|
+
WRITE_DROPPED = [
|
|
9
|
+
IO::WaitWritable, # buffer full — consumer is behind, skip
|
|
10
|
+
Errno::EAGAIN, # same as above on some platforms
|
|
11
|
+
IOError, # our own writer end has been closed
|
|
12
|
+
Errno::EPIPE # the reader end is gone (consumer crashed) — skip
|
|
13
|
+
].freeze
|
|
14
|
+
|
|
15
|
+
READ_EXHAUSTED = [
|
|
16
|
+
IO::WaitReadable, # nothing left in the buffer — normal exit
|
|
17
|
+
EOFError, # writer end closed — no more data ever
|
|
18
|
+
IOError # our own reader end has been closed
|
|
19
|
+
].freeze
|
|
8
20
|
|
|
9
21
|
attr_reader :reader, :writer
|
|
10
22
|
|
|
@@ -16,12 +28,14 @@ module Async
|
|
|
16
28
|
|
|
17
29
|
def notify
|
|
18
30
|
@writer.write_nonblock("\x01")
|
|
19
|
-
rescue
|
|
20
|
-
#
|
|
21
|
-
|
|
22
|
-
#
|
|
31
|
+
rescue *WRITE_DROPPED
|
|
32
|
+
# All write failures are non-fatal: the job is already in the
|
|
33
|
+
# store, and missing one wake-up only delays pickup by at most
|
|
34
|
+
# one poll interval.
|
|
23
35
|
end
|
|
24
36
|
|
|
37
|
+
alias notify_all notify
|
|
38
|
+
|
|
25
39
|
def wait(timeout: nil)
|
|
26
40
|
@reader.wait_readable(timeout)
|
|
27
41
|
drain
|
|
@@ -53,7 +67,7 @@ module Async
|
|
|
53
67
|
def drain
|
|
54
68
|
loop do
|
|
55
69
|
@reader.read_nonblock(256)
|
|
56
|
-
rescue *
|
|
70
|
+
rescue *READ_EXHAUSTED
|
|
57
71
|
break
|
|
58
72
|
end
|
|
59
73
|
nil
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'socket'
|
|
4
|
+
|
|
5
|
+
module Async
|
|
6
|
+
module Background
|
|
7
|
+
module Queue
|
|
8
|
+
class SocketNotifier
|
|
9
|
+
# Errors that indicate a worker is unavailable - silently skip
|
|
10
|
+
UNAVAILABLE = [
|
|
11
|
+
Errno::ENOENT, # Socket file doesn't exist (worker hasn't started yet)
|
|
12
|
+
Errno::ECONNREFUSED, # File exists but no one listening (worker died)
|
|
13
|
+
Errno::EPIPE, # Connection broken during write
|
|
14
|
+
Errno::EAGAIN, # Socket buffer full - wake-up already queued
|
|
15
|
+
IO::WaitWritable, # Same as EAGAIN on some platforms
|
|
16
|
+
Errno::ECONNRESET # Connection reset by peer
|
|
17
|
+
].freeze
|
|
18
|
+
|
|
19
|
+
def initialize(socket_dir:, total_workers:)
|
|
20
|
+
@socket_dir = socket_dir
|
|
21
|
+
@total_workers = total_workers
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def notify_all
|
|
25
|
+
(1..@total_workers).each do |worker_index|
|
|
26
|
+
notify_one(worker_index)
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
private
|
|
31
|
+
|
|
32
|
+
def notify_one(worker_index)
|
|
33
|
+
path = socket_path(worker_index)
|
|
34
|
+
sock = UNIXSocket.new(path)
|
|
35
|
+
begin
|
|
36
|
+
sock.write_nonblock("\x01")
|
|
37
|
+
ensure
|
|
38
|
+
sock.close rescue nil
|
|
39
|
+
end
|
|
40
|
+
rescue *UNAVAILABLE
|
|
41
|
+
# Worker is unavailable - not a problem.
|
|
42
|
+
# The job is already in the database. The worker will:
|
|
43
|
+
# - Pick it up on next poll (within QUEUE_POLL_INTERVAL seconds), or
|
|
44
|
+
# - Pick it up when it starts/restarts via normal fetch loop
|
|
45
|
+
rescue => e
|
|
46
|
+
# Unexpected error - log but don't crash the enqueue operation
|
|
47
|
+
Console.logger.warn(self) { "SocketNotifier#notify_one(#{worker_index}) failed: #{e.class} #{e.message}" } rescue nil
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
def socket_path(worker_index)
|
|
51
|
+
File.join(@socket_dir, "async_bg_worker_#{worker_index}.sock")
|
|
52
|
+
end
|
|
53
|
+
end
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'socket'
|
|
4
|
+
require 'async/notification'
|
|
5
|
+
require 'fileutils'
|
|
6
|
+
|
|
7
|
+
module Async
|
|
8
|
+
module Background
|
|
9
|
+
module Queue
|
|
10
|
+
class SocketWaker
|
|
11
|
+
attr_reader :path
|
|
12
|
+
|
|
13
|
+
def initialize(path)
|
|
14
|
+
@path = path
|
|
15
|
+
@server = nil
|
|
16
|
+
@notification = ::Async::Notification.new
|
|
17
|
+
@running = false
|
|
18
|
+
@accept_task = nil
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def open!
|
|
22
|
+
cleanup_stale_socket
|
|
23
|
+
ensure_directory
|
|
24
|
+
@server = UNIXServer.new(@path)
|
|
25
|
+
File.chmod(0600, @path)
|
|
26
|
+
@running = true
|
|
27
|
+
rescue Errno::EADDRINUSE
|
|
28
|
+
raise "Socket #{@path} is already in use by another process"
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def start_accept_loop(parent_task)
|
|
32
|
+
@accept_task = parent_task.async do |task|
|
|
33
|
+
while @running
|
|
34
|
+
begin
|
|
35
|
+
client = @server.accept_nonblock
|
|
36
|
+
handle_client(task, client)
|
|
37
|
+
rescue IO::WaitReadable
|
|
38
|
+
@server.wait_readable
|
|
39
|
+
rescue Errno::EBADF, IOError
|
|
40
|
+
break
|
|
41
|
+
rescue => e
|
|
42
|
+
Console.logger.error(self) { "SocketWaker accept error: #{e.class} #{e.message}" }
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
rescue => e
|
|
46
|
+
Console.logger.error(self) { "SocketWaker loop crashed: #{e.class} #{e.message}\n#{e.backtrace.join("\n")}" }
|
|
47
|
+
ensure
|
|
48
|
+
@accept_task = nil
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def wait(timeout: nil)
|
|
53
|
+
if timeout
|
|
54
|
+
::Async::Task.current.with_timeout(timeout) { @notification.wait }
|
|
55
|
+
else
|
|
56
|
+
@notification.wait
|
|
57
|
+
end
|
|
58
|
+
rescue ::Async::TimeoutError
|
|
59
|
+
# Timeout is normal - listener will fall back to polling
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def signal
|
|
63
|
+
@notification.signal
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
def close
|
|
67
|
+
@running = false
|
|
68
|
+
if @accept_task && !@accept_task.finished?
|
|
69
|
+
@accept_task.stop rescue nil
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
@server&.close rescue nil
|
|
73
|
+
@server = nil
|
|
74
|
+
File.unlink(@path) rescue nil
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
private
|
|
78
|
+
|
|
79
|
+
def handle_client(parent_task, client)
|
|
80
|
+
parent_task.async do
|
|
81
|
+
begin
|
|
82
|
+
loop do
|
|
83
|
+
client.read_nonblock(256)
|
|
84
|
+
rescue IO::WaitReadable
|
|
85
|
+
client.wait_readable
|
|
86
|
+
retry
|
|
87
|
+
rescue EOFError, Errno::ECONNRESET
|
|
88
|
+
break
|
|
89
|
+
end
|
|
90
|
+
rescue => e
|
|
91
|
+
Console.logger.warn(self) { "SocketWaker client handler error: #{e.class} #{e.message}" }
|
|
92
|
+
ensure
|
|
93
|
+
client.close rescue nil
|
|
94
|
+
@notification.signal
|
|
95
|
+
end
|
|
96
|
+
end
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
def cleanup_stale_socket
|
|
100
|
+
return unless File.exist?(@path)
|
|
101
|
+
|
|
102
|
+
begin
|
|
103
|
+
UNIXSocket.open(@path) { |s| s.close }
|
|
104
|
+
|
|
105
|
+
raise "Socket #{@path} is already in use by another process (worker_index conflict?)"
|
|
106
|
+
rescue Errno::ECONNREFUSED, Errno::ENOENT
|
|
107
|
+
File.unlink(@path) rescue nil
|
|
108
|
+
end
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
def ensure_directory
|
|
112
|
+
dir = File.dirname(@path)
|
|
113
|
+
FileUtils.mkdir_p(dir) unless File.exist?(dir)
|
|
114
|
+
end
|
|
115
|
+
end
|
|
116
|
+
end
|
|
117
|
+
end
|
|
118
|
+
end
|
|
@@ -53,6 +53,7 @@ module Async
|
|
|
53
53
|
def ensure_database!
|
|
54
54
|
require_sqlite3
|
|
55
55
|
db = SQLite3::Database.new(@path)
|
|
56
|
+
db.execute('PRAGMA busy_timeout = 5000')
|
|
56
57
|
db.execute_batch(PRAGMAS.call(@mmap ? MMAP_SIZE : 0))
|
|
57
58
|
db.execute_batch(SCHEMA)
|
|
58
59
|
db.execute("PRAGMA wal_checkpoint(TRUNCATE)")
|
|
@@ -128,6 +129,7 @@ module Async
|
|
|
128
129
|
require_sqlite3
|
|
129
130
|
finalize_statements
|
|
130
131
|
@db = SQLite3::Database.new(@path)
|
|
132
|
+
@db.execute('PRAGMA busy_timeout = 5000')
|
|
131
133
|
@db.execute_batch(PRAGMAS.call(@mmap ? MMAP_SIZE : 0))
|
|
132
134
|
|
|
133
135
|
unless @schema_checked
|
|
@@ -19,7 +19,7 @@ module Async
|
|
|
19
19
|
|
|
20
20
|
def initialize(
|
|
21
21
|
config_path:, job_count: 2, worker_index:, total_workers:,
|
|
22
|
-
|
|
22
|
+
queue_socket_dir: nil, queue_db_path: nil, queue_mmap: true
|
|
23
23
|
)
|
|
24
24
|
@logger = Console.logger
|
|
25
25
|
@worker_index = worker_index
|
|
@@ -33,7 +33,7 @@ module Async
|
|
|
33
33
|
@semaphore = ::Async::Semaphore.new(job_count)
|
|
34
34
|
@heap = build_heap(config_path)
|
|
35
35
|
|
|
36
|
-
setup_queue(
|
|
36
|
+
setup_queue(queue_socket_dir, queue_db_path, queue_mmap)
|
|
37
37
|
end
|
|
38
38
|
|
|
39
39
|
def run
|
|
@@ -46,6 +46,7 @@ module Async
|
|
|
46
46
|
|
|
47
47
|
semaphore.acquire {}
|
|
48
48
|
@queue_store&.close
|
|
49
|
+
@queue_waker&.close
|
|
49
50
|
end
|
|
50
51
|
end
|
|
51
52
|
|
|
@@ -55,7 +56,7 @@ module Async
|
|
|
55
56
|
@running = false
|
|
56
57
|
logger.info { "Async::Background: stopping gracefully" }
|
|
57
58
|
shutdown.signal
|
|
58
|
-
@
|
|
59
|
+
@queue_waker&.signal
|
|
59
60
|
end
|
|
60
61
|
|
|
61
62
|
def running?
|
|
@@ -64,35 +65,40 @@ module Async
|
|
|
64
65
|
|
|
65
66
|
private
|
|
66
67
|
|
|
67
|
-
def setup_queue(
|
|
68
|
+
def setup_queue(queue_socket_dir, queue_db_path, queue_mmap)
|
|
68
69
|
@listen_queue = false
|
|
69
|
-
return unless
|
|
70
|
+
return unless queue_socket_dir
|
|
70
71
|
|
|
71
72
|
# Lazy require — only loaded when queue is actually used
|
|
72
73
|
require_relative 'queue/store'
|
|
73
|
-
require_relative 'queue/
|
|
74
|
+
require_relative 'queue/socket_waker'
|
|
74
75
|
require_relative 'queue/client'
|
|
75
76
|
|
|
76
77
|
isolated = ENV.fetch("ISOLATION_FORKS", "").split(",").map(&:to_i)
|
|
77
78
|
return if isolated.include?(worker_index)
|
|
78
79
|
|
|
79
|
-
@listen_queue
|
|
80
|
-
@
|
|
81
|
-
@queue_store = Queue::Store.new(
|
|
80
|
+
@listen_queue = true
|
|
81
|
+
@queue_store = Queue::Store.new(
|
|
82
82
|
path: queue_db_path || Queue::Store.default_path,
|
|
83
83
|
mmap: queue_mmap
|
|
84
84
|
)
|
|
85
85
|
|
|
86
|
+
socket_path = File.join(queue_socket_dir, "async_bg_worker_#{worker_index}.sock")
|
|
87
|
+
@queue_waker = Queue::SocketWaker.new(socket_path)
|
|
88
|
+
@queue_waker.open!
|
|
89
|
+
|
|
86
90
|
recovered = @queue_store.recover(worker_index)
|
|
87
91
|
logger.info { "Async::Background queue: recovered #{recovered} stale jobs" } if recovered > 0
|
|
88
92
|
end
|
|
89
93
|
|
|
90
94
|
def start_queue_listener(task)
|
|
95
|
+
@queue_waker.start_accept_loop(task)
|
|
96
|
+
|
|
91
97
|
task.async do
|
|
92
98
|
logger.info { "Async::Background queue: listening on worker #{worker_index}" }
|
|
93
99
|
|
|
94
100
|
while running?
|
|
95
|
-
@
|
|
101
|
+
@queue_waker.wait(timeout: QUEUE_POLL_INTERVAL)
|
|
96
102
|
|
|
97
103
|
while running?
|
|
98
104
|
job = @queue_store.fetch(worker_index)
|
|
@@ -196,7 +202,7 @@ module Async
|
|
|
196
202
|
@signal_r.wait_readable
|
|
197
203
|
@signal_r.read_nonblock(256) rescue nil
|
|
198
204
|
shutdown.signal
|
|
199
|
-
@
|
|
205
|
+
@queue_waker&.signal
|
|
200
206
|
break unless running?
|
|
201
207
|
end
|
|
202
208
|
end
|
data/lib/async/background.rb
CHANGED
|
@@ -12,5 +12,6 @@ require_relative 'background/min_heap'
|
|
|
12
12
|
require_relative 'background/entry'
|
|
13
13
|
require_relative 'background/metrics'
|
|
14
14
|
require_relative 'background/runner'
|
|
15
|
+
require_relative 'background/queue/store'
|
|
15
16
|
require_relative 'background/queue/client'
|
|
16
17
|
require_relative 'background/job'
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: async-background
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.6.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Roman Hajdarov
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-04-07 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: async
|
|
@@ -90,6 +90,10 @@ executables: []
|
|
|
90
90
|
extensions: []
|
|
91
91
|
extra_rdoc_files: []
|
|
92
92
|
files:
|
|
93
|
+
- CHANGELOG.md
|
|
94
|
+
- LICENSE
|
|
95
|
+
- README.md
|
|
96
|
+
- async-background.gemspec
|
|
93
97
|
- lib/async/background.rb
|
|
94
98
|
- lib/async/background/clock.rb
|
|
95
99
|
- lib/async/background/entry.rb
|
|
@@ -98,6 +102,8 @@ files:
|
|
|
98
102
|
- lib/async/background/min_heap.rb
|
|
99
103
|
- lib/async/background/queue/client.rb
|
|
100
104
|
- lib/async/background/queue/notifier.rb
|
|
105
|
+
- lib/async/background/queue/socket_notifier.rb
|
|
106
|
+
- lib/async/background/queue/socket_waker.rb
|
|
101
107
|
- lib/async/background/queue/store.rb
|
|
102
108
|
- lib/async/background/runner.rb
|
|
103
109
|
- lib/async/background/version.rb
|