async-background 0.6.0 → 0.6.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b7e4015afbfcd9ee8291c28caa34e69de838f283fd0d6374a81cc0d53a87b90d
-  data.tar.gz: b0333ec4f4626895e01c2b77da17abb569c912e6449def849fa8e0918fb06a58
+  metadata.gz: 0f4392db5a752c9a07da3c140fedec1ad2234950ccce48d127eb7c4de188bfee
+  data.tar.gz: 31d3c8f2cc5b303361c081e95162d13e7ed04917f7ae93187abd9c3b3c175445
 SHA512:
-  metadata.gz: 245fa669ebf1573e37770eb16f86904921cbe0f0effb40099c613bda9732e4e8732854f24cdf84421efa877d69e25271544bc279bf5e35ff3c7e364cd8689252
-  data.tar.gz: aff2eeba3e4793b4d5e686fb5bf3f27c185f61492aee837d7100a9dde35c3d97cf904c0c037c4dacd1655dd0da49b9f556d5003620e0e0f2c256735539805f29
+  metadata.gz: 90225c64139b4ec18ed9bb72bf82161f9642f14f39f9031ad1d64ddf1b6a074bb2b7690722017054aac521fda6395ddcae755ed17fb264fc32620639a7a64219
+  data.tar.gz: b7ff64e05abc6e5a9a4f0122c86f4e73b2f7f170978213b307104b96bd907384a5ab16ff61db96bb50c91bb5999cf98f0cc0318601741bd3b78357b6a0302f27

data/CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## 0.6.1
+
+### Bug Fixes
+- **Runner: cron jobs busy-loop on overlap skip** — when a scheduled run was skipped because the previous one was still active, the entry was re-pushed to the heap without calling `reschedule`. For cron jobs (where `interval` is `nil`), this meant `next_run_at` was never advanced to the next cron tick, causing the entry to be picked up again immediately on the next loop iteration. Skip branch now calls `entry.reschedule(monotonic_now)` like the normal path
+- **Store: prepared statement not reset on fetch error** — `@fetch_stmt.reset!` was called after `execute` returned, so an exception inside `execute` left the statement in a dirty state and the next `fetch` could fail. Wrapped in `begin/ensure` to guarantee reset
+
+### Improvements
+- **SocketNotifier: non-blocking enqueue with ring fallback** — `notify_all` no longer connects to all N worker sockets on every enqueue. `UNIXSocket.new` is a blocking, non-fiber-aware syscall, and notifying every worker blocked the Falcon reactor for N `connect()` calls on the hot HTTP enqueue path. Now wakes a single worker chosen by random offset, falling back through the ring only if the chosen worker is dead (`ECONNREFUSED` etc.). Happy path: 1 connect. Worst case (all workers down): N connects — same as before, but only when actually needed. Safe because the queue is shared in SQLite, not sharded per worker
+- **SocketNotifier: cleaned up `UNAVAILABLE` error list** — removed `IO::WaitWritable` and `Errno::EAGAIN`. They implied "socket buffer full", but `write_nonblock` of a single byte to a freshly-opened connection cannot fill the kernel buffer. Listing them only misled readers
+- **Store: partial index for pending lookup** — replaced `idx_jobs_status_run_at_id(status, run_at, id)` with partial index `idx_jobs_pending(run_at, id) WHERE status = 'pending'`. Smaller on disk, cheaper to update, and matches the only query that uses it (`fetch`). `done`/`failed`/`running` rows no longer occupy index pages
+
 ## 0.6.0
 
 ### Breaking Changes

data/README.md

@@ -1,76 +1,74 @@
 # Async::Background
 
-A lightweight, production-grade cron/interval scheduler for Ruby's [Async](https://github.com/socketry/async) ecosystem. Designed for [Falcon](https://github.com/socketry/falcon) but works with any Async-based application.
+A lightweight cron, interval, and job-queue scheduler for Ruby's [Async](https://github.com/socketry/async) ecosystem. Built for [Falcon](https://github.com/socketry/falcon), works with any Async app.
 
-## What It Does
-
-- **Cron & interval scheduling** — single event loop + min-heap, scales to hundreds of jobs
-- **Dynamic job queue** — enqueue jobs at runtime via SQLite, pick up by background workers
-- **Delayed jobs** — schedule jobs for future execution with `perform_in` / `perform_at` (Sidekiq-like API)
-- **Multi-process safe** — deterministic worker sharding via `Zlib.crc32`, no duplicate execution
-- **Skip overlapping** — if a job is still running when its next tick arrives, the tick is skipped
-- **Timeout protection** — per-job configurable timeout via `Async::Task#with_timeout`
-- **Startup jitter** — random delay to prevent thundering herd after restart
-- **Optional metrics** — shared memory performance tracking with `async-utilization`
+- **Cron & interval scheduling** on a single event loop with a min-heap
+- **Dynamic job queue** backed by SQLite, with delayed jobs (`perform_in` / `perform_at`)
+- **Cross-process wake-ups** over Unix domain sockets — web workers can enqueue and instantly wake background workers
+- **Multi-process safe** — deterministic worker sharding, no duplicate execution
+- **Per-job timeouts**, skip-on-overlap, startup jitter, optional metrics
 
 ## Requirements
 
-- **Ruby >= 3.3** — Fiber Scheduler production-ready ([why?](#why-ruby-33))
-- **Async ~> 2.0** — Fiber Scheduler-based concurrency
-- **Fugit ~> 1.0** — cron expression parsing
+- Ruby >= 3.3
+- `async ~> 2.0`, `fugit ~> 1.0`
+- `sqlite3 ~> 2.0` (optional, for the job queue)
+- `async-utilization ~> 0.3` (optional, for metrics)
 
-## Installation
+## Install
 
 ```ruby
 # Gemfile
 gem "async-background"
-
-# Optional
-gem "sqlite3", "~> 2.0"           # for dynamic job queue
-gem "async-utilization", "~> 0.3"  # for metrics
+gem "sqlite3", "~> 2.0"            # optional
+gem "async-utilization", "~> 0.3"  # optional
 ```
 
 ## ➡️ [Get Started](docs/GET_STARTED.md)
 
-Step-by-step setup guide: schedule config, Falcon integration, Docker, dynamic queue, delayed jobs.
+Full setup walkthrough: schedule config, Falcon integration, Docker, queue, delayed jobs.
 
 ---
 
-## Quick Example: Job Module
-
-Include `Async::Background::Job` for a Sidekiq-like interface:
+## Quick Look
 
 ```ruby
 class SendEmailJob
   include Async::Background::Job
 
   def perform(user_id, template)
-    user = User.find(user_id)
-    Mailer.send(user, template)
+    Mailer.send(User.find(user_id), template)
   end
 end
 
-# Immediate execution in the queue
 SendEmailJob.perform_async(user_id, "welcome")
-
-# Execute after 5 minutes
 SendEmailJob.perform_in(300, user_id, "reminder")
-
-# Execute at a specific time
-SendEmailJob.perform_at(Time.new(2026, 4, 1, 9, 0, 0), user_id, "scheduled")
+SendEmailJob.perform_at(Time.new(2026, 4, 1, 9), user_id, "scheduled")
 ```
 
-Or use the lower-level API directly:
+Schedule recurring jobs in `config/schedule.yml`:
 
-```ruby
-Async::Background::Queue.enqueue(SendEmailJob, user_id, "welcome")
-Async::Background::Queue.enqueue_in(300, SendEmailJob, user_id, "reminder")
-Async::Background::Queue.enqueue_at(Time.new(2026, 4, 1, 9, 0, 0), SendEmailJob, user_id, "scheduled")
+```yaml
+sync_products:
+  class: SyncProductsJob
+  every: 60
+
+daily_report:
+  class: DailyReportJob
+  cron: "0 3 * * *"
+  timeout: 120
 ```
 
+| Key | Description |
+|---|---|
+| `class` | Job class — must include `Async::Background::Job` |
+| `every` / `cron` | One of: interval in seconds, or cron expression |
+| `timeout` | Max execution time in seconds (default: 30) |
+| `worker` | Pin to a specific worker. Default: `crc32(name) % total_workers` |
+
 ---
 
-## ⚠️ Important Notes
+## Gotchas
 
 ### Docker: SQLite requires a named volume
 
@@ -78,18 +76,20 @@ The SQLite database **must not** live on Docker's `overlay2` filesystem. The `ov
 
 ```yaml
 # docker-compose.yml
-volumes:
-  - queue-data:/app/tmp/queue   # ← named volume, NOT overlay2
+services:
+  app:
+    volumes:
+      - queue-data:/app/tmp/queue   # ← named volume, NOT overlay2
 
 volumes:
   queue-data:
 ```
 
-Without this, you will get database crashes in multi-process mode. See [Get Started → Step 3](docs/GET_STARTED.md#step-3-docker) for details.
+Without this, you will get database crashes in multi-process mode. See [Get Started → Step 3](docs/GET_STARTED.md#step-3-docker) for details. If you can't use a named volume, pass `queue_mmap: false` to disable mmap entirely.
 
-### Fork safety
+### Other gotchas
 
-SQLite connections **must not** cross `fork()` boundaries. Always open connections **after** fork (inside `container.run` block), never before. The gem handles this internally via lazy `ensure_connection`, but if you create a `Queue::Store` manually for schema setup, close it before fork:
+**Don't share SQLite connections across `fork()`.** The gem opens connections lazily after fork, but if you create a `Queue::Store` manually for schema setup, close it before forking:
 
 ```ruby
 store = Async::Background::Queue::Store.new(path: db_path)
@@ -97,96 +97,49 @@ store.ensure_database!
 store.close  # ← before fork
 ```
 
-### Clock handling
-
-The `Clock` module provides shared time helpers used across the codebase:
-
-- **`monotonic_now`** (`CLOCK_MONOTONIC`) — for in-process intervals and durations, immune to NTP drift / wall-clock jumps
-- **`realtime_now`** (`CLOCK_REALTIME`) — for persisted timestamps (SQLite `run_at`, `created_at`, `locked_at`)
-
-Interval jobs use monotonic clock. Cron jobs use `Time.now` because "every day at 3am" must respect real time. These are different clocks by design.
+**Two clocks, on purpose.** Interval jobs use `CLOCK_MONOTONIC` (immune to NTP drift). Cron jobs use wall-clock time, because "every day at 3am" needs to mean 3am.
 
 ---
 
-## Architecture
+## How it works
 
 ```
-schedule.yml
-     │
-     ▼
- build_heap          ← parse config, validate, assign workers
-     │
-     ▼
- MinHeap<Entry>      ← O(log N) push/pop, sorted by next_run_at
-     │
-     ▼
- 1 scheduler loop    ← single Async task, sleeps until next entry
-     │
-     ▼
- Semaphore           ← limits concurrent job execution
-     │
-     ▼
- run_job             ← timeout, logging, error handling
+schedule.yml ─► build_heap ─► MinHeap<Entry> ─► scheduler loop ─► Semaphore ─► run_job
 ```
 
-### Queue Architecture
+A single Async task sleeps until the next entry is due, then dispatches it under a semaphore that caps concurrency. Overlapping ticks are skipped and rescheduled.
+
+The dynamic queue runs alongside it:
 
 ```
-Producer (web/console)          Consumer (background worker)
-     │                                │
-     ▼                                ▼
- Queue::Client                   Queue::Store.fetch
-     │                           (WHERE run_at <= now)
-     ├─ push(class, args, run_at)     │
-     ├─ push_in(delay, class, args)   ▼
-     └─ push_at(time, class, args)  run_queue_job
-     │                                │
-     ▼                                ▼
- Queue::Store ──── SQLite ──── Queue::Notifier
- (INSERT job)    (jobs table)    (IO.pipe wakeup)
+   Producer (web/console)              Consumer (background worker)
+          │                                       │
+          ▼                                       ▼
+    Queue::Client                          Queue::Store#fetch
+   push / push_in / push_at                (run_at <= now)
+          │                                       ▲
+          ▼                                       │
+    Queue::Store ──── SQLite (jobs) ──── SocketWaker
+          │                                       ▲
+          └───────► SocketNotifier ───────────────┘
+                    (UNIX socket wake-up, ~80µs)
 ```
 
-## Schedule Config
-
-| Key | Required | Description |
-|---|---|---|
-| `class` | yes | Must include `Async::Background::Job` |
-| `every` | one of | Interval in seconds between runs |
-| `cron` | one of | Cron expression (parsed by Fugit) |
-| `timeout` | no | Max execution time in seconds (default: 30) |
-| `worker` | no | Pin to specific worker index. If omitted — `crc32(name) % total_workers` |
-
-## SQLite Pragmas
-
-| Pragma | Value | Why |
-|---|---|---|
-| `journal_mode` | WAL | Concurrent reads during writes |
-| `synchronous` | NORMAL | Safe with WAL, lower fsync overhead |
-| `mmap_size` | 256 MB | Fast reads ([requires proper filesystem](#docker-sqlite-requires-a-named-volume)) |
-| `cache_size` | 16000 pages | ~64 MB page cache |
-| `busy_timeout` | 5000 ms | Wait instead of failing on lock contention |
+Jobs are persisted in SQLite, so a missed wake-up is never a lost job — workers also poll every 5 seconds as a safety net.
 
 ## Metrics
 
-When `async-utilization` gem is available, metrics are collected in shared memory (`/tmp/async-background.shm`) with lock-free updates per worker.
+With `async-utilization` installed, per-worker stats land in shared memory at `/tmp/async-background.shm` with lock-free updates.
 
 ```ruby
 runner.metrics.values
 # => { total_runs: 142, total_successes: 140, total_failures: 2,
-#      total_timeouts: 0, total_skips: 5, active_jobs: 1,
-#      last_run_at: 1774445243, last_duration_ms: 1250 }
+#      total_timeouts: 0, total_skips: 5, active_jobs: 1, ... }
 
-# Read all workers at once (no server needed)
 Async::Background::Metrics.read_all(total_workers: 2)
 ```
 
-Without the gem — metrics are silently disabled, zero overhead.
-
-## Why Ruby 3.3?
-
-- Ruby 3.0 introduced Fiber Scheduler but had critical bugs
-- Ruby 3.2 is the first production-ready release (per Samuel Williams)
-- `io-event >= 1.14` (pulled by latest `async`) requires Ruby `>= 3.3`
+Without the gem, metrics are silently disabled — zero overhead.
 
 ## License
 

data/lib/async/background/queue/socket_notifier.rb

@@ -6,13 +6,11 @@ module Async
   module Background
     module Queue
       class SocketNotifier
-        # Errors that indicate a worker is unavailable - silently skip
+        # Errors that indicate a worker is unavailable - silently skip and try the next.
         UNAVAILABLE = [
           Errno::ENOENT,        # Socket file doesn't exist (worker hasn't started yet)
           Errno::ECONNREFUSED,  # File exists but no one listening (worker died)
           Errno::EPIPE,         # Connection broken during write
-          Errno::EAGAIN,        # Socket buffer full - wake-up already queued
-          IO::WaitWritable,     # Same as EAGAIN on some platforms
           Errno::ECONNRESET     # Connection reset by peer
         ].freeze
 
@@ -22,8 +20,12 @@ module Async
         end
 
         def notify_all
-          (1..@total_workers).each do |worker_index|
-            notify_one(worker_index)
+          return if @total_workers <= 0
+
+          start = rand(@total_workers)
+          @total_workers.times do |i|
+            worker_index = ((start + i) % @total_workers) + 1
+            return if notify_one(worker_index)
           end
         end
 
@@ -37,14 +39,12 @@ module Async
           ensure
             sock.close rescue nil
           end
+          true
         rescue *UNAVAILABLE
-          # Worker is unavailable - not a problem.
-          # The job is already in the database. The worker will:
-          # - Pick it up on next poll (within QUEUE_POLL_INTERVAL seconds), or
-          # - Pick it up when it starts/restarts via normal fetch loop
+          false
         rescue => e
-          # Unexpected error - log but don't crash the enqueue operation
           Console.logger.warn(self) { "SocketNotifier#notify_one(#{worker_index}) failed: #{e.class} #{e.message}" } rescue nil
+          false
         end
 
         def socket_path(worker_index)

data/lib/async/background/queue/store.rb

@@ -21,7 +21,7 @@ module Async
             locked_by  INTEGER,
             locked_at  REAL
           );
-          CREATE INDEX IF NOT EXISTS idx_jobs_status_run_at_id ON jobs(status, run_at, id);
+          CREATE INDEX IF NOT EXISTS idx_jobs_pending ON jobs(run_at, id) WHERE status = 'pending';
         SQL
 
         MMAP_SIZE = 268_435_456
@@ -72,9 +72,14 @@ module Async
           ensure_connection
           now = realtime_now
           @db.execute("BEGIN IMMEDIATE")
-          results = @fetch_stmt.execute(worker_id, now, now)
-          row = results.first
-          @fetch_stmt.reset!
+
+          begin
+            results = @fetch_stmt.execute(worker_id, now, now)
+            row = results.first
+          ensure
+            @fetch_stmt.reset! rescue nil
+          end
+
           @db.execute("COMMIT")
           return unless row
 

data/lib/async/background/runner.rb

@@ -170,13 +170,16 @@ module Async
             if entry.running
               logger.warn('Async::Background') { "#{entry.name}: skipped, previous run still active" }
               metrics.job_skipped(entry)
-            else
-              entry.running = true
-              semaphore.async do |job_task|
-                run_job(job_task, entry)
-              ensure
-                entry.running = false
-              end
+              entry.reschedule(monotonic_now)
+              heap.replace_top(entry)
+              next
+            end
+
+            entry.running = true
+            semaphore.async do |job_task|
+              run_job(job_task, entry)
+            ensure
+              entry.running = false
             end
 
             entry.reschedule(monotonic_now)

data/lib/async/background/version.rb

@@ -2,6 +2,6 @@
 
 module Async
   module Background
-    VERSION = '0.6.0'
+    VERSION = '0.6.1'
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: async-background
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.6.1
 platform: ruby
 authors:
 - Roman Hajdarov
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-04-07 00:00:00.000000000 Z
+date: 2026-04-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: async