async-background 1.0.1 → 1.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 823b13ecbf3740e79a15a9c24f0195a2d09bbc49bb189885d0738a0636cf913c
-  data.tar.gz: 4424725707108aaf8249b8ea62e500ccaa884bbc5392a4691cdcdc540ea5fda4
+  metadata.gz: 233f46d3617b40d9a8a27f83689050b0127603679342ddfa80a18eea0137651c
+  data.tar.gz: 5d70fe357702dff97b51734c1fddf64178f3ea0d9d9d88449fe3af9b32fc59d3
 SHA512:
-  metadata.gz: 3e0d13f6b354efdd47fe4d93c464eb4411eb4658bd91fc9f3a2a154fd0300d72982680d98b454180c958ac93c15c3a7500df6c0ec346508c0791f6503525c042
-  data.tar.gz: 53a18f4ce6f47db152a303f456f5ba80cd3b706ceaee4e1aac35d752e8067c2e1eb807765117a77837e838376740a61c91690ac4714dd5853ae062c41cbd6c3e
+  metadata.gz: 148770cb2522083b5944a5ee0f5830a1e1453f9c06647feaf0b92842e6008e2654aefd7e8c4a4a97509fc823702d64176a7bf4e30e8a08d409308f57792bbb1d
+  data.tar.gz: 99e64f1f1066528429617270d54ae64109896d1318634f5b4b001a9d10e4ed19b88239dc4b7a089c885d4f481d2799b22b70495b763d64df51702c82b15865f8

data/CHANGELOG.md

@@ -1,5 +1,91 @@
 # Changelog
 
+## 1.0.2
+
+Queue maintenance bug fix plus profiler-driven work on the hot paths that
+actually showed up in StackProf: the sqlite3 statement wrapper, transaction
+control, and the socket notifier.
+
+### Fixed
+
+- `Store#cleanup_finished_jobs` tested `@db.changes` after running *both*
+  DELETEs. `sqlite3_changes()` reports only the most recent statement, so the
+  incremental-vacuum decision saw the failed-job count alone and ignored every
+  deleted done-job. On a busy queue, where done-jobs vastly outnumber failed
+  ones, the vacuum effectively never ran and the database file grew without
+  bound. Both counts are now summed.
+- `PRAGMA incremental_vacuum` ran without a page limit, releasing every free
+  page in a single blocking call — an unbounded reactor stall proportional to
+  the accumulated free list. Now capped at 64 pages per call
+  (`SQL::INCREMENTAL_VACUUM_PAGES`).
+- `SocketWaker` signalled its notification only from the `ensure` block, so a
+  wake-up was really driven by the client disconnecting rather than by the
+  wake byte arriving. It happened to work because `SocketNotifier` closes
+  immediately after writing, but it made the protocol depend on a disconnect.
+  The signal now fires for every byte received; the `ensure` still signals so a
+  disconnect racing with a read cannot drop a wake-up.
+
+### Performance
+
+Measured on the `enqueue_stress` CI scenario (2.1M inserts / 30s, two producer
+processes). `SQLite3::Statement#step` accounts for ~82% of producer wall time
+and is irreducible; the changes below target what surrounded it.
+
+- `Store#enqueue` and `Store#fetch` bind and step their prepared statements
+  directly instead of calling `Statement#execute`. The wrapper built a splat
+  array, ran `Array#flatten` over it and allocated a `ResultSet` — roughly 8%
+  of producer wall time, and `Array#flatten` alone was 5.8% of all object
+  allocations. Behaviour is unchanged: `execute` only steps when
+  `column_count == 0`, which is exactly what the INSERT path needed, and the
+  `UPDATE ... RETURNING` fetch still consumes a single row.
+- `SocketNotifier` caches its socket paths at construction. It previously ran
+  `File.join` plus a string interpolation on every enqueue attempt —
+  13.3% of allocations in the normal scenario.
+- `SocketNotifier` remembers unreachable workers for `DEAD_WORKER_TTL` (5s)
+  instead of reconnecting to them on every enqueue. Because the scan started at
+  a random index, a dead worker was re-probed indefinitely and each attempt
+  raised and swallowed an `Errno`; `SystemCallError#initialize` alone was 6.8%
+  of allocations. A worker that starts during the TTL window is still picked up
+  by the listener's own polling fallback.
+- `SocketNotifier` rotates its scan start with a cursor rather than calling
+  `rand` per enqueue, which also spreads wake-ups deterministically.
+- `Errno::EAGAIN` on the wake byte (`IO::WaitWritable`) is now treated as
+  success rather than falling through to the generic rescue and logging a
+  warning: a full send buffer means the worker already has unread wake-ups.
+- `Store#transaction` runs `BEGIN IMMEDIATE`, `COMMIT` and `ROLLBACK` as
+  prepared statements. They previously went through `Database#execute`, which
+  compiles a fresh `Statement` and builds a `ResultSet` on every call: 26
+  allocations per BEGIN+COMMIT pair against 2 prepared, and 6x the wall time
+  over 30k pairs. Every `fetch` paid this twice, and it was 11.8% of worker
+  allocations in the object profile.
+- `mark_started!`, `complete`, `fail`, `retry_job!`, `recover`,
+  `stored_options_for`, `lease_alive?`, `next_pending_run_at` and the two
+  cleanup DELETEs now bind and step directly, like `enqueue` and `fetch`
+  already did. `Statement#execute` is gone from the Store entirely;
+  `Database#execute` remains only on the cold pragma path. This is what was
+  still leaving `Array#flatten` at 4.9% of worker allocations and
+  `Statement#execute` at 10.8% of worker wall time.
+- Combined effect on a full job cycle (enqueue omitted; fetch + mark_started +
+  complete, 8000 jobs, measured twice): **49 -> 18 allocations per job**, wall
+  time down roughly 10-20% depending on run.
+- `@db.changes` and `@db.last_insert_row_id` are read after the statement is
+  reset; both were verified to survive `sqlite3_reset()`, so the claim-token
+  lease checks behave exactly as before (valid token true, stale token false,
+  repeat completion false).
+
+### Notes
+
+- `SocketNotifier` still opens a fresh connection per notification. This is
+  the largest remaining win and it is paid on both sides: 8.5% of producer wall
+  time in the scenario where workers are actually up, plus 16.4% of worker
+  allocations, because every connection makes `SocketWaker#handle_client` spawn
+  a fresh `Async::Task` and fiber. Persistent connections require the
+  `SocketWaker` fix above to be deployed on every worker first — a 1.0.2 producer holding a connection open
+  to a 1.0.1 worker would never wake it, silently degrading queue latency to
+  the 5s poll interval. Deferred until 1.0.2 is the deployed floor.
+- No schema change; `Schema::VERSION` is untouched and 1.0.2 is a drop-in
+  replacement for 1.0.1 on an existing database.
+
 ## 1.0.1
 
 Dashboard security headers and a fiber-native rewrite of the SSE stream.

data/lib/async/background/metrics.rb

@@ -127,9 +127,7 @@ module Async
         end
 
         def decode_workers(buffer, schema, segment, total_workers)
-          (1..total_workers).map do |worker|
-            decode_worker(buffer, schema, segment, worker)
-          end.freeze
+          (1..total_workers).map { |worker| decode_worker(buffer, schema, segment, worker) }.freeze
         end
 
         def decode_worker(buffer, schema, segment, worker)

data/lib/async/background/queue/socket_notifier.rb

@@ -2,10 +2,14 @@
 
 require 'socket'
 
+require_relative '../clock'
+
 module Async
   module Background
     module Queue
       class SocketNotifier
+        include Clock
+
         # Errors that indicate a worker is unavailable - silently skip and try the next.
         UNAVAILABLE = [
           Errno::ENOENT,        # Socket file doesn't exist (worker hasn't started yet)
@@ -14,41 +18,68 @@ module Async
           Errno::ECONNRESET     # Connection reset by peer
         ].freeze
 
+        DEAD_WORKER_TTL = 5.0
+        WAKE_BYTE = "\x01".freeze
+
         def initialize(socket_dir:, total_workers:)
           @socket_dir = socket_dir
           @total_workers = total_workers
+          @paths = build_paths
+          @dead_until = Array.new([@total_workers, 0].max, 0.0)
+          @cursor = 0
         end
 
         def notify_all
-          return if @total_workers <= 0
+          return false if @total_workers <= 0
+
+          now = monotonic_now
+          start = advance_cursor
 
-          start = rand(@total_workers)
-          @total_workers.times do |i|
-            worker_index = ((start + i) % @total_workers) + 1
-            return if notify_one(worker_index)
+          @total_workers.times do |offset|
+            index = (start + offset) % @total_workers
+            return true if notify_one(index, now)
           end
+
+          false
         end
 
         private
 
-        def notify_one(worker_index)
-          path = socket_path(worker_index)
-          sock = UNIXSocket.new(path)
+        def build_paths
+          return [].freeze if @total_workers <= 0
+
+          Array.new(@total_workers) do |index|
+            File.join(@socket_dir, "async_bg_worker_#{index + 1}.sock").freeze
+          end.freeze
+        end
+
+        def advance_cursor
+          @cursor = (@cursor + 1) % @total_workers
+        end
+
+        def notify_one(index, now)
+          return false if @dead_until[index] > now
+
+          socket = UNIXSocket.new(@paths[index])
           begin
-            sock.write_nonblock("\x01")
+            socket.write_nonblock(WAKE_BYTE)
           ensure
-            sock.close rescue nil
+            socket.close rescue nil
           end
           true
+        rescue IO::WaitWritable
+          true
         rescue *UNAVAILABLE
+          mark_dead(index, now)
           false
         rescue => e
-          Console.logger.warn(self) { "SocketNotifier#notify_one(#{worker_index}) failed: #{e.class} #{e.message}" } rescue nil
+          mark_dead(index, now)
+          Console.logger.warn(self) { "SocketNotifier#notify_one(#{index + 1}) failed: #{e.class} #{e.message}" } rescue nil
           false
         end
 
-        def socket_path(worker_index)
-          File.join(@socket_dir, "async_bg_worker_#{worker_index}.sock")
+        def mark_dead(index, now)
+          @dead_until[index] = now + DEAD_WORKER_TTL
         end
       end
     end

data/lib/async/background/queue/socket_waker.rb

@@ -80,12 +80,15 @@ module Async
           parent_task.async do
             begin
               loop do
-                client.read_nonblock(256)
-              rescue IO::WaitReadable
-                client.wait_readable
-                retry
-              rescue EOFError, Errno::ECONNRESET
-                break
+                begin
+                  client.read_nonblock(256)
+                  @notification.signal
+                rescue IO::WaitReadable
+                  client.wait_readable
+                  retry
+                rescue EOFError, Errno::ECONNRESET
+                  break
+                end
               end
             rescue => e
               Console.logger.warn(self) { "SocketWaker client handler error: #{e.class} #{e.message}" }

data/lib/async/background/queue/sql.rb

@@ -9,7 +9,8 @@ module Async
         BUSY_TIMEOUT = 'PRAGMA busy_timeout'.freeze
         TABLE_INFO = 'PRAGMA table_info(jobs)'.freeze
         OPTIMIZE = 'PRAGMA optimize'.freeze
-        INCREMENTAL_VACUUM = 'PRAGMA incremental_vacuum'.freeze
+        INCREMENTAL_VACUUM_PAGES = 64
+        INCREMENTAL_VACUUM = "PRAGMA incremental_vacuum(#{INCREMENTAL_VACUUM_PAGES})".freeze
         AUTO_VACUUM_INCREMENTAL = 'PRAGMA auto_vacuum = INCREMENTAL'.freeze
         BEGIN_IMMEDIATE = 'BEGIN IMMEDIATE'.freeze
         COMMIT = 'COMMIT'.freeze

data/lib/async/background/queue/store.rb

@@ -22,6 +22,7 @@ module Async
         CLEANUP_INTERVAL = 300
         CLEANUP_AGE = 3600
         FAILED_RETENTION_AGE = 7 * 24 * 3600
+        CLEANUP_VACUUM_THRESHOLD = 100
         ERROR_MESSAGE_MAX_LEN = 2_000
         EMPTY_ARGS_JSON = '[]'.freeze
 
@@ -80,7 +81,13 @@ module Async
         def enqueue(class_name, args = EMPTY_ARGS, run_at = nil, options: EMPTY_OPTIONS)
           ensure_connection
           now = realtime_now
-          @enqueue_stmt.execute(class_name, dump_args(args), dump_options(options), now, run_at || now)
+          stepped(@enqueue_stmt) do |statement|
+            statement.bind_param(1, class_name)
+            statement.bind_param(2, dump_args(args))
+            statement.bind_param(3, dump_options(options))
+            statement.bind_param(4, now)
+            statement.bind_param(5, run_at || now)
+          end
           @db.last_insert_row_id
         end
 
@@ -90,9 +97,14 @@ module Async
           now = realtime_now
 
           row = transaction do
-            with_statement(@fetch_stmt) { |statement| statement.execute(worker_id, now, token, now).first }
+            stepped(@fetch_stmt) do |statement|
+              statement.bind_param(1, worker_id)
+              statement.bind_param(2, now)
+              statement.bind_param(3, token)
+              statement.bind_param(4, now)
+            end
           end
-          return unless row
+          return if row.nil? || row.empty?
 
           maybe_cleanup
           job_from_row(row, token)
@@ -100,26 +112,28 @@ module Async
 
         def mark_started!(job_id, claim_token:, started_at: realtime_now)
           ensure_connection
-          @mark_started_stmt.execute(started_at, job_id, claim_token)
+          stepped(@mark_started_stmt) do |statement|
+            statement.bind_param(1, started_at)
+            statement.bind_param(2, job_id)
+            statement.bind_param(3, claim_token)
+          end
           @db.changes.positive?
         end
 
         def complete(job_id, claim_token:, finished_at: realtime_now, duration_ms: nil)
           ensure_connection
-          @complete_stmt.execute(finished_at, duration_ms, job_id, claim_token)
+          stepped(@complete_stmt) do |statement|
+            statement.bind_param(1, finished_at)
+            statement.bind_param(2, duration_ms)
+            statement.bind_param(3, job_id)
+            statement.bind_param(4, claim_token)
+          end
           @db.changes.positive?
         end
 
         def fail(job_id, claim_token:, error_class: nil, error_message: nil, finished_at: realtime_now, duration_ms: nil)
           ensure_connection
-          @fail_stmt.execute(
-            finished_at,
-            duration_ms,
-            error_class&.to_s,
-            truncate_message(error_message),
-            job_id,
-            claim_token
-          )
+          bind_failure(@fail_stmt, finished_at, duration_ms, error_class, error_message, job_id, claim_token)
           @db.changes.positive?
         end
 
@@ -146,13 +160,13 @@ module Async
 
         def recover(worker_id)
           ensure_connection
-          @requeue_stmt.execute(worker_id)
+          stepped(@requeue_stmt) { |statement| statement.bind_param(1, worker_id) }
           @db.changes
         end
 
         def next_pending_run_at
           ensure_connection
-          with_statement(@next_pending_stmt) { |statement| statement.execute.first&.first }
+          stepped(@next_pending_stmt)&.first
         end
 
         def data_version
@@ -243,9 +257,11 @@ module Async
         end
 
         def stored_options_for(job_id, claim_token)
-          with_statement(@retry_state_stmt) do |statement|
-            load_options(statement.execute(job_id, claim_token).first&.first)
+          row = stepped(@retry_state_stmt) do |statement|
+            statement.bind_param(1, job_id)
+            statement.bind_param(2, claim_token)
           end
+          load_options(row&.first)
         end
 
         def retry_policy(stored_options, fallback_options)
@@ -260,29 +276,33 @@ module Async
 
         def retry_job!(job_id, claim_token, policy, error_class, error_message)
           advanced = policy.with_attempt(policy.next_attempt)
-          @retry_stmt.execute(
-            realtime_now + advanced.next_retry_delay(advanced.attempt),
-            dump_options(advanced.to_h.compact),
-            error_class&.to_s,
-            truncate_message(error_message),
-            job_id,
-            claim_token
-          )
+          stepped(@retry_stmt) do |statement|
+            statement.bind_param(1, realtime_now + advanced.next_retry_delay(advanced.attempt))
+            statement.bind_param(2, dump_options(advanced.to_h.compact))
+            statement.bind_param(3, error_class&.to_s)
+            statement.bind_param(4, truncate_message(error_message))
+            statement.bind_param(5, job_id)
+            statement.bind_param(6, claim_token)
+          end
           @db.changes.positive? ? :retried : nil
         end
 
         def fail_job!(job_id, claim_token, error_class, error_message, finished_at, duration_ms)
-          @fail_stmt.execute(
-            finished_at,
-            duration_ms,
-            error_class&.to_s,
-            truncate_message(error_message),
-            job_id,
-            claim_token
-          )
+          bind_failure(@fail_stmt, finished_at, duration_ms, error_class, error_message, job_id, claim_token)
           @db.changes.positive? ? :failed : nil
         end
 
+        def bind_failure(statement, finished_at, duration_ms, error_class, error_message, job_id, claim_token)
+          stepped(statement) do |s|
+            s.bind_param(1, finished_at)
+            s.bind_param(2, duration_ms)
+            s.bind_param(3, error_class&.to_s)
+            s.bind_param(4, truncate_message(error_message))
+            s.bind_param(5, job_id)
+            s.bind_param(6, claim_token)
+          end
+        end
+
         def generate_claim_token = SecureRandom.hex(16)
 
         def truncate_message(message)
@@ -293,23 +313,30 @@ module Async
         end
 
         def lease_alive?(job_id, claim_token)
-          with_statement(@lease_check_stmt) do |statement|
-            !statement.execute(job_id, claim_token).first.nil?
-          end
+          !stepped(@lease_check_stmt) do |statement|
+            statement.bind_param(1, job_id)
+            statement.bind_param(2, claim_token)
+          end.nil?
         end
 
         def transaction
-          @db.execute(SQL::BEGIN_IMMEDIATE)
+          stepped(@begin_stmt)
           result = yield
-          @db.execute(SQL::COMMIT)
+          stepped(@commit_stmt)
           result
         rescue StandardError
-          @db.execute(SQL::ROLLBACK) rescue nil
+          begin
+            stepped(@rollback_stmt)
+          rescue StandardError
+            nil
+          end
           raise
         end
 
-        def with_statement(statement)
-          yield statement
+        def stepped(statement)
+          statement.reset!
+          yield statement if block_given?
+          statement.step
         ensure
           statement.reset! rescue nil
         end
@@ -346,6 +373,9 @@ module Async
           @cleanup_done_stmt = @db.prepare(SQL::CLEANUP_DONE)
           @cleanup_failed_stmt = @db.prepare(SQL::CLEANUP_FAILED)
           @next_pending_stmt = @db.prepare(SQL::NEXT_PENDING_RUN_AT)
+          @begin_stmt = @db.prepare(SQL::BEGIN_IMMEDIATE)
+          @commit_stmt = @db.prepare(SQL::COMMIT)
+          @rollback_stmt = @db.prepare(SQL::ROLLBACK)
         end
 
         def finalize_statements
@@ -366,7 +396,10 @@ module Async
             @requeue_stmt,
             @cleanup_done_stmt,
             @cleanup_failed_stmt,
-            @next_pending_stmt
+            @next_pending_stmt,
+            @begin_stmt,
+            @commit_stmt,
+            @rollback_stmt
           ]
         end
 
@@ -376,6 +409,7 @@ module Async
           @retry_stmt = @requeue_stmt = nil
           @cleanup_done_stmt = @cleanup_failed_stmt = nil
           @next_pending_stmt = nil
+          @begin_stmt = @commit_stmt = @rollback_stmt = nil
         end
 
         def maybe_cleanup
@@ -387,9 +421,15 @@ module Async
         end
 
         def cleanup_finished_jobs(now)
-          @cleanup_done_stmt.execute(now - CLEANUP_AGE)
-          @cleanup_failed_stmt.execute(now - FAILED_RETENTION_AGE)
-          @db.execute(SQL::INCREMENTAL_VACUUM) if @db.changes > 100
+          deleted = 0
+
+          stepped(@cleanup_done_stmt) { |statement| statement.bind_param(1, now - CLEANUP_AGE) }
+          deleted += @db.changes
+          stepped(@cleanup_failed_stmt) { |statement| statement.bind_param(1, now - FAILED_RETENTION_AGE) }
+          deleted += @db.changes
+
+          @db.execute(SQL::INCREMENTAL_VACUUM) if deleted > CLEANUP_VACUUM_THRESHOLD
+          deleted
         end
       end
     end

data/lib/async/background/runner.rb

@@ -47,11 +47,7 @@ module Async
         @total_workers = total_workers
         @running = true
         @shutdown = ::Async::Condition.new
-        @metrics = Metrics.new(
-          worker_index: worker_index,
-          total_workers: total_workers,
-          shm_path: metrics_shm_path
-        )
+        @metrics = Metrics.new(worker_index: worker_index, total_workers: total_workers, shm_path: metrics_shm_path)
         logger.info { "Async::Background worker_index=#{worker_index}/#{total_workers}, job_count=#{job_count}" }
 
         @drain_barrier = ::Async::Barrier.new
@@ -91,7 +87,7 @@ module Async
         # continues independently in its own Async task.
         return shutdown.wait if heap.empty? && @listen_queue
 
-        loop do
+        while running?
           entry = heap.peek
           break unless entry
 
@@ -123,12 +119,7 @@ module Async
       end
 
       def dispatch_entry(entry)
-        if entry.running
-          skip_entry(entry)
-        else
-          execute_entry(entry)
-        end
-
+        entry.running ? skip_entry(entry) : execute_entry(entry)
         entry.reschedule(monotonic_now)
         heap.replace_top(entry)
       end
@@ -182,12 +173,11 @@ module Async
 
       def start_signal_watcher(task)
         task.async(transient: true) do
-          loop do
+          while running?
             @signal_r.wait_readable
             @signal_r.read_nonblock(256) rescue nil
             shutdown.signal
             @queue_waker&.signal
-            break unless running?
           end
         end
       end

data/lib/async/background/version.rb

@@ -2,6 +2,6 @@
 
 module Async
   module Background
-    VERSION = '1.0.1'
+    VERSION = '1.0.2'
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: async-background
 version: !ruby/object:Gem::Version
-  version: 1.0.1
+  version: 1.0.2
 platform: ruby
 authors:
 - Roman Hajdarov
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-06-29 00:00:00.000000000 Z
+date: 2026-08-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: async
@@ -198,7 +198,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.3.27
+rubygems_version: 3.4.22
 signing_key:
 specification_version: 4
 summary: Lightweight heap-based cron/interval scheduler for Async.