async-background 1.0.0 → 1.0.1

data/README.md

@@ -1,36 +1,67 @@
 # Async::Background
 
-A lightweight cron, interval, and job-queue scheduler for Ruby's [Async](https://github.com/socketry/async) ecosystem. Built for [Falcon](https://github.com/socketry/falcon), works with any Async app.
+A lightweight cron, interval, and job-queue scheduler for Ruby's
+[Async](https://github.com/socketry/async) ecosystem. Built for
+[Falcon](https://github.com/socketry/falcon), works with any Async app.
+
+- **Cron & interval scheduling** on a single event loop with a min-heap.
+- **Dynamic job queue** backed by SQLite, with delayed jobs
+  (`perform_in` / `perform_at`).
+- **Cross-process wake-ups** over Unix domain sockets — web workers can
+  enqueue and instantly wake background workers.
+- **Multi-process safe** — deterministic worker sharding, no duplicate
+  execution.
+- **Per-job timeouts**, skip-on-overlap, startup jitter, optional metrics.
 
-- **Cron & interval scheduling** on a single event loop with a min-heap
-- **Dynamic job queue** backed by SQLite, with delayed jobs (`perform_in` / `perform_at`)
-- **Cross-process wake-ups** over Unix domain sockets — web workers can enqueue and instantly wake background workers
-- **Multi-process safe** — deterministic worker sharding, no duplicate execution
-- **Per-job timeouts**, skip-on-overlap, startup jitter, optional metrics
+---
+
+## Why Async? Why fibers?
+
+The whole gem is built around the assumption that Falcon's reactor schedules
+many fibers on top of one OS thread per process — so the dashboard's SSE
+stream, the cron scheduler, and the queue worker all share that one thread
+cooperatively. A blocked fiber yields; a blocked thread doesn't.
+
+![Threads vs fibers under different Ruby web servers](docs/fibers-vs-threads.svg)
+
+That's also why the dashboard (since 1.0.1) runs its SSE loop entirely inside
+the request fiber: zero extra threads, zero `ConditionVariable`, ~4 KB per
+open tab.
+
+---
 
 ## Requirements
 
-- Ruby >= 3.3
-- `async ~> 2.0`, `fugit ~> 1.0`
-- `sqlite3 ~> 2.0` (optional, storage)
-- `async-utilization >= 0.3, < 0.5` (optional, for metrics)
+| Dependency           | Version          | Required?            |
+| -------------------- | ---------------- | -------------------- |
+| Ruby                 | `>= 3.3`         | yes                  |
+| `async`              | `~> 2.0`         | yes                  |
+| `fugit`              | `~> 1.0`         | yes                  |
+| `sqlite3`            | `~> 2.0`         | for the queue & dashboard |
+| `async-utilization`  | `>= 0.3, < 0.5`  | for metrics          |
+
+---
 
 ## Install
 
 ```ruby
 # Gemfile
 gem "async-background"
-gem "sqlite3", "~> 2.0"  # optional
-gem "async-utilization", ">= 0.3", "< 0.5"  # optional
+
+gem "sqlite3",           "~> 2.0"          # if you use the queue or dashboard
+gem "async-utilization", ">= 0.3", "< 0.5" # if you want worker metrics
 ```
 
-## ➡️ [Get Started](docs/GET_STARTED.md)
+---
+
+## ➡️ [Get started](docs/GET_STARTED.md)
 
-Full setup walkthrough: schedule config, Falcon integration, Docker, queue, delayed jobs.
+A four-step walkthrough: schedule config, Falcon integration, Docker, queue,
+delayed jobs.
 
 ---
 
-## Quick Look
+## Quick look
 
 ```ruby
 class SendEmailJob
@@ -59,114 +90,71 @@ daily_report:
   timeout: 120
 ```
 
-| Key | Description |
-|---|---|
-| `class` | Job class — must include `Async::Background::Job` |
-| `every` / `cron` | One of: interval in seconds, or cron expression |
-| `timeout` | Max execution time in seconds (default: 30) |
-| `worker` | Pin to a specific worker. Default: `crc32(name) % total_workers` |
+| Key              | Description                                                     |
+| ---------------- | --------------------------------------------------------------- |
+| `class`          | Job class — must include `Async::Background::Job`.              |
+| `every` / `cron` | Interval in seconds, or a cron expression. Exactly one.         |
+| `timeout`        | Max execution time in seconds. Default: 30.                     |
+| `worker`         | Pin to a specific worker. Default: `crc32(name) % total_workers`. |
 
 ---
 
 ## Gotchas
 
-### Docker: SQLite requires a named volume
-
-The SQLite database **must not** live on Docker's `overlay2` filesystem. The `overlay2` driver breaks coherence between `write()` and `mmap()`, which corrupts SQLite WAL under concurrent access.
-
-```yaml
-# docker-compose.yml
-services:
-  app:
-    volumes:
-      - queue-data:/app/tmp/queue   # ← named volume, NOT overlay2
-
-volumes:
-  queue-data:
-```
-
-Without this, you will get database crashes in multi-process mode. See [Get Started → Step 3](docs/GET_STARTED.md#step-3-docker) for details. If you can't use a named volume, pass `queue_mmap: false` to disable mmap entirely.
-
-### Other gotchas
+**Docker + SQLite — use a named volume.**
+SQLite's database must not live on Docker's default `overlay2` filesystem:
+`overlay2` breaks coherence between `write()` and `mmap()`, which corrupts
+the WAL under concurrent access. Mount the queue directory as a named volume,
+or pass `mmap: false` to `Store.new`. See
+[Get Started → Docker](docs/GET_STARTED.md#step-3--docker-setup).
 
-**Don't share SQLite connections across `fork()`.** The gem opens connections lazily after fork, but if you create a `Queue::Store` manually for schema setup, close it before forking:
-
-```ruby
-Async::Background::Queue.migrate!(path: db_path) # ← once, before fork
-# Every process opens its own Store lazily after fork.
-```
+**Don't share SQLite connections across `fork()`.**
+The gem opens connections lazily after fork. If you build a `Store` manually
+for schema setup, close it before forking.
 
-**Two clocks, on purpose.** Interval jobs use `CLOCK_MONOTONIC` (immune to NTP drift). Cron jobs use wall-clock time, because "every day at 3am" needs to mean 3am.
+**Two clocks, on purpose.**
+Interval jobs use `CLOCK_MONOTONIC` so NTP drift can't fire them twice. Cron
+jobs use wall-clock time, because "every day at 3am" needs to mean 3am.
 
 ---
 
 ## How it works
 
+A single Async task sleeps until the next entry is due, then dispatches it
+under a semaphore that caps concurrency. Overlapping ticks are skipped and
+rescheduled.
+
 ```
-schedule.yml ─► build_heap ─► MinHeap<Entry> ─► scheduler loop ─► Semaphore ─► run_job
+schedule.yml → build_heap → MinHeap<Entry> → scheduler loop → Semaphore → run_job
 ```
 
-A single Async task sleeps until the next entry is due, then dispatches it under a semaphore that caps concurrency. Overlapping ticks are skipped and rescheduled.
-
 The dynamic queue runs alongside it:
 
 ```
-   Producer (web/console)              Consumer (background worker)
-          │                                       │
-          ▼                                       ▼
-    Queue::Client                          Queue::Store#fetch
-   push / push_in / push_at                (run_at <= now)
-          │                                       ▲
-          ▼                                       │
-    Queue::Store ──── SQLite (jobs) ──── SocketWaker
-          │                                       ▲
-          └───────► SocketNotifier ───────────────┘
-                    (UNIX socket wake-up, ~80µs)
-```
-
-Jobs are persisted in SQLite, so a missed wake-up is never a lost job — workers also poll every 5 seconds as a safety net.
-
-### Queue-only workers
-
-Recurring schedules are optional. A worker that serves only dynamic jobs starts with
-`config_path: nil` and a `queue_socket_dir`; it does not need a placeholder schedule file.
-A supplied schedule path stays strict and raises when the file is missing or empty.
-
-### Schema migration during deploy
-
-Run queue migrations once in the release/pre-deploy step, before starting new web or worker
-processes. This serializes the schema upgrade with `BEGIN IMMEDIATE`, records the version in
-SQLite, and avoids a first producer doing DDL under live queue traffic:
-
-```ruby
-Async::Background::Queue.migrate!(path: ENV.fetch("QUEUE_DB_PATH"))
+   Producer (web / console)                Consumer (background worker)
+          │                                          │
+          ▼                                          ▼
+    Queue::Client                            Queue::Store#fetch
+   push / push_in / push_at                  (run_at <= now)
+          │                                          ▲
+          ▼                                          │
+    Queue::Store ──── SQLite (jobs) ──── SocketWaker │
+          │                                          ▲
+          └─────────► SocketNotifier ────────────────┘
+                      (UNIX socket wake-up, ~80µs)
 ```
 
-A fresh database still self-initializes on first use for local development, but explicit
-migration is the production path. For an existing queue, finish or stop 0.7.1 producers/workers,
-run the migration once, then start 0.7.2 processes.
-
-### Dashboard indexes
-
-The queue does **not** install dashboard indexes by default. They slow every enqueue even though
-pending rows never enter terminal or in-flight read-model indexes. Enable them once before
-mounting the dashboard:
-
-```ruby
-Async::Background::Queue.prepare_dashboard!(path: ENV.fetch("QUEUE_DB_PATH"))
-```
+Jobs are persisted in SQLite, so a missed wake-up is never a lost job —
+workers also poll every 5 seconds as a safety net.
 
-It adds four compact indexes: cursor-sorted `done` / `failed` history plus separate
-`executing` / `claimed` in-flight lists. It does not change queue behavior or rerun the core migration.
+---
 
 ## Metrics
 
-Metrics are an optional integration with `async-utilization` (`>= 0.3`, `< 0.5`). The
-background worker remains fully functional when that gem is absent. With it installed, each
-worker publishes counters to a shared-memory segment.
+Metrics are an optional integration with `async-utilization`. With the gem
+installed, each worker publishes counters to a shared-memory segment:
 
 ```ruby
-runner.metrics.enabled?
 runner.metrics.values
 # => { total_runs: 142, total_successes: 140, total_failures: 2,
 #      total_timeouts: 0, total_skips: 5, active_jobs: 1, ... }
@@ -175,19 +163,13 @@ Async::Background::Metrics.read_all(total_workers: 2)
 # => [{ worker: 1, ... }, { worker: 2, ... }]
 ```
 
-`Metrics.read_all` returns `[]` until the optional gem is installed and a worker has created
-the file, so an observer can render an unavailable state without rescuing `LoadError`. Its
-snapshot is lock-free best effort: cumulative fields (`total_runs`, `total_successes`,
-`total_failures`, `total_timeouts`, `total_skips`) are counters; `active_jobs`,
-`last_run_at`, and `last_duration_ms` are gauges. Fields can describe adjacent moments in time
-rather than one globally atomic instant.
-
-By default the file is `/tmp/async-background.shm`. Set `ASYNC_BACKGROUND_METRICS_PATH`
-or pass `metrics_shm_path:` to `Runner.new` when another observer runs in a separate process
-or container; both sides must see the same mounted file.
-
+Without the gem, `runner.metrics.enabled?` is `false` and `read_all` returns
+`[]` — no `LoadError` to rescue. Configuration and the cross-container
+shared-memory path are covered in
+[Get Started → Optional metrics](docs/GET_STARTED.md#appendix-optional-metrics).
 
+---
 
 ## License
 
-MIT
+MIT.

data/lib/async/background/version.rb

@@ -2,6 +2,6 @@
 
 module Async
   module Background
-    VERSION = '1.0.0'
+    VERSION = '1.0.1'
   end
 end

data/lib/async/background/web/app.rb

@@ -6,7 +6,8 @@ module Async
       class App
         def initialize(config)
           @config = config.validate!
-          @auth = Auth.new(@config.auth)
+          @logger = @config.logger
+          @auth = Auth.new(@config.auth, logger: @logger)
           @snapshot = Snapshot.new(path: @config.queue_path, counts_cache_ttl: @config.counts_cache_ttl).open!
           @metrics_reader = build_metrics_reader
           @serializer = Serializer.new(@config)
@@ -15,30 +16,45 @@ module Async
         end
 
         def call(env)
+          head = env['REQUEST_METHOD'] == 'HEAD'
+          response = handle(env, head: head)
+          return response unless head
+
+          status, headers, _body = response
+          [status, headers, []]
+        end
+
+        def close
+          @event_hub&.close
+          @snapshot.close
+          self
+        end
+
+        private
+
+        def handle(env, head:)
           return Response.unauthorized unless @auth.authorized?(env)
 
           route = @router.match(env)
           return Response.not_found unless route
 
+          if head && route == :stream
+            return @config.transport == :sse ? [200, Response.sse_headers, []] : Response.not_found
+          end
+
           dispatch(route, env)
         rescue RequestError => error
           Response.bad_request(error.message)
         rescue UnavailableError, ClosedError
           Response.unavailable
-        rescue StandardError
+        rescue StandardError => error
           # Do not turn internal class names, paths or database errors into an
-          # unauthenticated information disclosure channel.
+          # unauthenticated information disclosure channel — but do surface
+          # them to the operator via the configured logger.
+          log_internal_error(env, error)
           Response.internal_error
         end
 
-        def close
-          @event_hub&.close
-          @snapshot.close
-          self
-        end
-
-        private
-
         def build_metrics_reader
           return unless @config.metrics_enabled?
 
@@ -48,12 +64,7 @@ module Async
         def build_event_hub
           return unless @config.transport == :sse
 
-          EventHub.new(
-            @snapshot,
-            @serializer,
-            metrics_reader: @metrics_reader,
-            poll_seconds: @config.stream_poll_seconds
-          )
+          EventHub.new(@snapshot, @serializer, metrics_reader: @metrics_reader)
         end
 
         def dispatch(route, env)
@@ -128,10 +139,20 @@ module Async
             Stream.new(
               @event_hub,
               heartbeat_seconds: @config.stream_heartbeat_seconds,
-              retry_ms: @config.stream_retry_ms
+              retry_ms: @config.stream_retry_ms,
+              poll_seconds: @config.stream_poll_seconds,
+              logger: @logger
             )
           )
         end
+
+        def log_internal_error(env, error)
+          @logger&.error(
+            "[async-background-web] internal error on " \
+            "#{env['REQUEST_METHOD']} #{env['PATH_INFO']}: " \
+            "#{error.class}: #{error.message}"
+          )
+        end
       end
     end
   end

data/lib/async/background/web/auth.rb

@@ -4,13 +4,18 @@ module Async
   module Background
     module Web
       class Auth
-        def initialize(callable)
+        def initialize(callable, logger: nil)
           @callable = callable
+          @logger = logger
         end
 
         def authorized?(env)
           !!@callable.call(env)
-        rescue StandardError
+        rescue StandardError => error
+          @logger&.warn(
+            "[async-background-web] auth callable raised: " \
+            "#{error.class}: #{error.message}"
+          )
           false
         end
       end

data/lib/async/background/web/configuration.rb

@@ -31,7 +31,8 @@ module Async
                       :stream_heartbeat_seconds,
                       :stream_retry_ms,
                       :title,
-                      :mount_path
+                      :mount_path,
+                      :logger
 
         def initialize
           @queue_path = Queue::Store.default_path
@@ -49,6 +50,7 @@ module Async
           @stream_retry_ms = DEFAULT_STREAM_RETRY_MS
           @title = 'Async::Background'
           @mount_path = ''
+          @logger = nil
         end
 
         def validate!
@@ -62,6 +64,7 @@ module Async
           validate_redactor!
           validate_metrics!
           validate_mount_path!
+          validate_logger!
           self
         end
 
@@ -148,9 +151,20 @@ module Async
         end
 
         def validate_mount_path!
-          return if mount_path.is_a?(String)
+          raise ConfigurationError, 'mount_path must be a String' unless mount_path.is_a?(String)
+          return if mount_path.empty?
 
-          raise ConfigurationError, 'mount_path must be a String'
+          raise ConfigurationError, 'mount_path must start with "/" or be empty' unless mount_path.start_with?('/')
+          raise ConfigurationError, 'mount_path must not end with "/"' if mount_path.end_with?('/')
+          raise ConfigurationError, 'mount_path must not contain control characters' if mount_path.match?(/[[:cntrl:]]/)
+          raise ConfigurationError, 'mount_path must not contain whitespace' if mount_path.match?(/\s/)
+        end
+
+        def validate_logger!
+          return if logger.nil?
+          return if logger.respond_to?(:warn) && logger.respond_to?(:error)
+
+          raise ConfigurationError, 'logger must respond to #warn and #error'
         end
       end
     end

data/lib/async/background/web/event_hub.rb

@@ -9,184 +9,61 @@ module Async
         HEARTBEAT_FRAME = ":keepalive\n\n"
         UNAVAILABLE_FRAME = "event: unavailable\ndata: #{JSON.generate(error: 'unavailable')}\n\n".freeze
 
-        class Subscription
-          def initialize(clock: nil)
-            @clock = clock || -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) }
-            @mutex = Mutex.new
-            @condition = ConditionVariable.new
-            @frame = nil
-            @closed = false
-          end
-
-          def publish(frame)
-            @mutex.synchronize do
-              return false if @closed
-
-              @frame = frame
-              @condition.signal
-              true
-            end
-          end
-
-          def pop(timeout:)
-            deadline = @clock.call + timeout
-
-            @mutex.synchronize do
-              while @frame.nil? && !@closed
-                remaining = deadline - @clock.call
-                break if remaining <= 0
-
-                @condition.wait(@mutex, remaining)
-              end
-
-              frame = @frame
-              @frame = nil
-              frame
-            end
-          end
-
-          def close
-            @mutex.synchronize do
-              return if @closed
-
-              @closed = true
-              @condition.broadcast
-            end
-          end
-
-          def closed?
-            @mutex.synchronize { @closed }
-          end
-        end
-
-        def initialize(snapshot, serializer, metrics_reader: nil, poll_seconds:, sleeper: nil)
+        def initialize(snapshot, serializer, metrics_reader: nil)
           @snapshot = snapshot
           @serializer = serializer
           @metrics_reader = metrics_reader
-          @poll_seconds = poll_seconds
-          @sleeper = sleeper || ->(seconds) { sleep(seconds) }
           @mutex = Mutex.new
-          @condition = ConditionVariable.new
-          @subscribers = {}
+          @cached_version = nil
+          @cached_frame = nil
           @closed = false
-          @monitor = nil
-          @last_data_version = nil
-          @unavailable = false
         end
 
-        def subscribe
-          subscription = Subscription.new
-          frame, data_version = current_overview
+        def current_version
+          @mutex.synchronize { raise ClosedError, 'event hub is closed' if @closed }
+          @snapshot.data_version
+        end
 
+        def frame_for(version)
           @mutex.synchronize do
             raise ClosedError, 'event hub is closed' if @closed
+            return @cached_frame if @cached_version == version && @cached_frame
 
-            @subscribers[subscription.object_id] = subscription
-            @last_data_version ||= data_version
-            start_monitor_unless_running!
-            @condition.signal
+            refresh_frame_locked!
+            @cached_frame
           end
-
-          [subscription, frame]
         end
 
-        def unsubscribe(subscription)
+        def initial_frame
           @mutex.synchronize do
-            @subscribers.delete(subscription.object_id)
+            raise ClosedError, 'event hub is closed' if @closed
+
+            refresh_frame_locked!
+            [@cached_version, @cached_frame]
           end
-          subscription.close
-          nil
         end
 
         def close
-          monitor = nil
-          subscribers = nil
-
           @mutex.synchronize do
-            return if @closed
-
             @closed = true
-            subscribers = @subscribers.values
-            @subscribers.clear
-            monitor = @monitor
-            @condition.broadcast
+            @cached_frame = nil
+            @cached_version = nil
           end
-
-          subscribers.each(&:close)
-          monitor&.join(1) unless monitor == Thread.current
-          nil
+          self
         end
 
-        private
-
-        def start_monitor_unless_running!
-          return if @monitor&.alive?
-
-          @monitor = Thread.new { monitor_loop }
-          @monitor.name = 'async-background-web-events' if @monitor.respond_to?(:name=)
-          @monitor.abort_on_exception = false
-        end
-
-        def monitor_loop
-          loop do
-            break unless wait_for_subscribers
-
-            begin
-              detect_change
-            rescue ClosedError, UnavailableError
-              notify_unavailable
-            end
-
-            @sleeper.call(@poll_seconds)
-          end
-        ensure
-          @mutex.synchronize { @monitor = nil if @monitor == Thread.current }
-        end
-
-        def wait_for_subscribers
-          @mutex.synchronize do
-            @condition.wait(@mutex) while !@closed && @subscribers.empty?
-            !@closed
-          end
+        def closed?
+          @mutex.synchronize { @closed }
         end
 
-        def detect_change
-          version = @snapshot.data_version
-          previous_version = @mutex.synchronize { @last_data_version }
-          if version == previous_version
-            @mutex.synchronize { @unavailable = false }
-            return
-          end
-
-          frame, observed_version = current_overview
-          @mutex.synchronize do
-            @last_data_version = observed_version
-            @unavailable = false
-          end
-          broadcast(frame)
-        end
+        private
 
-        def current_overview
+        def refresh_frame_locked!
           overview = @snapshot.overview(force: true)
           metrics = @metrics_reader&.aggregated
           payload = @serializer.overview(overview, metrics)
-          ["event: overview\ndata: #{JSON.generate(payload)}\n\n", payload.fetch(:data_version)]
-        end
-
-        def notify_unavailable
-          should_broadcast = @mutex.synchronize do
-            next false if @unavailable
-
-            @unavailable = true
-            true
-          end
-          broadcast(UNAVAILABLE_FRAME) if should_broadcast
-        end
-
-        def broadcast(frame)
-          subscribers = @mutex.synchronize { @subscribers.values.dup }
-          subscribers.each { |subscription| subscription.publish(frame) }
-          nil
+          @cached_version = payload.fetch(:data_version)
+          @cached_frame = "event: overview\ndata: #{JSON.generate(payload)}\n\n"
         end
       end
     end