jetstream_bridge 7.0.5 → 7.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7d7a08f0c887494700158188558b2e686d7a55fe773d20d2716f13741f92de67
-  data.tar.gz: 6ebea4aecaeefdb2b03d07445313dd7faba8a09cc5b51c2a1cb2ab60a8a4f86a
+  metadata.gz: 290f604d47d1605f6c2b2abcde100159327a7418b103c0c054b0b92f5e7fbf56
+  data.tar.gz: 485e9cf4070d9671a1994ffbde32ab8ca6e41fbfb7d94d8cf571e2e8ec04cd3b
 SHA512:
-  metadata.gz: 661aefd62030b960a4d370226145406c1429391ac0f8be94b6c1c93dd8e575f095227efe8469b7a9bf4c9d9c086ff65574182a47c28371dd1c7a290dbccdcb66
-  data.tar.gz: efff08f8624bf540027b0ce5a049c839db02e4c041ddf33d131f3fb6e69cd9a92187ecde3e6ee04afd435bcbd5ec26d5efd72e7aec8a639f779fa3fab2f03e17
+  metadata.gz: a1a17ed6a75d8f9b32de3efbe23da1797824c4e846c0d4d73ee1014d9ea20048367088cfaabd3f9a02179314090741237d21dffc0c618a429a47510597bb1ed8
+  data.tar.gz: f3ae4a9e9b256f4dfa1d1e84a32d66140ae01c52d07f21b8fac7976c61bb76a81cb83a53aae6631af9c0affa81eb24d81b7167ebb18875489e398f7869ebd2d6

data/CHANGELOG.md

@@ -5,6 +5,21 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [7.1.0] - 2026-02-01
+
+### Added
+
+- **Auto-create consumer on subscription** - Consumers are now automatically created if they don't exist when subscribing, regardless of the `auto_provision` setting. This eliminates the need for manual consumer provisioning while `auto_provision` continues to control only stream topology creation.
+- `SubscriptionManager#stream_exists?` - Public method to check if a stream exists.
+- `SubscriptionManager#consumer_exists?` - Public method to check if a consumer exists in the stream.
+- `SubscriptionManager#create_consumer_if_missing!` - Public method to create a consumer only if it doesn't already exist (race-condition safe). Raises `StreamNotFoundError` if the stream doesn't exist.
+
+### Changed
+
+- `ensure_consumer!` now always auto-creates the consumer if it doesn't exist. The `auto_provision` setting only controls stream topology (stream creation), not consumer creation.
+- Consumer recovery automatically attempts to create the consumer when a "consumer not found" error is detected.
+- **Stream must exist** - Auto-create consumer fails fast with `StreamNotFoundError` if the stream doesn't exist. Streams must be provisioned separately via `auto_provision=true` or manual provisioning.
+
 ## [7.0.0] - 2026-01-30
 
 ### Added

data/README.md

@@ -26,16 +26,16 @@
 
 - Transactional outbox and idempotent inbox (optional) for exactly-once pipelines.
 - Durable pull (default) or push consumers with retries, backoff, and DLQ routing.
-- Auto stream/consumer provisioning with overlap protection.
+- Auto stream provisioning with overlap protection; consumers auto-created on subscription.
 - Rails-native: generators, migrations, health check, and eager-loading safety.
-- Least-privilege friendly: run with `auto_provision=false` plus pre-created consumers.
+- Least-privilege friendly: run with `auto_provision=false` (stream must exist; consumers are auto-created).
 - Mock NATS for fast, no-infra testing.
 
 ## Quick Start
 
 ```ruby
 # Gemfile
-gem "jetstream_bridge", "~> 7.0"
+gem "jetstream_bridge", "~> 7.1"
 ```
 
 ```bash

data/docs/API.md

@@ -40,7 +40,7 @@ JetstreamBridge.configure do |config|
   config.backoff      = ["1s", "5s", "15s", "30s", "60s"]
 
   # Provisioning
-  config.auto_provision = true  # Auto-create stream/consumer on startup
+  config.auto_provision = true  # Auto-create stream on startup (consumers are always auto-created)
 
   # Connection behavior
   config.lazy_connect = false  # Set true to skip autostart
@@ -224,6 +224,28 @@ provisioner.provision_stream!
 provisioner.provision_consumer!
 ```
 
+### `JetstreamBridge::SubscriptionManager`
+
+Manages consumer lifecycle during subscription. Used internally by `Consumer`, but also available for advanced use cases.
+
+```ruby
+# Create subscription manager
+sub_mgr = JetstreamBridge::SubscriptionManager.new(jetstream_context, "my-durable")
+
+# Check if stream exists
+sub_mgr.stream_exists?  # => true/false
+
+# Check if consumer exists
+sub_mgr.consumer_exists?  # => true/false
+
+# Create consumer if missing (raises StreamNotFoundError if stream doesn't exist)
+sub_mgr.create_consumer_if_missing!
+```
+
+**Note:** Consumers are automatically created when subscribing, regardless of the `auto_provision` setting. The `auto_provision` setting only controls stream topology creation.
+
+**Raises:** `JetstreamBridge::StreamNotFoundError` if the stream doesn't exist when calling `create_consumer_if_missing!`
+
 ## Health & Diagnostics
 
 ### `JetstreamBridge.health_check`
@@ -347,6 +369,20 @@ rescue JetstreamBridge::PublishError => e
 end
 ```
 
+### `JetstreamBridge::StreamNotFoundError`
+
+Raised when attempting to create a consumer on a stream that doesn't exist.
+
+```ruby
+begin
+  consumer = JetstreamBridge::Consumer.new { |event| process(event) }
+  consumer.run!
+rescue JetstreamBridge::StreamNotFoundError => e
+  logger.error("Stream not found: #{e.message}")
+  # Stream must be provisioned separately (use auto_provision=true or run provisioning with admin credentials)
+end
+```
+
 ### Custom Error Handler
 
 ```ruby

data/docs/GETTING_STARTED.md

@@ -6,7 +6,7 @@ This guide covers installation, Rails setup, configuration, and basic publish/co
 
 ```ruby
 # Gemfile
-gem "jetstream_bridge", "~> 7.0"
+gem "jetstream_bridge", "~> 7.1"
 ```
 
 ```bash
@@ -109,6 +109,12 @@ JetstreamBridge.configure do |config|
   config.max_deliver = 5
   config.ack_wait    = "30s"
   config.backoff     = %w[1s 5s 15s 30s 60s]
+
+  # Provisioning (default: true)
+  # - When true: auto-creates stream on startup
+  # - When false: stream must exist (provision separately with admin credentials)
+  # Note: Consumers are always auto-created on subscription regardless of this setting
+  config.auto_provision = true
 end
 
 # Note: `configure` only sets options; it does not connect. Rails will start
@@ -119,6 +125,8 @@ end
 
 Rails autostart runs after initialization (including in console). You can opt out for rake tasks or other tooling with `config.lazy_connect = true` or `JETSTREAM_BRIDGE_DISABLE_AUTOSTART=1`; it will then connect on first publish/subscribe.
 
+**Important:** When `auto_provision=false`, the stream must exist before subscribing. Consumers are automatically created when they don't exist, but attempting to create a consumer on a missing stream will raise `StreamNotFoundError`.
+
 ### Push consumer mode (restricted credentials)
 
 If your NATS user cannot publish to `$JS.API.*`, switch to push consumers and pre-create the durable consumer:

data/docs/RESTRICTED_PERMISSIONS.md

@@ -36,13 +36,15 @@ When you cannot modify NATS server permissions, you have **two options**:
 
 For both options, you need to:
 
-0. **Turn off runtime provisioning** so the app never calls `$JS.API.*`:
+0. **Turn off runtime provisioning** so the app never calls `$JS.API.*` for stream creation:
    - Set `config.auto_provision = false`
-   - Provision stream + consumer once with admin credentials (CLI below or `bundle exec rake jetstream_bridge:provision`)
-1. **Pre-create the consumer** using a privileged NATS account
-2. **Ensure the consumer configuration matches** what your app expects
+   - Provision the **stream** once with admin credentials (CLI below or `bundle exec rake jetstream_bridge:provision`)
+1. **Ensure the stream exists** - consumers are auto-created when subscribing, but the stream must be provisioned separately
+2. **Optionally pre-create the consumer** using a privileged NATS account (consumers are auto-created if missing)
 
-> Tip: When `auto_provision=false`, the app still connects/publishes/consumes but skips JetStream management APIs (account_info, stream_info, consumer_info). Health checks will report basic connectivity only.
+> **Note (v7.1.0+):** Consumers are now auto-created on subscription regardless of the `auto_provision` setting. The `auto_provision` setting only controls stream topology creation. If the stream doesn't exist, subscription will fail with `StreamNotFoundError`.
+
+> Tip: When `auto_provision=false`, the app still connects/publishes/consumes but skips JetStream management APIs (account_info, stream_info) for stream creation. Health checks will report basic connectivity only.
 
 ---
 
@@ -543,19 +545,21 @@ nats stream info pwas-heavyworth-sync
 
 Check if the issue is:
 
-1. **Consumer doesn't exist** - Pre-create it with NATS CLI
+1. **Stream doesn't exist** - Provision it with NATS CLI or admin credentials (consumers are auto-created)
 2. **Filter subject mismatch** - Verify with `nats consumer info`
 3. **Permissions still insufficient** - User needs subscribe permissions on the filtered subject
 4. **Wrong stream name** - Ensure stream name matches your configured `config.stream_name`
 
+**Note (v7.1.0+):** Consumers are auto-created if they don't exist, so "consumer doesn't exist" is no longer a common cause. However, the **stream must exist** or you'll see `StreamNotFoundError`.
+
 ---
 
 ## Security Considerations
 
-1. **Consumer pre-creation requires privileged access** - Keep admin credentials secure
+1. **Stream pre-creation requires privileged access** - Keep admin credentials secure
 2. **Configuration drift risk** - If app config doesn't match consumer config, message delivery may fail silently
-3. **No automatic recovery** - If consumer is deleted, it won't be recreated automatically
-4. **Consider automation** - Use infrastructure-as-code (Terraform, Ansible) to manage consumer creation
+3. **Automatic consumer recovery (v7.1.0+)** - If consumer is deleted, it will be auto-recreated on subscription. Stream must exist.
+4. **Consider automation** - Use infrastructure-as-code (Terraform, Ansible) to manage stream creation
 
 ---
 

data/lib/generators/jetstream_bridge/migrations/migrations_generator.rb

@@ -40,10 +40,11 @@ module JetstreamBridge
         # Prefer the Rails application configuration which is available
         # without hitting ActiveRecord's dynamic matchers (that would raise
         # when no connection is established).
-        if defined?(Rails) &&
-           Rails.application &&
-           Rails.application.config.respond_to?(:active_record)
-          ar_config = Rails.application.config.active_record
+        if defined?(::Rails) &&
+           ::Rails.respond_to?(:application) &&
+           ::Rails.application &&
+           ::Rails.application.config.respond_to?(:active_record)
+          ar_config = ::Rails.application.config.active_record
           return ar_config.timestamped_migrations if ar_config.respond_to?(:timestamped_migrations)
         end
 

data/lib/jetstream_bridge/consumer/consumer.rb

@@ -429,6 +429,9 @@ module JetstreamBridge
           tag: 'JetstreamBridge::Consumer'
         )
 
+        # If this looks like a consumer-not-found error, try to auto-create it
+        auto_create_consumer_on_error(error) if consumer_not_found_error?(error)
+
         sleep(backoff_secs)
         ensure_subscription!
 
@@ -440,6 +443,27 @@ module JetstreamBridge
       0
     end
 
+    def consumer_not_found_error?(error)
+      msg = error.message.to_s.downcase
+      (msg.include?('consumer') && (msg.include?('not found') || msg.include?('does not exist'))) ||
+        msg.include?('no responders')
+    end
+
+    def auto_create_consumer_on_error(_error)
+      Logging.info(
+        "Consumer not found error detected, attempting auto-creation for #{@durable}...",
+        tag: 'JetstreamBridge::Consumer'
+      )
+
+      @sub_mgr.create_consumer_if_missing!
+    rescue StandardError => e
+      Logging.warn(
+        "Auto-create consumer failed: #{e.class} #{e.message}",
+        tag: 'JetstreamBridge::Consumer'
+      )
+      # Don't re-raise - let the normal recovery flow continue
+    end
+
     def calculate_reconnect_backoff(attempt)
       # Exponential backoff: 0.1s, 0.2s, 0.4s, 0.8s, 1.6s, ... up to 30s max
       base_delay = 0.1

data/lib/jetstream_bridge/consumer/subscription_manager.rb

@@ -28,14 +28,96 @@ module JetstreamBridge
       @desired_cfg
     end
 
-    def ensure_consumer!(force: false)
-      # Runtime path: never hit JetStream management APIs to avoid admin permissions.
-      unless force || @cfg.auto_provision
-        log_runtime_skip
+    # Ensure consumer exists, auto-creating if missing.
+    #
+    # @param force [Boolean] Kept for backward compatibility but no longer used.
+    #   Consumers are always auto-created regardless of this parameter.
+    def ensure_consumer!(**_options)
+      # Always auto-create consumer if it doesn't exist, regardless of auto_provision setting.
+      # auto_provision only controls stream topology creation, not consumer creation.
+      create_consumer_if_missing!
+    end
+
+    # Check if stream exists.
+    #
+    # @return [Boolean] true if stream exists, false otherwise
+    def stream_exists?
+      @jts.stream_info(stream_name)
+      true
+    rescue StandardError => e
+      msg = e.message.to_s.downcase
+      return false if msg.include?('not found') || msg.include?('does not exist') || msg.include?('no responders')
+
+      # Re-raise unexpected errors
+      raise unless msg.include?('stream')
+
+      false
+    end
+
+    # Check if consumer exists in the stream.
+    #
+    # @return [Boolean] true if consumer exists, false otherwise
+    def consumer_exists?
+      @jts.consumer_info(stream_name, @durable)
+      true
+    rescue StandardError => e
+      msg = e.message.to_s.downcase
+      return false if msg.include?('not found') || msg.include?('does not exist') || msg.include?('no responders')
+
+      # Re-raise unexpected errors
+      raise unless msg.include?('consumer')
+
+      false
+    end
+
+    # Create consumer only if it doesn't already exist.
+    #
+    # Fails if the stream doesn't exist - streams must be provisioned separately.
+    #
+    # This is a safer alternative to create_consumer! that won't fail
+    # if the consumer was already created by another process.
+    #
+    # @raise [StreamNotFoundError] if the stream doesn't exist
+    def create_consumer_if_missing!
+      # First, verify stream exists - fail fast with clear error if not
+      unless stream_exists?
+        raise StreamNotFoundError,
+              "Stream '#{stream_name}' does not exist. " \
+              'Streams must be provisioned separately ' \
+              '(use auto_provision=true or run provisioning with admin credentials).'
+      end
+
+      if consumer_exists?
+        Logging.info(
+          "Consumer #{@durable} already exists (stream=#{stream_name})",
+          tag: 'JetstreamBridge::Consumer'
+        )
         return
       end
 
+      Logging.info(
+        "Consumer #{@durable} not found, auto-creating on stream #{stream_name}...",
+        tag: 'JetstreamBridge::Consumer'
+      )
       create_consumer!
+    rescue StreamNotFoundError
+      raise
+    rescue StandardError => e
+      # If creation fails due to consumer already existing (race condition), that's OK
+      msg = e.message.to_s.downcase
+      if msg.include?('already') || msg.include?('exists')
+        Logging.info(
+          "Consumer #{@durable} was created by another process",
+          tag: 'JetstreamBridge::Consumer'
+        )
+        return
+      end
+
+      Logging.error(
+        "Failed to auto-create consumer #{@durable}: #{e.class} #{e.message}",
+        tag: 'JetstreamBridge::Consumer'
+      )
+      raise
     end
 
     # Bind a subscriber to the existing durable consumer.
@@ -136,14 +218,6 @@ module JetstreamBridge
       )
     end
 
-    def log_runtime_skip
-      Logging.info(
-        "Skipping consumer provisioning/verification for #{@durable} at runtime to avoid JetStream API usage. " \
-        'Ensure it is pre-created via provisioning.',
-        tag: 'JetstreamBridge::Consumer'
-      )
-    end
-
     def resolve_nc
       return @jts.nc if @jts.respond_to?(:nc)
       return @jts.instance_variable_get(:@nc) if @jts.instance_variable_defined?(:@nc)

data/lib/jetstream_bridge/version.rb

@@ -4,5 +4,5 @@
 #
 # Version constant for the gem.
 module JetstreamBridge
-  VERSION = '7.0.5'
+  VERSION = '7.1.0'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: jetstream_bridge
 version: !ruby/object:Gem::Version
-  version: 7.0.5
+  version: 7.1.0
 platform: ruby
 authors:
 - Mike Attara