jetstream_bridge 4.5.0 → 4.5.1

data/docs/PRODUCTION.md

@@ -78,9 +78,9 @@ JetstreamBridge.configure do |config|
   config.connect_retry_delay = 3     # Default: 2 seconds
 
   # Required configuration
+  config.stream_name = ENV.fetch("JETSTREAM_STREAM_NAME", "jetstream-bridge-stream")
   config.app_name = ENV.fetch("APP_NAME", "myapp")
   config.destination_app = ENV.fetch("DESTINATION_APP")
-  config.stream_name = ENV.fetch("STREAM_NAME", "myapp-stream")
 
   # Enable reliability features
   config.use_outbox = true
@@ -100,31 +100,6 @@ JetstreamBridge.configure do |config|
 end
 ```
 
-### Permissions and Inbox Prefix
-
-If your NATS account restricts `_INBOX.>` subscriptions, set an allowed prefix:
-
-```ruby
-JetstreamBridge.configure do |config|
-  config.inbox_prefix = "$RPC"
-end
-```
-
-For pre-provisioned streams and consumers:
-
-```ruby
-JetstreamBridge.configure do |config|
-  config.stream_name = "my-stream"      # required
-  config.durable_name = "my-durable"    # optional
-  config.disable_js_api = true          # skip JetStream management APIs
-end
-```
-
-Minimum NATS permissions:
-
-- **Publish**: `$JS.API.>`, `$JS.ACK.>`, source/destination subjects, DLQ subject
-- **Subscribe**: `_INBOX.>` (or custom inbox_prefix), destination subject
-
 ---
 
 ## Consumer Tuning
@@ -167,7 +142,7 @@ Long-running consumers automatically:
 
 - Log health checks every 10 minutes (iterations, memory, uptime)
 - Warn when memory exceeds 1GB
-- Warn once when heap object counts grow large so you can profile/trigger GC in the host app
+- Suggest garbage collection when heap grows large
 
 Monitor these logs to detect memory leaks early.
 
@@ -178,7 +153,7 @@ Monitor these logs to detect memory leaks early.
 ### Key Metrics to Track
 
 | Metric | Description | Alert Threshold |
-| -------- | ------------- | ----------------- |
+| --- | --- | --- |
 | Consumer Lag | Pending messages in stream | > 1000 messages |
 | DLQ Size | Messages in dead letter queue | > 100 messages |
 | Connection Status | Health check failures | 2 consecutive failures |
@@ -194,7 +169,7 @@ Use the built-in health check for monitoring:
 # config/routes.rb
 Rails.application.routes.draw do
   get '/health/jetstream', to: proc { |env|
-    health = JetstreamBridge.health
+    health = JetstreamBridge.health_check
     status = health[:healthy] ? 200 : 503
     [status, { 'Content-Type' => 'application/json' }, [health.to_json]]
   }
@@ -213,8 +188,8 @@ end
   },
   "stream": {
     "exists": true,
-    "name": "production-jetstream-bridge-stream",
-    "subjects": ["production.app.sync.worker"],
+    "name": "jetstream-bridge-stream",
+    "subjects": ["app.sync.worker"],
     "messages": 1523
   },
   "performance": {
@@ -222,15 +197,14 @@ end
     "health_check_duration_ms": 45.2
   },
   "config": {
-    "env": "production",
+    "stream_name": "jetstream-bridge-stream",
     "app_name": "app",
     "destination_app": "worker",
     "use_outbox": true,
     "use_inbox": true,
-    "use_dlq": true,
-    "disable_js_api": true
+    "use_dlq": true
   },
-  "version": "4.4.0"
+  "version": "4.0.3"
 }
 ```
 
@@ -267,6 +241,10 @@ jetstream_dlq = prometheus.gauge(
 
 ## Security Hardening
 
+### Permissions & Topology
+
+Keep runtime credentials least-privileged: set `config.auto_provision = false` and provision the stream/consumer during deploy (`bundle exec rake jetstream_bridge:provision` or NATS CLI). The exact durable/subject layout and minimal publish/subscribe permissions live in [RESTRICTED_PERMISSIONS.md](RESTRICTED_PERMISSIONS.md). Health checks skip stream info when `auto_provision=false`; rely on your provisioning pipeline for validation.
+
 ### Rate Limiting
 
 The health check endpoint has built-in rate limiting (1 uncached request per 5 seconds). For HTTP endpoints, add additional protection:

data/docs/RESTRICTED_PERMISSIONS.md

@@ -0,0 +1,399 @@
+# Working with Restricted NATS Permissions
+
+This guide explains how to use JetStream Bridge when your NATS user lacks JetStream API permissions (`$JS.API.*` subjects).
+
+## Problem
+
+When the NATS user has restricted permissions and cannot access JetStream API subjects, you'll see errors like:
+
+```markdown
+ERROR -- : [JetstreamBridge::ConnectionManager] NATS error: 'Permissions Violation for Publish to "$JS.API.CONSUMER.INFO.{stream}.{consumer}"'
+NATS::IO::Timeout: nats: timeout
+```
+
+This happens because:
+
+1. JetStream Bridge tries to verify consumer configuration using `$JS.API.CONSUMER.INFO.*`
+2. The NATS user doesn't have permission to publish to these API subjects
+3. The connection is terminated, causing timeout errors
+
+## Solution Overview
+
+When you cannot modify NATS server permissions, you need to:
+
+0. **Turn off runtime provisioning** so the app never calls `$JS.API.*`:
+   - Set `config.auto_provision = false`
+   - Provision stream + consumer once with admin credentials (CLI below or `bundle exec rake jetstream_bridge:provision`)
+1. **Pre-create the consumer** using a privileged NATS account
+2. **Ensure the consumer configuration matches** what your app expects
+
+> Tip: When `auto_provision=false`, the app still connects/publishes/consumes but skips JetStream management APIs (account_info, stream_info, consumer_info). Health checks will report basic connectivity only.
+
+---
+
+## Runtime requirements (least privilege)
+
+- Config: `config.auto_provision = false`, `config.stream_name` set explicitly.
+- Topology: stream + durable consumer must be pre-provisioned (via `bundle exec rake jetstream_bridge:provision` or NATS CLI).
+- NATS permissions for runtime creds:
+  - publish allow: `">"` (or narrowed to your business subjects) and `$JS.API.CONSUMER.MSG.NEXT.{stream_name}.{app_name}-workers`
+  - subscribe allow: `">"` (or narrowed) and `_INBOX.>` (responses for pull consumers)
+- Health check will only report connectivity (stream info skipped).
+
+### Topology required
+
+- Stream: `config.stream_name` (retention: workqueue, storage: file).
+- Subjects:
+  - Publish: `{app_name}.sync.{destination_app}`
+  - Consume: `{destination_app}.sync.{app_name}`
+  - DLQ (if enabled): `{app_name}.sync.dlq`
+- Durable consumer: `{app_name}-workers` filtering on `{destination_app}.sync.{app_name}`.
+
+---
+
+## Option A: Provision with the built-in task (creates stream + consumer)
+
+Run this from CI/deploy with admin NATS credentials:
+
+```bash
+# Uses your configured stream/app/destination to create the stream + durable consumer
+NATS_URLS="nats://admin:pass@10.199.12.34:4222" \
+bundle exec rake jetstream_bridge:provision
+```
+
+This is the easiest way to keep `auto_provision=false` in runtime while still reusing the bridge’s topology logic (subjects, DLQ, overlap guard).
+
+## Option B: Pre-create the Consumer using NATS CLI
+
+### Install NATS CLI
+
+```bash
+# Download from https://github.com/nats-io/natscli/releases
+curl -sf https://binaries.nats.dev/nats-io/natscli/nats@latest | sh
+
+# Or using Homebrew (macOS)
+brew install nats-io/nats-tools/nats
+```
+
+### Create the Consumer
+
+You need to create a durable pull consumer with the exact configuration your app expects.
+
+**Required values from your JetStream Bridge config:**
+
+- **Stream name**: `JETSTREAM_STREAM_NAME` (e.g., `jetstream-bridge-stream`)
+- **Consumer name**: `{app_name}-workers` (e.g., `pwas-workers`)
+- **Filter subject**: `{app_name}.sync.{destination_app}` (e.g., `pwas-workers.sync.heavyworth`)
+
+**Create consumer command:**
+
+```bash
+# Connect using a privileged NATS account
+nats context save admin \
+  --server=nats://admin-user:admin-pass@10.199.12.34:4222 \
+  --description="Admin account for consumer creation"
+
+# Select the context
+nats context select admin
+
+# Create the consumer
+nats consumer add production-jetstream-bridge-stream production-pwas-workers \
+  --filter "production.pwas-workers.sync.heavyworth" \
+  --ack explicit \
+  --pull \
+  --deliver all \
+  --max-deliver 5 \
+  --ack-wait 30s \
+  --replay instant \
+  --max-pending 25000
+```
+
+**With backoff (recommended for production):**
+
+```bash
+nats consumer add production-jetstream-bridge-stream production-pwas-workers \
+  --filter "production.pwas-workers.sync.heavyworth" \
+  --ack explicit \
+  --pull \
+  --deliver all \
+  --max-deliver 5 \
+  --ack-wait 30s \
+  --backoff 1s,5s,15s,30s,60s \
+  --replay instant \
+  --max-pending 25000
+```
+
+### Verify Consumer Creation
+
+```bash
+nats consumer info production-jetstream-bridge-stream production-pwas-workers
+```
+
+Expected output:
+
+```bash
+Information for Consumer production-jetstream-bridge-stream > production-pwas-workers
+
+Configuration:
+
+        Durable Name: production-pwas-workers
+      Filter Subject: production.pwas-workers.sync.heavyworth
+          Ack Policy: explicit
+            Ack Wait: 30s
+       Replay Policy: instant
+  Maximum Deliveries: 5
+             Backoff: [1s 5s 15s 30s 60s]
+
+State:
+
+  Last Delivered Message: Consumer sequence: 0 Stream sequence: 0
+    Acknowledgment Floor: Consumer sequence: 0 Stream sequence: 0
+        Pending Messages: 0
+    Redelivered Messages: 0
+```
+
+---
+
+## Step 2: Configure JetStream Bridge
+
+Configure JetStream Bridge to avoid JetStream management APIs at runtime (pre-provisioning handles them instead):
+
+```ruby
+# config/initializers/jetstream_bridge.rb
+JetstreamBridge.configure do |config|
+  config.nats_urls = ENV.fetch("NATS_URLS")
+  config.stream_name = "jetstream-bridge-stream"
+  config.app_name = "pwas-workers"
+  config.destination_app = "heavyworth"
+  config.auto_provision = false
+
+  config.use_outbox = true
+  config.use_inbox = true
+  config.use_dlq = true
+
+  # These settings MUST match the pre-created consumer
+  config.max_deliver = 5
+  config.ack_wait = "30s"
+  config.backoff = %w[1s 5s 15s 30s 60s]
+end
+```
+
+**Critical:** The `max_deliver`, `ack_wait`, and `backoff` values in your config **must exactly match** the pre-created consumer configuration. If they don't match, messages may be redelivered incorrectly.
+
+---
+
+## Step 3: Deploy and Verify
+
+### Deploy the Updated Gem
+
+If you're working on the jetstream_bridge gem itself:
+
+```bash
+# Build the gem
+gem build jetstream_bridge.gemspec
+
+# Install locally for testing
+gem install ./jetstream_bridge-4.5.0.gem
+
+# Or update in your application's Gemfile.lock
+bundle update jetstream_bridge
+```
+
+If this is a local modification, you can point your Gemfile to the local path:
+
+```ruby
+# Gemfile (temporary for testing)
+gem 'jetstream_bridge', path: '/path/to/local/jetstream_bridge'
+```
+
+### Restart the Service
+
+```bash
+sudo systemctl restart pwas_production_sync
+```
+
+### Monitor the Logs
+
+```bash
+sudo journalctl -u pwas_production_sync -f
+```
+
+**Expected success output:**
+
+```bash
+INFO -- : [JetstreamBridge::ConnectionManager] Connected to NATS (1 server): nats://pwas:***@10.199.12.34:4222
+INFO -- : [DataSync] Consumer starting (durable=production-pwas-workers, batch=25, dest="heavyworth")
+INFO -- : [DataSync] run! started successfully
+```
+
+Health checks will report connectivity only when `auto_provision=false` (stream info is skipped to avoid `$JS.API.STREAM.INFO`).
+
+**If you still see errors:**
+
+1. **Timeout during subscribe** - The pre-created consumer name/filter might not match. Verify:
+
+   ```bash
+   nats consumer ls production-jetstream-bridge-stream
+   ```
+
+2. **Permission violations on fetch** - The NATS user also needs permission to fetch messages. Minimum required:
+
+   ```conf
+   subscribe: {
+     allow: ["production.>", "_INBOX.>"]
+   }
+   ```
+
+## Maintenance
+
+### Updating Consumer Configuration
+
+If you need to change consumer settings (e.g., increase `max_deliver`):
+
+1. **Stop your application** to prevent message processing
+
+   ```bash
+   sudo systemctl stop pwas_production_sync
+   ```
+
+2. **Delete the old consumer** (using privileged account)
+
+   ```bash
+   nats consumer rm production-jetstream-bridge-stream production-pwas-workers -f
+   ```
+
+3. **Create the new consumer** with updated settings
+
+   ```bash
+   nats consumer add production-jetstream-bridge-stream production-pwas-workers \
+     --filter "production.pwas-workers.sync.heavyworth" \
+     --ack explicit \
+     --pull \
+     --deliver all \
+     --max-deliver 10 \
+     --ack-wait 60s \
+     --backoff 2s,10s,30s,60s,120s \
+     --replay instant \
+     --max-pending 25000
+   ```
+
+4. **Update your application config** to match
+
+   ```ruby
+   config.max_deliver = 10
+   config.ack_wait = "60s"
+   config.backoff = %w[2s 10s 30s 60s 120s]
+   ```
+
+5. **Restart your application**
+
+   ```bash
+   sudo systemctl start pwas_production_sync
+   ```
+
+### Monitoring
+
+Add monitoring to detect configuration drift:
+
+```ruby
+# Check consumer exists and is healthy
+health = JetstreamBridge.health_check
+unless health[:healthy]
+  alert("JetStream consumer unhealthy: #{health}")
+end
+```
+
+---
+
+## Troubleshooting
+
+### Consumer doesn't receive messages
+
+**Check stream has messages:**
+
+```bash
+nats stream info production-jetstream-bridge-stream
+```
+
+**Verify filter subject matches your topology:**
+
+```bash
+# App publishes to:   {app_name}.sync.{destination_app}
+# Consumer filters on: {destination_app}.sync.{app_name}
+```
+
+### Service keeps restarting
+
+Check if the issue is:
+
+1. **Consumer doesn't exist** - Pre-create it with NATS CLI
+2. **Filter subject mismatch** - Verify with `nats consumer info`
+3. **Permissions still insufficient** - User needs subscribe permissions on the filtered subject
+4. **Wrong stream name** - Ensure stream name matches your configured `config.stream_name`
+
+---
+
+## Security Considerations
+
+1. **Consumer pre-creation requires privileged access** - Keep admin credentials secure
+2. **Configuration drift risk** - If app config doesn't match consumer config, message delivery may fail silently
+3. **No automatic recovery** - If consumer is deleted, it won't be recreated automatically
+4. **Consider automation** - Use infrastructure-as-code (Terraform, Ansible) to manage consumer creation
+
+---
+
+## Example: Production Setup for pwas-api
+
+Based on your logs, here's the exact setup:
+
+```bash
+# 1. Pre-create the consumer (as admin)
+nats consumer add pwas-heavyworth-sync production-pwas-workers \
+  --filter "production.pwas-workers.sync.heavyworth" \
+  --ack explicit \
+  --pull \
+  --deliver all \
+  --max-deliver 5 \
+  --ack-wait 30s \
+  --backoff 1s,5s,15s,30s,60s \
+  --replay instant
+```
+
+```ruby
+# 2. Update config/initializers/jetstream_bridge.rb
+JetstreamBridge.configure do |config|
+  config.nats_urls = "nats://pwas:***@10.199.12.34:4222"
+  config.stream_name = "jetstream-bridge-stream"
+  config.app_name = "pwas-workers"
+  config.destination_app = "heavyworth"
+  config.max_deliver = 5
+  config.ack_wait = "30s"
+  config.backoff = %w[1s 5s 15s 30s 60s]
+end
+```
+
+**Note:** Verify your exact stream name and consumer durable (`app_name-workers`) match what was provisioned.
+
+---
+
+## Alternative: Request Minimal Permissions
+
+If you have any influence over NATS permissions, request only these minimal subjects:
+
+```conf
+# Minimal permissions needed for JetStream Bridge consumer
+publish: {
+  allow: [
+    ">",                                   # Your app subjects (narrow if desired)
+    "$JS.API.CONSUMER.MSG.NEXT.{stream_name}.{app_name}-workers",  # Fetch messages (required)
+  ]
+}
+subscribe: {
+  allow: [
+    ">",                                   # Your app subjects (narrow if desired)
+    "_INBOX.>",                            # Request-reply responses (required)
+  ]
+}
+```
+
+These are read-only operations and don't allow creating/modifying streams or consumers.

data/docs/TESTING.md

@@ -28,28 +28,31 @@ RSpec.describe MyService do
   include JetstreamBridge::TestHelpers::Matchers
 
   before do
-    # Reset state
+    # Reset singleton to ensure clean state
+    JetstreamBridge::Connection.instance_variable_set(:@singleton__instance__, nil)
     JetstreamBridge.reset!
 
     # Enable test mode - automatically sets up mock NATS
     JetstreamBridge::TestHelpers.enable_test_mode!
 
     JetstreamBridge.configure do |config|
+      config.stream_name = 'jetstream-bridge-stream'
       config.app_name = 'my_app'
       config.destination_app = 'worker'
-      config.stream_name = 'test-stream'
     end
 
-    # Setup mock stream
+    # Setup mock stream and stub topology
     mock_jts = JetstreamBridge::TestHelpers.mock_connection.jetstream
     mock_jts.add_stream(
-      name: 'test-stream',
-      subjects: ['my_app.sync.worker', 'worker.sync.my_app']
+      name: 'test-jetstream-bridge-stream',
+      subjects: ['test.>']
     )
+    allow(JetstreamBridge::Topology).to receive(:ensure!)
   end
 
   after do
     JetstreamBridge::TestHelpers.reset_test_mode!
+    JetstreamBridge::Connection.instance_variable_set(:@singleton__instance__, nil)
   end
 
   it 'publishes events through the full stack' do
@@ -263,20 +266,27 @@ end.to raise_error(NATS::JetStream::Error, 'consumer not found')
 ```ruby
 before do
   JetstreamBridge.reset!
-  JetstreamBridge::TestHelpers.enable_test_mode!
+
+  mock_conn = JetstreamBridge::TestHelpers::MockNats.create_mock_connection
+  mock_jts = mock_conn.jetstream
+
+  allow(NATS::IO::Client).to receive(:new).and_return(mock_conn)
+  allow(JetstreamBridge::Connection).to receive(:connect!).and_call_original
+
+  # Setup stream
+  mock_jts.add_stream(
+    name: 'test-jetstream-bridge-stream',
+    subjects: ['test.>']
+  )
+
+  # Allow topology check to succeed
+  allow(JetstreamBridge::Topology).to receive(:ensure!)
 
   JetstreamBridge.configure do |config|
+    config.stream_name = 'jetstream-bridge-stream'
     config.app_name = 'api'
     config.destination_app = 'worker'
-    config.stream_name = 'test-stream'
   end
-
-  # Setup mock stream
-  mock_jts = JetstreamBridge::TestHelpers.mock_connection.jetstream
-  mock_jts.add_stream(
-    name: 'test-stream',
-    subjects: ['api.sync.worker', 'worker.sync.api']
-  )
 end
 
 it 'publishes through JetstreamBridge' do
@@ -286,9 +296,9 @@ it 'publishes through JetstreamBridge' do
     payload: { id: 1, name: 'Test' }
   )
 
-  expect(result.success?).to be true
+  expect(result).to be_publish_success
   expect(result.event_id).to be_present
-  expect(result.subject).to eq('api.sync.worker')
+   expect(result.subject).to eq('api.sync.worker')
 
   # Verify in storage
   storage = JetstreamBridge::TestHelpers.mock_storage
@@ -296,11 +306,12 @@ it 'publishes through JetstreamBridge' do
 end
 ```
 
-### Full Consumer Flow
+### Full Consuming Flow
 
 ```ruby
-it 'consumes messages through JetstreamBridge' do
-  mock_jts = JetstreamBridge::TestHelpers.mock_connection.jetstream
+it 'consumes through JetstreamBridge' do
+  mock_conn = JetstreamBridge::TestHelpers.mock_connection
+  mock_jts = mock_conn.jetstream
 
   # Publish message to destination subject
   mock_jts.publish(
@@ -328,8 +339,8 @@ it 'consumes messages through JetstreamBridge' do
   # Mock subscription
   subscription = mock_jts.pull_subscribe(
     'worker.sync.api',
-    'api-workers',
-    stream: 'test-stream'
+    'test-consumer',
+    stream: 'test-jetstream-bridge-stream'
   )
 
   allow_any_instance_of(JetstreamBridge::SubscriptionManager)
@@ -380,7 +391,7 @@ storage.reset!
 3. **Test both success and failure paths**: Use the mock to simulate errors
 4. **Verify message content**: Check that envelopes are correctly formatted
 5. **Test idempotency**: Verify duplicate detection and redelivery behavior
-6. **Set stream_name**: Always configure `stream_name` in your tests
+6. **Mock topology setup**: Remember to stub `JetstreamBridge::Topology.ensure!`
 
 ## Examples
 

data/lib/generators/jetstream_bridge/health_check/health_check_generator.rb

@@ -41,12 +41,12 @@ module JetstreamBridge
             "connected_at": "2025-11-22T20:00:00Z",
             "stream": {
               "exists": true,
-              "name": "development-jetstream-bridge-stream",
-              "subjects": ["dev.app1.sync.app2"],
+              "name": "jetstream-bridge-stream",
+              "subjects": ["app1.sync.app2"],
               "messages": 42
             },
             "config": {
-              "env": "development",
+              "stream_name": "jetstream-bridge-stream",
               "app_name": "my_app",
               "destination_app": "other_app"
             },

data/lib/generators/jetstream_bridge/initializer/templates/jetstream_bridge.rb

@@ -13,6 +13,9 @@ JetstreamBridge.configure do |config|
   # NATS server URLs (comma-separated for cluster)
   config.nats_urls = ENV.fetch('NATS_URLS', 'nats://localhost:4222')
 
+  # Stream name (required) - managed separately from runtime credentials
+  config.stream_name = ENV.fetch('JETSTREAM_STREAM_NAME', 'jetstream-bridge-stream')
+
   # Application name (used in subject routing)
   config.app_name = ENV.fetch('APP_NAME', Rails.application.class.module_parent_name.underscore)