lepus 0.0.1.beta2 → 0.1.0

data/docs/supervisor.md

@@ -0,0 +1,112 @@
+# Supervisor
+
+The supervisor is the parent process when you run `lepus start`. It forks workers, monitors them, and handles signals.
+
+## Process model
+
+```
+Supervisor (pid = $$)
+├── Worker[:default]        pid = 1001
+│   ├── Thread OrdersConsumer
+│   └── Thread OrdersConsumer
+└── Worker[:high_priority]  pid = 1002
+    └── Thread PaymentsConsumer
+```
+
+- **One worker per named pool.** Consumers with `process.name = :default` share a worker; consumers with `process.name = :high_priority` share a different one. Unnamed consumers default to `:default`.
+- **Multiple threads per worker.** Each worker runs a thread pool sized by `config.worker(:name).pool_size`.
+- **One channel per consumer.** Each consumer gets its own Bunny channel to avoid cross-consumer interference.
+
+Fork boundaries matter because child processes inherit open file descriptors, sockets, and DB connections. See the `before_fork` / `after_fork` hooks below.
+
+## Fork hooks
+
+```ruby
+Lepus.configure do |config|
+  config.worker(:default) do |w|
+    # Runs in the SUPERVISOR just before fork()
+    w.before_fork do
+      ActiveRecord::Base.connection_handler.clear_all_connections!
+      Redis.current.disconnect!
+    end
+
+    # Runs in the CHILD after fork()
+    w.after_fork do
+      ActiveRecord::Base.establish_connection
+      Redis.current = Redis.new
+      SecureRandom.hex(4)  # implicit reseed
+    end
+  end
+end
+```
+
+The classic pattern: close long-lived connections in `before_fork`, reopen them in `after_fork`. Otherwise all children end up sharing the same socket.
+
+## Signals
+
+| Signal | Supervisor response |
+|--------|---------------------|
+| `SIGTERM` | Graceful shutdown — see below. |
+| `SIGINT` | Graceful shutdown (same as SIGTERM). |
+| `SIGQUIT` | Graceful shutdown, slightly more aggressive. |
+| `SIGTTIN` | Dump thread backtraces of every worker (debugging). |
+| `SIGUSR1` | Reopen log files (useful for logrotate). |
+
+Child workers respond to the same signals, but normally the supervisor handles signals and forwards the shutdown to children.
+
+## Graceful shutdown
+
+1. Supervisor writes a shutdown message to each worker's pipe.
+2. Each worker stops accepting new messages (`basic.cancel` on its channels).
+3. In-flight `perform` calls are allowed to finish.
+4. Workers close connections and exit with code 0.
+5. If a worker takes longer than `shutdown_timeout` (default 5 seconds per message, tuned via consumer `channel.shutdown_timeout`), it's killed with `SIGKILL`.
+6. Supervisor exits.
+
+Rule of thumb for deploy scripts: send SIGTERM, wait at least `pool_size × average_message_duration + 5 seconds`, then the supervisor should have exited cleanly.
+
+## Worker crash recovery
+
+If a worker exits unexpectedly (raised exception outside a `perform` call, OOM, killed by OOM killer):
+
+1. Supervisor detects the pipe close.
+2. Logs the exit status.
+3. Forks a new worker using the same `before_fork` / `after_fork` hooks.
+4. Worker re-declares its queues and resumes consuming.
+
+Within a `perform`, exceptions are caught and routed to `on_thread_error`; the worker is not restarted for those. Only crashes at the worker level (outside message handling) trigger a restart.
+
+## Heartbeats and the process registry
+
+Each worker (and the supervisor itself) emits heartbeats to the process registry:
+
+```ruby
+config.process_heartbeat_interval = 60     # seconds
+config.process_alive_threshold    = 5 * 60 # seconds
+```
+
+The web dashboard reads from this registry to display process status. Processes whose last heartbeat is older than `process_alive_threshold` are considered dead and hidden from the UI (but kept briefly so transient restarts don't flash).
+
+## Pidfile
+
+```bash
+bundle exec lepus start --pidfile /var/run/lepus.pid
+```
+
+Written by the supervisor at boot, removed at graceful shutdown. If the file exists when lepus starts, the supervisor checks if the PID is alive; if so, it exits (don't run two supervisors). If not, it overwrites.
+
+## Running under a process manager
+
+- **systemd:** `ExecStart=/path/to/bundle exec lepus start --pidfile /run/lepus.pid`, `KillSignal=SIGTERM`, `TimeoutStopSec=60`.
+- **Foreman / Procfile:** `worker: bundle exec lepus start`. No pidfile needed.
+- **Kubernetes:** set a `preStop` lifecycle hook to send SIGTERM, and tune `terminationGracePeriodSeconds` to be larger than your longest `perform`.
+
+## Debugging a stuck worker
+
+Send SIGTTIN:
+
+```bash
+kill -TTIN <worker-pid>
+```
+
+The worker prints thread backtraces to the log. Useful when you suspect a deadlock or a `perform` stuck on a slow network call.

data/docs/testing.md

@@ -0,0 +1,141 @@
+# Testing
+
+Lepus ships a test-mode module that captures publishes and runs consumer `perform` methods synchronously — no RabbitMQ connection required.
+
+## Enabling
+
+```ruby
+# spec/spec_helper.rb (RSpec)
+require 'lepus/testing'
+
+RSpec.configure do |config|
+  config.before(:each) { Lepus::Testing.enable! }
+  config.after(:each)  { Lepus::Testing.reset! }
+end
+```
+
+Once enabled:
+
+- Publishes don't hit RabbitMQ. They're captured in an in-memory buffer keyed by producer class.
+- Consumer handling is synchronous when invoked through `Lepus::Testing.consumer_perform`.
+
+## Testing a consumer
+
+```ruby
+describe OrdersConsumer do
+  it 'creates an order' do
+    result = Lepus::Testing.consumer_perform(
+      OrdersConsumer,
+      { order_id: 42, total: 99.99 }
+    )
+
+    expect(result).to eq(:ack)
+    expect(Order.find(42).total).to eq(99.99)
+  end
+
+  it 'rejects invalid payloads' do
+    result = Lepus::Testing.consumer_perform(OrdersConsumer, { bad: 'data' })
+    expect(result).to eq(:reject)
+  end
+
+  it 'sets delivery info and metadata' do
+    result = Lepus::Testing.consumer_perform(
+      OrdersConsumer,
+      { order_id: 7 },
+      delivery_info: { routing_key: 'order.created' },
+      metadata:      { correlation_id: 'abc-123' }
+    )
+    expect(result).to eq(:ack)
+  end
+end
+```
+
+`consumer_perform` signature:
+
+```ruby
+Lepus::Testing.consumer_perform(
+  ConsumerClass,
+  payload,
+  delivery_info: {},
+  metadata: {}
+)
+```
+
+It builds a `Lepus::Message`, runs the full middleware chain (including global middlewares), and returns the disposition symbol.
+
+## Testing a producer
+
+```ruby
+describe OrdersProducer do
+  it 'publishes the order' do
+    order = Order.create!(id: 42, total: 99.99)
+    OrdersProducer.order_created(order)
+
+    messages = Lepus::Testing.producer_messages(OrdersProducer)
+    expect(messages.size).to eq(1)
+    expect(messages[0][:payload]).to include(order_id: 42)
+    expect(messages[0][:routing_key]).to eq('order.created')
+  end
+
+  it 'runs through middleware' do
+    OrdersProducer.publish({ foo: 'bar' }, routing_key: 'x')
+
+    msg = Lepus::Testing.producer_messages(OrdersProducer).last
+    expect(msg[:metadata][:correlation_id]).to be_present  # set by :correlation_id middleware
+    expect(msg[:metadata][:content_type]).to eq('application/json')
+  end
+end
+```
+
+`producer_messages(ProducerClass)` returns an array of hashes with `:payload`, `:routing_key`, `:delivery_info`, `:metadata`.
+
+## RSpec matchers
+
+```ruby
+# spec/spec_helper.rb
+require 'lepus/testing/rspec_matchers'
+```
+
+Then:
+
+```ruby
+expect { OrdersProducer.order_created(order) }
+  .to have_published_message(OrdersProducer)
+  .with_payload(include(order_id: order.id))
+  .to_routing_key('order.created')
+```
+
+## Testing middleware in isolation
+
+Middlewares are plain Ruby objects with a `call(message, app)` method. Unit-test them directly:
+
+```ruby
+describe LogLevelMiddleware do
+  it 'logs before calling down the chain' do
+    middleware = LogLevelMiddleware.new(level: :debug)
+    message   = Lepus::Testing::MessageBuilder.build(payload: { x: 1 })
+    captured  = nil
+
+    allow(Lepus.logger).to receive(:debug) { |msg| captured = msg }
+    middleware.call(message, ->(m) { :ack })
+
+    expect(captured).to include('Processing:')
+  end
+end
+```
+
+`Lepus::Testing::MessageBuilder.build(**kwargs)` builds a realistic `Lepus::Message` for unit tests.
+
+## Resetting between tests
+
+`Lepus::Testing.reset!` clears captured publishes. In shared setup:
+
+```ruby
+RSpec.configure do |config|
+  config.before(:each) { Lepus::Testing.enable!; Lepus::Testing.reset! }
+end
+```
+
+## When you do need a real RabbitMQ
+
+Integration tests that exercise the full round-trip (publish → RabbitMQ → consume) benefit from a real broker. Use `docker run rabbitmq:3-management` or your test infra's existing one, and skip `Lepus::Testing.enable!` for those specs.

data/docs/web.md

@@ -0,0 +1,85 @@
+# Web Dashboard
+
+![Lepus web dashboard](images/lepus-web.png)
+
+Lepus ships a Rack-based monitoring UI showing consumer status, throughput, and recent activity.
+
+## Running it standalone
+
+```bash
+bundle exec lepus web --port 9292 --host 0.0.0.0
+```
+
+Visit http://localhost:9292.
+
+## Mounting in Rails
+
+```ruby
+# config/routes.rb
+require 'lepus/web'
+
+authenticate :user, ->(u) { u.admin? } do
+  mount Lepus::Web::App, at: '/lepus'
+end
+```
+
+Or with Devise:
+
+```ruby
+authenticate :admin_user do
+  mount Lepus::Web::App, at: '/admin/lepus'
+end
+```
+
+**Important:** the dashboard has no built-in auth. Wrap it with whatever your app already uses.
+
+## What it shows
+
+- **Supervisors.** Every running `lepus start` process.
+- **Workers.** Subprocesses per supervisor, with their named pool and PID.
+- **Consumers.** Per-consumer message counts (processed, rejected, errored), queue names, routing keys.
+- **Exchanges & queues.** RabbitMQ topology as seen by the gem.
+- **Recent activity.** Last N messages — timestamps, routing keys, dispositions.
+
+## Registry backend
+
+The dashboard reads from the process registry, configured at `Lepus.configure`:
+
+```ruby
+config.process_registry_backend = :file      # single-host
+# or
+config.process_registry_backend = :rabbitmq  # multi-host
+```
+
+- `:file` — metadata stored under `/tmp/lepus/...`. Works out of the box but only for a single host.
+- `:rabbitmq` — metadata stored in RabbitMQ itself. Multiple `lepus start` processes across multiple hosts show up in one dashboard.
+
+For `:rabbitmq`, also set:
+
+```ruby
+config.management_api_url = 'http://rabbitmq:15672'
+```
+
+## Heartbeats
+
+Each worker heartbeats into the registry every `config.process_heartbeat_interval` (default 60 seconds). Processes are considered "alive" if their last heartbeat is within `config.process_alive_threshold` (default 5 minutes).
+
+## API endpoints
+
+The dashboard exposes a minimal read-only JSON API at `/api/...` — the UI is a single-page app that consumes it. You can also consume the API directly for custom dashboards or alerting.
+
+Endpoints include (subject to change):
+
+- `GET /api/processes` — all tracked processes
+- `GET /api/consumers` — all registered consumers with counts
+- `GET /api/queues` — queue metadata from the RabbitMQ management API (if `management_api_url` is set)
+
+## Prometheus metrics
+
+See [prometheus.md](prometheus.md) for the metric list, label cardinality notes, and how to wire the collector server inside a Lepus supervisor process.
+
+## Operating in production
+
+- Put the dashboard behind your existing auth layer (OAuth proxy, Rails authentication, Basic auth).
+- Use `:rabbitmq` registry backend for multi-node visibility.
+- Retain logs separately — the dashboard is for live state, not audit trails.