typed_bus 0.0.1 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +158 -4
- data/LICENSE +21 -0
- data/README.md +25 -13
- data/lib/typed_bus/channel.rb +170 -91
- data/lib/typed_bus/configuration.rb +51 -8
- data/lib/typed_bus/dead_letter_queue.rb +32 -8
- data/lib/typed_bus/delivery.rb +1 -1
- data/lib/typed_bus/delivery_tracker.rb +28 -19
- data/lib/typed_bus/message_bus.rb +55 -14
- data/lib/typed_bus/version.rb +1 -1
- data/lib/typed_bus.rb +3 -0
- metadata +23 -62
- data/.envrc +0 -1
- data/.github/workflows/deploy-github-pages.yml +0 -52
- data/.github/workflows/deploy-yard-docs.yml +0 -52
- data/.irbrc +0 -10
- data/COMMITS.md +0 -196
- data/Rakefile +0 -12
- data/docs/api/channel.md +0 -94
- data/docs/api/configuration.md +0 -90
- data/docs/api/dead-letter-queue.md +0 -52
- data/docs/api/delivery-tracker.md +0 -66
- data/docs/api/delivery.md +0 -73
- data/docs/api/index.md +0 -45
- data/docs/api/message-bus.md +0 -101
- data/docs/api/stats.md +0 -50
- data/docs/architecture/concurrency-model.md +0 -38
- data/docs/architecture/configuration-cascade.md +0 -76
- data/docs/architecture/index.md +0 -7
- data/docs/architecture/publish-flow.md +0 -67
- data/docs/assets/css/custom.css +0 -11
- data/docs/assets/images/typed_bus.gif +0 -0
- data/docs/assets/images/typed_bus.jpg +0 -0
- data/docs/assets/images/typed_bus.mp4 +0 -0
- data/docs/concepts.md +0 -47
- data/docs/examples/index.md +0 -37
- data/docs/getting-started/configuration.md +0 -87
- data/docs/getting-started/index.md +0 -7
- data/docs/getting-started/installation.md +0 -33
- data/docs/getting-started/quick-start.md +0 -51
- data/docs/guides/adaptive-throttling.md +0 -76
- data/docs/guides/backpressure.md +0 -58
- data/docs/guides/dead-letter-queues.md +0 -78
- data/docs/guides/delivery-and-ack.md +0 -89
- data/docs/guides/error-handling.md +0 -91
- data/docs/guides/graceful-shutdown.md +0 -75
- data/docs/guides/index.md +0 -13
- data/docs/guides/logging.md +0 -93
- data/docs/guides/stats-and-monitoring.md +0 -75
- data/docs/guides/typed-channels.md +0 -63
- data/docs/index.md +0 -56
- data/examples/00_config.rb +0 -156
- data/examples/01_basic_usage.rb +0 -26
- data/examples/02_typed_channels.rb +0 -41
- data/examples/03_multiple_subscribers.rb +0 -44
- data/examples/04_nack_and_dead_letters.rb +0 -47
- data/examples/05_delivery_timeout.rb +0 -44
- data/examples/06_backpressure.rb +0 -32
- data/examples/07_message_bus.rb +0 -55
- data/examples/08_error_handling.rb +0 -82
- data/examples/09_stats_and_monitoring.rb +0 -71
- data/examples/10_concurrent_pipeline.rb +0 -116
- data/examples/11_logging_basic.rb +0 -60
- data/examples/12_logging_failures.rb +0 -65
- data/examples/13_logging_backpressure.rb +0 -56
- data/examples/14_adaptive_throttling.rb +0 -101
- data/examples/README.md +0 -27
- data/examples/temp.md +0 -95
- data/mkdocs.yml +0 -183
|
@@ -1,67 +0,0 @@
|
|
|
1
|
-
# Publish Flow
|
|
2
|
-
|
|
3
|
-
When you call `channel.publish(message)`, the following steps execute in order:
|
|
4
|
-
|
|
5
|
-
```mermaid
|
|
6
|
-
flowchart TD
|
|
7
|
-
A[publish called] --> B{Channel closed?}
|
|
8
|
-
B -- yes --> ERR[raise RuntimeError]
|
|
9
|
-
B -- no --> C[Type check]
|
|
10
|
-
C --> D{Throttle enabled?}
|
|
11
|
-
D -- yes --> E[Apply throttle delay]
|
|
12
|
-
D -- no --> F{Bounded channel?}
|
|
13
|
-
E --> F
|
|
14
|
-
F -- yes --> G[Wait for capacity]
|
|
15
|
-
F -- no --> H{Subscribers?}
|
|
16
|
-
G --> H
|
|
17
|
-
H -- none --> I[Route to DLQ]
|
|
18
|
-
H -- 1+ --> J[Create DeliveryTracker]
|
|
19
|
-
J --> K[Fan out Delivery per subscriber]
|
|
20
|
-
K --> L[Each subscriber runs in Async task]
|
|
21
|
-
L --> M{ack! or nack!}
|
|
22
|
-
M -- ack --> N[Tracker records ack]
|
|
23
|
-
M -- nack --> O[Tracker records nack → DLQ]
|
|
24
|
-
M -- timeout --> P[Auto-nack → DLQ]
|
|
25
|
-
N --> Q{All resolved?}
|
|
26
|
-
O --> Q
|
|
27
|
-
P --> Q
|
|
28
|
-
Q -- yes, all acked --> R[on_complete → delivered stat]
|
|
29
|
-
Q -- yes, some nacked --> S[on_resolved → cleanup]
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
## Step-by-Step
|
|
33
|
-
|
|
34
|
-
### 1. Guard: Closed Channel
|
|
35
|
-
|
|
36
|
-
If the channel is closed, `publish` raises `RuntimeError`. No message is delivered.
|
|
37
|
-
|
|
38
|
-
### 2. Type Check
|
|
39
|
-
|
|
40
|
-
If the channel has a `type:` constraint, the message's class is verified with `is_a?`. A mismatch raises `ArgumentError`.
|
|
41
|
-
|
|
42
|
-
### 3. Throttle
|
|
43
|
-
|
|
44
|
-
If `throttle` is configured and the channel is bounded, the remaining capacity ratio is checked. Below the threshold, the publisher sleeps for `1 / (max_pending * remaining_ratio)` seconds — an asymptotic delay that approaches infinity as remaining capacity approaches zero.
|
|
45
|
-
|
|
46
|
-
### 4. Backpressure
|
|
47
|
-
|
|
48
|
-
If the channel is bounded (`max_pending` is set) and the pending delivery count has reached the limit, the publishing fiber blocks on an `Async::Condition` until a subscriber ACKs and frees a slot.
|
|
49
|
-
|
|
50
|
-
### 5. Fan-Out
|
|
51
|
-
|
|
52
|
-
A `DeliveryTracker` is created for this message, tracking the set of current subscriber IDs. Each subscriber receives its own `Delivery` envelope in a separate `Async` task.
|
|
53
|
-
|
|
54
|
-
### 6. Resolution
|
|
55
|
-
|
|
56
|
-
Each subscriber must call `ack!` or `nack!` on its Delivery. If neither is called before the timeout, the delivery auto-nacks. The tracker aggregates responses:
|
|
57
|
-
|
|
58
|
-
- **All acked** — the `on_complete` callback fires, incrementing the `:delivered` stat.
|
|
59
|
-
- **Any nacked** — the `on_resolved` callback fires for cleanup. NACKed deliveries are pushed to the dead letter queue.
|
|
60
|
-
|
|
61
|
-
### 7. Backpressure Release
|
|
62
|
-
|
|
63
|
-
When a tracker resolves, the pending delivery is removed and `@backpressure.signal` wakes any blocked publisher fiber.
|
|
64
|
-
|
|
65
|
-
## No-Subscriber Path
|
|
66
|
-
|
|
67
|
-
If there are no subscribers when `publish` is called, the message is wrapped in a Delivery, immediately NACKed, and pushed to the dead letter queue. The `:nacked` and `:dead_lettered` stats are incremented.
|
data/docs/assets/css/custom.css
DELETED
|
Binary file
|
|
Binary file
|
|
Binary file
|
data/docs/concepts.md
DELETED
|
@@ -1,47 +0,0 @@
|
|
|
1
|
-
# Concepts
|
|
2
|
-
|
|
3
|
-
TypedBus is built around a small set of core concepts. Understanding these makes the entire library predictable.
|
|
4
|
-
|
|
5
|
-
## Channels
|
|
6
|
-
|
|
7
|
-
A **channel** is a named pub/sub topic. Publishers send messages to a channel; subscribers receive them. Each channel can optionally enforce a type constraint on its messages.
|
|
8
|
-
|
|
9
|
-
Channels are the unit of configuration — each has its own timeout, backpressure limit, and throttle setting.
|
|
10
|
-
|
|
11
|
-
## Deliveries
|
|
12
|
-
|
|
13
|
-
When a message is published to a channel with subscribers, each subscriber receives a **Delivery** envelope wrapping the original message. The subscriber must explicitly resolve the delivery by calling `ack!` (success) or `nack!` (failure).
|
|
14
|
-
|
|
15
|
-
If neither is called before the channel's timeout expires, the delivery auto-nacks.
|
|
16
|
-
|
|
17
|
-
## Delivery Tracker
|
|
18
|
-
|
|
19
|
-
A **DeliveryTracker** aggregates the responses from all subscribers for a single published message. A message is considered "delivered" only when every subscriber has ACKed. If any subscriber NACKs, the failed deliveries route to the dead letter queue.
|
|
20
|
-
|
|
21
|
-
## Dead Letter Queue
|
|
22
|
-
|
|
23
|
-
Every channel has a **Dead Letter Queue** (DLQ) that collects failed deliveries. Entries arrive from:
|
|
24
|
-
|
|
25
|
-
- Explicit `nack!` calls
|
|
26
|
-
- Delivery timeouts (auto-nack)
|
|
27
|
-
- Messages published with no subscribers
|
|
28
|
-
|
|
29
|
-
The DLQ supports iteration, draining for retry, and callbacks for real-time alerting.
|
|
30
|
-
|
|
31
|
-
## MessageBus
|
|
32
|
-
|
|
33
|
-
The **MessageBus** is a registry facade that manages multiple named channels with shared statistics. It provides a single entry point for publish, subscribe, and lifecycle operations.
|
|
34
|
-
|
|
35
|
-
## Configuration Cascade
|
|
36
|
-
|
|
37
|
-
TypedBus resolves configurable parameters through three tiers:
|
|
38
|
-
|
|
39
|
-
```
|
|
40
|
-
Global → Bus → Channel
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
Each tier inherits from the one above unless explicitly overridden. See [Configuration Cascade](architecture/configuration-cascade.md) for details.
|
|
44
|
-
|
|
45
|
-
## Concurrency Model
|
|
46
|
-
|
|
47
|
-
TypedBus uses **fiber-only concurrency** powered by the `async` gem. There are no mutexes and no threads. All state is managed within a single Async reactor. Operations like backpressure waits and throttle sleeps yield the calling fiber, not the thread.
|
data/docs/examples/index.md
DELETED
|
@@ -1,37 +0,0 @@
|
|
|
1
|
-
# Examples
|
|
2
|
-
|
|
3
|
-
TypedBus ships with runnable examples in the `examples/` directory. Run any example from the project root:
|
|
4
|
-
|
|
5
|
-
```bash
|
|
6
|
-
bundle exec ruby examples/01_basic_usage.rb
|
|
7
|
-
```
|
|
8
|
-
|
|
9
|
-
## Demo Index
|
|
10
|
-
|
|
11
|
-
| # | File | Description |
|
|
12
|
-
|---|------|-------------|
|
|
13
|
-
| 00 | `00_config.rb` | Three-tier configuration cascade. Global defaults, bus-level overrides, per-channel overrides, inheritance, and `reset_configuration!`. |
|
|
14
|
-
| 01 | `01_basic_usage.rb` | Basic pub/sub with explicit acknowledgment. One channel, one subscriber, `ack!` after each message. |
|
|
15
|
-
| 02 | `02_typed_channels.rb` | Type-constrained channels. Publishing the wrong type raises `ArgumentError` at the publish site. |
|
|
16
|
-
| 03 | `03_multiple_subscribers.rb` | Fan-out delivery. Every subscriber gets its own `Delivery`; a message is only "delivered" when all subscribers ACK. Shows `DeliveryTracker` state queries. |
|
|
17
|
-
| 04 | `04_nack_and_dead_letters.rb` | Explicit rejection with `nack!`. Rejected deliveries route to the dead letter queue. Shows DLQ iteration and `drain` for retry. |
|
|
18
|
-
| 05 | `05_delivery_timeout.rb` | Delivery deadlines. Subscribers that don't respond within the timeout are auto-NACKed. Shows `timed_out?` flag. |
|
|
19
|
-
| 06 | `06_backpressure.rb` | Bounded channels with `max_pending`. The publisher fiber blocks when the limit is reached and resumes as subscribers ACK. |
|
|
20
|
-
| 07 | `07_message_bus.rb` | `MessageBus` registry facade. Manages multiple typed channels, aggregates stats, single entry point for publish/subscribe. |
|
|
21
|
-
| 08 | `08_error_handling.rb` | Edge cases: subscriber exceptions (auto-nack), no subscribers (DLQ), unsubscribe mid-stream, close force-nacks, publishing to closed channel. |
|
|
22
|
-
| 09 | `09_stats_and_monitoring.rb` | Per-channel stat counters. Combines DLQ callbacks with stats for real-time alerting. Shows `reset!`. |
|
|
23
|
-
| 10 | `10_concurrent_pipeline.rb` | Multi-stage event pipeline: ingest, validate, process. Typed messages, backpressure, timeouts, DLQ monitoring, graceful shutdown. |
|
|
24
|
-
| 11 | `11_logging_basic.rb` | Structured logging via `TypedBus.logger`. Lifecycle log output for channel creation, subscriber registration, publish, delivery, ack, tracker resolution. |
|
|
25
|
-
| 12 | `12_logging_failures.rb` | Failure-path logging: explicit nacks, delivery timeouts, no-subscriber dead-lettering, DLQ drain. Colorized output. |
|
|
26
|
-
| 13 | `13_logging_backpressure.rb` | Backpressure logging. DEBUG-level messages as the publisher fiber blocks and resumes while a slow subscriber drains a bounded channel. |
|
|
27
|
-
| 14 | `14_adaptive_throttling.rb` | Adaptive rate limiting with `throttle: Float`. Asymptotic backoff as a bounded channel fills, bus-level throttle defaults with per-channel overrides. |
|
|
28
|
-
|
|
29
|
-
## Running All Examples
|
|
30
|
-
|
|
31
|
-
```bash
|
|
32
|
-
for f in examples/*.rb; do
|
|
33
|
-
echo "=== $f ==="
|
|
34
|
-
bundle exec ruby "$f"
|
|
35
|
-
echo
|
|
36
|
-
done
|
|
37
|
-
```
|
|
@@ -1,87 +0,0 @@
|
|
|
1
|
-
# Configuration
|
|
2
|
-
|
|
3
|
-
TypedBus resolves parameters through a three-tier cascade: **Global → Bus → Channel**. Each tier inherits from the one above unless explicitly overridden.
|
|
4
|
-
|
|
5
|
-
## Parameters
|
|
6
|
-
|
|
7
|
-
| Parameter | Default | Description |
|
|
8
|
-
|-----------|---------|-------------|
|
|
9
|
-
| `timeout` | `30` | Delivery ACK deadline in seconds |
|
|
10
|
-
| `max_pending` | `nil` | Backpressure limit; `nil` = unbounded |
|
|
11
|
-
| `throttle` | `0.0` | Remaining-capacity ratio where backoff begins; `0.0` = disabled |
|
|
12
|
-
| `logger` | `nil` | Ruby Logger instance (global-only) |
|
|
13
|
-
| `log_level` | `Logger::INFO` | Log level applied to the logger (global-only) |
|
|
14
|
-
|
|
15
|
-
## Global Defaults
|
|
16
|
-
|
|
17
|
-
Set defaults that apply to every bus and channel:
|
|
18
|
-
|
|
19
|
-
```ruby
|
|
20
|
-
TypedBus.configure do |config|
|
|
21
|
-
config.timeout = 60
|
|
22
|
-
config.max_pending = 500
|
|
23
|
-
config.throttle = 0.5
|
|
24
|
-
config.logger = Logger.new($stdout)
|
|
25
|
-
config.log_level = Logger::DEBUG
|
|
26
|
-
end
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
## Bus-Level Overrides
|
|
30
|
-
|
|
31
|
-
Pass parameters to `MessageBus.new` to override globals for this bus:
|
|
32
|
-
|
|
33
|
-
```ruby
|
|
34
|
-
# Inherits global max_pending and throttle, overrides timeout
|
|
35
|
-
bus = TypedBus::MessageBus.new(timeout: 10)
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
Each bus gets an independent config snapshot. Mutating one does not affect the global or other buses.
|
|
39
|
-
|
|
40
|
-
## Channel-Level Overrides
|
|
41
|
-
|
|
42
|
-
Pass parameters to `add_channel` to override the bus defaults for this channel:
|
|
43
|
-
|
|
44
|
-
```ruby
|
|
45
|
-
# Inherits bus timeout, overrides max_pending
|
|
46
|
-
bus.add_channel(:alerts, max_pending: 50)
|
|
47
|
-
```
|
|
48
|
-
|
|
49
|
-
## Cascade Example
|
|
50
|
-
|
|
51
|
-
```ruby
|
|
52
|
-
TypedBus.configure do |config|
|
|
53
|
-
config.timeout = 60 # global default
|
|
54
|
-
config.max_pending = 500
|
|
55
|
-
end
|
|
56
|
-
|
|
57
|
-
bus = TypedBus::MessageBus.new(timeout: 30) # bus overrides timeout
|
|
58
|
-
|
|
59
|
-
bus.add_channel(:orders) # timeout=30, max_pending=500
|
|
60
|
-
bus.add_channel(:alerts, timeout: 5) # timeout=5, max_pending=500
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
## Logger Shortcuts
|
|
64
|
-
|
|
65
|
-
```ruby
|
|
66
|
-
# These are equivalent:
|
|
67
|
-
TypedBus.configure { |c| c.logger = Logger.new($stdout) }
|
|
68
|
-
TypedBus.logger = Logger.new($stdout)
|
|
69
|
-
|
|
70
|
-
# Reading:
|
|
71
|
-
TypedBus.logger # => the Logger instance
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
Setting `log_level` automatically applies it to the current logger. Setting a logger automatically applies the current `log_level` to it.
|
|
75
|
-
|
|
76
|
-
## Reset
|
|
77
|
-
|
|
78
|
-
Restore factory defaults (useful in tests):
|
|
79
|
-
|
|
80
|
-
```ruby
|
|
81
|
-
TypedBus.reset_configuration!
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
## Further Reading
|
|
85
|
-
|
|
86
|
-
- [Configuration Cascade](../architecture/configuration-cascade.md) — architecture details
|
|
87
|
-
- [API Reference: Configuration](../api/configuration.md) — full method reference
|
|
@@ -1,7 +0,0 @@
|
|
|
1
|
-
# Getting Started
|
|
2
|
-
|
|
3
|
-
This section covers installation, a quick start tutorial, and configuration basics to get you productive with TypedBus.
|
|
4
|
-
|
|
5
|
-
- [Installation](installation.md) — add TypedBus to your project
|
|
6
|
-
- [Quick Start](quick-start.md) — publish and subscribe in under 20 lines
|
|
7
|
-
- [Configuration](configuration.md) — set global, bus, and channel defaults
|
|
@@ -1,33 +0,0 @@
|
|
|
1
|
-
# Installation
|
|
2
|
-
|
|
3
|
-
## Requirements
|
|
4
|
-
|
|
5
|
-
- Ruby >= 3.2.0
|
|
6
|
-
- [async](https://github.com/socketry/async) ~> 2.0
|
|
7
|
-
|
|
8
|
-
## Bundler
|
|
9
|
-
|
|
10
|
-
Add to your Gemfile:
|
|
11
|
-
|
|
12
|
-
```ruby
|
|
13
|
-
gem "typed_bus"
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
Then run:
|
|
17
|
-
|
|
18
|
-
```bash
|
|
19
|
-
bundle install
|
|
20
|
-
```
|
|
21
|
-
|
|
22
|
-
## Manual
|
|
23
|
-
|
|
24
|
-
```bash
|
|
25
|
-
gem install typed_bus
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
## Verify
|
|
29
|
-
|
|
30
|
-
```ruby
|
|
31
|
-
require "typed_bus"
|
|
32
|
-
puts TypedBus::VERSION
|
|
33
|
-
```
|
|
@@ -1,51 +0,0 @@
|
|
|
1
|
-
# Quick Start
|
|
2
|
-
|
|
3
|
-
## Minimal Example
|
|
4
|
-
|
|
5
|
-
```ruby
|
|
6
|
-
require "typed_bus"
|
|
7
|
-
|
|
8
|
-
bus = TypedBus::MessageBus.new
|
|
9
|
-
bus.add_channel(:events, timeout: 5)
|
|
10
|
-
|
|
11
|
-
bus.subscribe(:events) do |delivery|
|
|
12
|
-
puts "Received: #{delivery.message}"
|
|
13
|
-
delivery.ack!
|
|
14
|
-
end
|
|
15
|
-
|
|
16
|
-
Async do
|
|
17
|
-
bus.publish(:events, "hello world")
|
|
18
|
-
end
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
## What Just Happened
|
|
22
|
-
|
|
23
|
-
1. **Created a bus** — `MessageBus.new` creates a registry for channels with shared stats.
|
|
24
|
-
2. **Added a channel** — `:events` is a named topic with a 5-second delivery timeout.
|
|
25
|
-
3. **Subscribed** — the block receives a `Delivery` envelope for each published message. The subscriber must call `ack!` or `nack!`.
|
|
26
|
-
4. **Published inside Async** — channel operations create Async tasks internally, so `publish` must run within an Async reactor.
|
|
27
|
-
|
|
28
|
-
## Using Channels Directly
|
|
29
|
-
|
|
30
|
-
You don't need a MessageBus. Channels work standalone:
|
|
31
|
-
|
|
32
|
-
```ruby
|
|
33
|
-
require "typed_bus"
|
|
34
|
-
|
|
35
|
-
channel = TypedBus::Channel.new(:greetings, timeout: 5)
|
|
36
|
-
|
|
37
|
-
channel.subscribe do |delivery|
|
|
38
|
-
puts delivery.message
|
|
39
|
-
delivery.ack!
|
|
40
|
-
end
|
|
41
|
-
|
|
42
|
-
Async do
|
|
43
|
-
channel.publish("Hello, world!")
|
|
44
|
-
end
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
## Next Steps
|
|
48
|
-
|
|
49
|
-
- [Configuration](configuration.md) — set defaults at global, bus, and channel levels
|
|
50
|
-
- [Typed Channels](../guides/typed-channels.md) — constrain messages to a specific class
|
|
51
|
-
- [Delivery and Acknowledgment](../guides/delivery-and-ack.md) — understand ack/nack lifecycle
|
|
@@ -1,76 +0,0 @@
|
|
|
1
|
-
# Adaptive Throttling
|
|
2
|
-
|
|
3
|
-
When a bounded channel approaches capacity, adaptive throttling progressively slows publishers before they hit the hard backpressure block.
|
|
4
|
-
|
|
5
|
-
## Enabling Throttling
|
|
6
|
-
|
|
7
|
-
Throttling requires `max_pending` to be set:
|
|
8
|
-
|
|
9
|
-
```ruby
|
|
10
|
-
channel = TypedBus::Channel.new(:pipeline,
|
|
11
|
-
max_pending: 10,
|
|
12
|
-
throttle: 0.5, # begin backoff at 50% remaining capacity
|
|
13
|
-
timeout: 5
|
|
14
|
-
)
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
The `throttle` value is a `Float` between `0.0` (exclusive) and `1.0` (exclusive), representing the remaining-capacity ratio at which backoff begins.
|
|
18
|
-
|
|
19
|
-
## Backoff Curve
|
|
20
|
-
|
|
21
|
-
Below the threshold, each publish sleeps for `1 / (max_pending * remaining_ratio)` seconds — an asymptotic delay that approaches infinity as remaining capacity approaches zero.
|
|
22
|
-
|
|
23
|
-
With `max_pending: 10, throttle: 0.5`:
|
|
24
|
-
|
|
25
|
-
```
|
|
26
|
-
Publish Pending Remaining Ratio Delay
|
|
27
|
-
───────────────────────────────────────────
|
|
28
|
-
1 0 10 1.0 —
|
|
29
|
-
2 1 9 0.9 —
|
|
30
|
-
3 2 8 0.8 —
|
|
31
|
-
4 3 7 0.7 —
|
|
32
|
-
5 4 6 0.6 —
|
|
33
|
-
6 5 5 0.5 0.200s
|
|
34
|
-
7 6 4 0.4 0.250s
|
|
35
|
-
8 7 3 0.3 0.333s
|
|
36
|
-
9 8 2 0.2 0.500s
|
|
37
|
-
10 9 1 0.1 1.000s
|
|
38
|
-
11 10 0 0.0 hard block
|
|
39
|
-
```
|
|
40
|
-
|
|
41
|
-
The first 50% of capacity fills at full speed. The last 50% applies increasing delays.
|
|
42
|
-
|
|
43
|
-
## Threshold Selection
|
|
44
|
-
|
|
45
|
-
Higher values begin backoff earlier (more conservative):
|
|
46
|
-
|
|
47
|
-
```ruby
|
|
48
|
-
TypedBus::Channel.new(:conservative, max_pending: 100, throttle: 0.8) # backoff at 80%
|
|
49
|
-
TypedBus::Channel.new(:aggressive, max_pending: 100, throttle: 0.3) # backoff at 30%
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
## Bus-Level Default
|
|
53
|
-
|
|
54
|
-
Set a default throttle for all channels on a bus:
|
|
55
|
-
|
|
56
|
-
```ruby
|
|
57
|
-
bus = TypedBus::MessageBus.new(throttle: 0.5)
|
|
58
|
-
|
|
59
|
-
bus.add_channel(:orders, max_pending: 100) # inherits 0.5
|
|
60
|
-
bus.add_channel(:alerts, max_pending: 20, throttle: 0.3) # overrides to 0.3
|
|
61
|
-
bus.add_channel(:logs, max_pending: 50, throttle: 0.0) # disables throttle
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
## Validation
|
|
65
|
-
|
|
66
|
-
- `throttle` must be between `0.0` and `1.0` (exclusive)
|
|
67
|
-
- `throttle` requires `max_pending` to be set (raises `ArgumentError` otherwise)
|
|
68
|
-
- `throttle: 0.0` disables throttling (the default)
|
|
69
|
-
|
|
70
|
-
## Stats
|
|
71
|
-
|
|
72
|
-
Each throttled publish increments the `:throttled` stat:
|
|
73
|
-
|
|
74
|
-
```ruby
|
|
75
|
-
bus.stats[:orders_throttled]
|
|
76
|
-
```
|
data/docs/guides/backpressure.md
DELETED
|
@@ -1,58 +0,0 @@
|
|
|
1
|
-
# Backpressure
|
|
2
|
-
|
|
3
|
-
Set `max_pending` to bound how many unresolved deliveries a channel allows. When the limit is reached, `publish` blocks the calling fiber until subscribers ACK and free capacity.
|
|
4
|
-
|
|
5
|
-
## Enabling Backpressure
|
|
6
|
-
|
|
7
|
-
```ruby
|
|
8
|
-
channel = TypedBus::Channel.new(:work,
|
|
9
|
-
max_pending: 100,
|
|
10
|
-
timeout: 10
|
|
11
|
-
)
|
|
12
|
-
```
|
|
13
|
-
|
|
14
|
-
Or via MessageBus:
|
|
15
|
-
|
|
16
|
-
```ruby
|
|
17
|
-
bus.add_channel(:work, max_pending: 100)
|
|
18
|
-
```
|
|
19
|
-
|
|
20
|
-
Or via the configuration cascade:
|
|
21
|
-
|
|
22
|
-
```ruby
|
|
23
|
-
TypedBus.configure { |c| c.max_pending = 500 }
|
|
24
|
-
bus = TypedBus::MessageBus.new
|
|
25
|
-
bus.add_channel(:work) # inherits max_pending=500
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
## How It Works
|
|
29
|
-
|
|
30
|
-
```mermaid
|
|
31
|
-
sequenceDiagram
|
|
32
|
-
participant P as Publisher Fiber
|
|
33
|
-
participant C as Channel
|
|
34
|
-
participant S as Subscriber Fiber
|
|
35
|
-
|
|
36
|
-
P->>C: publish(msg)
|
|
37
|
-
C-->>C: pending_count >= max_pending?
|
|
38
|
-
Note over C: YES — block publisher
|
|
39
|
-
S->>C: delivery.ack!
|
|
40
|
-
C-->>C: pending_count drops
|
|
41
|
-
C->>P: resume (signal)
|
|
42
|
-
P->>C: publish continues
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
1. Before fan-out, the channel checks `pending_deliveries.size >= max_pending`
|
|
46
|
-
2. If at capacity, the publishing fiber waits on an `Async::Condition`
|
|
47
|
-
3. When any delivery tracker resolves (all subscribers responded), the condition signals
|
|
48
|
-
4. The publisher wakes and proceeds
|
|
49
|
-
|
|
50
|
-
## Unbounded Channels
|
|
51
|
-
|
|
52
|
-
Set `max_pending: nil` (the default) for unbounded channels. Publishers never block.
|
|
53
|
-
|
|
54
|
-
## Interaction with Throttling
|
|
55
|
-
|
|
56
|
-
When both `max_pending` and `throttle` are set, throttling runs first. The throttle progressively slows publishers as capacity fills; backpressure is the hard stop at zero remaining capacity.
|
|
57
|
-
|
|
58
|
-
See [Adaptive Throttling](adaptive-throttling.md) for details.
|
|
@@ -1,78 +0,0 @@
|
|
|
1
|
-
# Dead Letter Queues
|
|
2
|
-
|
|
3
|
-
Every channel has a dead letter queue (DLQ) that collects failed deliveries.
|
|
4
|
-
|
|
5
|
-
## What Goes to the DLQ
|
|
6
|
-
|
|
7
|
-
- Deliveries explicitly NACKed by subscribers
|
|
8
|
-
- Deliveries that time out (auto-nack)
|
|
9
|
-
- Messages published with no subscribers
|
|
10
|
-
|
|
11
|
-
## Accessing the DLQ
|
|
12
|
-
|
|
13
|
-
Via MessageBus:
|
|
14
|
-
|
|
15
|
-
```ruby
|
|
16
|
-
dlq = bus.dead_letters(:orders)
|
|
17
|
-
```
|
|
18
|
-
|
|
19
|
-
Via Channel directly:
|
|
20
|
-
|
|
21
|
-
```ruby
|
|
22
|
-
dlq = channel.dead_letter_queue
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
## Inspecting Entries
|
|
26
|
-
|
|
27
|
-
```ruby
|
|
28
|
-
dlq.size # number of entries
|
|
29
|
-
dlq.empty? # true if no entries
|
|
30
|
-
|
|
31
|
-
dlq.each do |delivery|
|
|
32
|
-
puts "#{delivery.message} failed on subscriber ##{delivery.subscriber_id}"
|
|
33
|
-
puts " timed out? #{delivery.timed_out?}"
|
|
34
|
-
end
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
The DLQ includes `Enumerable`, so you can use `map`, `select`, `count`, etc.
|
|
38
|
-
|
|
39
|
-
## Draining for Retry
|
|
40
|
-
|
|
41
|
-
`drain` yields each entry and removes it from the queue:
|
|
42
|
-
|
|
43
|
-
```ruby
|
|
44
|
-
channel.dead_letter_queue.drain do |delivery|
|
|
45
|
-
retry_message(delivery.message)
|
|
46
|
-
end
|
|
47
|
-
```
|
|
48
|
-
|
|
49
|
-
Without a block, `drain` returns the array of drained entries:
|
|
50
|
-
|
|
51
|
-
```ruby
|
|
52
|
-
entries = channel.dead_letter_queue.drain
|
|
53
|
-
entries.each { |d| reprocess(d.message) }
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
## Real-Time Callback
|
|
57
|
-
|
|
58
|
-
Register a callback that fires whenever a new entry arrives:
|
|
59
|
-
|
|
60
|
-
```ruby
|
|
61
|
-
channel.dead_letter_queue.on_dead_letter do |delivery|
|
|
62
|
-
alert("Dead letter on :#{delivery.channel_name}: #{delivery.message}")
|
|
63
|
-
end
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
## Clearing
|
|
67
|
-
|
|
68
|
-
Discard all DLQ entries:
|
|
69
|
-
|
|
70
|
-
```ruby
|
|
71
|
-
channel.dead_letter_queue.clear!
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
Or clear the entire channel (including DLQ, pending deliveries, and timeout tasks):
|
|
75
|
-
|
|
76
|
-
```ruby
|
|
77
|
-
channel.clear!
|
|
78
|
-
```
|
|
@@ -1,89 +0,0 @@
|
|
|
1
|
-
# Delivery and Acknowledgment
|
|
2
|
-
|
|
3
|
-
Every subscriber receives a `Delivery` envelope wrapping the published message. The subscriber must explicitly resolve the delivery.
|
|
4
|
-
|
|
5
|
-
## The Delivery Envelope
|
|
6
|
-
|
|
7
|
-
```ruby
|
|
8
|
-
channel.subscribe do |delivery|
|
|
9
|
-
delivery.message # the published object
|
|
10
|
-
delivery.channel_name # :orders
|
|
11
|
-
delivery.subscriber_id # integer ID assigned at subscribe time
|
|
12
|
-
|
|
13
|
-
delivery.ack! # success
|
|
14
|
-
# or
|
|
15
|
-
delivery.nack! # failure → routes to dead letter queue
|
|
16
|
-
end
|
|
17
|
-
```
|
|
18
|
-
|
|
19
|
-
## Acknowledgment Rules
|
|
20
|
-
|
|
21
|
-
- Each delivery must be resolved exactly once with `ack!` or `nack!`
|
|
22
|
-
- Calling `ack!` or `nack!` a second time raises `RuntimeError`
|
|
23
|
-
- If neither is called before the channel's `timeout`, the delivery auto-nacks
|
|
24
|
-
|
|
25
|
-
## Delivery States
|
|
26
|
-
|
|
27
|
-
| Method | Description |
|
|
28
|
-
|--------|-------------|
|
|
29
|
-
| `pending?` | Not yet resolved |
|
|
30
|
-
| `acked?` | Successfully acknowledged |
|
|
31
|
-
| `nacked?` | Explicitly rejected or timed out |
|
|
32
|
-
| `timed_out?` | `true` if the nack was caused by timeout |
|
|
33
|
-
|
|
34
|
-
## Timeout Behavior
|
|
35
|
-
|
|
36
|
-
When a delivery times out:
|
|
37
|
-
|
|
38
|
-
1. `timed_out?` returns `true`
|
|
39
|
-
2. The delivery is automatically NACKed
|
|
40
|
-
3. It routes to the dead letter queue
|
|
41
|
-
4. The `:timed_out` and `:dead_lettered` stats increment
|
|
42
|
-
|
|
43
|
-
```ruby
|
|
44
|
-
channel = TypedBus::Channel.new(:work, timeout: 2)
|
|
45
|
-
|
|
46
|
-
channel.subscribe do |delivery|
|
|
47
|
-
sleep 5 # exceeds the 2s timeout
|
|
48
|
-
delivery.ack! # raises — already resolved by timeout
|
|
49
|
-
end
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
Use `timed_out?` to distinguish timeouts from explicit nacks when processing the DLQ:
|
|
53
|
-
|
|
54
|
-
```ruby
|
|
55
|
-
channel.dead_letter_queue.each do |delivery|
|
|
56
|
-
if delivery.timed_out?
|
|
57
|
-
# retry — subscriber was too slow
|
|
58
|
-
else
|
|
59
|
-
# investigate — subscriber explicitly rejected
|
|
60
|
-
end
|
|
61
|
-
end
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
## Multiple Subscribers
|
|
65
|
-
|
|
66
|
-
When a message has multiple subscribers, each gets its own independent Delivery:
|
|
67
|
-
|
|
68
|
-
```ruby
|
|
69
|
-
bus.add_channel(:events, timeout: 5)
|
|
70
|
-
bus.subscribe(:events) { |d| d.ack! } # subscriber 1
|
|
71
|
-
bus.subscribe(:events) { |d| d.nack! } # subscriber 2
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
The message is "delivered" (`:delivered` stat) only when **all** subscribers ACK. If any subscriber NACKs, only that subscriber's Delivery goes to the DLQ.
|
|
75
|
-
|
|
76
|
-
## DeliveryTracker
|
|
77
|
-
|
|
78
|
-
`publish` returns a `DeliveryTracker` that aggregates subscriber responses:
|
|
79
|
-
|
|
80
|
-
```ruby
|
|
81
|
-
Async do
|
|
82
|
-
tracker = channel.publish(message)
|
|
83
|
-
# tracker.fully_delivered? — all acked?
|
|
84
|
-
# tracker.fully_resolved? — all responded?
|
|
85
|
-
# tracker.pending_count — still waiting on N
|
|
86
|
-
end
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
Returns `nil` if there are no subscribers (message goes directly to DLQ).
|