outbox_machines 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +106 -0
- data/Cargo.lock +314 -0
- data/Cargo.toml +28 -0
- data/LICENSE.txt +21 -0
- data/README.md +331 -0
- data/ext/outbox_machines_native/core/CHANGELOG.md +81 -0
- data/ext/outbox_machines_native/core/Cargo.toml +23 -0
- data/ext/outbox_machines_native/core/README.md +226 -0
- data/ext/outbox_machines_native/core/src/lib.rs +558 -0
- data/ext/outbox_machines_native/core/src/refusal.rs +48 -0
- data/ext/outbox_machines_native/core/src/values.rs +298 -0
- data/ext/outbox_machines_native/extconf.rb +2 -0
- data/ext/outbox_machines_native/ffi/Cargo.toml +21 -0
- data/ext/outbox_machines_native/ffi/extconf.rb +14 -0
- data/ext/outbox_machines_native/ffi/src/classes.rs +37 -0
- data/ext/outbox_machines_native/ffi/src/init.rs +35 -0
- data/ext/outbox_machines_native/ffi/src/lib.rs +300 -0
- data/ext/outbox_machines_native/ffi/src/values.rs +149 -0
- data/lib/outbox_machines/version.rb +5 -0
- data/lib/outbox_machines.rb +131 -0
- data/sig/outbox_machines.rbs +192 -0
- metadata +92 -0
data/README.md
ADDED
|
@@ -0,0 +1,331 @@
|
|
|
1
|
+
# OutboxMachines
|
|
2
|
+
|
|
3
|
+
> Sent, probably, is not a delivery status.
|
|
4
|
+
|
|
5
|
+
A delivery ledger for Ruby. Every message you owe the outside world gets a record: pending, claimed by one attempt, confirmed, or dead. Each claim carries an attempt number the outbox issues once and never again, so a worker that woke up late can't confirm or fail an attempt that has since been handed to someone else. Attempts that never report back are surfaced as what they are: unknown.
|
|
6
|
+
|
|
7
|
+
This is an initial release and the API is unstable. It is delivery policy, not a transactional outbox and not a retry engine: it does not write to your database, relay anything, sleep, or schedule.
|
|
8
|
+
|
|
9
|
+
## Quick Start
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
gem 'outbox_machines'
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
```ruby
|
|
16
|
+
SECOND = 1_000_000_000
|
|
17
|
+
def now = Process.clock_gettime(Process::CLOCK_MONOTONIC, :nanosecond)
|
|
18
|
+
|
|
19
|
+
outbox = OutboxMachines::Outbox.new(5) # five attempts, then dead
|
|
20
|
+
|
|
21
|
+
# Once your transaction has committed the row:
|
|
22
|
+
outbox.enqueue(event.id, now)
|
|
23
|
+
|
|
24
|
+
def relay(outbox, broker)
|
|
25
|
+
claimed = outbox.claim(now, 30 * SECOND)
|
|
26
|
+
return unless claimed.is_a?(OutboxMachines::Claimed) # nil: nothing due
|
|
27
|
+
|
|
28
|
+
broker.publish(payload_for(claimed.id))
|
|
29
|
+
outbox.confirm(claimed.id, claimed.attempt)
|
|
30
|
+
rescue BrokerRefused
|
|
31
|
+
outbox.fail(claimed.id, claimed.attempt, now, 5 * SECOND)
|
|
32
|
+
end
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
One outbox, six verbs. Ids and times are Integers. Every time, instant or span, in or out, is nanoseconds from a monotonic clock: `now`, `lease`, `retry_after`, a claim's `lease_expires`, the instant in a `Retry`. They take the whole range a Rust `Duration` holds, so an instant the outbox answered past `2**64 - 1` (a `lease_expires` is `now + lease`) goes back in as `now`; a negative one or one past `Duration::MAX` raises `RangeError`, and a sum past it is the refusal `:invalid_duration`.
|
|
36
|
+
|
|
37
|
+
The examples below use small numbers so the arithmetic fits on a line. They are nanoseconds like any other.
|
|
38
|
+
|
|
39
|
+
## A Message to the Resistance
|
|
40
|
+
|
|
41
|
+
So you saved the order and published `order.created` in the same request. The commit went through and the publish timed out. Or the publish went through and the commit rolled back. Either way, two systems now disagree about whether an order exists, and you find out from a customer.
|
|
42
|
+
|
|
43
|
+
You know what the docs told you to do. "Use the outbox pattern." So you add an `outbox` table and a relay loop. Then two relays pick up the same row. So you add a `locked_at` column. Then a relay dies holding the lock, so you add a timeout. Then the relay that wasn't dead, only slow, wakes up and marks the row sent, over the top of the retry that is halfway through sending it again.
|
|
44
|
+
|
|
45
|
+
The piece you keep rebuilding is a small state machine per message: waiting, out with one attempt, done, given up. With a number on each attempt, so yesterday's attempt can't speak for today's. And a deadline, after which the honest answer is "we don't know what happened". That machine is this gem.
|
|
46
|
+
|
|
47
|
+
## Chapter 1: Stardate 2026.316, the AMS Helland
|
|
48
|
+
|
|
49
|
+
> **AMS Helland**, NCC-4221, *Helland*-class dispatch courier.
|
|
50
|
+
> A dispatch with no receipt was neither delivered nor lost.
|
|
51
|
+
|
|
52
|
+
The courier doesn't carry the dispatches. It keeps the board. Every dispatch on it is waiting, out with a named runner, signed for, or written off. A runner who comes back after his run was given to someone else finds his chit is no longer good. The ship is named for Pat Helland, who spent a career explaining that the far side of a network is another country.
|
|
53
|
+
|
|
54
|
+
On a bridge somewhere, a cadet volunteers:
|
|
55
|
+
|
|
56
|
+
```ruby
|
|
57
|
+
def relay(row)
|
|
58
|
+
broker.publish(row.payload)
|
|
59
|
+
row.update!(sent: true)
|
|
60
|
+
rescue Timeout::Error
|
|
61
|
+
retry
|
|
62
|
+
end
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
"That," says the courier's dispatcher, "is a timeout you have decided to read as a no."
|
|
66
|
+
|
|
67
|
+
## The Flight Plan
|
|
68
|
+
|
|
69
|
+
Each record the outbox holds is in one of these states:
|
|
70
|
+
|
|
71
|
+
```mermaid
|
|
72
|
+
stateDiagram-v2
|
|
73
|
+
[*] --> pending: enqueue
|
|
74
|
+
pending --> claimed: claim, returns a Claimed
|
|
75
|
+
claimed --> confirmed: confirm(id, attempt)
|
|
76
|
+
claimed --> pending: fail, attempts remain
|
|
77
|
+
claimed --> pending: expire_claims, attempts remain
|
|
78
|
+
claimed --> dead: fail or expire_claims, attempts spent
|
|
79
|
+
confirmed --> [*]: forget
|
|
80
|
+
dead --> [*]: forget
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
And the afternoon a relay went quiet with message 42:
|
|
84
|
+
|
|
85
|
+
```mermaid
|
|
86
|
+
sequenceDiagram
|
|
87
|
+
participant R1 as Relay 1
|
|
88
|
+
participant O as Outbox
|
|
89
|
+
participant R2 as Relay 2
|
|
90
|
+
|
|
91
|
+
R1->>O: claim
|
|
92
|
+
O-->>R1: Claimed 42, attempt 1
|
|
93
|
+
Note over R1: publishes, then stalls
|
|
94
|
+
|
|
95
|
+
R2->>O: expire_claims
|
|
96
|
+
O-->>R2: Expired 42, attempt 1, retry
|
|
97
|
+
Note over R2: outcome unknown. Reconcile first.
|
|
98
|
+
|
|
99
|
+
R2->>O: claim
|
|
100
|
+
O-->>R2: Claimed 42, attempt 2
|
|
101
|
+
|
|
102
|
+
R1->>O: confirm(42, 1)
|
|
103
|
+
O-->>R1: :stale
|
|
104
|
+
|
|
105
|
+
R2->>O: confirm(42, 2)
|
|
106
|
+
O-->>R2: :confirmed
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
## How the Outbox Works
|
|
110
|
+
|
|
111
|
+
### Enqueue
|
|
112
|
+
|
|
113
|
+
```ruby
|
|
114
|
+
outbox = OutboxMachines::Outbox.new(3) # max attempts
|
|
115
|
+
|
|
116
|
+
outbox.enqueue(42, 0) # => :enqueued
|
|
117
|
+
outbox.enqueue(42, 0) # => :duplicate
|
|
118
|
+
outbox.get(42) # => #<data OutboxMachines::Entry id=42, state=:pending, attempts=0, attempt=nil, available_at=0>
|
|
119
|
+
outbox.get(7) # => nil, no such record
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
`get(id)` answers the core's `Entry`: the record's `id`, its `state` (`:pending`, `:claimed`, `:confirmed` or `:dead`), the `attempts` claimed so far (what `max_attempts` counts), the `attempt` number of the latest one (`nil` until the first claim) and `available_at`, the earliest instant a pending record may be claimed. It answers `nil` for an id the outbox does not hold, and never refuses.
|
|
123
|
+
|
|
124
|
+
A record is an id and nothing else. The payload stays in your table. An id is taken for as long as its record exists, in any state.
|
|
125
|
+
|
|
126
|
+
`:duplicate` is not a refusal. You asked for the id to be on the board and it is, so the call is honoured and changes nothing, the outbox's clock included.
|
|
127
|
+
|
|
128
|
+
### Claim
|
|
129
|
+
|
|
130
|
+
```ruby
|
|
131
|
+
claimed = outbox.claim(0, 30_000) # now 0, lease 30_000 ns
|
|
132
|
+
claimed # => #<data OutboxMachines::Claimed id=42, attempt=1, lease_expires=30000>
|
|
133
|
+
outbox.claim(0, 30_000) # => nil, nothing else is due
|
|
134
|
+
outbox.get(42).state # => :claimed
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
`claim(now, lease)` hands out the due pending record that was enqueued first. `nil` means nothing is due, which is an answer and not a refusal. `attempt` is the identity of this attempt: the outbox numbers every claim it hands out from one sequence, starting at 1, and never uses a number twice. An outbox built with `Outbox.with_scope(max_attempts, scope)` stamps its scope into the top 16 bits of that number. It is not the count of claims this record has had; that is `get(id).attempts`. `lease_expires` is `now + lease`, the instant from which `expire_claims` may reap the attempt. It is a deadline, not a timer: nothing happens when it passes until you ask.
|
|
138
|
+
|
|
139
|
+
### Fail
|
|
140
|
+
|
|
141
|
+
```ruby
|
|
142
|
+
failed = outbox.fail(42, claimed.attempt, 10, 5_000)
|
|
143
|
+
failed # => #<data OutboxMachines::Failed disposition=#<data OutboxMachines::Retry at=5010>>
|
|
144
|
+
failed.disposition # => #<data OutboxMachines::Retry at=5010>
|
|
145
|
+
|
|
146
|
+
# The reply got lost, so the relay reports the same attempt again:
|
|
147
|
+
outbox.fail(42, claimed.attempt, 12, 5_000)
|
|
148
|
+
# => #<data OutboxMachines::AlreadyFailed disposition=#<data OutboxMachines::Retry at=5010>>
|
|
149
|
+
outbox.fail(42, claimed.attempt, 12, 9_000) # => :mismatch, that is not what was reported
|
|
150
|
+
|
|
151
|
+
outbox.claim(5_009, 30_000) # => nil, not due yet
|
|
152
|
+
second = outbox.claim(5_010, 30_000)
|
|
153
|
+
second.attempt # => 2
|
|
154
|
+
outbox.fail(42, claimed.attempt, 5_010, 0) # => :stale, attempt 1 was superseded
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
`fail(id, attempt, now, retry_after)` is for an attempt you know did nothing. You pass the delay, and the answer is a `Failed` wrapping the core's `Disposition`: a `Retry(at)` carrying the instant the record is claimable again, `now + retry_after`, or the symbol `:dead`. The outbox does not compute backoff.
|
|
158
|
+
|
|
159
|
+
Report the same attempt twice and the second answer is an `AlreadyFailed`. It is not a refusal: the attempt is over, as you asked, so the call is honoured and changes nothing, the clock included. Its disposition is the one from the first call. A repeat is the same report made again: same attempt, same `retry_after`. `now` is when you called, so it may differ, but a different `retry_after` is a different report and the answer is the refusal `:mismatch`, even for a record that died. The arguments of a repeat are still checked. The outbox honours that repeat only while it can prove it: until the record is claimed again, and only when it was `fail` that ended the attempt. An attempt reaped by `expire_claims` was never failed, so a `fail` for it is `:stale`.
|
|
160
|
+
|
|
161
|
+
Once the attempts are spent the answer changes:
|
|
162
|
+
|
|
163
|
+
```ruby
|
|
164
|
+
single = OutboxMachines::Outbox.new(1)
|
|
165
|
+
single.enqueue(1, 0)
|
|
166
|
+
only = single.claim(0, 10)
|
|
167
|
+
|
|
168
|
+
single.fail(1, only.attempt, 5, 1_000) # => #<data OutboxMachines::Failed disposition=:dead>
|
|
169
|
+
single.fail(1, only.attempt, 6, 1_000) # => #<data OutboxMachines::AlreadyFailed disposition=:dead>
|
|
170
|
+
single.get(1).state # => :dead
|
|
171
|
+
single.claim(10_000, 10) # => nil
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
A dead record is never claimed again. It sits there, countable with `count(:dead)`, until you `forget` it.
|
|
175
|
+
|
|
176
|
+
### Confirm
|
|
177
|
+
|
|
178
|
+
```ruby
|
|
179
|
+
outbox.confirm(42, 1) # => :stale, attempt 1 was superseded
|
|
180
|
+
outbox.confirm(42, 2) # => :confirmed
|
|
181
|
+
outbox.confirm(42, 2) # => :already_confirmed, and nothing changed
|
|
182
|
+
outbox.confirm(42, 1) # => :stale, still not the attempt that confirmed
|
|
183
|
+
outbox.confirm(7, 1) # => :unknown
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
`confirm` is your word that the effect is persisted on the other side. The outbox can't check. Call it after the broker's acknowledgement or your durable write, never before.
|
|
187
|
+
|
|
188
|
+
`:already_confirmed` is not a refusal. A confirmed record is never claimed again, so its latest attempt is the one that confirmed it, and the outbox can tell that attempt's repeat from anyone else's for as long as it holds the record. After `forget` the record and the proof are both gone, and the same call answers `:unknown`.
|
|
189
|
+
|
|
190
|
+
### Expire claims
|
|
191
|
+
|
|
192
|
+
```ruby
|
|
193
|
+
outbox = OutboxMachines::Outbox.new(2)
|
|
194
|
+
outbox.enqueue(1, 0)
|
|
195
|
+
lost = outbox.claim(0, 10)
|
|
196
|
+
|
|
197
|
+
sweep = outbox.expire_claims(9, 64)
|
|
198
|
+
sweep.expired # => []
|
|
199
|
+
sweep.scanned # => 64
|
|
200
|
+
sweep.pass # => :complete
|
|
201
|
+
|
|
202
|
+
sweep = outbox.expire_claims(10, 64)
|
|
203
|
+
sweep.expired # => [#<data OutboxMachines::Expired id=1, attempt=1, disposition=#<data OutboxMachines::Retry at=10>>]
|
|
204
|
+
outbox.get(1).state # => :pending
|
|
205
|
+
|
|
206
|
+
fresh = outbox.claim(10, 10)
|
|
207
|
+
fresh.attempt # => 2
|
|
208
|
+
outbox.confirm(1, lost.attempt) # => :stale
|
|
209
|
+
outbox.confirm(1, fresh.attempt) # => :confirmed
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
`expire_claims(now, max_scan)` reaps claims whose lease has ended and answers a `Sweep`: `expired`, `scanned` and `pass`. Each entry of `expired` is an `Expired` with an `id`, an `attempt` and a `disposition`, which is a `Retry(at)` or `:dead`. An expired record is due again immediately, so `at` is the `now` you passed. One call looks at no more than `max_scan` slots, says how many it looked at in `scanned`, and reaps no more than 16 claims. Keep calling until `pass` is `:complete` (it is `:partial` until the cursor has looked at every slot since the last full pass). A `max_scan` the platform's `usize` cannot hold raises `RangeError`; it is never narrowed.
|
|
213
|
+
|
|
214
|
+
An expired attempt did not fail. Nobody knows what it did. Until it is reaped, a late `confirm` for it is still accepted, because if the effect happened the record must not go out again.
|
|
215
|
+
|
|
216
|
+
### Forget
|
|
217
|
+
|
|
218
|
+
```ruby
|
|
219
|
+
outbox.enqueue(1, 10) # => :duplicate, confirmed records keep their id
|
|
220
|
+
outbox.forget(1) # => :forgotten
|
|
221
|
+
outbox.forget(1) # => :not_held
|
|
222
|
+
outbox.enqueue(1, 10) # => :enqueued
|
|
223
|
+
outbox.forget(1) # => :not_terminal
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
Only confirmed and dead records can be forgotten. Forgetting frees the slot and the id. `:not_held` is not a refusal: you asked for the record to be gone and it is, whether it was forgotten earlier or never enqueued.
|
|
227
|
+
|
|
228
|
+
Forgetting does not free the attempt numbers the record used. Enqueue the id again and it is a new record whose first claim gets a number nobody has held:
|
|
229
|
+
|
|
230
|
+
```ruby
|
|
231
|
+
outbox = OutboxMachines::Outbox.new(3)
|
|
232
|
+
outbox.enqueue(7, 0)
|
|
233
|
+
old = outbox.claim(0, 10) # attempt 1
|
|
234
|
+
outbox.confirm(7, old.attempt) # => :confirmed
|
|
235
|
+
outbox.forget(7) # => :forgotten
|
|
236
|
+
outbox.enqueue(7, 0) # => :enqueued, a new record
|
|
237
|
+
fresh = outbox.claim(0, 10)
|
|
238
|
+
fresh.attempt # => 2
|
|
239
|
+
outbox.get(7).attempts # => 1
|
|
240
|
+
outbox.confirm(7, old.attempt) # => :stale, a report about the record that is gone
|
|
241
|
+
outbox.fail(7, old.attempt, 0, 0) # => :stale
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
### Capacity
|
|
245
|
+
|
|
246
|
+
The outbox holds 64 records at a time, in any state. Past that, `enqueue` refuses with `:full` until something is forgotten:
|
|
247
|
+
|
|
248
|
+
```ruby
|
|
249
|
+
outbox = OutboxMachines::Outbox.new(3)
|
|
250
|
+
outbox.capacity # => 64
|
|
251
|
+
OutboxMachines::CAPACITY # => 64
|
|
252
|
+
outbox.max_attempts # => 3
|
|
253
|
+
|
|
254
|
+
64.times { |i| outbox.enqueue(i, 0) }
|
|
255
|
+
outbox.enqueue(64, 0) # => :full
|
|
256
|
+
outbox.size # => 64
|
|
257
|
+
outbox.count(:pending) # => 64
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
Nothing is ever evicted to make room, confirmed and dead records included. Forget what you have dealt with.
|
|
261
|
+
|
|
262
|
+
### Time only moves forward
|
|
263
|
+
|
|
264
|
+
The outbox remembers the latest time it has seen. Hand it an earlier one and it refuses:
|
|
265
|
+
|
|
266
|
+
```ruby
|
|
267
|
+
outbox = OutboxMachines::Outbox.new(3)
|
|
268
|
+
outbox.enqueue(1, 10) # => :enqueued
|
|
269
|
+
outbox.enqueue(2, 9) # => :time_regression
|
|
270
|
+
outbox.claim(9, 5) # => :time_regression
|
|
271
|
+
outbox.expire_claims(9, 1) # => :time_regression
|
|
272
|
+
outbox.claim(10, 0) # => :invalid_duration
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
A zero lease is `:invalid_duration`. Feed it `Process.clock_gettime(Process::CLOCK_MONOTONIC, :nanosecond)`, never wall-clock time.
|
|
276
|
+
|
|
277
|
+
Only a call that is honoured and takes effect moves that clock. A refusal doesn't, and neither does a repeat: a `:duplicate` enqueue or an `AlreadyFailed`. (`confirm` and `forget` take no time at all.) A `claim` that finds nothing due and an `expire_claims` that reaps nothing were both honoured, so the time they were given counts.
|
|
278
|
+
|
|
279
|
+
### What each verb can answer
|
|
280
|
+
|
|
281
|
+
| Verb | Honoured | Records `now` | Refusals |
|
|
282
|
+
| --- | --- | --- | --- |
|
|
283
|
+
| `enqueue` | `:enqueued`, `:duplicate` | `:enqueued` only (the record is claimable from `now`) | `:time_regression`, `:full` |
|
|
284
|
+
| `claim` | a `Claimed`, `nil` | both (`nil` means nothing was due at `now`) | `:time_regression`, `:invalid_duration`, `:exhausted` |
|
|
285
|
+
| `confirm` | `:confirmed`, `:already_confirmed` | takes no time | `:stale`, `:unknown` |
|
|
286
|
+
| `fail` | `Failed(disposition)`, `AlreadyFailed(disposition)` | `Failed` only (it retries from `now`) | `:stale`, `:unknown`, `:time_regression`, `:invalid_duration`, `:mismatch` |
|
|
287
|
+
| `expire_claims` | a `Sweep` | always, also when it reaped nothing (an answer about `now`) | `:time_regression` |
|
|
288
|
+
| `forget` | `:forgotten`, `:not_held` | takes no time | `:not_terminal` |
|
|
289
|
+
|
|
290
|
+
An honoured answer records `now` exactly when it depends on `now`. A repeat that holds whatever the time (`:duplicate`, `AlreadyFailed`) records nothing, and no refusal records anything.
|
|
291
|
+
|
|
292
|
+
A `disposition` is a `Retry(at)` or `:dead`. The queries `get`, `count`, `size`, `empty?`, `capacity`, `max_attempts`, `scope` and `issued` never refuse.
|
|
293
|
+
|
|
294
|
+
`:duplicate`, `:already_confirmed`, `AlreadyFailed` and `:not_held` are idempotent repeats: what you asked for already holds, so the call is honoured and changes nothing.
|
|
295
|
+
|
|
296
|
+
`:unknown` comes from `confirm` and `fail` only, for an id the outbox does not hold, reported with an attempt the outbox did issue. That includes a record you have since forgotten: the outbox keeps nothing about it, so it cannot call your report a repeat. `:stale` means the id is held but the attempt is neither its live one nor the one whose outcome the call repeats; an attempt that belonged to an earlier record with the same id is exactly that. An attempt this outbox never issued is `:stale` too, and that is answered first, before the id is even looked up, so it is `:stale` for an id the outbox does not hold as well: an attempt from an outbox of another scope, a number of its own scope it has not reached yet, or a number whose low 48 bits are zero. `:mismatch` comes from `fail` only: the attempt was already failed, with a different `retry_after`.
|
|
297
|
+
|
|
298
|
+
`:exhausted` comes from `claim` only: a record is due but the outbox has handed out every attempt number of its scope (2**48 - 1 claims) in its lifetime. It cannot be reached from Ruby in practice; you will not see it. Each refusal is the snake_case of one core `Refusal` variant; there is no catch-all symbol.
|
|
299
|
+
|
|
300
|
+
## The Rules of the Courier
|
|
301
|
+
|
|
302
|
+
- **Ids and times are Integers** in `0..2**64-1`. Floats, nil, and strings raise `TypeError`. Negative or past `2**64-1` raises `RangeError`, and so does a `max_attempts` outside `1..2**32-1`.
|
|
303
|
+
- **Time is Integer nanoseconds**, for instants and spans alike. No floats, no milliseconds, no `Time`. An instant the outbox answers is exact, so `now + lease` can exceed `2**64-1` when you feed it numbers that large.
|
|
304
|
+
- **Values are valid or do not exist.** Every value class checks itself on `new` through the core's own constructor, so one you build by hand is held to the rule a returned one is. `Claimed.new(7, 0, 10)` (no outbox issues attempt sequence zero), `Claimed.new(7, 1, 0)` (a lease ends after the epoch), `Expired.new(7, 0, :dead)`, `Sweep.new([expired], 0, :complete)` (more reaped than scanned), `Entry.new(1, :claimed, 0, nil, 3)` (only a pending record is unclaimed), `Failed.new(:nope)` and `Retry.new(-1)` raise `RangeError`; a wrong type (`Retry.new(1.0)`, `Sweep.new([1], 3, :complete)`, `count('pending')`) raises `TypeError`. `count(:nope)` is `RangeError`.
|
|
305
|
+
- **Values are final, and the extension is the authority.** Subclassing a value class raises `TypeError` and `allocate` is gone. That stops accidents, not reflection: a named subclass still leaves its constant behind, `Marshal.load` and a hand-bound `Data#initialize` skip the check. So every value the extension reads back (the `Retry` in a `Failed` or an `Expired`, each `Expired` in a `Sweep`) must be exactly its class, and is read natively and checked again by the core: a forged one raises `TypeError` or `RangeError`. `Sweep` takes a plain `Array`, never a subclass.
|
|
306
|
+
- **The outbox never loses work.** Every argument is converted before the core is called, and the value classes are resolved once, when the extension loads. `claim`, `fail` and `expire_claims` answer Ruby objects, and building one runs Ruby code (`new`, which a reopened class can redefine), so the core decides on a copy, the answer is built, and the copy is kept only if the outbox is still unfrozen and unchanged. A `new` that raises applies nothing; one that freezes the outbox raises `FrozenError`, one that calls back in and changes it raises `RuntimeError`, and in both cases the outer call is not applied.
|
|
307
|
+
- **A Claimed is an id, an attempt, and `lease_expires`.** `Claimed`, `Expired`, `Sweep` and `Entry` are plain frozen `Data` values with the fields of the core structs they are named after; `Retry` is the core's `Disposition::Retry(at)`, and `Failed` and `AlreadyFailed` are the two variants of its `Failure`. `Sweep#expired` holds the reaped claims themselves where the core's `Sweep::expired` counts the ones it wrote to the caller's buffer. `confirm` and `fail` take the first two numbers of a `Claimed`, and an attempt that is neither the live one nor the one being repeated is `:stale`. Attempt numbers are never reissued, so a `Claimed` from last week cannot speak for a record enqueued today.
|
|
308
|
+
- **Scopes.** An attempt number is only meaningful to the outbox that issued it. Numbers count from 1, so two outboxes that hold the same record id hand out the same ones, and a report sent to the wrong outbox would settle the wrong attempt. Give each outbox its own scope: `Outbox.with_scope(max_attempts, scope)`, any Integer in `0..65_535` (a non-Integer raises `TypeError`, anything outside the range raises `RangeError`). `Outbox.new(max_attempts)` is scope zero, and `outbox.scope` reads it back. The scope rides in the top 16 bits of every `attempt`, and `confirm` and `fail` answer `:stale` to an attempt from another scope. The outbox can't pick a scope for you, and two outboxes left in the same scope still mix up each other's attempts.
|
|
309
|
+
- **Restarts.** A new outbox counts from 1 again, so an attempt that outlived a restart could name a new claim. Either save `outbox.issued` (how many attempt numbers it has handed out) and restart with `Outbox.resumed(max_attempts, scope, issued)`, which carries on at `issued + 1`, or give each incarnation a scope no earlier one used. Save the count before you act on a claim: resuming from too low a count hands numbers out twice. `issued` is an Integer in `0..2**48-1`; anything else raises `TypeError` or `RangeError`. Records are not restored, enqueue them again.
|
|
310
|
+
- **Refusals don't raise.** `:full`, `:stale`, `:unknown`, `:not_terminal`, `:time_regression`, `:invalid_duration`, `:exhausted` and `:mismatch` are answers. Only malformed input raises. A refused call changes nothing: no record, and not the clock.
|
|
311
|
+
- **Repeats are honoured.** `:duplicate`, `:already_confirmed`, `AlreadyFailed` and `:not_held` say the effect you asked for already holds. They change nothing either.
|
|
312
|
+
- **Lookup is a scan.** The table is small on purpose. Shard across outboxes before you ask for a bigger one.
|
|
313
|
+
- **Frozen means frozen.** `freeze` the outbox and `enqueue`, `claim`, `confirm`, `fail`, `expire_claims`, and `forget` all raise `FrozenError`. Reads still work.
|
|
314
|
+
- **Native is required.** There is no pure-Ruby fallback.
|
|
315
|
+
|
|
316
|
+
## What You Think You're Doing vs Reality
|
|
317
|
+
|
|
318
|
+
### You Think: "The lease expired, so the send failed. Retry."
|
|
319
|
+
### Reality: The send might be sitting in the broker right now.
|
|
320
|
+
|
|
321
|
+
A timeout tells you the worker stopped talking. It doesn't tell you what the worker did first. That's why expiry and failure are separate verbs here: `fail` is a fact you supply, `expire_claims` is a list of questions the outbox hands back. Answer them, with an idempotency key on the destination, a fence, or a lookup, before the next attempt goes out.
|
|
322
|
+
|
|
323
|
+
## What's Yours
|
|
324
|
+
|
|
325
|
+
The outbox lives in your process and forgets everything when the process does. Writing the outbox row in the same transaction as the business change is yours. So is the durable record of who claimed what, rebuilding the table after a restart, the relay loop, the backoff schedule, and working out what an expired attempt actually did. Receiving is a different problem: deduplicating what arrives is [dedup_machines](https://github.com/seuros/dedup_machines). This gem shares no code with it, or with the lease, chrono, and saga gems. They meet in your adapter.
|
|
326
|
+
|
|
327
|
+
## License
|
|
328
|
+
|
|
329
|
+
MIT License. See [LICENSE.txt](LICENSE.txt) for details.
|
|
330
|
+
|
|
331
|
+
**Remember: the relay will time out. The question is whether you call that a failure.**
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## [0.9.0](https://github.com/seuros/outbox_machines/compare/outbox-machines-v0.8.0...outbox-machines-v0.9.0) (2026-11-16)
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
### ⚠ BREAKING CHANGES
|
|
7
|
+
|
|
8
|
+
* rename Sweep::written to Sweep::expired; times take full Duration
|
|
9
|
+
|
|
10
|
+
### Features
|
|
11
|
+
|
|
12
|
+
* rename Sweep::written to Sweep::expired; times take full Duration ([6ed9a94](https://github.com/seuros/outbox_machines/commit/6ed9a94383a1f48efcbd0074900b2d79c717d425))
|
|
13
|
+
|
|
14
|
+
## [0.8.0](https://github.com/seuros/outbox_machines/compare/outbox-machines-v0.7.0...outbox-machines-v0.8.0) (2026-11-14)
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
### ⚠ BREAKING CHANGES
|
|
18
|
+
|
|
19
|
+
* adopt matryona-machines 0.2 and check frozen before converting arguments
|
|
20
|
+
|
|
21
|
+
### Features
|
|
22
|
+
|
|
23
|
+
* adopt matryona-machines 0.2 and check frozen before converting arguments ([06553b9](https://github.com/seuros/outbox_machines/commit/06553b93b8b47b79289cb85c1da24b44248146a0))
|
|
24
|
+
|
|
25
|
+
## [0.7.0](https://github.com/seuros/outbox_machines/compare/outbox-machines-v0.6.0...outbox-machines-v0.7.0) (2026-11-14)
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
### ⚠ BREAKING CHANGES
|
|
29
|
+
|
|
30
|
+
* build the binding on matryona-machines; accept a genuine value whose singleton class was opened; name every argument in conversion errors
|
|
31
|
+
|
|
32
|
+
### Features
|
|
33
|
+
|
|
34
|
+
* build the binding on matryona-machines; accept a genuine value whose singleton class was opened; name every argument in conversion errors ([79e565c](https://github.com/seuros/outbox_machines/commit/79e565c2859fd02c7ca7fc99057b0cca410aa988))
|
|
35
|
+
|
|
36
|
+
## [0.6.0](https://github.com/seuros/outbox_machines/compare/outbox-machines-v0.5.0...outbox-machines-v0.6.0) (2026-11-14)
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
### ⚠ BREAKING CHANGES
|
|
40
|
+
|
|
41
|
+
* private value fields, drop the enqueue sequence, Pass replaces the sweep flag, sealed exact-class values, clone-then-keep claim/fail/expire
|
|
42
|
+
|
|
43
|
+
### Features
|
|
44
|
+
|
|
45
|
+
* private value fields, drop the enqueue sequence, Pass replaces the sweep flag, sealed exact-class values, clone-then-keep claim/fail/expire ([e6c66dc](https://github.com/seuros/outbox_machines/commit/e6c66dcb726d2a211785504a9fff6ad1daa19611))
|
|
46
|
+
|
|
47
|
+
## [0.5.0](https://github.com/seuros/outbox_machines/compare/outbox-machines-v0.4.0...outbox-machines-v0.5.0) (2026-11-13)
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
### ⚠ BREAKING CHANGES
|
|
51
|
+
|
|
52
|
+
* handles are issued through periscope-machines. Every instance can be built with a scope; a handle from another scope is refused before any lookup; all handle kinds of an instance share one sequence; resumed restores the sequence after a restart. An attempt from another outbox, or one this outbox never issued, answers Stale.
|
|
53
|
+
|
|
54
|
+
### Features
|
|
55
|
+
|
|
56
|
+
* scope attempt identities through periscope-machines, refuse attempts never issued, add Outbox::resumed ([f47285b](https://github.com/seuros/outbox_machines/commit/f47285b5304cbc21d6378b7f448badca23f32196))
|
|
57
|
+
|
|
58
|
+
## [0.4.0](https://github.com/seuros/outbox_machines/compare/outbox-machines-v0.3.0...outbox-machines-v0.4.0) (2026-11-13)
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
### ⚠ BREAKING CHANGES
|
|
62
|
+
|
|
63
|
+
* see CONVENTIONS: one exhaustive Refusal per crate, Result from every mutator, honoured idempotent repeats, Ruby mirrors the core. No compatibility aliases. Claimed and Expired carry a u64 attempt instead of a per-record generation; refused or repeated calls never advance the clock; the Error alias is gone; Ruby times are Integer nanoseconds and values mirror the core.
|
|
64
|
+
|
|
65
|
+
### Features
|
|
66
|
+
|
|
67
|
+
* never reissue attempt identities, honour repeated confirm, fail and forget, Full and Mismatch refusals ([12af815](https://github.com/seuros/outbox_machines/commit/12af815e86a5ac549ec5149642afe35c91ff7a75))
|
|
68
|
+
|
|
69
|
+
## [0.3.0](https://github.com/seuros/outbox_machines/compare/outbox-machines-v0.2.0...outbox-machines-v0.3.0) (2026-11-13)
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+
### Features
|
|
73
|
+
|
|
74
|
+
* rename Error to Refusal and implement Default for Expired ([6f70b82](https://github.com/seuros/outbox_machines/commit/6f70b82ebad6b6f66ea67e520bbb54fcde79550a))
|
|
75
|
+
|
|
76
|
+
## [0.2.0](https://github.com/seuros/outbox_machines/compare/outbox-machines-v0.1.0...outbox-machines-v0.2.0) (2026-11-12)
|
|
77
|
+
|
|
78
|
+
|
|
79
|
+
### Features
|
|
80
|
+
|
|
81
|
+
* initial extraction ([1f6f6ac](https://github.com/seuros/outbox_machines/commit/1f6f6ac855f31b4f1da27c3709f71bc1f0d4522c))
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
[package]
|
|
2
|
+
name = "outbox-machines"
|
|
3
|
+
version = "0.9.0"
|
|
4
|
+
edition.workspace = true
|
|
5
|
+
rust-version.workspace = true
|
|
6
|
+
license.workspace = true
|
|
7
|
+
description = "Bounded delivery-policy metadata with generation-checked attempts"
|
|
8
|
+
readme = "README.md"
|
|
9
|
+
exclude = ["src/**/tests.rs", "src/tests/", "tests/"]
|
|
10
|
+
keywords = ["outbox", "delivery", "retry", "generation", "no-std"]
|
|
11
|
+
categories = ["no-std", "algorithms"]
|
|
12
|
+
|
|
13
|
+
[features]
|
|
14
|
+
default = []
|
|
15
|
+
|
|
16
|
+
[dependencies]
|
|
17
|
+
periscope-machines.workspace = true
|
|
18
|
+
|
|
19
|
+
[dev-dependencies]
|
|
20
|
+
matryona-machines = { workspace = true, features = ["testing"] }
|
|
21
|
+
|
|
22
|
+
[lints]
|
|
23
|
+
workspace = true
|