outbox_machines 0.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,226 @@
1
+ # outbox-machines
2
+
3
+ > **AMS Helland**, NCC-4221, *Helland*-class dispatch courier.
4
+
5
+ Engine-room flight manual: track owed dispatches through identity-checked attempts.
6
+
7
+ Pre-1.0. Bounded delivery-policy metadata for an outbox: records move
8
+ through `pending`, `claimed`, `confirmed` and `dead`, and every attempt carries
9
+ an identity the outbox issues once and never again, so a stale worker cannot
10
+ touch a newer attempt, nor a newer record that reuses the id.
11
+
12
+ This is **delivery policy only**. It is not a transactional outbox or inbox, not
13
+ a relay, and not a retry engine: it never sleeps, schedules, or performs I/O.
14
+ Atomic insertion with business writes, durable claims, crash recovery and
15
+ payload storage are adapter responsibilities. It composes with saga, dedup,
16
+ lease and chrono through explicit caller-side contracts only; this crate has no
17
+ dependency on any sibling.
18
+
19
+ Allocation-free `no_std`. Capacity `N` is a const generic; a full table refuses
20
+ with `Refusal::Full` and never evicts. Time is explicit and must not go
21
+ backwards: every instant (`now`, `Claimed::lease_expires`, `Disposition::Retry`,
22
+ `Entry::available_at`) and every span (`lease`, `retry_after`) is a
23
+ `core::time::Duration` since an epoch the caller picks. The Ruby gem passes
24
+ and answers the same values as Integer nanoseconds, and mirrors `Claimed`,
25
+ `Expired`, `Sweep` and `Entry` as `Data` values of the same names, and
26
+ `Failure`'s two variants as `Failed` and `AlreadyFailed` (its `Sweep#expired`
27
+ carries the expired entries where this one's `Sweep::expired` counts them). Lookup is a linear scan, so keep `N`
28
+ small. Ids are `u64`; payloads live outside.
29
+
30
+ - `enqueue(id, now) -> Result<Enqueue, Refusal>`: `Enqueued`, or `Duplicate`
31
+ for an id that is already present (an idempotent repeat: honoured, nothing
32
+ changes). An id stays taken, even when confirmed or dead, until
33
+ `forget(id)`. Refuses with `TimeRegression` or `Full`.
34
+ - `claim(now, lease) -> Result<Option<Claimed>, Refusal>`: the due pending
35
+ record enqueued first (a forgotten record's slot goes to a newcomer, its
36
+ place in line does not), or `None` when nothing is due. `Claimed::attempt` is
37
+ the identity of the attempt, drawn from one sequence for the whole outbox,
38
+ starting at 1 and carried under the outbox's scope. Refuses with `TimeRegression`, `InvalidDuration`, or
39
+ `Exhausted` when a record is due but that sequence is spent.
40
+ - `confirm(id, attempt) -> Result<Confirmation, Refusal>`: the caller
41
+ asserts the external effect was persisted. The outbox cannot verify it.
42
+ Accepted even after the lease deadline, as long as no newer attempt exists.
43
+ `Confirmed`, or `AlreadyConfirmed` when this same attempt already
44
+ confirmed the record. Refuses with `Stale` for an attempt this outbox never
45
+ issued (another scope, or a sequence it has not reached; before any
46
+ lookup), then `Unknown` or `Stale`.
47
+ - `fail(id, attempt, now, retry_after) -> Result<Failure, Refusal>`: a
48
+ *known* failure. `Failed(Disposition)`: back to pending at
49
+ `now + retry_after`, or `Dead` once `max_attempts` claims are spent.
50
+ `AlreadyFailed(Disposition)` when `fail` already ended this same attempt
51
+ with this same `retry_after` and no claim followed; it carries the standing
52
+ disposition. Refuses, in this order, with `Stale` for an attempt this outbox
53
+ never issued, `Unknown`, `Stale`,
54
+ `TimeRegression`, `InvalidDuration`, or `Mismatch` for a repeat whose
55
+ `retry_after` is not the one recorded.
56
+ - `expire_claims(now, max_scan, out)`: reaps claims whose lease ended, into a
57
+ caller buffer, with a persistent cursor. Bounded by `max_scan` and
58
+ `out.len()`; call again until `Sweep::pass()` is `Pass::Complete`. Writes the first
59
+ `Sweep::expired` entries of `out` and leaves the rest untouched. Build the
60
+ buffer as `[Expired::default(); K]`: the default is a placeholder with
61
+ attempt 0, which the outbox never emits. Refuses with `TimeRegression`.
62
+ - `forget(id) -> Result<Forgetting, Refusal>`: drops a confirmed or dead
63
+ record. `Forgotten`, or `NotHeld` when no record carries the id. Refuses
64
+ with `NotTerminal`.
65
+ - `get(id) -> Option<Entry>`: the record's state, `attempts` (the claims it
66
+ has had, which is what `max_attempts` counts) and `attempt` (the identity
67
+ of the latest one, `None` before the first claim).
68
+
69
+ ## Attempt identities
70
+
71
+ An attempt identity is issued once for the lifetime of the outbox. It is not
72
+ the record's attempt count, which restarts when an id is forgotten and
73
+ enqueued again; the identity does not. A worker that claimed id 7, went quiet,
74
+ and reports back after 7 was confirmed, forgotten, enqueued and claimed anew
75
+ presents an identity the new record never had: `Stale`, nothing changed.
76
+
77
+ ## Scopes
78
+
79
+ An attempt identity means something only to the outbox that issued it. It is
80
+ a `u64` in the [`periscope-machines`](https://crates.io/crates/periscope-machines)
81
+ layout: the outbox's scope in the top 16 bits, a sequence counted from 1 in
82
+ the low 48. `Outbox::new(max_attempts)` is scope zero and issues 1, 2, 3;
83
+ `Outbox::with_scope(max_attempts, scope)` issues the same sequence under
84
+ `scope`, and `Outbox::scope()`, `Claimed::scope()` and `Expired::scope()` read
85
+ it back.
86
+
87
+ Two outboxes that hold the same record id issue the same sequence numbers, so
88
+ without scopes a report sent to the wrong outbox settles the wrong attempt.
89
+ Give each outbox its own scope: `confirm` and `fail` then answer `Stale` to an
90
+ attempt of another scope before they look at any record, and nothing changes.
91
+ The outbox reads no clock and no random source, so it cannot pick a scope;
92
+ two outboxes left in the same scope still alias.
93
+
94
+ The check is `periscope_machines::Issuer::owns`, so it refuses more than
95
+ another scope: an attempt of this scope whose sequence the outbox has not
96
+ reached, or sequence zero, was never issued here and is `Stale` too, whatever
97
+ the id.
98
+
99
+ A fresh outbox starts its sequence again, so an attempt that outlived a
100
+ restart could name a new claim. Either save `Outbox::issued()` and restart
101
+ with `Outbox::resumed(max_attempts, scope, issued)`, which continues at
102
+ `issued + 1` (`None` when `issued` is past 2^48 - 1), or give each incarnation
103
+ a scope no earlier one used. Save the count before acting on a claim:
104
+ resuming from too low a count reissues identities.
105
+
106
+ A scope holds 2^48 - 1 identities. When they are spent `claim` refuses with
107
+ `Exhausted` rather than wrap, and only when a record is due.
108
+
109
+ ## Repeats and refusals
110
+
111
+ `Duplicate`, `AlreadyConfirmed`, `AlreadyFailed` and `NotHeld` are idempotent
112
+ repeats: the effect asked for already holds, so the call is honoured and
113
+ changes nothing, the observed clock included. The outbox honours only what it
114
+ can prove from the record it still holds:
115
+
116
+ - A confirmed record is never claimed again, so its latest attempt *is* the
117
+ confirming one. `confirm` with that attempt repeats; any other attempt is
118
+ `Stale`, an attempt of an earlier record with the same id included. The
119
+ attempt a record holds was issued by this outbox, and an attempt of another
120
+ scope is refused before the record is read, so a repeat is only ever
121
+ honoured for an identity of this scope.
122
+ - A record remembers whether its latest attempt was ended by `fail`, and with
123
+ which `retry_after`. `fail` with that attempt and that `retry_after` repeats
124
+ until the record is claimed again. `now` is the instant of the call, not
125
+ part of the report, so it may differ; a different `retry_after` is a
126
+ different report and is refused with `Mismatch`, also when the record died
127
+ and the delay shaped nothing. An attempt reaped by `expire_claims` has an
128
+ unknown outcome, not a failed one: `fail` for it is `Stale`, as is `fail`
129
+ for a confirmed attempt and `confirm` for a failed one.
130
+ - `claim` and `expire_claims` carry no identity a repeat could be matched on.
131
+ A second identical call is a new call: it claims the next due record, or
132
+ reaps nothing.
133
+
134
+ `Refusal::Unknown` is answered by `confirm` and `fail` only, when no record
135
+ carries the id and the attempt is one this outbox issued. That includes a record that was confirmed or dead and then
136
+ forgotten: nothing about it is kept, so a late repeat cannot be proven and is
137
+ `Unknown`. `enqueue`, `claim`, `expire_claims` and `forget` never answer it.
138
+
139
+ `Refusal::Stale` means the id is held but `attempt` is neither its live
140
+ attempt nor the attempt whose outcome the call repeats, or that this outbox
141
+ never issued `attempt` (another scope, or a sequence it has not reached),
142
+ whatever the id. The variants are
143
+ `Unknown`, `Stale`, `NotTerminal`, `TimeRegression`, `InvalidDuration`,
144
+ `Exhausted`, `Full` and `Mismatch`. `Refusal` is the only refusal type, no public enum is `#[non_exhaustive]`
145
+ (a new variant is a breaking change every caller matches on), and a refused call
146
+ changes nothing: no record, no sweep cursor, and not the observed clock.
147
+ `Outbox` is `PartialEq`, so `outbox == clone_taken_before` proves it.
148
+
149
+ An honoured answer records `now` exactly when it depends on `now`:
150
+
151
+ | Method | Records `now` | Leaves the clock alone |
152
+ | --- | --- | --- |
153
+ | `enqueue` | `Enqueued` (claimable from `now`) | `Duplicate` |
154
+ | `claim` | `Some(Claimed)`, `None` (nothing due at `now`) | |
155
+ | `confirm` | takes no time | `Confirmed`, `AlreadyConfirmed` |
156
+ | `fail` | `Failed` (retries from `now`) | `AlreadyFailed` |
157
+ | `expire_claims` | every `Sweep`, one that wrote nothing included | |
158
+ | `forget` | takes no time | `Forgotten`, `NotHeld` |
159
+
160
+ Every refusal leaves it alone.
161
+
162
+ ## Values
163
+
164
+ `Claimed::new`, `Expired::new`, `Sweep::new` and `Entry::new` build a value
165
+ only if some outbox could answer it, `None` otherwise: an attempt whose
166
+ sequence is zero (no scope issues it), a `Claimed` whose lease ends at the
167
+ epoch, a `Sweep` that wrote more than it scanned, an `Entry` whose attempt and
168
+ claim count disagree or that is claimed, confirmed or dead without an attempt.
169
+ Their fields are private and read through accessors of the same names
170
+ (`Sweep::pass()` answers `Pass::Partial` or `Pass::Complete`), so no value
171
+ skips that check; the Ruby gem validates every value it is handed through
172
+ these constructors. `Expired::default()` is the output-buffer
173
+ placeholder only.
174
+
175
+ ## Ambiguous effects
176
+
177
+ An expired lease is an unknown outcome, not a failure: the worker may still run
178
+ or may have already produced the effect. Expired claims are reported separately
179
+ so the adapter can reconcile (fence, query the destination, or use an
180
+ idempotency key) before trusting a retry. Use `fail` only for failures you know
181
+ did not take effect.
182
+
183
+ ```rust
184
+ use core::num::NonZeroU32;
185
+ use core::time::Duration;
186
+ use outbox_machines::{Confirmation, Disposition, Enqueue, Failure, Forgetting, Outbox, Refusal};
187
+
188
+ # fn main() -> Result<(), Refusal> {
189
+ let mut outbox = Outbox::<8>::new(NonZeroU32::MIN.saturating_add(2));
190
+ assert_eq!(outbox.enqueue(42, Duration::ZERO), Ok(Enqueue::Enqueued));
191
+ let Some(claimed) = outbox.claim(Duration::ZERO, Duration::from_secs(30))? else {
192
+ return Ok(());
193
+ };
194
+ assert_eq!((claimed.id(), claimed.attempt()), (42, 1));
195
+ let retry_at_5 = Disposition::Retry(Duration::from_secs(5));
196
+ assert_eq!(
197
+ outbox.fail(42, 1, Duration::ZERO, Duration::from_secs(5)),
198
+ Ok(Failure::Failed(retry_at_5))
199
+ );
200
+ assert_eq!(
201
+ outbox.fail(42, 1, Duration::ZERO, Duration::from_secs(5)),
202
+ Ok(Failure::AlreadyFailed(retry_at_5))
203
+ );
204
+ assert_eq!(
205
+ outbox.fail(42, 1, Duration::ZERO, Duration::from_secs(9)),
206
+ Err(Refusal::Mismatch)
207
+ );
208
+ let retry = outbox.claim(Duration::from_secs(5), Duration::from_secs(30))?;
209
+ assert_eq!(retry.map(|c| c.attempt()), Some(2));
210
+ assert_eq!(outbox.confirm(42, 1), Err(Refusal::Stale));
211
+ assert_eq!(outbox.confirm(42, 2), Ok(Confirmation::Confirmed));
212
+ assert_eq!(outbox.confirm(42, 2), Ok(Confirmation::AlreadyConfirmed));
213
+ assert_eq!(outbox.forget(42), Ok(Forgetting::Forgotten));
214
+ assert_eq!(outbox.forget(42), Ok(Forgetting::NotHeld));
215
+ assert_eq!(outbox.confirm(42, 2), Err(Refusal::Unknown));
216
+
217
+ // The id is free again, but the identities its attempts used are not.
218
+ assert_eq!(outbox.enqueue(42, Duration::from_secs(5)), Ok(Enqueue::Enqueued));
219
+ let anew = outbox.claim(Duration::from_secs(5), Duration::from_secs(30))?;
220
+ assert_eq!(anew.map(|c| c.attempt()), Some(3));
221
+ assert_eq!(outbox.get(42).map(|e| e.attempts()), Some(1));
222
+ assert_eq!(outbox.confirm(42, 1), Err(Refusal::Stale));
223
+ assert_eq!(outbox.confirm(42, 2), Err(Refusal::Stale));
224
+ # Ok(())
225
+ # }
226
+ ```