join_machines 0.6.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md ADDED
@@ -0,0 +1,278 @@
1
+ # JoinMachines
2
+
3
+ > Two streams, one key, and a window of time in which they are allowed to meet. 🤝
4
+
5
+ An interval-join ledger for Ruby. You tell it about every record on two inputs: an id, a key, an event time. It tells you which left and right records share a key and landed close enough in time, hands you those pairs in batches as small as you like, and tells you which records can never match anything again so you can throw their payloads away. It never sees a payload. It keeps the bookkeeping, you keep the data.
6
+
7
+ Pre-1.0: breaking changes land in minor versions. It is a policy core, not a stream processor.
8
+
9
+ ## Quick Start
10
+
11
+ ```bash
12
+ gem 'join_machines'
13
+ ```
14
+
15
+ ```ruby
16
+ # A click joins an impression of the same ad shown up to 30 s before it.
17
+ join = JoinMachines::Join.new(JoinMachines::Bounds.new(before: 30_000, after: 0))
18
+
19
+ def on_record(join, side, record, store)
20
+ case join.insert(side, JoinMachines::Record.new(record.id, record.ad_id, record.event_ms))
21
+ when :late then return count_late(record)
22
+ when :mismatch then return count_conflict(record) # this id is held with other contents
23
+ when :full then raise 'join is full: advance the watermarks and retire'
24
+ end # :accepted, or :duplicate for a replay of a held record: nothing new is owed
25
+ store[side][record.id] = record
26
+
27
+ until (pairs = join.drain(100)).empty?
28
+ pairs.each { |pair| emit(store[:left][pair.left], store[:right][pair.right]) }
29
+ end
30
+ end
31
+
32
+ def on_watermark(join, side, event_ms, store)
33
+ join.advance_watermark(side, event_ms)
34
+ until (gone = join.retire(100)).empty?
35
+ gone.each { |r| store[r.side].delete(r.record.id) }
36
+ end
37
+ end
38
+ ```
39
+
40
+ Ids, keys, and times are Integers. Time is event time in whatever unit you pick, as long as both inputs agree. Each side's watermark speaks for its own input only: one input running well ahead of the other makes nothing `:late` on the slower one.
41
+
42
+ ## A Message to the Resistance
43
+
44
+ So you have impressions on one topic and clicks on another, and somebody in a meeting said "just join them." Thirty seconds of attribution window. How hard can it be.
45
+
46
+ You put the impressions in a hash by ad id. A click arrives, you look it up, you emit. Then you notice the hash only grows, so you add a sweeper that deletes anything older than thirty seconds by the wall clock. Then a consumer lags for a minute, the sweeper deletes impressions whose clicks are still sitting in the partition, and attribution quietly drops by four percent. Nothing raises. The dashboard just gets a little sadder.
47
+
48
+ So you stop sweeping by wall clock and sweep by event time. Whose event time? The click that arrives after the sweep was allowed to be up to thirty seconds *behind* the impression's stream, not its own. Then one hot key matches nine thousand buffered records at once and the handler that was supposed to emit "a pair" blocks the consumer while it emits nine thousand.
49
+
50
+ The piece you keep rebuilding is small: which records are held, which pairs are owed, and when is a record provably finished. With a hard bound on all three. That piece is this gem.
51
+
52
+ ## Chapter 1: Stardate 2026.316, the AMS Widom
53
+
54
+ > **AMS Widom**, NCC-4232, *Widom*-class stream tender.
55
+ > Two convoys pass in the dark. The tender logs who was abeam of whom, and for how long it still matters.
56
+
57
+ The tender carries no cargo. Freighters from two lanes report in as they pass: hull number, consignment code, time of passage. The tender's log pairs every hull on one lane with every hull on the other that carried the same code within the agreed interval, and strikes a hull from the log only when the other lane's traffic control certifies that nothing that early is still inbound.
58
+
59
+ On a bridge somewhere, a cadet volunteers:
60
+
61
+ ```ruby
62
+ def on_click(click)
63
+ impression = @impressions[click.ad_id]
64
+ emit(impression, click) if impression && click.at - impression.at <= 30
65
+ end
66
+
67
+ every(30.seconds) { @impressions.delete_if { |_, i| i.at < Time.now - 30 } }
68
+ ```
69
+
70
+ "That," says the tender's logkeeper, "is a join whose correctness depends on nobody ever being late. Have you met a convoy?"
71
+
72
+ ## The Flight Plan
73
+
74
+ Each record the join holds goes through this:
75
+
76
+ ```mermaid
77
+ stateDiagram-v2
78
+ [*] --> refused: insert → :mismatch, :pending, :late, :full
79
+ [*] --> held: insert → :accepted
80
+ held --> held: insert of the same record → :duplicate
81
+ held --> held: pairs with it are drained
82
+ held --> retirable: opposite watermark passes its reach and its own watermark reaches it
83
+ retirable --> [*]: retire, returns a Retired
84
+ ```
85
+
86
+ And the join as a whole only ever owes pairs for one record at a time:
87
+
88
+ ```mermaid
89
+ sequenceDiagram
90
+ participant C as Caller
91
+ participant J as Join
92
+
93
+ C->>J: insert(:right, Record(1, key 9, t=8))
94
+ J-->>C: :accepted (nothing to match)
95
+ C->>J: insert(:right, Record(2, key 9, t=15))
96
+ J-->>C: :accepted
97
+ C->>J: insert(:left, Record(100, key 9, t=10))
98
+ J-->>C: :accepted, 2 pairs owed
99
+ C->>J: insert(:left, Record(101, ...))
100
+ J-->>C: :pending
101
+ C->>J: drain(1)
102
+ J-->>C: [Pair(9, 100, 1)]
103
+ C->>J: drain(1)
104
+ J-->>C: [Pair(9, 100, 2)]
105
+ C->>J: advance_watermark(:right, 15)
106
+ C->>J: retire(10)
107
+ J-->>C: [] (the left watermark has not reached 100)
108
+ C->>J: advance_watermark(:left, 10)
109
+ C->>J: retire(10)
110
+ J-->>C: [Retired(:left, Record(100, 9, 10)), Retired(:right, Record(1, 9, 8))]
111
+ ```
112
+
113
+ ## How the Join Works
114
+
115
+ ### Bounds
116
+
117
+ ```ruby
118
+ bounds = JoinMachines::Bounds.new(before: 2, after: 5) # or Bounds.new(2, 5)
119
+ join = JoinMachines::Join.new(bounds)
120
+ join.bounds # => #<data JoinMachines::Bounds before=2, after=5>
121
+ bounds.reach(:left, 10) # => #<data JoinMachines::Reach first=8, last=15>
122
+ bounds.reach(:right, 10) # => #<data JoinMachines::Reach first=5, last=12>
123
+ bounds.reach(:left, 10).include?(15) # => true
124
+ JoinMachines.other(:left) # => :right
125
+ ```
126
+
127
+ A left record at time `t` matches right records with the same key at times `t - 2` through `t + 5`, both ends included. `Bounds.new(0, 0)` is an exact-time join. `Join.new` takes the `Bounds` value, the way the core's constructor does; the bounds are fixed for the life of the join. `reach(side, timestamp)` is the range of opposite-side times a record on `side` can match.
128
+
129
+ ### Insert
130
+
131
+ ```ruby
132
+ Record = JoinMachines::Record # id, key, timestamp
133
+
134
+ join.insert(:right, Record.new(1, 9, 8)) # => :accepted
135
+ join.insert(:right, Record.new(2, 9, 15)) # => :accepted
136
+ join.insert(:right, Record.new(3, 9, 16)) # => :accepted
137
+ join.insert(:left, Record.new(100, 9, 10)) # => :accepted
138
+ join.pending? # => true
139
+ join.pending_pairs # => 2
140
+ ```
141
+
142
+ `insert(side, record)` takes a `Record`, the same value `retire` hands back. An accepted record is held, and the join now owes one pair for every held record on the other side with the same key inside the interval. The ids are yours. They only have to be unique among the records currently held on the same side.
143
+
144
+ Replaying a record the join still holds is fine. The same id with the same key and timestamp answers `:duplicate`: honoured, nothing changes, and no pair is owed a second time. The same id with a different key or timestamp answers `:mismatch`, a refusal. A held id is answered this way before anything else is looked at, so a replay is `:duplicate` even while pairs are owed or after the side's watermark has passed the record.
145
+
146
+ ```ruby
147
+ join.insert(:left, Record.new(100, 9, 10)) # => :duplicate, already held, still owes 2 pairs
148
+ join.insert(:left, Record.new(100, 9, 11)) # => :mismatch, id 100 is held with timestamp 10
149
+ ```
150
+
151
+ Held means held right now. A record that matched and was drained is still held. A record that was retired is not, but it cannot come back: it was let go only after its own side's watermark had reached it, and that watermark never moves back, so the same record replayed is `:late`, every time, and no pair is handed over twice. The join does this without keeping any history, which also means it does not remember the id itself: a *different* record under a retired id is judged like any new one. Keep one id for one record and replays across retirement are safe.
152
+
153
+ ### Drain
154
+
155
+ ```ruby
156
+ join.drain(1) # => [#<data JoinMachines::Pair key=9, left=100, right=1>]
157
+ join.pending? # => true
158
+ join.drain(8) # => [#<data JoinMachines::Pair key=9, left=100, right=2>]
159
+ join.pending? # => false
160
+ join.drain(8) # => []
161
+ ```
162
+
163
+ `drain(max)` hands over at most `max` owed pairs. Whatever didn't fit stays owed, in order, and the next call picks up exactly where this one stopped. Every pair comes out once. While pairs are owed, `insert` and `retire` answer `:pending`:
164
+
165
+ ```ruby
166
+ join.insert(:left, Record.new(101, 9, 14)) # => :accepted, owes pairs with 2 and 3
167
+ join.insert(:left, Record.new(102, 9, 14)) # => :pending
168
+ join.drain(8).size # => 2
169
+ ```
170
+
171
+ ### Watermarks
172
+
173
+ ```ruby
174
+ join.advance_watermark(:right, 14) # => :advanced
175
+ join.advance_watermark(:right, 14) # => :repeated, already there, nothing changed
176
+ join.advance_watermark(:right, 13) # => :watermark_regressed, time went backwards
177
+ join.watermark(:right) # => 14
178
+ join.watermark(:left) # => nil
179
+ join.insert(:right, Record.new(4, 9, 14)) # => :late
180
+ join.insert(:right, Record.new(4, 9, 15)) # => :accepted
181
+ join.drain(8)
182
+ ```
183
+
184
+ Each side has its own watermark: your promise that no more records at or below that time will arrive on that side. The join holds you to it. A record at or below its own side's watermark is `:late` and is not held. The *other* side's watermark never makes a record late: if its partner is still held, the record is accepted and the pair is owed, however far apart the two watermarks are. Advancing a watermark removes nothing by itself. Repeating the value a side already holds is not a refusal: the promise you are making already stands, so the answer is `:repeated`. Only a value behind the side's watermark is refused, with `:watermark_regressed`.
185
+
186
+ If you already track a stream's progress in a watermark object, hand it over instead of unwrapping it:
187
+
188
+ ```ruby
189
+ clicks = WindowMachines::Watermark.new # from the window_machines gem, if you use it
190
+ clicks.advance(14) # => :advanced
191
+ join.advance_watermark_from(:right, clicks) # => :repeated, the side already agrees
192
+ join.watermark(:right) # => 14
193
+ join.side_watermark(:right) # => #<data JoinMachines::Watermark value=14>
194
+ join.side_watermark(:left) # => #<data JoinMachines::Watermark value=nil>
195
+ ```
196
+
197
+ `advance_watermark_from(side, watermark)` reads `watermark.value` and answers like `advance_watermark`: `:advanced` when the side moved up to it, `:repeated` when the side already equals it, `:watermark_regressed` and no change when it is behind. Any object with a `value` reader returning an Integer or `nil` works; this gem does not depend on `window_machines`. A `WindowMachines::Watermark` fits as it is, whether you advanced it yourself, built it with `Watermark.at(saved)`, or took the copy `Sessions#watermark` answers; only its `value` is read, and it is never advanced for you. A `nil` value means nothing was observed yet: `:repeated` while the side is `nil` too, `:watermark_regressed` once the side has a value. An object without `value` raises `TypeError`. This gem's own `Watermark` is read natively instead, and a subclass of it is not taken for a sibling's: it raises `TypeError`. A `value` that freezes the join raises `FrozenError` and nothing is advanced.
198
+
199
+ `side_watermark(side)` is the other direction: the side's progress as a `JoinMachines::Watermark`, a frozen `Data` with the one reader `value` (an Integer, or `nil` before the first watermark). That is exactly the shape `advance_watermark_from` reads, so one join's side can be handed to another join. It is a copy taken at the call, not a live view, and it does not advance; `watermark(side)` is the same number without the wrapper.
200
+
201
+ ### Retire
202
+
203
+ ```ruby
204
+ join.retirable # => 0
205
+ join.advance_watermark(:right, 15) # left 100 reached right times 8..15
206
+ join.retirable # => 0, a replay of left 100 would not be late yet
207
+ join.advance_watermark(:left, 10) # now it would be; right 1 reached left times 3..10
208
+ join.retirable # => 2
209
+ join.retire(10)
210
+ # => [#<data JoinMachines::Retired side=:left,
211
+ # record=#<data JoinMachines::Record id=100, key=9, timestamp=10>>,
212
+ # #<data JoinMachines::Retired side=:right,
213
+ # record=#<data JoinMachines::Record id=1, key=9, timestamp=8>>]
214
+ ```
215
+
216
+ A held record is finished when two things are true. The *other* side's watermark has reached the last time it could match (`timestamp + after` for a left record, `timestamp + before` for a right one, stopping at `2**64-1`), so nothing that can still arrive pairs with it. And its *own* side's watermark has reached its timestamp, so the same record offered again is `:late`. The first makes dropping it lossless, the second makes it final. A record whose own side lags stays held until that side catches up. `retire(max)` removes up to `max` finished records and tells you which, so you can drop their payloads: each `Retired` is the `side` and the `Record` (`id`, `key`, `timestamp`) that was held. Anything over `max` stays `retirable` for the next call.
217
+
218
+ ### Capacity
219
+
220
+ ```ruby
221
+ JoinMachines::CAPACITY # => 64
222
+ join.capacity(:left) # => 64
223
+ join.capacity(:right) # => 64
224
+ join.size(:left) # => 1
225
+ ```
226
+
227
+ Each side holds 64 records. Past that, `insert` says `:full`. A record is never evicted to make room, because evicting it would silently lose its future pairs. Advance the watermarks, retire, and try again, or shard by key across joins.
228
+
229
+ ## The Rules of the Tender
230
+
231
+ - **Ids, keys, times, bounds, and limits are Integers** in `0..2**64-1`. Floats, nil, and strings raise `TypeError`. Out of range raises `RangeError`. A side that isn't `:left` or `:right` raises `RangeError` (or `TypeError` if it isn't a Symbol).
232
+ - **Refusals don't raise.** `:mismatch`, `:pending`, `:late`, `:full` and `:watermark_regressed` are answers, each with its own symbol and no catch-all, and a refused call changes nothing. Only malformed input raises.
233
+ - **Every answer, by method.**
234
+
235
+ | Method | Honoured | Refused | Records the clock |
236
+ | --- | --- | --- | --- |
237
+ | `insert(side, record)` | `:accepted`, or `:duplicate` when exactly that record is already held | `:mismatch`, `:pending`, `:late`, `:full` | no clock, nothing recorded |
238
+ | `drain(max)` | an Array of `Pair`, `[]` when none is owed | never refuses | no clock, nothing recorded |
239
+ | `advance_watermark(side, value)` | `:advanced`, or `:repeated` when the side already held `value` | `:watermark_regressed` | no clock; `:advanced` records the side's watermark, `:repeated` nothing |
240
+ | `advance_watermark_from(side, watermark)` | `:advanced`, or `:repeated` when the side already equals it | `:watermark_regressed` | no clock; `:advanced` records the side's watermark, `:repeated` nothing |
241
+ | `retire(max)` | an Array of `Retired`, `[]` when none is retirable | `:pending` | no clock, nothing recorded |
242
+ | `Reach#include?(timestamp)` | `true` or `false` | never refuses | no clock, nothing recorded |
243
+ - **Arguments are the core's values.** `Join.new` takes a `Bounds`, `insert` takes a `Record`. A Hash, an Array or loose Integers in their place raise `TypeError` (a wrong argument count is Ruby's own `ArgumentError`).
244
+ - **Values are valid or do not exist.** `Reach`, `Bounds`, `Pair`, `Record`, `Retired` and `Watermark` are frozen `Data` checked by the core when built, by hand or by the gem, `with` included: a wrong type raises `TypeError`, an Integer outside `0..2**64-1` or a side other than `:left`/`:right` raises `RangeError`, and `Reach.new(5, 3)` raises `RangeError` (a reach holds at least one timestamp). `Retired` holds a `Record` and nothing else. Every value class is sealed: `allocate` is gone and subclassing raises `TypeError`. Sealing stops accidents, not reflection (a named `class X < Record` still binds its constant, `Data#initialize` can be bound directly, `Marshal.load` skips `initialize`), so the native side is the authority: `Join.new`, `insert`, `Retired.new`, `Bounds#reach`, `Reach#include?` and `advance_watermark_from` (for this gem's own `Watermark`) accept only an instance of exactly the class, read its members without calling any method, and check them through the core again: `TypeError` for anything else, `RangeError` for an invalid member. The classes are resolved once when the gem loads; swapping a constant later changes nothing the gem builds or accepts.
245
+ - **A call that raises changed nothing.** `drain` and `retire` build their whole answer before the join keeps the change, so if building it raises every pair is still owed and every record still held. If building it changed the join (a redefined `Pair.new` or `Retired.new` calling back in), the outer call raises `RuntimeError`, is not applied, and that inner change stands. If building it froze the join, the call raises `FrozenError` and is not applied.
246
+ - **No clock is recorded.** Everything here is event time you pass in; nothing reads or records a monotonic clock. The only progress kept is each side's watermark, recorded on `:advanced` and left alone on `:repeated` or `:watermark_regressed`. `advance_watermark_from` reads a watermark (a `WindowMachines::Watermark` or `side_watermark`'s answer) and keeps only its value.
247
+ - **Repeats are honoured only while provable.** `:duplicate` means the join holds that exact record on that side right now, and changes nothing. It is never a refusal. A retired record leaves no trace, so a replay of it is not `:duplicate`; it is `:late`, because its own side's watermark had reached it before it was let go and never moves back.
248
+ - **One record's pairs at a time.** Drain what the last insert owes before inserting or retiring.
249
+ - **Late is judged on one watermark. Retirement is judged on both.** A record is `:late` at or below its own side's watermark and for no other reason, so a record whose partner is held is never turned away because of the other input. `retire` releases a held record at `t` only when the other side's watermark is at or past the end of its reach (`>= bounds.reach(side, t).last`: nothing still to come can match it) *and* its own side's watermark is at or past `t` (a replay is `:late`). So nothing matchable is ever refused, a retired record replayed is `:late` for good, and every pair is handed over exactly once.
250
+ - **A lagging side keeps its records.** A side whose own watermark is behind holds its records longer, however far the other side has run, and a side whose watermark never moves retires nothing. That costs slots, never pairs: the bound is the capacity, and past it `insert` says `:full`. Advance both watermarks to release records.
251
+ - **Time saturates at the edges.** A reach that would run below 0 or past `2**64-1` stops there.
252
+ - **Frozen means frozen.** `freeze` the join and `insert`, `drain`, `advance_watermark`, `advance_watermark_from`, and `retire` raise `FrozenError`.
253
+
254
+ ## What You Think You're Doing vs Reality
255
+
256
+ ### You Think: "I'll expire old records after the window has passed."
257
+ ### Reality: Passed according to whom?
258
+
259
+ Your wall clock says thirty seconds went by. The click stream, three partitions behind, has not been told. A record is finished when the stream it is waiting on says it has moved on, and the only thing that can say that is that stream's watermark. `retire` asks that, and asks the record's own stream whether it could still be delivered again.
260
+
261
+ ### You Think: "A record matches, I emit a pair."
262
+ ### Reality: A record matches four hundred, and your output buffer holds fifty.
263
+
264
+ A join's output per input is not bounded by one. `drain(max)` makes the batch size your decision, and keeps the remainder owed instead of dropped or buffered somewhere you can't see.
265
+
266
+ ## What's Yours
267
+
268
+ The payloads. The join stores three Integers per record and nothing else, so you keep a store keyed by side and id and delete from it when `retire` says so. Hashing a wide key down to an Integer is also yours, and so is checking the real key on a pair if your hash can collide.
269
+
270
+ The watermarks. The join believes them. Advance one too eagerly and that side's own records come back `:late`; the other side's never do. Never advance one and both sides fill up and stay `:full`.
271
+
272
+ Everything else a stream processor does: reading inputs, running the user's function on a pair, checkpointing, restoring after a crash. The join lives in your process and forgets everything when the process does. Pairs you drained before a crash are not remembered as drained by anyone but you.
273
+
274
+ ## License
275
+
276
+ MIT License. See [LICENSE.txt](LICENSE.txt) for details.
277
+
278
+ **Remember: the record you swept on a timer was thirty seconds old. Its match was thirty-one seconds late.**
@@ -0,0 +1,66 @@
1
+ # Changelog
2
+
3
+ ## [0.7.1](https://github.com/seuros/join_machines/compare/join-machines-v0.7.0...join-machines-v0.7.1) (2026-11-16)
4
+
5
+
6
+ ### Bug Fixes
7
+
8
+ * validate values from their stored members, not deconstruct ([7af4e49](https://github.com/seuros/join_machines/commit/7af4e498acb6f9e48b3b323bf0fb0ccc57331ab3))
9
+
10
+ ## [0.7.0](https://github.com/seuros/join_machines/compare/join-machines-v0.6.0...join-machines-v0.7.0) (2026-11-14)
11
+
12
+
13
+ ### ⚠ BREAKING CHANGES
14
+
15
+ * adopt matryona-machines 0.2 (exact_fields, symbols!, frozen-first)
16
+
17
+ ### Features
18
+
19
+ * adopt matryona-machines 0.2 (exact_fields, symbols!, frozen-first) ([84d94d2](https://github.com/seuros/join_machines/commit/84d94d2fc841d1dcbc3d0cdd8da938dcd3547298))
20
+
21
+ ## [0.6.0](https://github.com/seuros/join_machines/compare/join-machines-v0.5.0...join-machines-v0.6.0) (2026-11-14)
22
+
23
+
24
+ ### ⚠ BREAKING CHANGES
25
+
26
+ * seal values, native Bounds#reach, exact-class reads, re-check frozen before keeping
27
+
28
+ ### Features
29
+
30
+ * seal values, native Bounds#reach, exact-class reads, re-check frozen before keeping ([3432d10](https://github.com/seuros/join_machines/commit/3432d1010e389333e15a2adf9dd702f0f7adf64d))
31
+
32
+ ## [0.5.0](https://github.com/seuros/join_machines/compare/join-machines-v0.4.0...join-machines-v0.5.0) (2026-11-13)
33
+
34
+
35
+ ### ⚠ BREAKING CHANGES
36
+
37
+ * converted to the fleet conventions. One exhaustive Refusal per crate, Result from every operation that can refuse, a refusal changes nothing, provable idempotent repeats are honoured, Ruby mirrors the core. No compatibility aliases. Requires window-machines >= 0.3 and re-exports its Advancing. Refusal::Duplicate is gone; a retired record is Late on replay.
38
+
39
+ ### Features
40
+
41
+ * honour identical re-inserts as Duplicate, refuse changed ones as Mismatch, retire only when both watermarks have passed ([e06e9fe](https://github.com/seuros/join_machines/commit/e06e9fe31e3e93b80dd1fa99bfb5bc60839b2bd2))
42
+
43
+ ## [0.4.0](https://github.com/seuros/join_machines/compare/join-machines-v0.3.0...join-machines-v0.4.0) (2026-11-13)
44
+
45
+
46
+ ### ⚠ BREAKING CHANGES
47
+
48
+ * see CONVENTIONS: one exhaustive Refusal per crate, Result from every mutator, honoured idempotent repeats, Ruby mirrors the core. No compatibility aliases. advance_watermark and advance_watermark_from return Result<Advancing, Refusal>.
49
+
50
+ ### Features
51
+
52
+ * watermark advances answer Advancing or WatermarkRegressed, Ruby takes Bounds and Record ([eb4f557](https://github.com/seuros/join_machines/commit/eb4f557d1449c4791204000ca818b4a7d8bdec55))
53
+
54
+ ## [0.3.0](https://github.com/seuros/join_machines/compare/join-machines-v0.2.0...join-machines-v0.3.0) (2026-11-13)
55
+
56
+
57
+ ### Features
58
+
59
+ * expose the window-machines Watermark seam and default output elements ([51df744](https://github.com/seuros/join_machines/commit/51df744004344e4679adf8ccfe308ad68337eca8))
60
+
61
+ ## [0.2.0](https://github.com/seuros/join_machines/compare/join-machines-v0.1.0...join-machines-v0.2.0) (2026-11-12)
62
+
63
+
64
+ ### Features
65
+
66
+ * initial extraction ([a045343](https://github.com/seuros/join_machines/commit/a04534332b17bd40113de9bdcb41c6a24c6eefa6))
@@ -0,0 +1,17 @@
1
+ [package]
2
+ name = "join-machines"
3
+ version = "0.7.1"
4
+ edition.workspace = true
5
+ rust-version.workspace = true
6
+ license.workspace = true
7
+ description = "Bounded two-input interval-join metadata with resumable pair emission and watermark retirement"
8
+ readme = "README.md"
9
+ exclude = ["src/**/tests.rs", "src/tests/", "tests/"]
10
+ keywords = ["join", "event-time", "watermark", "streaming", "no-std"]
11
+ categories = ["no-std", "algorithms"]
12
+ [dependencies]
13
+ window-machines.workspace = true
14
+ [dev-dependencies]
15
+ matryona-machines = { workspace = true, features = ["testing"] }
16
+ [lints]
17
+ workspace = true
@@ -0,0 +1,176 @@
1
+ # join-machines
2
+
3
+ > **AMS Widom**, NCC-4232, *Widom*-class stream tender.
4
+
5
+ Engine-room flight manual: log which hulls on two lanes passed abeam of each other.
6
+
7
+ Allocation-free `no_std` two-input interval join over record metadata. Each
8
+ record is an identity, a `u64` key and an event time; a left record at `t`
9
+ matches right records with the same key at `t - before ..= t + after`. The join
10
+ holds at most `L` left and `R` right records and stores no payloads.
11
+
12
+ Inserting a record owes one pair per matching held opposite record. `drain`
13
+ writes owed pairs into a caller buffer of any size and resumes where it stopped,
14
+ so a full buffer never loses or repeats a pair. While pairs are owed, `insert`
15
+ and `retire` are refused with `Pending`; that is the only backpressure the core
16
+ needs to stay bounded.
17
+
18
+ Each input has its own `window_machines::Watermark`. A record at or below its
19
+ own side's watermark is `Late`; the opposite watermark never makes a record
20
+ late. A held record becomes retirable when both watermarks have finished it:
21
+ the opposite one has reached the end of its reach, the last time it can match,
22
+ and its own has reached its timestamp. `retire` removes such records into a
23
+ bounded buffer so the caller can release their payloads. So a record whose
24
+ partner is still held is always admitted and paired, and a retired record
25
+ offered again is `Late`. Reaches saturate at the `u64` edges. The exact
26
+ inequalities are in the Contract below.
27
+
28
+ Refusals (`Mismatch`, `Pending`, `Late`, `Full`, `WatermarkRegressed`) are one
29
+ `Refusal` enum and change nothing. A full side is never evicted.
30
+
31
+ `insert` returns `Result<Inserting, Refusal>`. A record whose identity is held
32
+ on that side is answered from the held record, before any other check:
33
+
34
+ | `insert(side, record)` | Answer |
35
+ | --- | --- |
36
+ | identity not held, admitted | `Ok(Inserting::Accepted)`: held, its pairs owed |
37
+ | identity held with the same key and timestamp | `Ok(Inserting::Duplicate)`: nothing changes, no pair is owed twice |
38
+ | identity held with another key or timestamp | `Err(Refusal::Mismatch)` |
39
+ | identity not held, pairs still owed | `Err(Refusal::Pending)` |
40
+ | identity not held, at or below its own side's watermark | `Err(Refusal::Late)` |
41
+ | identity not held, no free slot | `Err(Refusal::Full)` |
42
+
43
+ Held means live right now. A record that has matched and been drained is still
44
+ held, so replaying it is `Duplicate`, also while its pairs are still owed and
45
+ after its own watermark has passed it. A retired record is no longer held, and
46
+ the join keeps no history of identities, yet the same record can never be
47
+ admitted again: it was retired only once its own watermark had reached it, and
48
+ that watermark never moves back, so a replay is `Late`. The identity itself is
49
+ not remembered: other contents under it are judged as any new record is
50
+ (`Accepted` if they are above the side's watermark, never `Mismatch`).
51
+
52
+ `advance_watermark` returns `Result<Advancing, Refusal>`: `Advancing::Advanced`
53
+ when the side moved forward, `Advancing::Repeated` when it already held exactly
54
+ that value (an idempotent repeat that changes nothing), and
55
+ `Err(Refusal::WatermarkRegressed)` for a value behind it. `Advancing` is
56
+ `window_machines::Advancing`, re-exported. The join asks the side's
57
+ `Watermark` where it stands and then sets it with `Watermark::at`, so it never
58
+ has to translate sibling refusals (`Overflow`, `Undersized`, `Late`, `Full`)
59
+ that `Watermark::advance` can never give.
60
+
61
+ The sibling seam is public: `Watermark` and `Advancing` are re-exported, `side_watermark(side)`
62
+ returns a side's `Watermark` by value (next to `watermark(side)`, which still
63
+ reads the bare `Option<u64>`), and `advance_watermark_from(side, &Watermark)`
64
+ advances a side from a watermark the caller owns, with the same answers as
65
+ `advance_watermark`. A source watermark with no value yet is
66
+ `Ok(Advancing::Repeated)` while the side is also uninitialized (they already
67
+ agree) and `Err(Refusal::WatermarkRegressed)` once the side has a value; it
68
+ never clears a side.
69
+
70
+ `Reach` keeps its fields private and is valid or does not exist:
71
+ `Reach::new(first, last)` is `None` unless `first <= last`, and `Bounds::reach`
72
+ always answers one (it saturates at both edges). `Record`, `Pair`, `Retired`
73
+ and `Bounds` hold any value of their field types, so their fields stay public
74
+ (`Bounds` behind its infallible `new`).
75
+
76
+ `Pair` and `Retired` implement `Default` so an output buffer is
77
+ `[Pair::default(); N]`. The default is a placeholder, not a
78
+ sentinel: a real zero-valued output equals it, so read only the entries reported.
79
+
80
+ The Ruby gem mirrors `Bounds`, `Reach`, `Record`, `Pair` and `Retired` as
81
+ `Data` values of the same names and fields, checked by the core when built, and takes `Bounds` and `Record`
82
+ where this core does. `side_watermark` answers a `Data` with the one reader
83
+ `value`, which is what its `advance_watermark_from` reads.
84
+
85
+ This is join bookkeeping only: no payload buffering, transport, execution,
86
+ persistence, key hashing or clocks. Key collisions, watermark generation and
87
+ recovery belong to the caller. Pre-1.0: breaking changes land in minor versions.
88
+
89
+ ```rust
90
+ use join_machines::{
91
+ Advancing, Bounds, Inserting, Join, Pair, Record, Refusal, Side, Watermark,
92
+ };
93
+ let mut join = Join::<4, 4>::new(Bounds::new(2, 5));
94
+ let record = |id, timestamp| Record { id, key: 9, timestamp };
95
+ assert_eq!(join.insert(Side::Right, record(1, 8)), Ok(Inserting::Accepted));
96
+ assert_eq!(join.insert(Side::Right, record(2, 15)), Ok(Inserting::Accepted));
97
+ assert_eq!(join.insert(Side::Left, record(100, 10)), Ok(Inserting::Accepted));
98
+ assert_eq!(join.insert(Side::Left, record(101, 10)), Err(Refusal::Pending));
99
+ // A replay of a held record is honoured; the same id with other contents is not.
100
+ assert_eq!(join.insert(Side::Left, record(100, 10)), Ok(Inserting::Duplicate));
101
+ assert_eq!(join.insert(Side::Left, record(100, 11)), Err(Refusal::Mismatch));
102
+
103
+ let mut out = [Pair::default(); 1];
104
+ assert_eq!(join.drain(&mut out), 1);
105
+ assert_eq!(out[0], Pair { key: 9, left: 100, right: 1 });
106
+ assert_eq!(join.drain(&mut out), 1);
107
+ assert_eq!(out[0].right, 2);
108
+ assert!(!join.pending());
109
+
110
+ // The right input's progress is tracked by a watermark the caller owns.
111
+ let mut right = Watermark::default();
112
+ assert_eq!(right.advance(15), Ok(Advancing::Advanced));
113
+ assert_eq!(join.advance_watermark_from(Side::Right, &right), Ok(Advancing::Advanced));
114
+ assert_eq!(join.advance_watermark(Side::Right, 15), Ok(Advancing::Repeated));
115
+ assert_eq!(join.advance_watermark(Side::Right, 14), Err(Refusal::WatermarkRegressed));
116
+ assert_eq!(join.side_watermark(Side::Right), right);
117
+ assert_eq!(join.watermark(Side::Right), Some(15));
118
+ // Right 15 ends left 100 (it reaches 8..=15), but a replay of it would not be
119
+ // late yet: it stays until the left watermark reaches it too.
120
+ assert_eq!(join.retirable(), 0);
121
+ assert_eq!(join.advance_watermark(Side::Left, 10), Ok(Advancing::Advanced));
122
+ // Left 100 goes, and so does right 1: it reaches left times 3..=10.
123
+ assert_eq!(join.retirable(), 2);
124
+ ```
125
+
126
+ ## Contract
127
+
128
+ Write `wl` and `wr` for the left and right watermarks (unset until first
129
+ advanced) and `⊕` for addition saturating at `u64::MAX`.
130
+
131
+ - **Match.** Left `l` and right `r` match when the keys are equal and
132
+ `l.timestamp - before <= r.timestamp <= l.timestamp + after`.
133
+ - **Late.** `insert` of a record at `t` that is not held answers `Late` when
134
+ its own side's watermark is set and `t <= w_own`, and on no other count. The
135
+ opposite watermark never refuses a record. Watermarks never move back, so a
136
+ record that is `Late` once is `Late` for good.
137
+ - **Nothing matchable is refused.** A record whose partner is held is admitted
138
+ and paired however far the two watermarks have drifted apart, unless its own
139
+ side has declared it late (or the side is `Full`, or pairs are `Pending`).
140
+ - **Retire.** A held record at `t` is retirable when both hold:
141
+ 1. *Nothing can match it any more.* The opposite watermark has reached the
142
+ end of its reach: `wr >= t ⊕ after` for a left record, `wl >= t ⊕ before`
143
+ for a right one. An opposite record is admitted only strictly above its
144
+ own watermark, so every one still to come lies past the reach. The
145
+ comparison is inclusive for that reason, and exact: one short, a record
146
+ at the end of the reach is still admitted and matches. A reach that
147
+ saturates is reached only by a watermark of `u64::MAX`, where no opposite
148
+ record can be admitted at all.
149
+ 2. *A replay of it is late.* Its own watermark has reached it: `wl >= t`
150
+ for a left record, `wr >= t` for a right one. This is `insert`'s `Late`
151
+ test itself, so the two cannot disagree.
152
+
153
+ `retire` and `retirable` use this one test. With `Bounds::new(0, 0)` a
154
+ record at `t` is retirable once both watermarks are at or past `t`.
155
+ - **A lagging side keeps its records.** A record is held until its *own*
156
+ side's watermark reaches it, whatever the opposite one says: a side whose
157
+ watermark is never advanced never retires anything. Memory stays bounded all
158
+ the same: the side fills and `insert` answers `Full`. Advance both
159
+ watermarks to release records.
160
+ - **Exactly once, and complete.** As long as an identity names one record
161
+ (same side, key and timestamp) for the life of the join: no `(left, right)`
162
+ pair is drained twice, because a pair is owed only when one of its records
163
+ is `Accepted`, a held record replayed is `Duplicate`, and a retired one is
164
+ `Late`; and every two accepted records that match are drained as a pair,
165
+ because the earlier one is still held when the later one is accepted
166
+ (rule 1 kept it).
167
+ - **No clock.** Every time here is event time; the core reads and records no
168
+ monotonic clock. An honoured `advance_watermark`/`advance_watermark_from`
169
+ records the side's watermark (progress, not a clock) on `Advanced` and
170
+ nothing on `Repeated`.
171
+ - **No unreachable arms.** No `panic!`, `assert!`, `unreachable!` or `expect`
172
+ outside tests. The owed pair carries the opposite record it names, so
173
+ `drain` never looks a slot up again.
174
+ - **Refusals change nothing**, and none is remembered: each is an answer about
175
+ the join as it is now. `Late` stays `Late` only because watermarks, which
176
+ honoured calls record, never regress.