merkle_machines 0.6.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md ADDED
@@ -0,0 +1,326 @@
1
+ # MerkleMachines
2
+
3
+ > Two replicas disagree. Find out where before you ship the whole dataset. 🧭
4
+
5
+ A divergence survey for Ruby. You have two digest trees of the same shape, one per replica, and their roots differ. The survey tells you which nodes to fetch child digests for, compares what you bring back, and hands you the ranges of leaves that actually differ. It holds a fixed number of nodes and a fixed number of ranges, so a badly diverged pair slows the survey down instead of growing it.
6
+
7
+ ## Quick Start
8
+
9
+ ```bash
10
+ gem 'merkle_machines'
11
+ ```
12
+
13
+ ```ruby
14
+ schema = MerkleMachines::Schema.new(fanout: 16, depth: 3, partitioning: 1, digest: 1)
15
+ versions = MerkleMachines::Versions.new(local: mine.version, remote: theirs.version)
16
+ # One scope per survey whose requests could meet: two Survey.new are both
17
+ # scope zero, open the same sessions, and honour each other's requests.
18
+ survey = MerkleMachines::Survey.with_scope(1)
19
+
20
+ case survey.begin(schema, theirs.schema, versions, mine.root, theirs.root)
21
+ when :identical then return
22
+ when :diverged then nil
23
+ else raise 'trees are not comparable'
24
+ end
25
+
26
+ until survey.phase == :complete
27
+ if (request = survey.next_request)
28
+ survey.answer(request, versions,
29
+ mine.children(request.node.level, request.node.index),
30
+ theirs.children(request.node.level, request.node.index))
31
+ end
32
+ survey.drain(MerkleMachines::OUTPUT_CAPACITY).each { |leaves| repair(leaves) }
33
+ end
34
+ ```
35
+
36
+ One survey, a handful of methods. Digests are Strings you computed. Everything else is an Integer.
37
+
38
+ ## A Message to the Resistance
39
+
40
+ So your two replicas drifted. A write landed on one during a partition, or a repair job died halfway, and now the row counts match and the checksums don't.
41
+
42
+ You know the textbook answer. "Use a Merkle tree." Hash the leaves, hash the hashes, compare the roots, walk down where they differ. Then you write the walk. It recurses, so a pair of replicas that disagree everywhere allocates a frontier the size of the tree. It talks to the other node from inside the recursion, so a dropped connection loses your place. Someone compacts the remote tree halfway through and you spend an afternoon repairing leaves that were never different. And one day a replica is rebuilt with sixteen children per node instead of eight, the walk compares child 3 with child 3 anyway, and every leaf looks wrong.
43
+
44
+ The walk is a small state machine: which nodes still need their children fetched, which requests are out, which leaf ranges are ready. With a refusal for trees that can't be compared, and a refusal for a tree that moved. That machine is this gem.
45
+
46
+ ## Chapter 1: Stardate 2026.316, the AMS Merkle
47
+
48
+ > **AMS Merkle**, NCC-4224, *Merkle*-class survey cutter.
49
+ > Two charts of the same sector. Mark where they disagree, and nothing else.
50
+
51
+ The cutter doesn't draw charts and doesn't correct them. Two ships surveyed the same sector and came back with two charts. The cutter lays them side by side, quadrant by quadrant, and asks for a closer look only where the summaries differ. It carries a small plotting table, so it asks for a few quadrants at a time and files its findings as it goes.
52
+
53
+ On a bridge somewhere, a cadet volunteers:
54
+
55
+ ```ruby
56
+ def diff(node)
57
+ return [node.leaf] if node.leaf?
58
+ node.children.zip(remote.children(node)).flat_map { |a, b| a.digest == b.digest ? [] : diff(a) }
59
+ end
60
+ ```
61
+
62
+ "That," says the cutter's navigator, "is a network call inside a recursion inside a hope that nobody edits the other chart."
63
+
64
+ ## The Flight Plan
65
+
66
+ A survey moves through four phases:
67
+
68
+ ```mermaid
69
+ stateDiagram-v2
70
+ [*] --> idle
71
+ idle --> complete: begin, roots equal
72
+ idle --> surveying: begin, roots differ
73
+ surveying --> surveying: next_request / answer / requeue
74
+ surveying --> draining: last node resolved
75
+ draining --> complete: drain empties the buffer
76
+ surveying --> complete: last node resolved, buffer already empty
77
+ complete --> surveying: begin again
78
+ surveying --> idle: abort
79
+ draining --> idle: abort
80
+ ```
81
+
82
+ And one exchange, over a tree of four leaves where only leaf 2 differs:
83
+
84
+ ```mermaid
85
+ sequenceDiagram
86
+ participant A as Your code
87
+ participant S as Survey
88
+ participant R as Remote replica
89
+
90
+ A->>S: begin(schema, schema, versions, root, root)
91
+ S-->>A: :diverged
92
+ A->>S: next_request
93
+ S-->>A: Request(level 0, index 0)
94
+ A->>R: children of the root, at version 57
95
+ R-->>A: two digests
96
+ A->>S: answer(request, versions, mine, theirs)
97
+ S-->>A: :accepted
98
+ A->>S: next_request
99
+ S-->>A: Request(level 1, index 1)
100
+ A->>R: children of node 1, at version 57
101
+ R-->>A: two digests
102
+ A->>S: answer(request, versions, mine, theirs)
103
+ S-->>A: :accepted
104
+ A->>S: drain(8)
105
+ S-->>A: [2...3]
106
+ ```
107
+
108
+ ## How the Survey Works
109
+
110
+ ### Schema
111
+
112
+ ```ruby
113
+ schema = MerkleMachines::Schema.new(fanout: 2, depth: 2, partitioning: 1, digest: 1)
114
+ ```
115
+
116
+ `fanout` is children per node, from 2 to 64. `depth` is the number of levels below the root, so this tree has four leaves. `partitioning` and `digest` are numbers you choose to name how keys map to leaves and how digests are computed. The survey never interprets them. It only checks that both sides say the same thing.
117
+
118
+ A shape that is not a tree is not a `Schema`. A fanout outside 2..64, or more leaves than fit in 64 bits, raises `RangeError` from `Schema.new`:
119
+
120
+ ```ruby
121
+ MerkleMachines::Schema.new(fanout: 1, depth: 2, partitioning: 1, digest: 1) # RangeError
122
+ MerkleMachines::Schema.new(fanout: 2, depth: 64, partitioning: 1, digest: 1) # RangeError
123
+ ```
124
+
125
+ ### Begin
126
+
127
+ ```ruby
128
+ versions = MerkleMachines::Versions.new(local: 40, remote: 57)
129
+ survey = MerkleMachines::Survey.new
130
+
131
+ survey.begin(schema, schema, versions, 'root-a', 'root-b') # => :diverged
132
+ survey.phase # => :surveying
133
+ ```
134
+
135
+ A schema answers the questions the core's does:
136
+
137
+ ```ruby
138
+ schema.leaves # => 4
139
+ schema.width(1) # => 2 nodes at level 1
140
+ schema.span(MerkleMachines::Node.new(1, 1)) # => 2...4, the leaves below that node
141
+ schema.span(MerkleMachines::Node.new(3, 0)) # => nil, not a node of this tree
142
+ schema.span(MerkleMachines::Node::ROOT) # => 0...4
143
+ schema.compatible(schema.with(fanout: 4)) # => :fanout_mismatch
144
+ schema.compatible(schema) # => :compatible
145
+ survey.schema # => the open session's Schema, nil when idle
146
+ survey.versions # => the Versions it pinned, nil when idle
147
+ ```
148
+
149
+ Equal roots answer `:identical` and the survey is `:complete` on the spot. Trees that can't be compared are refused by name, and no session opens:
150
+
151
+ ```ruby
152
+ other = schema.with(fanout: 4)
153
+ MerkleMachines::Survey.new.begin(schema, other, versions, 'a', 'b') # => :fanout_mismatch
154
+ ```
155
+
156
+ The others are `:depth_mismatch`, `:partitioning_mismatch` and `:digest_mismatch`. A survey that is still working answers `:busy`.
157
+
158
+ ### Request
159
+
160
+ ```ruby
161
+ request = survey.next_request
162
+ # => #<data MerkleMachines::Request session=1, node=#<data MerkleMachines::Node level=0, index=0>>
163
+ ```
164
+
165
+ A request names one node: `request.node` is a `Node` with a `level` (0 is the root) and an `index` within that level. Fetch that node's child digests from both trees, in child order. `nil` means nothing is queued right now: answers are still out, the range buffer is full, or the survey is finished. Call it as many times as you want requests in the air.
166
+
167
+ ### Answer
168
+
169
+ ```ruby
170
+ survey.answer(request, versions, %w[left same], %w[left changed]) # => :accepted
171
+
172
+ request = survey.next_request
173
+ # => #<data MerkleMachines::Request session=1, node=#<data MerkleMachines::Node level=1, index=1>>
174
+ survey.answer(request, versions, %w[x y], %w[z y]) # => :accepted
175
+ ```
176
+
177
+ The survey compares the two lists position by position and queues the children that differ. Children that match are never visited again. An answer it can't use is refused and changes nothing:
178
+
179
+ ```ruby
180
+ survey = MerkleMachines::Survey.new
181
+ survey.begin(schema, schema, versions, 'root-a', 'root-b')
182
+ request = survey.next_request
183
+
184
+ survey.answer(request, versions.with(remote: 58), %w[a b], %w[a c]) # => :version_changed
185
+ survey.answer(request, versions, %w[a], %w[a c]) # => :arity
186
+ survey.answer(request, versions, %w[a b], %w[a b]) # => :inconsistent
187
+ elsewhere = request.with(node: request.node.with(level: 1))
188
+ survey.answer(elsewhere, versions, %w[a b], %w[a c]) # => :unexpected
189
+ ```
190
+
191
+ `:inconsistent` means the parents differed and no child does, so the digests didn't come from one tree. `:stale` is the answer to a request from a session that was aborted or replaced, or from a survey in another scope.
192
+
193
+ ### Drain
194
+
195
+ ```ruby
196
+ survey.answer(request, versions, %w[a b], %w[a c]) # => :accepted
197
+ survey.answer(survey.next_request, versions, %w[x y], %w[z y])
198
+
199
+ survey.phase # => :draining
200
+ survey.drain(8) # => [2...3]
201
+ survey.phase # => :complete
202
+ survey.pop_range # => nil, nothing buffered
203
+ ```
204
+
205
+ Ranges are leaf indices, end excluded. `drain(limit)` hands back at most `limit` of them, sorted, with neighbours joined, and keeps the rest buffered; `MerkleMachines::OUTPUT_CAPACITY` takes everything there can be. `pop_range` takes just the lowest one, or `nil`. Two calls can return ranges that touch each other, because the second one was found after you took the first.
206
+
207
+ ### Requeue
208
+
209
+ A request you sent and never heard back about goes back in the queue:
210
+
211
+ ```ruby
212
+ survey = MerkleMachines::Survey.new
213
+ survey.begin(schema, schema, versions, 'root-a', 'root-b')
214
+ request = survey.next_request
215
+
216
+ survey.requeue(request) # => :requeued
217
+ survey.requeue(request) # => :unexpected, it is not in flight any more
218
+ survey.next_request == request # => true
219
+ survey.requeue_all # => 1, for when the whole connection dropped
220
+ ```
221
+
222
+ `requeue` answers `:stale` for a request from a session that was aborted or replaced, or from a survey in another scope, and `:unexpected` for a node that is not in flight. Neither changes anything.
223
+
224
+ ### Abort
225
+
226
+ ```ruby
227
+ survey.abort # => :aborted
228
+ survey.phase # => :idle
229
+ survey.abort # => :already_idle, there was no session to drop and nothing changed
230
+ ```
231
+
232
+ Abort when a tree moved under you. The buffered ranges are discarded, every outstanding request becomes `:stale`, and the next `begin` pins the new versions.
233
+
234
+ ### Scopes
235
+
236
+ A request is only meaningful to the survey that issued it. Run more than one survey (one per peer, say) and each needs its own scope, an Integer in `0..65535` carried by every session it opens and every request of that session:
237
+
238
+ ```ruby
239
+ north = MerkleMachines::Survey.with_scope(1)
240
+ south = MerkleMachines::Survey.with_scope(2)
241
+ [north, south].each { |survey| survey.begin(schema, schema, versions, 'mine', 'theirs') }
242
+
243
+ request = north.next_request
244
+ south.next_request
245
+ request.scope # => 1
246
+ request.session # => 281474976710657, scope 1 in the top 16 bits, session 1 below
247
+ south.answer(request, versions, %w[a b], %w[a c]) # => :stale, not one of mine
248
+ south.requeue(request) # => :stale
249
+ north.answer(request, versions, %w[a b], %w[a c]) # => :accepted
250
+ ```
251
+
252
+ `Survey.new` is scope zero, where sessions are plain `1, 2, 3`. The survey can't pick a scope for you: it reads no clock and rolls no dice. Leave two surveys in the same scope and they open the same sessions, so an answer meant for one lands in the other.
253
+
254
+ A restarted survey opens session 1 again, so a request that outlived the restart could name a node in flight in a new session. Resume from a saved count with `Survey.resumed(scope, survey.issued)`, or give each incarnation a new scope. `resumed` brings back the session sequence and nothing else: the survey is idle, earlier requests are `:stale`, and an `issued` above 2**48-1 raises `RangeError`.
255
+
256
+ ### What each call answers
257
+
258
+ | Call | Honoured | Refused, nothing changed |
259
+ | --- | --- | --- |
260
+ | `begin` | `:identical`, `:diverged` | `:fanout_mismatch`, `:depth_mismatch`, `:partitioning_mismatch`, `:digest_mismatch`, `:busy`, `:undersized`, `:exhausted` |
261
+ | `answer` | `:accepted` | `:stale`, `:unexpected`, `:version_changed`, `:arity`, `:inconsistent` |
262
+ | `requeue` | `:requeued` | `:stale`, `:unexpected` |
263
+ | `requeue_all` | the number requeued | never refuses |
264
+ | `drain(limit)` | an Array of Ranges, empty when none is buffered | never refuses |
265
+ | `pop_range` | a Range, or `nil` when none is buffered | never refuses |
266
+ | `next_request` | a `Request`, or `nil` when nothing is queued | never refuses |
267
+ | `abort` | `:aborted`, or `:already_idle` when no session was open (an idempotent repeat) | never refuses |
268
+ | `schema`, `versions` | the open session's `Schema` / `Versions`, or `nil` when the survey is idle | a query, never refuses |
269
+ | `session` | the latest session id issued in this scope, open or not; `nil` while none was ever begun | a query, never refuses |
270
+
271
+ No answer records a clock: the survey takes no time at all. `Versions` are tree versions the caller pins, not instants, and nothing about them is remembered beyond the open session.
272
+
273
+ `session` after `abort` is the aborted session. A resumed survey is idle until its next `begin` and answers the last session issued before the restart, or `nil` when it was resumed from a count of zero.
274
+
275
+ `:undersized` means the tree is deeper than `PENDING_CAPACITY`, and `:exhausted` means the survey has opened all 2**48-1 sessions of its scope. Every refusal has its own symbol; there is no catch-all. The core's `Refusal::Schema(Mismatch)` answers with the mismatch's symbol, the same four `Schema#compatible` uses.
276
+
277
+ ### Capacity
278
+
279
+ ```ruby
280
+ MerkleMachines::PENDING_CAPACITY # => 64
281
+ MerkleMachines::OUTPUT_CAPACITY # => 64
282
+ MerkleMachines::MAX_FANOUT # => 64
283
+
284
+ survey.pending # nodes held: queued, in flight, or answered and waiting for room
285
+ survey.in_flight # requests out
286
+ survey.buffered # ranges waiting for drain
287
+ ```
288
+
289
+ A survey holds at most 64 nodes and 64 ranges. When the range buffer is full, answered nodes wait and `next_request` returns `nil` until you call `drain` or `pop_range`. Nothing is dropped and no answer is refused for lack of room.
290
+
291
+ ## The Rules of the Navigator
292
+
293
+ - **Digests are Strings.** Use binary strings (`.b`). They are compared byte for byte, and two strings with the same bytes in incompatible encodings are different digests. Anything else raises `TypeError`.
294
+ - **Numbers are Integers** in `0..2**64-1`. `fanout`, `depth`, and `level` stop at 255. Floats, nil, and strings raise `TypeError`. Out of range raises `RangeError`.
295
+ - **Schemas, versions, nodes, and requests are value objects.** `Schema`, `Versions`, `Node` and `Request` are frozen `Data` values with the fields of the core types they are named after. An Array in their place raises `TypeError`. They are sealed (subclassing raises `TypeError`, `allocate` is undefined), and since sealing stops accidents, not reflection, the binding is the authority: every value it reads must be an instance of exactly its class, its fields are read straight from the struct and checked again by the core's constructors, so a forged value raises `TypeError` or `RangeError` and never reaches the survey.
296
+ - **Values are valid or do not exist.** Every value is checked by the core when it is built, by you or by the gem: a wrong type raises `TypeError`, a value the core has none for raises `RangeError` (`Schema.new(fanout: 1, ...)`, `Node.new(256, 0)`, `Request.new(0, Node::ROOT)`: sequence zero is no session).
297
+ - **A call that raises changed nothing.** Every check, and every class lookup, runs before the survey moves; the `Request` from `next_request` and the Ranges from `pop_range` and `drain` are built natively from what the core returned, without running any Ruby code (not even `initialize`), so no work is lost to a raise after it.
298
+ - **Scopes.** A `Request` is only meaningful to the survey that issued it. Give each survey its own scope with `Survey.with_scope(scope)`; the core cannot pick one. A `Request` from a survey in another scope is `:stale`. Two surveys in the same scope, `Survey.new` twice included, open the same sessions and alias each other's requests.
299
+ - **Restarts.** Resume from a saved count with `Survey.resumed(scope, issued)`, or give each incarnation a new scope. `resumed` restores the session sequence only; the survey is idle and comparing again is yours.
300
+ - **Refusals don't raise.** Every refusal is a Symbol, and a refused call changes nothing. Only malformed input raises.
301
+ - **One version of each tree per session.** Every answer names the versions its digests were read at. Different versions are refused.
302
+ - **Frozen means frozen.** `freeze` the survey and every method that would change it raises `FrozenError`.
303
+
304
+ ## What You Think You're Doing vs Reality
305
+
306
+ ### You Think: "Merkle trees find the difference with the fewest messages."
307
+ ### Reality: They find it without sending everything.
308
+
309
+ The survey visits the nodes whose digests differ and skips the rest. That is all it promises. It makes no claim about minimal exchange or round count: two replicas that disagree on every leaf cost you every internal node, which is more than shipping the leaves outright. How many requests you keep in the air, how you batch them, and when you give up and copy the lot are your decisions.
310
+
311
+ ### You Think: "Equal digests, equal data."
312
+ ### Reality: Equal digests, and you chose the hash.
313
+
314
+ The survey trusts a matching digest and never looks below it. A collision in your digest hides a difference, and the survey can't know.
315
+
316
+ ## What's Yours
317
+
318
+ This is an early release and the API will change. It is a policy core, not a sync runtime.
319
+
320
+ You build the trees and compute every digest. The survey does no hashing. You talk to the other replica, in whatever protocol and batch size you like, and you decide when a request is lost. You keep each tree readable at the version you pinned for as long as the session runs, or you abort and start over. You pick the `partitioning` and `digest` numbers and make sure equal numbers really mean equal layouts. You decide what a differing leaf range means: the survey locates divergence, it doesn't merge, repair, or pick a winner. And the survey lives in your process, so a restart means a new `begin`.
321
+
322
+ ## License
323
+
324
+ MIT License. See [LICENSE.txt](LICENSE.txt) for details.
325
+
326
+ **Remember: the root told you they differ. The survey tells you where. Fixing it is still your job.**
@@ -0,0 +1,66 @@
1
+ # Changelog
2
+
3
+ ## [0.7.1](https://github.com/seuros/merkle_machines/compare/merkle-machines-v0.7.0...merkle-machines-v0.7.1) (2026-11-16)
4
+
5
+
6
+ ### Bug Fixes
7
+
8
+ * index-free merkle survey core; Request#scope reads natively ([5e40549](https://github.com/seuros/merkle_machines/commit/5e40549c8b2bf82827a563797aa3551863da6b66))
9
+
10
+ ## [0.7.0](https://github.com/seuros/merkle_machines/compare/merkle-machines-v0.6.0...merkle-machines-v0.7.0) (2026-11-14)
11
+
12
+
13
+ ### âš  BREAKING CHANGES
14
+
15
+ * adopt matryona-machines 0.2: native exact_fields readers for every value, native begin/answer/requeue, no reader or deconstruct runs
16
+
17
+ ### Features
18
+
19
+ * adopt matryona-machines 0.2: native exact_fields readers for every value, native begin/answer/requeue, no reader or deconstruct runs ([b360c77](https://github.com/seuros/merkle_machines/commit/b360c775def2c804b7a4c5145cd3b7e855ff667d))
20
+
21
+ ## [0.6.0](https://github.com/seuros/merkle_machines/compare/merkle-machines-v0.5.0...merkle-machines-v0.6.0) (2026-11-14)
22
+
23
+
24
+ ### âš  BREAKING CHANGES
25
+
26
+ * seal values, check exact classes natively, clone-then-keep requests and drains, native fields helper
27
+
28
+ ### Features
29
+
30
+ * seal values, check exact classes natively, clone-then-keep requests and drains, native fields helper ([eb6ebfa](https://github.com/seuros/merkle_machines/commit/eb6ebfa88033b8f934743af11f8b370d88d60baa))
31
+
32
+ ## [0.5.0](https://github.com/seuros/merkle_machines/compare/merkle-machines-v0.4.0...merkle-machines-v0.5.0) (2026-11-13)
33
+
34
+
35
+ ### âš  BREAKING CHANGES
36
+
37
+ * handles are issued through periscope-machines. Every instance can be built with a scope; a handle from another scope is refused before any lookup; all handle kinds of an instance share one sequence; resumed restores the sequence after a restart. A request from another survey answers Stale.
38
+
39
+ ### Features
40
+
41
+ * scope sessions through periscope-machines, add Survey::resumed and issued ([0e54402](https://github.com/seuros/merkle_machines/commit/0e54402d670ce201fe4d18cf61544276005b87be))
42
+
43
+ ## [0.4.0](https://github.com/seuros/merkle_machines/compare/merkle-machines-v0.3.0...merkle-machines-v0.4.0) (2026-11-13)
44
+
45
+
46
+ ### âš  BREAKING CHANGES
47
+
48
+ * see CONVENTIONS: one exhaustive Refusal per crate, Result from every mutator, honoured idempotent repeats, Ruby mirrors the core. No compatibility aliases. requeue returns Result<(), Refusal>; abort returns Aborting; Schema.new raises where the core has no schema.
49
+
50
+ ### Features
51
+
52
+ * requeue and abort answer their real outcome, Ruby Request carries a Node and drain replaces take_ranges ([445d667](https://github.com/seuros/merkle_machines/commit/445d667f19c51a2bbf225af73d2897420a9a4d9a))
53
+
54
+ ## [0.3.0](https://github.com/seuros/merkle_machines/compare/merkle-machines-v0.2.0...merkle-machines-v0.3.0) (2026-11-13)
55
+
56
+
57
+ ### Features
58
+
59
+ * implement Default for LeafRange and name the drain buffer out ([d6e4c5e](https://github.com/seuros/merkle_machines/commit/d6e4c5ec9b67a705d52dcffea34d3a95fad059e7))
60
+
61
+ ## [0.2.0](https://github.com/seuros/merkle_machines/compare/merkle-machines-v0.1.0...merkle-machines-v0.2.0) (2026-11-12)
62
+
63
+
64
+ ### Features
65
+
66
+ * initial extraction ([43dd5e9](https://github.com/seuros/merkle_machines/commit/43dd5e92b117d600f3828fb106b9e4fa798f3f5e))
@@ -0,0 +1,20 @@
1
+ [package]
2
+ name = "merkle-machines"
3
+ version = "0.7.1"
4
+ edition.workspace = true
5
+ rust-version.workspace = true
6
+ license.workspace = true
7
+ description = "Bounded resumable digest-tree comparison that locates differing leaf ranges"
8
+ readme = "README.md"
9
+ exclude = ["src/**/tests.rs", "src/tests/", "tests/"]
10
+ keywords = ["merkle", "anti-entropy", "reconciliation", "digest", "no-std"]
11
+ categories = ["no-std", "algorithms"]
12
+
13
+ [dependencies]
14
+ periscope-machines.workspace = true
15
+
16
+ [dev-dependencies]
17
+ matryona-machines = { workspace = true, features = ["testing"] }
18
+
19
+ [lints]
20
+ workspace = true
@@ -0,0 +1,114 @@
1
+ # merkle-machines
2
+
3
+ > **AMS Merkle**, NCC-4224, *Merkle*-class survey cutter.
4
+
5
+ Engine-room flight manual: chart where two digest trees disagree, a bounded
6
+ number of nodes at a time.
7
+
8
+ Allocation-free `no_std` comparison traversal over two fixed-depth digest trees
9
+ of the same shape. The survey hands out requests for the child digests of nodes
10
+ whose digests differ, the caller answers with both trees' children, and the
11
+ survey emits the ranges of leaves that differ. It stores no digests and computes
12
+ none: hashing, storage and the exchange with the other replica belong to the
13
+ caller.
14
+
15
+ Two trees are comparable only when their `Schema` matches: fanout, depth, and the
16
+ caller's identifiers for key-to-leaf partitioning and digest algorithm. A
17
+ mismatch is refused by name and opens no session. A session pins one version of
18
+ each tree; an answer read from any other version is refused, because digests
19
+ from a tree that moved cannot be compared with digests from before it moved.
20
+
21
+ Buffers are fixed. `P` bounds the nodes held at once (queued, in flight or
22
+ partly expanded) and must be at least the tree depth; `O` bounds the buffered
23
+ leaf ranges and must be at least one. An answered node keeps its differing
24
+ children as a bit mask and releases them only as slots and output space allow,
25
+ so an answer is never refused for lack of room and a full buffer delays the
26
+ traversal instead of losing work. Lost requests are requeued one by one or all
27
+ at once. Every refusal is a `Refusal`, the crate's one refusal type, and leaves
28
+ the survey unchanged: `begin`, `answer` and `requeue` return
29
+ `Result<_, Refusal>`. Requests carry a session
30
+ id, so answers to an aborted or replaced session are stale. `abort` never
31
+ refuses: it answers `Aborting::Aborted`, or `Aborting::AlreadyIdle` when no
32
+ session was open, an idempotent repeat that changes nothing.
33
+
34
+ **Scopes.** A request is only meaningful to the survey that issued it. Session
35
+ ids come from a `periscope_machines::Issuer`: the survey's scope in the top 16
36
+ bits, a sequence counted from one below, never reissued.
37
+ `Survey::with_scope(scope)` chooses the scope and `new()` is scope zero;
38
+ `Survey::scope` and `Request::scope` read it. `answer` and `requeue` answer
39
+ `Stale` for a request whose session the survey did not issue, before looking at
40
+ any node. Give each survey its own scope: the core reads no clock and no random
41
+ source, so it cannot pick one, and two surveys left in the same scope (two
42
+ `new()` included) issue the same sessions and honour each other's requests. A
43
+ scope is `Exhausted` after 2^48 - 1 sessions. `session()` is the latest session
44
+ issued in the scope, open or not (`Issuer::last`): `None` until the first
45
+ session begins, the aborted session after `abort`. `Request::new` rebuilds a
46
+ request that crossed a wire and is `None` for a session of sequence zero, which
47
+ no survey issues.
48
+
49
+ **Restarts.** A fresh survey opens session one again, so a request that
50
+ outlived a restart could name a node in flight in a new session. Save
51
+ `Survey::issued` and rebuild with `Survey::resumed(scope, issued)`, or give
52
+ each incarnation a new scope. `resumed` restores only the session sequence: the
53
+ survey is idle, every earlier request is `Stale`, and comparing again is the
54
+ caller's job. It is `None` when `issued` is beyond
55
+ `periscope_machines::SEQUENCE_MAX`. A resumed survey's `session()` is the last
56
+ session issued before the restart, which is not open, or `None` when resumed
57
+ from zero.
58
+
59
+ **No clock.** The survey reads and records no time; `Versions` are tree
60
+ versions the caller pins for one session, not instants.
61
+
62
+ The Ruby gem mirrors `Schema`, `Versions`, `Node` and `Request` as `Data`
63
+ values of the same names and fields, each checked by the core when built: where
64
+ `Schema::new` or `Request::new` answers `None`, or a field does not fit the
65
+ core's type, `new` raises `RangeError` (`TypeError` for a wrong type). A
66
+ `LeafRange` is a half-open Ruby `Range`.
67
+
68
+ Buffered ranges are sorted, disjoint and joined when they touch. Ranges taken at
69
+ different times may still touch each other. The traversal visits only nodes
70
+ whose digests differ, deepest first; it makes no claim about minimal exchange or
71
+ round count. Equal digests are trusted to mean equal subtrees, so a digest
72
+ collision hides a difference, and a parent that differs while none of its
73
+ children do is refused as inconsistent.
74
+
75
+ `pop_range` takes one range; `drain(out)` writes as many as fit into the front
76
+ of `out`, returns the count and leaves the rest untouched. Build the buffer as
77
+ `[LeafRange::default(); N]`: the default is the empty range `0..0`, a
78
+ placeholder the survey never emits. A `LeafRange` never runs backwards: its
79
+ fields are private, read through `start()` and `end()`, and
80
+ `LeafRange::new(start, end)` answers `None` when `start > end`, so `len()`
81
+ cannot underflow.
82
+
83
+ ```rust
84
+ use merkle_machines::{Begun, LeafRange, Phase, Schema, Survey, Versions};
85
+
86
+ // Two trees of four leaves; only leaf 2 differs.
87
+ let ours = [[10_u64, 20], [1, 2], [3, 4]];
88
+ let theirs = [[10_u64, 21], [1, 2], [9, 4]];
89
+
90
+ let schema = Schema::new(2, 2, 1, 1).expect("valid shape");
91
+ let versions = Versions { local: 40, remote: 57 };
92
+ let mut survey = Survey::<4, 4>::new();
93
+ assert_eq!(survey.begin(schema, schema, versions, &100_u64, &101_u64), Ok(Begun::Diverged));
94
+
95
+ while let Some(request) = survey.next_request() {
96
+ let node = request.node();
97
+ let row = if node.level == 0 { 0 } else { 1 + node.index as usize };
98
+ assert_eq!(survey.answer(request, versions, &ours[row], &theirs[row]), Ok(()));
99
+ }
100
+ assert_eq!(survey.pop_range(), LeafRange::new(2, 3));
101
+ assert_eq!(survey.phase(), Phase::Complete);
102
+
103
+ // Another survey, another scope: it refuses a request it did not issue.
104
+ use merkle_machines::Refusal;
105
+ let mut north = Survey::<4, 4>::with_scope(1);
106
+ let mut south = Survey::<4, 4>::with_scope(2);
107
+ north.begin(schema, schema, versions, &100_u64, &101_u64).expect("comparable");
108
+ south.begin(schema, schema, versions, &100_u64, &101_u64).expect("comparable");
109
+ let request = north.next_request().expect("the root");
110
+ south.next_request().expect("the root");
111
+ assert_eq!(request.scope(), 1);
112
+ assert_eq!(south.answer(request, versions, &ours[0], &theirs[0]), Err(Refusal::Stale));
113
+ assert_eq!(north.answer(request, versions, &ours[0], &theirs[0]), Ok(()));
114
+ ```
@@ -0,0 +1,16 @@
1
+ #![no_std]
2
+ #![cfg_attr(not(test), deny(clippy::indexing_slicing))]
3
+ #![doc = include_str!("../README.md")]
4
+
5
+ mod refusal;
6
+ mod request;
7
+ mod schema;
8
+ mod survey;
9
+
10
+ pub use refusal::Refusal;
11
+ pub use request::{Request, Versions};
12
+ pub use schema::{LeafRange, MAX_FANOUT, Mismatch, Node, Schema};
13
+ pub use survey::{Aborting, Begun, Phase, Survey};
14
+
15
+ #[cfg(test)]
16
+ mod tests;
@@ -0,0 +1,54 @@
1
+ //! The one refusal type of the crate.
2
+
3
+ use core::fmt;
4
+
5
+ use crate::schema::Mismatch;
6
+
7
+ /// Why a call was refused. A refused call changes nothing.
8
+ ///
9
+ /// Every operation that can refuse answers with this type:
10
+ /// [`Survey::begin`](crate::Survey::begin), [`Survey::answer`](crate::Survey::answer)
11
+ /// and [`Survey::requeue`](crate::Survey::requeue).
12
+ #[derive(Debug, Clone, Copy, PartialEq, Eq)]
13
+ pub enum Refusal {
14
+ /// The two trees are not comparable.
15
+ Schema(Mismatch),
16
+ /// Pending capacity is below the tree depth, or output capacity is zero.
17
+ Undersized,
18
+ /// A session is still surveying or draining; abort it first.
19
+ Busy,
20
+ /// Every session id of this survey's scope has been issued.
21
+ Exhausted,
22
+ /// The request does not belong to the open session: it is from an earlier
23
+ /// session, from another survey's scope, or no session is open.
24
+ Stale,
25
+ /// The node is not in flight: never requested, already answered or requeued.
26
+ Unexpected,
27
+ /// The digests were read from a different tree version than the session pinned.
28
+ VersionChanged,
29
+ /// A child digest list does not hold exactly `fanout` digests.
30
+ Arity,
31
+ /// The parents differed but no child does: the digests are not from one tree.
32
+ Inconsistent,
33
+ }
34
+
35
+ impl fmt::Display for Refusal {
36
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
37
+ f.write_str(match self {
38
+ Self::Schema(Mismatch::Fanout) => "the trees differ in fanout",
39
+ Self::Schema(Mismatch::Depth) => "the trees differ in depth",
40
+ Self::Schema(Mismatch::Partitioning) => "the trees differ in partitioning",
41
+ Self::Schema(Mismatch::Digest) => "the trees differ in digest",
42
+ Self::Undersized => "capacity is below what the tree needs",
43
+ Self::Busy => "a session is still open",
44
+ Self::Exhausted => "every session of this scope has been issued",
45
+ Self::Stale => "request does not belong to the open session",
46
+ Self::Unexpected => "node is not in flight",
47
+ Self::VersionChanged => "digests were read from another tree version",
48
+ Self::Arity => "child digest list does not match the fanout",
49
+ Self::Inconsistent => "parents differ but no child does",
50
+ })
51
+ }
52
+ }
53
+
54
+ impl core::error::Error for Refusal {}
@@ -0,0 +1,60 @@
1
+ //! What travels between a survey and the replica answering it.
2
+
3
+ use periscope_machines::{scope_of, sequence_of};
4
+
5
+ use crate::schema::Node;
6
+
7
+ /// Tree versions the digests of one session must all be read from.
8
+ #[derive(Debug, Clone, Copy, PartialEq, Eq)]
9
+ pub struct Versions {
10
+ /// Version of the local tree.
11
+ pub local: u64,
12
+ /// Version of the remote tree.
13
+ pub remote: u64,
14
+ }
15
+
16
+ /// A request for the child digests of one node, on both trees.
17
+ #[derive(Debug, Clone, Copy, PartialEq, Eq)]
18
+ pub struct Request {
19
+ session: u64,
20
+ node: Node,
21
+ }
22
+
23
+ impl Request {
24
+ /// Rebuild a request that crossed a wire. A forged request is refused by
25
+ /// [`Survey::answer`] unless it names a node that is in flight.
26
+ ///
27
+ /// `None` when `session` has sequence number zero: no survey issues that
28
+ /// id, so it names no session at all.
29
+ #[must_use]
30
+ pub const fn new(session: u64, node: Node) -> Option<Self> {
31
+ if sequence_of(session) == 0 {
32
+ return None;
33
+ }
34
+ Some(Self { session, node })
35
+ }
36
+
37
+ /// A request for a session the survey issued, so never sequence zero.
38
+ pub(crate) const fn issued(session: u64, node: Node) -> Self {
39
+ Self { session, node }
40
+ }
41
+
42
+ /// Session the request belongs to: a scoped id, the issuing survey's
43
+ /// scope in the top 16 bits and its session count (from one) below.
44
+ #[must_use]
45
+ pub const fn session(&self) -> u64 {
46
+ self.session
47
+ }
48
+
49
+ /// The scope of the survey that issued the request's session.
50
+ #[must_use]
51
+ pub const fn scope(&self) -> u16 {
52
+ scope_of(self.session)
53
+ }
54
+
55
+ /// Node whose children are wanted.
56
+ #[must_use]
57
+ pub const fn node(&self) -> Node {
58
+ self.node
59
+ }
60
+ }