@sovereignbase/sequencer 0.0.1 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/INTERNALS.md +1530 -0
- package/LICENSE +201 -201
- package/README.md +128 -90
- package/dist/index.cjs +264 -160
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +118 -12
- package/dist/index.d.cts.map +1 -1
- package/dist/index.d.ts +118 -12
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +265 -160
- package/dist/index.js.map +1 -1
- package/package.json +27 -27
package/INTERNALS.md
ADDED
|
@@ -0,0 +1,1530 @@
|
|
|
1
|
+
# How Sequencer works
|
|
2
|
+
|
|
3
|
+
Sequencer gives applications an ordinary indexed sequence while internally maintaining a stable distributed history.
|
|
4
|
+
|
|
5
|
+
An application sees something simple:
|
|
6
|
+
|
|
7
|
+
```text
|
|
8
|
+
A B C D
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
and edits it using positions:
|
|
12
|
+
|
|
13
|
+
```text
|
|
14
|
+
insert at 2
|
|
15
|
+
remove 1..3
|
|
16
|
+
replace 4..6
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Those positions are useful locally, but they cannot be replicated directly.
|
|
20
|
+
|
|
21
|
+
Two replicas may edit concurrently. By the time an operation reaches another replica, content may have been inserted or removed before the original position. Position `5` on one replica may therefore refer to something completely different on another.
|
|
22
|
+
|
|
23
|
+
Sequencer solves this by translating every local edit from a temporary visible position into a stable relationship with existing history.
|
|
24
|
+
|
|
25
|
+
The basic model is:
|
|
26
|
+
|
|
27
|
+
```text
|
|
28
|
+
visible Projection
|
|
29
|
+
↓
|
|
30
|
+
historical logical point
|
|
31
|
+
↓
|
|
32
|
+
canonical Insertion
|
|
33
|
+
↓
|
|
34
|
+
deterministic Structural Order
|
|
35
|
+
↓
|
|
36
|
+
visible Projection
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Replicas do not need to agree on temporary positions, network arrival order, runtime fragmentation, or local indexes.
|
|
40
|
+
|
|
41
|
+
They only need to eventually receive the same canonical history.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Projection is the visible sequence
|
|
46
|
+
|
|
47
|
+
The **Projection** is what the application sees.
|
|
48
|
+
|
|
49
|
+
For example:
|
|
50
|
+
|
|
51
|
+
```text
|
|
52
|
+
A B C D
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
contains four visible values, called **Frames**:
|
|
56
|
+
|
|
57
|
+
```text
|
|
58
|
+
A B C D
|
|
59
|
+
0 1 2 3
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Applications read and edit these visible positions.
|
|
63
|
+
|
|
64
|
+
Insertion positions are boundaries between Frames:
|
|
65
|
+
|
|
66
|
+
```text
|
|
67
|
+
| A | B | C | D |
|
|
68
|
+
0 1 2 3 4
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
This distinction is important.
|
|
72
|
+
|
|
73
|
+
A Frame position identifies visible content.
|
|
74
|
+
|
|
75
|
+
A boundary identifies a place where new history can be attached.
|
|
76
|
+
|
|
77
|
+
Projection positions are deliberately temporary. They describe the current visible result, not the permanent identity of anything inside it.
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## History must use stable coordinates
|
|
82
|
+
|
|
83
|
+
Suppose two replicas begin with:
|
|
84
|
+
|
|
85
|
+
```text
|
|
86
|
+
A B
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Replica 1 inserts:
|
|
90
|
+
|
|
91
|
+
```text
|
|
92
|
+
X
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
between them:
|
|
96
|
+
|
|
97
|
+
```text
|
|
98
|
+
A X B
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
At the same time, Replica 2 inserts:
|
|
102
|
+
|
|
103
|
+
```text
|
|
104
|
+
Y
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
at the same visible position:
|
|
108
|
+
|
|
109
|
+
```text
|
|
110
|
+
A Y B
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
Both operations were authored at position `1`.
|
|
114
|
+
|
|
115
|
+
If Sequencer replicated only:
|
|
116
|
+
|
|
117
|
+
```text
|
|
118
|
+
insert X at 1
|
|
119
|
+
insert Y at 1
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
the final result could depend on which packet happened to arrive first.
|
|
123
|
+
|
|
124
|
+
Instead, both edits are translated into a relationship with the same historical boundary between `A` and `B`.
|
|
125
|
+
|
|
126
|
+
That boundary remains meaningful even after the visible sequence changes.
|
|
127
|
+
|
|
128
|
+
Sequencer can then deterministically order the two competing Insertions.
|
|
129
|
+
|
|
130
|
+
---
|
|
131
|
+
|
|
132
|
+
# Canonical Insertions
|
|
133
|
+
|
|
134
|
+
The fundamental replicated unit in Sequencer is an **Insertion**.
|
|
135
|
+
|
|
136
|
+
Conceptually, every Insertion contains:
|
|
137
|
+
|
|
138
|
+
```text
|
|
139
|
+
where it belongs
|
|
140
|
+
+
|
|
141
|
+
what history it introduces
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Its canonical coordinates are:
|
|
145
|
+
|
|
146
|
+
```text
|
|
147
|
+
anchorSession
|
|
148
|
+
anchorStart
|
|
149
|
+
anchorDiff
|
|
150
|
+
|
|
151
|
+
insertionSession
|
|
152
|
+
insertionStart
|
|
153
|
+
insertionDiff
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
These six values describe two independent things.
|
|
157
|
+
|
|
158
|
+
The anchor:
|
|
159
|
+
|
|
160
|
+
```text
|
|
161
|
+
(anchorSession, anchorStart, anchorDiff)
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
describes where the Insertion belongs.
|
|
165
|
+
|
|
166
|
+
The identity:
|
|
167
|
+
|
|
168
|
+
```text
|
|
169
|
+
(insertionSession, insertionStart)
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
describes which Insertion it is.
|
|
173
|
+
|
|
174
|
+
The effect:
|
|
175
|
+
|
|
176
|
+
```text
|
|
177
|
+
insertionDiff
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
describes how many original Frames it contributes.
|
|
181
|
+
|
|
182
|
+
A positive Insertion introduces Frames.
|
|
183
|
+
|
|
184
|
+
A negative Insertion represents removal.
|
|
185
|
+
|
|
186
|
+
---
|
|
187
|
+
|
|
188
|
+
## Insertion identity never depends on location
|
|
189
|
+
|
|
190
|
+
A canonical Insertion is identified by:
|
|
191
|
+
|
|
192
|
+
```text
|
|
193
|
+
(insertionSession, insertionStart)
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Its identity does not include its anchor.
|
|
197
|
+
|
|
198
|
+
This means:
|
|
199
|
+
|
|
200
|
+
```text
|
|
201
|
+
identity
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
and:
|
|
205
|
+
|
|
206
|
+
```text
|
|
207
|
+
position in history
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
are separate concepts.
|
|
211
|
+
|
|
212
|
+
That separation is useful for several reasons.
|
|
213
|
+
|
|
214
|
+
The same operation can be recognized after retransmission.
|
|
215
|
+
|
|
216
|
+
Fragments of one Insertion still belong to the same canonical identity.
|
|
217
|
+
|
|
218
|
+
An Insertion can be found independently from whichever Strip currently represents its anchor.
|
|
219
|
+
|
|
220
|
+
And duplicate delivery does not create duplicate visible content.
|
|
221
|
+
|
|
222
|
+
---
|
|
223
|
+
|
|
224
|
+
# Logical time
|
|
225
|
+
|
|
226
|
+
Every sequencing Session owns its own logical coordinate space.
|
|
227
|
+
|
|
228
|
+
A newly authored Insertion receives a start position inside that Session.
|
|
229
|
+
|
|
230
|
+
For example:
|
|
231
|
+
|
|
232
|
+
```text
|
|
233
|
+
Session 12
|
|
234
|
+
|
|
235
|
+
Insertion A starts at 0
|
|
236
|
+
Insertion B starts at 6
|
|
237
|
+
Insertion C starts at 11
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
These are not wall-clock timestamps.
|
|
241
|
+
|
|
242
|
+
They are logical coordinates used only to identify history.
|
|
243
|
+
|
|
244
|
+
A canonical Insertion normally reserves both:
|
|
245
|
+
|
|
246
|
+
```text
|
|
247
|
+
its Frame extent
|
|
248
|
+
+
|
|
249
|
+
a following logical boundary
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
before the next independent Insertion in the same Session.
|
|
253
|
+
|
|
254
|
+
That following boundary gives later history a stable point to reference.
|
|
255
|
+
|
|
256
|
+
---
|
|
257
|
+
|
|
258
|
+
## Frames and logical points are different
|
|
259
|
+
|
|
260
|
+
Consider an Insertion containing:
|
|
261
|
+
|
|
262
|
+
```text
|
|
263
|
+
A B C
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
It contains three Frames:
|
|
267
|
+
|
|
268
|
+
```text
|
|
269
|
+
0 1 2
|
|
270
|
+
```
|
|
271
|
+
|
|
272
|
+
but four logical boundary points:
|
|
273
|
+
|
|
274
|
+
```text
|
|
275
|
+
| A | B | C |
|
|
276
|
+
0 1 2 3
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
An anchor refers to one of these logical points.
|
|
280
|
+
|
|
281
|
+
For example:
|
|
282
|
+
|
|
283
|
+
```text
|
|
284
|
+
anchorDiff = 2
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
means:
|
|
288
|
+
|
|
289
|
+
```text
|
|
290
|
+
A B | C
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
inside that specific historical Insertion.
|
|
294
|
+
|
|
295
|
+
It does **not** mean:
|
|
296
|
+
|
|
297
|
+
```text
|
|
298
|
+
Projection position 2
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
That distinction is one of the most important ideas in Sequencer.
|
|
302
|
+
|
|
303
|
+
Visible positions move.
|
|
304
|
+
|
|
305
|
+
Historical logical points do not.
|
|
306
|
+
|
|
307
|
+
---
|
|
308
|
+
|
|
309
|
+
# From a local edit to canonical history
|
|
310
|
+
|
|
311
|
+
Suppose the current Projection is:
|
|
312
|
+
|
|
313
|
+
```text
|
|
314
|
+
Hello world
|
|
315
|
+
```
|
|
316
|
+
|
|
317
|
+
and the application inserts:
|
|
318
|
+
|
|
319
|
+
```text
|
|
320
|
+
big
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
here:
|
|
324
|
+
|
|
325
|
+
```text
|
|
326
|
+
Hello | world
|
|
327
|
+
```
|
|
328
|
+
|
|
329
|
+
The application supplies only the visible boundary.
|
|
330
|
+
|
|
331
|
+
Sequencer resolves that boundary to the historical Insertion that owns it.
|
|
332
|
+
|
|
333
|
+
It then records the edit relative to that original Insertion:
|
|
334
|
+
|
|
335
|
+
```text
|
|
336
|
+
specific historical Insertion
|
|
337
|
+
+
|
|
338
|
+
specific logical point inside it
|
|
339
|
+
```
|
|
340
|
+
|
|
341
|
+
The operation therefore means:
|
|
342
|
+
|
|
343
|
+
```text
|
|
344
|
+
insert here in history
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
rather than:
|
|
348
|
+
|
|
349
|
+
```text
|
|
350
|
+
insert at visible index 6
|
|
351
|
+
```
|
|
352
|
+
|
|
353
|
+
Another replica may have more visible content before that point.
|
|
354
|
+
|
|
355
|
+
The numeric Projection position may differ.
|
|
356
|
+
|
|
357
|
+
The historical relationship does not.
|
|
358
|
+
|
|
359
|
+
---
|
|
360
|
+
|
|
361
|
+
# Structural Order
|
|
362
|
+
|
|
363
|
+
Canonical history must eventually form one deterministic order.
|
|
364
|
+
|
|
365
|
+
Sequencer calls this **Structural Order**.
|
|
366
|
+
|
|
367
|
+
Structural Order contains everything required to explain the visible sequence, including history that may no longer itself be visible.
|
|
368
|
+
|
|
369
|
+
It may contain:
|
|
370
|
+
|
|
371
|
+
```text
|
|
372
|
+
positive Insertions
|
|
373
|
+
removal history
|
|
374
|
+
fragments
|
|
375
|
+
concurrent siblings
|
|
376
|
+
causal descendants
|
|
377
|
+
hidden historical boundaries
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
The Projection is derived from this structure.
|
|
381
|
+
|
|
382
|
+
Conceptually:
|
|
383
|
+
|
|
384
|
+
```text
|
|
385
|
+
Structural Order
|
|
386
|
+
↓
|
|
387
|
+
surviving positive Frames
|
|
388
|
+
↓
|
|
389
|
+
Projection
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
Structural Order is therefore richer than the visible sequence.
|
|
393
|
+
|
|
394
|
+
The Projection answers:
|
|
395
|
+
|
|
396
|
+
```text
|
|
397
|
+
what is visible now?
|
|
398
|
+
```
|
|
399
|
+
|
|
400
|
+
Structural Order answers:
|
|
401
|
+
|
|
402
|
+
```text
|
|
403
|
+
what historical relationships produced that result?
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
---
|
|
407
|
+
|
|
408
|
+
# Concurrent Insertions
|
|
409
|
+
|
|
410
|
+
Two Insertions directly compete when they use exactly the same canonical anchor.
|
|
411
|
+
|
|
412
|
+
For example, both of these may attach to:
|
|
413
|
+
|
|
414
|
+
```text
|
|
415
|
+
A | B
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
independently:
|
|
419
|
+
|
|
420
|
+
```text
|
|
421
|
+
X
|
|
422
|
+
Y
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
The order between them cannot depend on which replica received which packet first.
|
|
426
|
+
|
|
427
|
+
Sequencer therefore defines a deterministic competitor ordering.
|
|
428
|
+
|
|
429
|
+
Closer to the shared anchor:
|
|
430
|
+
|
|
431
|
+
1. positive Insertions come before reducing Insertions;
|
|
432
|
+
2. within the same sign, larger Session identifiers come first;
|
|
433
|
+
3. within the same Session, larger start coordinates come first.
|
|
434
|
+
|
|
435
|
+
Every replica can calculate this from the canonical history alone.
|
|
436
|
+
|
|
437
|
+
So one replica may receive:
|
|
438
|
+
|
|
439
|
+
```text
|
|
440
|
+
X
|
|
441
|
+
Y
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
while another receives:
|
|
445
|
+
|
|
446
|
+
```text
|
|
447
|
+
Y
|
|
448
|
+
X
|
|
449
|
+
```
|
|
450
|
+
|
|
451
|
+
but both eventually construct the same Structural Order.
|
|
452
|
+
|
|
453
|
+
---
|
|
454
|
+
|
|
455
|
+
# Causal subtrees stay together
|
|
456
|
+
|
|
457
|
+
A concurrent competitor may later receive children of its own.
|
|
458
|
+
|
|
459
|
+
Those children form its **causal subtree**.
|
|
460
|
+
|
|
461
|
+
Consider two replicas that independently create root content:
|
|
462
|
+
|
|
463
|
+
```text
|
|
464
|
+
Document A
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
and:
|
|
468
|
+
|
|
469
|
+
```text
|
|
470
|
+
Document B
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
Before synchronizing, both continue editing their own document.
|
|
474
|
+
|
|
475
|
+
The two roots compete at the same virtual root anchor.
|
|
476
|
+
|
|
477
|
+
Sequencer deterministically decides which root comes first.
|
|
478
|
+
|
|
479
|
+
But the descendants of each root must remain with that root.
|
|
480
|
+
|
|
481
|
+
The final Structural Order may be:
|
|
482
|
+
|
|
483
|
+
```text
|
|
484
|
+
[Document A and its descendants]
|
|
485
|
+
|
|
486
|
+
[Document B and its descendants]
|
|
487
|
+
```
|
|
488
|
+
|
|
489
|
+
or the reverse.
|
|
490
|
+
|
|
491
|
+
It must not become:
|
|
492
|
+
|
|
493
|
+
```text
|
|
494
|
+
part of A
|
|
495
|
+
part of B
|
|
496
|
+
rest of A
|
|
497
|
+
rest of B
|
|
498
|
+
```
|
|
499
|
+
|
|
500
|
+
Competitor ordering therefore works on causal subtrees, not merely isolated nodes.
|
|
501
|
+
|
|
502
|
+
This is what lets independently developed branches of history remain coherent when they meet.
|
|
503
|
+
|
|
504
|
+
---
|
|
505
|
+
|
|
506
|
+
# History can arrive in any causal order
|
|
507
|
+
|
|
508
|
+
Distributed delivery may be reordered.
|
|
509
|
+
|
|
510
|
+
Suppose one replica first creates:
|
|
511
|
+
|
|
512
|
+
```text
|
|
513
|
+
Hello
|
|
514
|
+
```
|
|
515
|
+
|
|
516
|
+
and later extends it into:
|
|
517
|
+
|
|
518
|
+
```text
|
|
519
|
+
Hello world
|
|
520
|
+
```
|
|
521
|
+
|
|
522
|
+
Another replica may receive the history for:
|
|
523
|
+
|
|
524
|
+
```text
|
|
525
|
+
world
|
|
526
|
+
```
|
|
527
|
+
|
|
528
|
+
before it receives the history containing:
|
|
529
|
+
|
|
530
|
+
```text
|
|
531
|
+
Hello
|
|
532
|
+
```
|
|
533
|
+
|
|
534
|
+
The child relationship is perfectly valid, but its parent does not yet exist locally.
|
|
535
|
+
|
|
536
|
+
Sequencer keeps that history unresolved until the required canonical parent becomes available.
|
|
537
|
+
|
|
538
|
+
When the missing parent later arrives, the relationship becomes resolvable.
|
|
539
|
+
|
|
540
|
+
No retransmission or special causal network ordering is required.
|
|
541
|
+
|
|
542
|
+
The same principle applies when the parent exists but has not yet grown far enough to contain the logical point referenced by the incoming history.
|
|
543
|
+
|
|
544
|
+
---
|
|
545
|
+
|
|
546
|
+
# Author-time relationships survive receiver-time differences
|
|
547
|
+
|
|
548
|
+
Consider an author producing:
|
|
549
|
+
|
|
550
|
+
```text
|
|
551
|
+
world
|
|
552
|
+
Hello world
|
|
553
|
+
Well, Hello world
|
|
554
|
+
Well, actually Hello world
|
|
555
|
+
Now, Well, actually Hello world
|
|
556
|
+
Okay, Now, Well, actually Hello world
|
|
557
|
+
```
|
|
558
|
+
|
|
559
|
+
Another replica might receive the individual pieces in a very different order:
|
|
560
|
+
|
|
561
|
+
```text
|
|
562
|
+
world
|
|
563
|
+
Hello
|
|
564
|
+
actually
|
|
565
|
+
Now
|
|
566
|
+
Well
|
|
567
|
+
Okay
|
|
568
|
+
```
|
|
569
|
+
|
|
570
|
+
The operations do not depend on the receiver seeing the same intermediate Projection that the author saw.
|
|
571
|
+
|
|
572
|
+
Each operation describes where it belonged relative to existing canonical history at author time.
|
|
573
|
+
|
|
574
|
+
Once all necessary relationships are available, they reconstruct:
|
|
575
|
+
|
|
576
|
+
```text
|
|
577
|
+
Okay, Now, Well, actually Hello world
|
|
578
|
+
```
|
|
579
|
+
|
|
580
|
+
Sequencer therefore preserves:
|
|
581
|
+
|
|
582
|
+
```text
|
|
583
|
+
author-time relationships
|
|
584
|
+
```
|
|
585
|
+
|
|
586
|
+
rather than:
|
|
587
|
+
|
|
588
|
+
```text
|
|
589
|
+
author-time numeric indexes
|
|
590
|
+
```
|
|
591
|
+
|
|
592
|
+
---
|
|
593
|
+
|
|
594
|
+
# Canonical history and runtime fragmentation
|
|
595
|
+
|
|
596
|
+
A canonical Insertion is a logical piece of history.
|
|
597
|
+
|
|
598
|
+
It does not need to remain one physical runtime object forever.
|
|
599
|
+
|
|
600
|
+
Suppose one Insertion originally contains:
|
|
601
|
+
|
|
602
|
+
```text
|
|
603
|
+
A B C D
|
|
604
|
+
```
|
|
605
|
+
|
|
606
|
+
Later edits may require the runtime to split it:
|
|
607
|
+
|
|
608
|
+
```text
|
|
609
|
+
[A B] [C D]
|
|
610
|
+
```
|
|
611
|
+
|
|
612
|
+
These pieces are called **Strips**.
|
|
613
|
+
|
|
614
|
+
The important point is that they are not new canonical Insertions.
|
|
615
|
+
|
|
616
|
+
Both still belong to the same original history:
|
|
617
|
+
|
|
618
|
+
```text
|
|
619
|
+
same anchor
|
|
620
|
+
same insertion identity
|
|
621
|
+
same original coordinate space
|
|
622
|
+
```
|
|
623
|
+
|
|
624
|
+
They merely represent different portions of it at runtime.
|
|
625
|
+
|
|
626
|
+
Conceptually:
|
|
627
|
+
|
|
628
|
+
```text
|
|
629
|
+
canonical Insertion
|
|
630
|
+
|
|
631
|
+
| A | B | C | D |
|
|
632
|
+
0 1 2 3 4
|
|
633
|
+
|
|
634
|
+
↓
|
|
635
|
+
|
|
636
|
+
runtime materialization
|
|
637
|
+
|
|
638
|
+
[A B] [C D]
|
|
639
|
+
```
|
|
640
|
+
|
|
641
|
+
The logical points remain:
|
|
642
|
+
|
|
643
|
+
```text
|
|
644
|
+
0 1 2 3 4
|
|
645
|
+
```
|
|
646
|
+
|
|
647
|
+
even if the current physical representation is split around point `2`.
|
|
648
|
+
|
|
649
|
+
This allows old operations to continue referring to historical locations after arbitrary later fragmentation.
|
|
650
|
+
|
|
651
|
+
---
|
|
652
|
+
|
|
653
|
+
# Canonical coordinates survive fragmentation
|
|
654
|
+
|
|
655
|
+
This separation between canonical identity and runtime representation is essential.
|
|
656
|
+
|
|
657
|
+
A replica may represent one Insertion as:
|
|
658
|
+
|
|
659
|
+
```text
|
|
660
|
+
[A B C D]
|
|
661
|
+
```
|
|
662
|
+
|
|
663
|
+
while another currently represents the same canonical Insertion as:
|
|
664
|
+
|
|
665
|
+
```text
|
|
666
|
+
[A] [B C] [D]
|
|
667
|
+
```
|
|
668
|
+
|
|
669
|
+
The two replicas still agree on:
|
|
670
|
+
|
|
671
|
+
```text
|
|
672
|
+
which Insertion it is
|
|
673
|
+
```
|
|
674
|
+
|
|
675
|
+
and:
|
|
676
|
+
|
|
677
|
+
```text
|
|
678
|
+
which original logical point an anchor refers to
|
|
679
|
+
```
|
|
680
|
+
|
|
681
|
+
They do not need identical runtime layouts.
|
|
682
|
+
|
|
683
|
+
The receiver resolves canonical coordinates through whatever fragmentation exists locally.
|
|
684
|
+
|
|
685
|
+
Runtime fragmentation is therefore invisible to replication.
|
|
686
|
+
|
|
687
|
+
---
|
|
688
|
+
|
|
689
|
+
# Consecutive editing can extend history
|
|
690
|
+
|
|
691
|
+
Without any coalescing, typing:
|
|
692
|
+
|
|
693
|
+
```text
|
|
694
|
+
H
|
|
695
|
+
He
|
|
696
|
+
Hel
|
|
697
|
+
Hell
|
|
698
|
+
Hello
|
|
699
|
+
```
|
|
700
|
+
|
|
701
|
+
could create one canonical Insertion for every character.
|
|
702
|
+
|
|
703
|
+
That would preserve correctness, but history would grow unnecessarily quickly.
|
|
704
|
+
|
|
705
|
+
Sequencer can instead extend a recent positive Insertion when the new edit is a genuine continuation of that same history.
|
|
706
|
+
|
|
707
|
+
Conceptually:
|
|
708
|
+
|
|
709
|
+
```text
|
|
710
|
+
H
|
|
711
|
+
↓
|
|
712
|
+
He
|
|
713
|
+
↓
|
|
714
|
+
Hel
|
|
715
|
+
↓
|
|
716
|
+
Hell
|
|
717
|
+
↓
|
|
718
|
+
Hello
|
|
719
|
+
```
|
|
720
|
+
|
|
721
|
+
may remain one growing canonical Insertion.
|
|
722
|
+
|
|
723
|
+
The key requirement is that extending it must not change previously established structural relationships.
|
|
724
|
+
|
|
725
|
+
If history has already attached to the old endpoint in a way that requires that boundary to remain distinct, a new independent Insertion is created instead.
|
|
726
|
+
|
|
727
|
+
If the canonical Insertion has fragmented at runtime, a valid extension can still become another fragment of the same grown canonical identity.
|
|
728
|
+
|
|
729
|
+
Extension is therefore a history-growth optimization that preserves the same underlying model.
|
|
730
|
+
|
|
731
|
+
---
|
|
732
|
+
|
|
733
|
+
# Removal is not deletion of history
|
|
734
|
+
|
|
735
|
+
Removing visible content does not immediately erase its original Insertion.
|
|
736
|
+
|
|
737
|
+
Instead, removal is represented as new history.
|
|
738
|
+
|
|
739
|
+
A removal is a reducing Insertion:
|
|
740
|
+
|
|
741
|
+
```text
|
|
742
|
+
insertionDiff < 0
|
|
743
|
+
```
|
|
744
|
+
|
|
745
|
+
which targets original Frames belonging to a positive Insertion.
|
|
746
|
+
|
|
747
|
+
This is best understood as **masking**.
|
|
748
|
+
|
|
749
|
+
Suppose the original content is:
|
|
750
|
+
|
|
751
|
+
```text
|
|
752
|
+
A B C D
|
|
753
|
+
```
|
|
754
|
+
|
|
755
|
+
and `B C` are removed.
|
|
756
|
+
|
|
757
|
+
The historical positive Insertion still exists.
|
|
758
|
+
|
|
759
|
+
A reducing Insertion records:
|
|
760
|
+
|
|
761
|
+
```text
|
|
762
|
+
mask original Frames B and C
|
|
763
|
+
```
|
|
764
|
+
|
|
765
|
+
The Projection then contains:
|
|
766
|
+
|
|
767
|
+
```text
|
|
768
|
+
A D
|
|
769
|
+
```
|
|
770
|
+
|
|
771
|
+
but Structural Order still knows what happened.
|
|
772
|
+
|
|
773
|
+
---
|
|
774
|
+
|
|
775
|
+
# Why removal must target original Frames
|
|
776
|
+
|
|
777
|
+
Consider two replicas starting with:
|
|
778
|
+
|
|
779
|
+
```text
|
|
780
|
+
The old house
|
|
781
|
+
```
|
|
782
|
+
|
|
783
|
+
Replica A removes:
|
|
784
|
+
|
|
785
|
+
```text
|
|
786
|
+
old
|
|
787
|
+
```
|
|
788
|
+
|
|
789
|
+
while Replica B concurrently inserts:
|
|
790
|
+
|
|
791
|
+
```text
|
|
792
|
+
small
|
|
793
|
+
```
|
|
794
|
+
|
|
795
|
+
after it:
|
|
796
|
+
|
|
797
|
+
```text
|
|
798
|
+
The old small house
|
|
799
|
+
```
|
|
800
|
+
|
|
801
|
+
Replica A never observed:
|
|
802
|
+
|
|
803
|
+
```text
|
|
804
|
+
small
|
|
805
|
+
```
|
|
806
|
+
|
|
807
|
+
Its removal should therefore not remove that concurrent content.
|
|
808
|
+
|
|
809
|
+
Because removal targets original Frames rather than a mutable visible interval, it masks only:
|
|
810
|
+
|
|
811
|
+
```text
|
|
812
|
+
old
|
|
813
|
+
```
|
|
814
|
+
|
|
815
|
+
After synchronization the result becomes:
|
|
816
|
+
|
|
817
|
+
```text
|
|
818
|
+
The small house
|
|
819
|
+
```
|
|
820
|
+
|
|
821
|
+
regardless of delivery order.
|
|
822
|
+
|
|
823
|
+
This is one of the core consequences of using historical coordinates rather than Projection indexes.
|
|
824
|
+
|
|
825
|
+
---
|
|
826
|
+
|
|
827
|
+
# A visible removal may cross several histories
|
|
828
|
+
|
|
829
|
+
What looks like one contiguous visible range may internally contain Frames from several canonical Insertions.
|
|
830
|
+
|
|
831
|
+
For example:
|
|
832
|
+
|
|
833
|
+
```text
|
|
834
|
+
Hello world
|
|
835
|
+
```
|
|
836
|
+
|
|
837
|
+
might structurally contain:
|
|
838
|
+
|
|
839
|
+
```text
|
|
840
|
+
[Hello]
|
|
841
|
+
[ ]
|
|
842
|
+
[world]
|
|
843
|
+
```
|
|
844
|
+
|
|
845
|
+
from three different historical Insertions.
|
|
846
|
+
|
|
847
|
+
Removing:
|
|
848
|
+
|
|
849
|
+
```text
|
|
850
|
+
lo wo
|
|
851
|
+
```
|
|
852
|
+
|
|
853
|
+
therefore cannot be represented as one blind operation against a Projection range.
|
|
854
|
+
|
|
855
|
+
Sequencer expresses the removal against the actual original Insertions whose Frames are being consumed.
|
|
856
|
+
|
|
857
|
+
Each reducing Insertion targets only Frames belonging to its own historical parent.
|
|
858
|
+
|
|
859
|
+
This prevents removals from accidentally claiming content they never observed.
|
|
860
|
+
|
|
861
|
+
---
|
|
862
|
+
|
|
863
|
+
# Removal can also fragment
|
|
864
|
+
|
|
865
|
+
A single canonical removal may encounter an original Insertion that has already fragmented.
|
|
866
|
+
|
|
867
|
+
Suppose original history contains:
|
|
868
|
+
|
|
869
|
+
```text
|
|
870
|
+
A B C D E F
|
|
871
|
+
```
|
|
872
|
+
|
|
873
|
+
but the current runtime representation is:
|
|
874
|
+
|
|
875
|
+
```text
|
|
876
|
+
[A B] [C D] [E F]
|
|
877
|
+
```
|
|
878
|
+
|
|
879
|
+
A removal of:
|
|
880
|
+
|
|
881
|
+
```text
|
|
882
|
+
B C D E
|
|
883
|
+
```
|
|
884
|
+
|
|
885
|
+
may need to materialize across several runtime regions.
|
|
886
|
+
|
|
887
|
+
Those pieces are still one canonical removal.
|
|
888
|
+
|
|
889
|
+
They do not become independent replicated operations merely because the runtime needed several Strips to express their effect.
|
|
890
|
+
|
|
891
|
+
Again:
|
|
892
|
+
|
|
893
|
+
```text
|
|
894
|
+
canonical history
|
|
895
|
+
```
|
|
896
|
+
|
|
897
|
+
and:
|
|
898
|
+
|
|
899
|
+
```text
|
|
900
|
+
runtime representation
|
|
901
|
+
```
|
|
902
|
+
|
|
903
|
+
remain separate.
|
|
904
|
+
|
|
905
|
+
---
|
|
906
|
+
|
|
907
|
+
# Overlapping removals
|
|
908
|
+
|
|
909
|
+
Concurrent replicas may independently remove overlapping Frames.
|
|
910
|
+
|
|
911
|
+
For example, from:
|
|
912
|
+
|
|
913
|
+
```text
|
|
914
|
+
the brown fox jumps
|
|
915
|
+
```
|
|
916
|
+
|
|
917
|
+
one replica removes:
|
|
918
|
+
|
|
919
|
+
```text
|
|
920
|
+
brown fox
|
|
921
|
+
```
|
|
922
|
+
|
|
923
|
+
while another removes:
|
|
924
|
+
|
|
925
|
+
```text
|
|
926
|
+
fox jumps
|
|
927
|
+
```
|
|
928
|
+
|
|
929
|
+
Both removals refer to original Frames.
|
|
930
|
+
|
|
931
|
+
The Frames belonging to:
|
|
932
|
+
|
|
933
|
+
```text
|
|
934
|
+
fox
|
|
935
|
+
```
|
|
936
|
+
|
|
937
|
+
were targeted twice.
|
|
938
|
+
|
|
939
|
+
They must still disappear only once from the Projection.
|
|
940
|
+
|
|
941
|
+
A reducing Insertion therefore retains its canonical authored meaning while its current runtime effect is resolved against Frames that remain available to consume.
|
|
942
|
+
|
|
943
|
+
This guarantees:
|
|
944
|
+
|
|
945
|
+
```text
|
|
946
|
+
one original Frame cannot reduce the Projection twice
|
|
947
|
+
```
|
|
948
|
+
|
|
949
|
+
Delivery order cannot cause repeated subtraction or make deleted content reappear.
|
|
950
|
+
|
|
951
|
+
---
|
|
952
|
+
|
|
953
|
+
# Replacement is composition
|
|
954
|
+
|
|
955
|
+
Replacement does not require its own structural concept.
|
|
956
|
+
|
|
957
|
+
Replacing:
|
|
958
|
+
|
|
959
|
+
```text
|
|
960
|
+
Draft
|
|
961
|
+
```
|
|
962
|
+
|
|
963
|
+
with:
|
|
964
|
+
|
|
965
|
+
```text
|
|
966
|
+
Final
|
|
967
|
+
```
|
|
968
|
+
|
|
969
|
+
is simply:
|
|
970
|
+
|
|
971
|
+
```text
|
|
972
|
+
remove Draft
|
|
973
|
+
insert Final
|
|
974
|
+
```
|
|
975
|
+
|
|
976
|
+
The important part is the relationship between those two operations.
|
|
977
|
+
|
|
978
|
+
The new content is anchored against the history that exists after the removal relationship has been established.
|
|
979
|
+
|
|
980
|
+
This keeps replacement semantics consistent with ordinary insertion and removal instead of introducing another independent ordering model.
|
|
981
|
+
|
|
982
|
+
---
|
|
983
|
+
|
|
984
|
+
# Visible content and historical content are deliberately different
|
|
985
|
+
|
|
986
|
+
After enough editing, Structural Order may contain much more history than the Projection shows.
|
|
987
|
+
|
|
988
|
+
For example:
|
|
989
|
+
|
|
990
|
+
```text
|
|
991
|
+
Structural Order
|
|
992
|
+
|
|
993
|
+
[A B C]
|
|
994
|
+
[mask B]
|
|
995
|
+
[X]
|
|
996
|
+
[mask C]
|
|
997
|
+
[Y]
|
|
998
|
+
```
|
|
999
|
+
|
|
1000
|
+
could produce a Projection such as:
|
|
1001
|
+
|
|
1002
|
+
```text
|
|
1003
|
+
A X Y
|
|
1004
|
+
```
|
|
1005
|
+
|
|
1006
|
+
The hidden history is not accidental garbage.
|
|
1007
|
+
|
|
1008
|
+
It may still be required to understand:
|
|
1009
|
+
|
|
1010
|
+
```text
|
|
1011
|
+
future delayed operations
|
|
1012
|
+
historical anchor points
|
|
1013
|
+
concurrent relationships
|
|
1014
|
+
removal ownership
|
|
1015
|
+
ordering of descendants
|
|
1016
|
+
```
|
|
1017
|
+
|
|
1018
|
+
Sequencer therefore cannot simply delete every hidden node as soon as it stops being visible.
|
|
1019
|
+
|
|
1020
|
+
---
|
|
1021
|
+
|
|
1022
|
+
# Replication transports canonical history
|
|
1023
|
+
|
|
1024
|
+
Replicas exchange canonical Insertions and acknowledgement information.
|
|
1025
|
+
|
|
1026
|
+
The transport itself is not part of the sequencing model.
|
|
1027
|
+
|
|
1028
|
+
History may be delivered:
|
|
1029
|
+
|
|
1030
|
+
```text
|
|
1031
|
+
immediately
|
|
1032
|
+
later
|
|
1033
|
+
out of order
|
|
1034
|
+
more than once
|
|
1035
|
+
through a central server
|
|
1036
|
+
peer to peer
|
|
1037
|
+
after being persisted
|
|
1038
|
+
```
|
|
1039
|
+
|
|
1040
|
+
None of these should affect the eventual ordering.
|
|
1041
|
+
|
|
1042
|
+
The important property is that the same canonical history eventually reaches the replicas that need it.
|
|
1043
|
+
|
|
1044
|
+
---
|
|
1045
|
+
|
|
1046
|
+
# Duplicate delivery is harmless
|
|
1047
|
+
|
|
1048
|
+
Because canonical Insertions have stable identities, receiving the same history multiple times does not duplicate its visible effect.
|
|
1049
|
+
|
|
1050
|
+
This allows replication systems to favor reliability over exactly-once transport.
|
|
1051
|
+
|
|
1052
|
+
A sender can retransmit uncertain history.
|
|
1053
|
+
|
|
1054
|
+
A relay can replay stored history.
|
|
1055
|
+
|
|
1056
|
+
A reconnecting replica can receive data it already has.
|
|
1057
|
+
|
|
1058
|
+
Sequencer recognizes the canonical identity and keeps the resulting history idempotent.
|
|
1059
|
+
|
|
1060
|
+
---
|
|
1061
|
+
|
|
1062
|
+
# Actors and Sessions have different roles
|
|
1063
|
+
|
|
1064
|
+
A **Session** identifies authored sequencing history.
|
|
1065
|
+
|
|
1066
|
+
An **Actor** participates in acknowledgement of removal history.
|
|
1067
|
+
|
|
1068
|
+
These concepts are intentionally separate.
|
|
1069
|
+
|
|
1070
|
+
One Actor may have several independent Sessions.
|
|
1071
|
+
|
|
1072
|
+
Different replicas belonging to the same logical Actor can therefore create history without sharing one mutable sequencing clock.
|
|
1073
|
+
|
|
1074
|
+
Session identities answer:
|
|
1075
|
+
|
|
1076
|
+
```text
|
|
1077
|
+
who authored this canonical history stream?
|
|
1078
|
+
```
|
|
1079
|
+
|
|
1080
|
+
Actor identities answer:
|
|
1081
|
+
|
|
1082
|
+
```text
|
|
1083
|
+
whose acknowledgement is required before old removal history can be forgotten?
|
|
1084
|
+
```
|
|
1085
|
+
|
|
1086
|
+
---
|
|
1087
|
+
|
|
1088
|
+
# Why acknowledgements are needed
|
|
1089
|
+
|
|
1090
|
+
Suppose Replica A removes some content.
|
|
1091
|
+
|
|
1092
|
+
Replica B is currently offline.
|
|
1093
|
+
|
|
1094
|
+
Replica B may still contain unsent history anchored to the content that A removed.
|
|
1095
|
+
|
|
1096
|
+
If A permanently discarded every trace of that historical content immediately, B's delayed operation could later refer to a point that no longer exists anywhere in A's retained state.
|
|
1097
|
+
|
|
1098
|
+
Sequencer therefore separates:
|
|
1099
|
+
|
|
1100
|
+
```text
|
|
1101
|
+
content is no longer visible
|
|
1102
|
+
```
|
|
1103
|
+
|
|
1104
|
+
from:
|
|
1105
|
+
|
|
1106
|
+
```text
|
|
1107
|
+
history is safe to forget
|
|
1108
|
+
```
|
|
1109
|
+
|
|
1110
|
+
Those are not the same event.
|
|
1111
|
+
|
|
1112
|
+
---
|
|
1113
|
+
|
|
1114
|
+
# Removal frontiers
|
|
1115
|
+
|
|
1116
|
+
Actors acknowledge received prefixes of removal Sessions.
|
|
1117
|
+
|
|
1118
|
+
Conceptually an acknowledgement says:
|
|
1119
|
+
|
|
1120
|
+
```text
|
|
1121
|
+
Actor A has observed this removal Session
|
|
1122
|
+
through logical point N
|
|
1123
|
+
```
|
|
1124
|
+
|
|
1125
|
+
A frontier summarizes a whole contiguous prefix rather than listing every individual removal.
|
|
1126
|
+
|
|
1127
|
+
This keeps acknowledgement state small while still allowing Sequencer to determine which removal history is globally known among the active Actors.
|
|
1128
|
+
|
|
1129
|
+
---
|
|
1130
|
+
|
|
1131
|
+
# Compaction
|
|
1132
|
+
|
|
1133
|
+
Once all active Actors agree on a removal frontier, the acknowledged removal history becomes eligible for compaction.
|
|
1134
|
+
|
|
1135
|
+
Compaction does not simply delete mask nodes.
|
|
1136
|
+
|
|
1137
|
+
It constructs an equivalent retained history in which obsolete removed Frames and the historical machinery surrounding them can be omitted.
|
|
1138
|
+
|
|
1139
|
+
This may involve:
|
|
1140
|
+
|
|
1141
|
+
```text
|
|
1142
|
+
removing acknowledged masks
|
|
1143
|
+
removing Footage that can no longer reappear
|
|
1144
|
+
shortening positive Insertions
|
|
1145
|
+
adjusting surviving mask coordinates
|
|
1146
|
+
collapsing obsolete logical regions
|
|
1147
|
+
re-anchoring surviving history
|
|
1148
|
+
```
|
|
1149
|
+
|
|
1150
|
+
The required invariant is simple:
|
|
1151
|
+
|
|
1152
|
+
```text
|
|
1153
|
+
Projection before compaction
|
|
1154
|
+
==
|
|
1155
|
+
Projection reconstructed from compacted history
|
|
1156
|
+
```
|
|
1157
|
+
|
|
1158
|
+
But preserving that invariant under future editing is the harder part.
|
|
1159
|
+
|
|
1160
|
+
The compacted history must not only look the same now.
|
|
1161
|
+
|
|
1162
|
+
It must also preserve the ordering relationships that remain relevant later.
|
|
1163
|
+
|
|
1164
|
+
---
|
|
1165
|
+
|
|
1166
|
+
# Compaction can collapse historical space
|
|
1167
|
+
|
|
1168
|
+
Consider surviving children originally attached to two different logical points:
|
|
1169
|
+
|
|
1170
|
+
```text
|
|
1171
|
+
point 1
|
|
1172
|
+
↓
|
|
1173
|
+
removed historical region
|
|
1174
|
+
↓
|
|
1175
|
+
point 2
|
|
1176
|
+
```
|
|
1177
|
+
|
|
1178
|
+
After the region between them becomes safe to forget, both historical points may collapse onto the same retained boundary.
|
|
1179
|
+
|
|
1180
|
+
At that point the original geometric distinction no longer exists.
|
|
1181
|
+
|
|
1182
|
+
But the children may already have a resolved order.
|
|
1183
|
+
|
|
1184
|
+
Compaction must preserve that order.
|
|
1185
|
+
|
|
1186
|
+
Sequencer can therefore re-anchor surviving positive history according to its already resolved retained Structural Order.
|
|
1187
|
+
|
|
1188
|
+
In other words, once obsolete history is discarded:
|
|
1189
|
+
|
|
1190
|
+
```text
|
|
1191
|
+
resolved order
|
|
1192
|
+
```
|
|
1193
|
+
|
|
1194
|
+
can become more important than:
|
|
1195
|
+
|
|
1196
|
+
```text
|
|
1197
|
+
the exact historical path that originally produced it
|
|
1198
|
+
```
|
|
1199
|
+
|
|
1200
|
+
This is what lets retained state become smaller without reopening old ordering decisions.
|
|
1201
|
+
|
|
1202
|
+
---
|
|
1203
|
+
|
|
1204
|
+
# Compaction does not rewrite live history
|
|
1205
|
+
|
|
1206
|
+
Compaction operates on retained state.
|
|
1207
|
+
|
|
1208
|
+
A live Projection does not need to restructure itself every time a snapshot is produced.
|
|
1209
|
+
|
|
1210
|
+
Instead:
|
|
1211
|
+
|
|
1212
|
+
```text
|
|
1213
|
+
live Structural Order
|
|
1214
|
+
↓
|
|
1215
|
+
derive equivalent compacted history
|
|
1216
|
+
↓
|
|
1217
|
+
persist / transfer snapshot
|
|
1218
|
+
```
|
|
1219
|
+
|
|
1220
|
+
A later Projection can hydrate from that compacted history and produce the same visible state and compatible future ordering.
|
|
1221
|
+
|
|
1222
|
+
This keeps persistence optimization separate from the current editing runtime.
|
|
1223
|
+
|
|
1224
|
+
---
|
|
1225
|
+
|
|
1226
|
+
# Actor retirement
|
|
1227
|
+
|
|
1228
|
+
A permanently gone Actor would otherwise block removal compaction forever.
|
|
1229
|
+
|
|
1230
|
+
Sequencer therefore allows an Actor to be retired from future acknowledgement consensus.
|
|
1231
|
+
|
|
1232
|
+
Retirement means:
|
|
1233
|
+
|
|
1234
|
+
```text
|
|
1235
|
+
this Actor is no longer expected to acknowledge future compaction
|
|
1236
|
+
```
|
|
1237
|
+
|
|
1238
|
+
It does not mean:
|
|
1239
|
+
|
|
1240
|
+
```text
|
|
1241
|
+
delete history authored by this Actor
|
|
1242
|
+
```
|
|
1243
|
+
|
|
1244
|
+
Those are unrelated concepts.
|
|
1245
|
+
|
|
1246
|
+
Retirement is an application-level decision because Sequencer cannot know whether a disconnected replica is temporarily offline or permanently abandoned.
|
|
1247
|
+
|
|
1248
|
+
---
|
|
1249
|
+
|
|
1250
|
+
# Persistence retains history, not runtime state
|
|
1251
|
+
|
|
1252
|
+
A persisted Sequence contains the history needed to reconstruct the Projection.
|
|
1253
|
+
|
|
1254
|
+
It does not need to persist every runtime optimization.
|
|
1255
|
+
|
|
1256
|
+
For example, persistence needs the equivalent of:
|
|
1257
|
+
|
|
1258
|
+
```text
|
|
1259
|
+
canonical Insertions
|
|
1260
|
+
retained acknowledgement state
|
|
1261
|
+
positive Footage
|
|
1262
|
+
```
|
|
1263
|
+
|
|
1264
|
+
It does not need to preserve things such as:
|
|
1265
|
+
|
|
1266
|
+
```text
|
|
1267
|
+
current fragmentation layout
|
|
1268
|
+
temporary traversal indexes
|
|
1269
|
+
recent lookup positions
|
|
1270
|
+
allocation addresses
|
|
1271
|
+
```
|
|
1272
|
+
|
|
1273
|
+
Those can be reconstructed.
|
|
1274
|
+
|
|
1275
|
+
The persistence boundary therefore follows the same conceptual separation as replication:
|
|
1276
|
+
|
|
1277
|
+
```text
|
|
1278
|
+
stable history is durable
|
|
1279
|
+
runtime materialization is disposable
|
|
1280
|
+
```
|
|
1281
|
+
|
|
1282
|
+
---
|
|
1283
|
+
|
|
1284
|
+
# Hydration reconstructs an equivalent Projection
|
|
1285
|
+
|
|
1286
|
+
When retained history is loaded again, Sequencer reconstructs the structural relationships encoded by that history and derives the visible Projection.
|
|
1287
|
+
|
|
1288
|
+
The important requirement is not that the new runtime representation be byte-for-byte identical to the previous one.
|
|
1289
|
+
|
|
1290
|
+
It only needs to reconstruct the same meaningful state:
|
|
1291
|
+
|
|
1292
|
+
```text
|
|
1293
|
+
same visible content
|
|
1294
|
+
same retained canonical relationships
|
|
1295
|
+
same deterministic future ordering
|
|
1296
|
+
```
|
|
1297
|
+
|
|
1298
|
+
Runtime Strip boundaries or indexes may differ.
|
|
1299
|
+
|
|
1300
|
+
They are not part of the distributed identity of the sequence.
|
|
1301
|
+
|
|
1302
|
+
---
|
|
1303
|
+
|
|
1304
|
+
# The same history can have different runtime shapes
|
|
1305
|
+
|
|
1306
|
+
This is a useful general principle.
|
|
1307
|
+
|
|
1308
|
+
Replica A might internally represent some history as:
|
|
1309
|
+
|
|
1310
|
+
```text
|
|
1311
|
+
[A B C] [D E]
|
|
1312
|
+
```
|
|
1313
|
+
|
|
1314
|
+
Replica B might represent the same canonical history as:
|
|
1315
|
+
|
|
1316
|
+
```text
|
|
1317
|
+
[A] [B C] [D] [E]
|
|
1318
|
+
```
|
|
1319
|
+
|
|
1320
|
+
A hydrated replica may create yet another local representation.
|
|
1321
|
+
|
|
1322
|
+
As long as all of them agree on:
|
|
1323
|
+
|
|
1324
|
+
```text
|
|
1325
|
+
canonical identities
|
|
1326
|
+
canonical anchors
|
|
1327
|
+
original Frame coordinates
|
|
1328
|
+
deterministic Structural Order
|
|
1329
|
+
```
|
|
1330
|
+
|
|
1331
|
+
they represent the same Sequence.
|
|
1332
|
+
|
|
1333
|
+
Convergence is therefore defined over logical history and Projection, not over internal memory layout.
|
|
1334
|
+
|
|
1335
|
+
---
|
|
1336
|
+
|
|
1337
|
+
# Three coordinate spaces
|
|
1338
|
+
|
|
1339
|
+
Many apparent complexities in Sequencer become much simpler when three different coordinate spaces are kept separate.
|
|
1340
|
+
|
|
1341
|
+
## Projection positions
|
|
1342
|
+
|
|
1343
|
+
These are the indexes applications use:
|
|
1344
|
+
|
|
1345
|
+
```text
|
|
1346
|
+
0, 1, 2, 3, ...
|
|
1347
|
+
```
|
|
1348
|
+
|
|
1349
|
+
They describe the visible sequence right now.
|
|
1350
|
+
|
|
1351
|
+
They are temporary.
|
|
1352
|
+
|
|
1353
|
+
---
|
|
1354
|
+
|
|
1355
|
+
## Canonical logical coordinates
|
|
1356
|
+
|
|
1357
|
+
These identify Frames and boundaries inside original Insertions.
|
|
1358
|
+
|
|
1359
|
+
They survive:
|
|
1360
|
+
|
|
1361
|
+
```text
|
|
1362
|
+
concurrent edits
|
|
1363
|
+
fragmentation
|
|
1364
|
+
removal
|
|
1365
|
+
reordered delivery
|
|
1366
|
+
restart
|
|
1367
|
+
```
|
|
1368
|
+
|
|
1369
|
+
These are the coordinates replication cares about.
|
|
1370
|
+
|
|
1371
|
+
---
|
|
1372
|
+
|
|
1373
|
+
## Runtime structural positions
|
|
1374
|
+
|
|
1375
|
+
A live implementation also has some local way to identify materialized pieces of Structural Order.
|
|
1376
|
+
|
|
1377
|
+
Those positions are purely runtime state.
|
|
1378
|
+
|
|
1379
|
+
Different replicas do not need to share them.
|
|
1380
|
+
|
|
1381
|
+
---
|
|
1382
|
+
|
|
1383
|
+
So:
|
|
1384
|
+
|
|
1385
|
+
```text
|
|
1386
|
+
Projection position
|
|
1387
|
+
!=
|
|
1388
|
+
canonical logical coordinate
|
|
1389
|
+
!=
|
|
1390
|
+
runtime structural position
|
|
1391
|
+
```
|
|
1392
|
+
|
|
1393
|
+
Keeping those concepts separate is the easiest way to reason about Sequencer.
|
|
1394
|
+
|
|
1395
|
+
---
|
|
1396
|
+
|
|
1397
|
+
# The model in one example
|
|
1398
|
+
|
|
1399
|
+
Start with an empty Projection.
|
|
1400
|
+
|
|
1401
|
+
Replica A inserts:
|
|
1402
|
+
|
|
1403
|
+
```text
|
|
1404
|
+
Hello
|
|
1405
|
+
```
|
|
1406
|
+
|
|
1407
|
+
This creates positive canonical history.
|
|
1408
|
+
|
|
1409
|
+
Replica B later receives that history.
|
|
1410
|
+
|
|
1411
|
+
Both now derive:
|
|
1412
|
+
|
|
1413
|
+
```text
|
|
1414
|
+
Hello
|
|
1415
|
+
```
|
|
1416
|
+
|
|
1417
|
+
Replica A inserts:
|
|
1418
|
+
|
|
1419
|
+
```text
|
|
1420
|
+
world
|
|
1421
|
+
```
|
|
1422
|
+
|
|
1423
|
+
at the end.
|
|
1424
|
+
|
|
1425
|
+
If eligible, the existing positive history may simply extend.
|
|
1426
|
+
|
|
1427
|
+
Meanwhile Replica B inserts:
|
|
1428
|
+
|
|
1429
|
+
```text
|
|
1430
|
+
beautiful
|
|
1431
|
+
```
|
|
1432
|
+
|
|
1433
|
+
at the same historical boundary.
|
|
1434
|
+
|
|
1435
|
+
Now the replicas contain concurrent history.
|
|
1436
|
+
|
|
1437
|
+
They may receive each other's edits in opposite orders.
|
|
1438
|
+
|
|
1439
|
+
Because both operations describe stable historical relationships, Sequencer deterministically resolves them.
|
|
1440
|
+
|
|
1441
|
+
Suppose the resulting Projection is:
|
|
1442
|
+
|
|
1443
|
+
```text
|
|
1444
|
+
Hello beautiful world
|
|
1445
|
+
```
|
|
1446
|
+
|
|
1447
|
+
Replica A removes:
|
|
1448
|
+
|
|
1449
|
+
```text
|
|
1450
|
+
beautiful
|
|
1451
|
+
```
|
|
1452
|
+
|
|
1453
|
+
while Replica B concurrently inserts:
|
|
1454
|
+
|
|
1455
|
+
```text
|
|
1456
|
+
wide
|
|
1457
|
+
```
|
|
1458
|
+
|
|
1459
|
+
inside nearby content.
|
|
1460
|
+
|
|
1461
|
+
The removal masks only Frames actually observed by its author.
|
|
1462
|
+
|
|
1463
|
+
It cannot accidentally consume the concurrent insertion.
|
|
1464
|
+
|
|
1465
|
+
Later, all active Actors acknowledge the relevant removal history.
|
|
1466
|
+
|
|
1467
|
+
The removed Frames and their masks become eligible for compaction.
|
|
1468
|
+
|
|
1469
|
+
A persisted snapshot can then omit the obsolete historical region while retaining the already resolved order of everything that survived.
|
|
1470
|
+
|
|
1471
|
+
A newly hydrated replica reconstructs:
|
|
1472
|
+
|
|
1473
|
+
```text
|
|
1474
|
+
Hello wide world
|
|
1475
|
+
```
|
|
1476
|
+
|
|
1477
|
+
and continues editing from equivalent history.
|
|
1478
|
+
|
|
1479
|
+
That lifecycle contains almost the entire Sequencer model:
|
|
1480
|
+
|
|
1481
|
+
```text
|
|
1482
|
+
visible positions
|
|
1483
|
+
→ canonical anchors
|
|
1484
|
+
→ deterministic history
|
|
1485
|
+
→ concurrency
|
|
1486
|
+
→ fragmentation
|
|
1487
|
+
→ masking
|
|
1488
|
+
→ acknowledgement
|
|
1489
|
+
→ compaction
|
|
1490
|
+
→ equivalent reconstruction
|
|
1491
|
+
```
|
|
1492
|
+
|
|
1493
|
+
---
|
|
1494
|
+
|
|
1495
|
+
# The central idea
|
|
1496
|
+
|
|
1497
|
+
Sequencer works because temporary visible positions are never treated as permanent distributed identities.
|
|
1498
|
+
|
|
1499
|
+
Every edit is translated into stable historical relationships.
|
|
1500
|
+
|
|
1501
|
+
Those relationships can survive:
|
|
1502
|
+
|
|
1503
|
+
```text
|
|
1504
|
+
concurrent editing
|
|
1505
|
+
different packet orders
|
|
1506
|
+
duplicate delivery
|
|
1507
|
+
offline replicas
|
|
1508
|
+
fragmentation
|
|
1509
|
+
removal
|
|
1510
|
+
restart
|
|
1511
|
+
compaction
|
|
1512
|
+
```
|
|
1513
|
+
|
|
1514
|
+
The overall transformation is:
|
|
1515
|
+
|
|
1516
|
+
```text
|
|
1517
|
+
local Projection position
|
|
1518
|
+
↓
|
|
1519
|
+
canonical historical relationship
|
|
1520
|
+
↓
|
|
1521
|
+
deterministic Structural Order
|
|
1522
|
+
↓
|
|
1523
|
+
local Projection
|
|
1524
|
+
```
|
|
1525
|
+
|
|
1526
|
+
Every replica may have a different temporary view while history is still arriving.
|
|
1527
|
+
|
|
1528
|
+
Once they know the same canonical history, they can independently derive the same result.
|
|
1529
|
+
|
|
1530
|
+
That is the core of Sequencer.
|