@homeostate/tool-simulator 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,129 @@
1
1
  # @homeostate/tool-simulator
2
2
 
3
+ ## 0.3.0
4
+
5
+ ### Minor Changes
6
+
7
+ - f3f22e8: `advance` and `settle` now check, after every delivery, that no peer's store and document both lose a synced top-level key that no peer deleted, and reject with a `LostKeyError` naming the peer, the key, the virtual time and the seed. `runRandomized` runs the check too, so its error names the step in which a key was lost. Before, a store that dropped a key for a while and got it back once a late update arrived still converged, so the run passed. Runs that passed before can now fail. A key deleted during a delivery, such as by a store subscriber or a timer, is reported as lost: pass `checkLostKeys: false` to `createNetwork`, or in `runRandomized`'s `network`, to turn the check off.
8
+
9
+ ### Patch Changes
10
+
11
+ - e98195d: **Breaking:** a smaller sync engine API, with one way to do each thing. Migrate as follows:
12
+
13
+ - `SyncEngine` drops `isConnected()`, which returned what `getStatus()` holds. Replace
14
+ `engine.isConnected()` with `engine.getStatus().connected`. This also applies to the engine that
15
+ Zustand's and Redux's `store.homeostate`, `syncMobxStore` and `syncStateTree` return. An object
16
+ that implements `SyncEngine` by hand can drop the method.
17
+ - `SyncEvent` gains `"error"`, sent when the engine records a failure, which
18
+ `getStatus().failure` then holds. Work that throws, which used to send no event, now sends
19
+ `"error"`, before `onError` runs or the error propagates. A write loop sends it after the
20
+ `"remote"` of the apply that stopped it. A `switch` over `SyncEvent` must handle it.
21
+ - `SyncEngine` drops `subscribeStatus()`: every change of the status comes with a sync event, so
22
+ `subscribe` tells when to read it. Replace `engine.subscribeStatus(listener)` with
23
+ `engine.subscribe(() => listener())`, and read `engine.getStatus()` in it as before; it returns
24
+ the same object while the status is unchanged, so compare by identity to skip the events that
25
+ change nothing. An object that implements `SyncEngine` by hand can drop the method.
26
+ - `SyncEngineConfig.seed` is a boolean, `true` by default, and the `SeedStrategy` type is gone.
27
+ `"if-empty"` seeded each synced key the backend lacks, not only an empty backend. Replace
28
+ `seed: "if-empty"` with `seed: true`, or leave it out, and `seed: "never"` with `seed: false`.
29
+ This also applies to the options of Zustand's and Redux's `homeostate`, `syncMobxStore`,
30
+ `syncStateTree` and `atomWithSync`. Remove imports of `SeedStrategy`; type a variable that holds
31
+ the option as `boolean`. In plain JavaScript, a leftover `seed: "never"` is truthy and now
32
+ seeds, so replace it.
33
+ - The `ChangeType` object and type are removed; compare a step's `type` with its string literal.
34
+ Replace `ChangeType.INSERT`, `ChangeType.UPDATE` and `ChangeType.DELETE` with `"insert"`,
35
+ `"update"` and `"delete"`, and type a variable that holds a step type as `Change["type"]`.
36
+ - The `"pending"` step type is renamed `"edit"`: a step that edits the container at its key in
37
+ place by its nested script `changes`, as before. `getChanges` returns `"edit"` steps where it
38
+ returned `"pending"` ones. Replace `ChangeType.PENDING` and `"pending"` with `"edit"` wherever
39
+ you build a script or switch on a step's `type`; a `switch` left on `"pending"` no longer
40
+ matches, and TypeScript reports the comparison.
41
+ - `applyStringChanges`, `EditScript` and `Diffable` are no longer exported. `applyStringChanges`
42
+ returned the string a text script leads to, which is the second string given to `getChanges`,
43
+ so use that string; to edit a text in place, use `applyChanges` with `ApplyOps.editText`.
44
+ Replace `EditScript` with the script type of the kind diffed, `RecordChange[]`, `ListChange[]`
45
+ or `TextChange[]`, as `getChanges` returns it, and `Diffable` with
46
+ `JsonObject | unknown[] | string`; `getChanges` still needs both values narrowed to one kind.
47
+ - `count` is required on an array or string delete step: `ListChange` and `TextChange` declare
48
+ `count: number`, no longer `count?: number`. `getChanges` already set it on every such delete,
49
+ so only a script built by hand changes: add `count: 1` to an array or string delete that left
50
+ it out before passing the script to `applyChanges`. In plain JavaScript, such a delete no
51
+ longer removes one item, so add the `count` there too. A record delete still has no `count`.
52
+ - `getChanges` replaces a changed nested string whole by default, as `createCrdtBackend` and
53
+ `mergeConcurrent` already did: without `options.text`, a string in a record or an array that
54
+ changed is one `"update"` step instead of an `"edit"` step with a character-level script, and
55
+ so is what `applyChanges` writes. Two strings passed directly are still diffed character by
56
+ character. Pass `text: () => true` to keep character-level scripts for every nested string, or
57
+ a `TextPolicy` for some. The CRDT backends pass their `text` option, so they are unchanged.
58
+ - `createCrdtBackend` takes the parts as one object, `text` among them, and
59
+ `CrdtBackendOptions` is removed. Replace
60
+ `createCrdtBackend((text) => ({ read, transact, watch, revision }), { text })` with
61
+ `createCrdtBackend({ read, transact, watch, revision, text })`, and build the `ApplyOps` that
62
+ `transact` hands over from the same `text`, which is `undefined` when no string is text: core
63
+ no longer passes a policy with its default applied. `CrdtBackendParts` gains the optional
64
+ `text`. The Yjs, Loro and Automerge backends take their `text` option as before.
65
+ - `CrdtBackend.revision` and `CrdtBackendParts.revision` are required. A backend written by hand,
66
+ or parts passed to `createCrdtBackend`, must implement `revision`; return `undefined` when it
67
+ cannot tell, which keeps the read before every local change that a backend without `revision`
68
+ had. Such a backend now also gets no `previous` when the engine writes back what the store
69
+ changed during a remote apply, or a local change merged into one, since the engine cannot tell
70
+ whether the document changed during `setState`; the backend diffs against its document then.
71
+ The Yjs, Loro, Automerge and memory backends already implement it.
72
+ - `createSyncEngine` takes one options object, as the setup helpers do: the `backend`, the
73
+ `adapter` and the `SyncEngineConfig` options, flat. Replace
74
+ `createSyncEngine(backend, adapter, config)` with
75
+ `createSyncEngine({ backend, adapter, ...config })`, or list the options inline, such as
76
+ `createSyncEngine({ backend, adapter, omit: ["draft"] })`, and
77
+ `createSyncEngine(backend, adapter)` with `createSyncEngine({ backend, adapter })`. The state
78
+ type is still taken from `adapter`. The new `SyncEngineOptions<S>` type names the object. The
79
+ setup helpers' options are unchanged.
80
+ - `filter` takes the key alone, `(key: keyof S & string) => boolean`, and a key the store holds as
81
+ a function is never synced, whatever `filter` returns: the engine neither writes it nor reads the
82
+ backend's value into it. A store that makes a synced key a function deletes it from the backend
83
+ with its next write, as before, and syncs it again once it holds a JSON value. A backend value is
84
+ no longer judged: a synced key always takes it. `defaultSyncFilter` is removed. Drop
85
+ `filter: defaultSyncFilter`; replace `(key, value) => defaultSyncFilter(key, value) && key !==
86
+ "draft"` with `(key) => key !== "draft"`, or with `omit: ["draft"]`; elsewhere, replace
87
+ `defaultSyncFilter(key, value)` with `typeof value !== "function"`. A custom filter no longer
88
+ syncs functions it keeps, so it need not exclude them. A rule on any other value has no
89
+ replacement: choose the keys by name, or sync a slice without them with `scopeAdapter`. A filter
90
+ that takes a second parameter fails to compile; in plain JavaScript, it gets `undefined` there.
91
+
92
+ - 1bfdf95: `runRandomized` no longer reports peers that converge as diverged when updates written after
93
+ its final heal are dropped. With a `dropRate`, a store that writes in answer to remote changes,
94
+ such as a subscriber that keeps a derived key in synced state, writes new updates as the
95
+ documents `heal()` exchanges arrive, and the network can drop those too. The run healed once,
96
+ so a dropped update stayed lost and the run failed with a `ConvergenceError`. It now heals and
97
+ settles again until a round drops nothing, at most 10 rounds, as a provider that resyncs would.
98
+ The steps make the same random choices, so a seed replays the same steps. When the tenth round
99
+ still dropped updates and the peers did not converge, the error says so. `expectConverged()`
100
+ now also says how many updates were dropped since the last `heal()`, so tests that heal by hand
101
+ get the same hint: repeat `heal()` and `settle()` until `stats().dropped` stops growing.
102
+ - Updated dependencies [e98195d]
103
+ - Updated dependencies [e4e66f6]
104
+ - Updated dependencies [d17efad]
105
+ - Updated dependencies [7172cf4]
106
+ - Updated dependencies [3edede4]
107
+ - Updated dependencies [ea86a91]
108
+ - Updated dependencies [b2c1521]
109
+ - Updated dependencies [038c0e1]
110
+ - Updated dependencies [988dec0]
111
+ - Updated dependencies [48b0532]
112
+ - Updated dependencies [bf8324b]
113
+ - Updated dependencies [e97b920]
114
+ - Updated dependencies [2756e79]
115
+ - Updated dependencies [44a47f2]
116
+ - Updated dependencies [fab156f]
117
+ - Updated dependencies [e51dad7]
118
+ - Updated dependencies [1a6cfb8]
119
+ - Updated dependencies [4008d1e]
120
+ - Updated dependencies [76ec475]
121
+ - Updated dependencies [106f2a4]
122
+ - Updated dependencies [b296a2f]
123
+ - Updated dependencies [987d676]
124
+ - Updated dependencies [3e8a239]
125
+ - @homeostate/core@0.5.0
126
+
3
127
  ## 0.2.0
4
128
 
5
129
  ### Minor Changes
package/README.md CHANGED
@@ -52,10 +52,16 @@ network.expectConverged();
52
52
 
53
53
  Each peer's `doc` is its backend's document as binary updates, from the CRDT package's
54
54
  `create*Persistable`: that is what the network carries. `settle()` delivers every message on
55
- its way, and `expectConverged()` throws unless every peer's store and document agree.
55
+ its way, and rejects if a peer's store and document both lose a synced key that no peer
56
+ deleted. `expectConverged()` throws unless every peer's store and document agree.
56
57
 
57
- `runRandomized` drives random edits, link cuts and reconnections across a fresh network, then
58
- checks convergence and names the seed of a run that fails:
58
+ With a `dropRate`, the updates stores write in answer to the exchange `heal()` starts, such as a
59
+ key derived from synced state, can be dropped too. Repeat `heal()` and `settle()` until
60
+ `stats().dropped` stops growing before you check convergence.
61
+
62
+ `runRandomized` drives random edits, link cuts and reconnections across a fresh network, checks
63
+ after every delivery that no peer's store loses a synced key, then heals and settles until
64
+ nothing more is dropped and checks convergence, and names the seed of a run that fails:
59
65
 
60
66
  ```ts
61
67
  await runRandomized({
package/dist/index.d.ts CHANGED
@@ -1,4 +1,5 @@
1
1
  export { ConvergenceError, createNetwork } from "./network.js";
2
+ export { LostKeyError } from "./lost-keys.js";
2
3
  export type { Network, NetworkOptions, NetworkStats, Peer, PeerSetup, } from "./network.js";
3
4
  export { randomEdit, runRandomized } from "./randomized.js";
4
5
  export type { RandomAction, RandomizedOptions } from "./randomized.js";
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAC/D,YAAY,EACV,OAAO,EACP,cAAc,EACd,YAAY,EACZ,IAAI,EACJ,SAAS,GACV,MAAM,cAAc,CAAC;AACtB,OAAO,EAAE,UAAU,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAC5D,YAAY,EAAE,YAAY,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AACvE,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,YAAY,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAC/D,OAAO,EAAE,YAAY,EAAE,MAAM,gBAAgB,CAAC;AAC9C,YAAY,EACV,OAAO,EACP,cAAc,EACd,YAAY,EACZ,IAAI,EACJ,SAAS,GACV,MAAM,cAAc,CAAC;AACtB,OAAO,EAAE,UAAU,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAC5D,YAAY,EAAE,YAAY,EAAE,iBAAiB,EAAE,MAAM,iBAAiB,CAAC;AACvE,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,YAAY,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC"}
package/dist/index.js CHANGED
@@ -1,4 +1,5 @@
1
1
  export { ConvergenceError, createNetwork } from "./network.js";
2
+ export { LostKeyError } from "./lost-keys.js";
2
3
  export { randomEdit, runRandomized } from "./randomized.js";
3
4
  export { createRandom } from "./random.js";
4
5
  //# sourceMappingURL=index.js.map
package/dist/index.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAQ/D,OAAO,EAAE,UAAU,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAE5D,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC"}
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAC/D,OAAO,EAAE,YAAY,EAAE,MAAM,gBAAgB,CAAC;AAQ9C,OAAO,EAAE,UAAU,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAE5D,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC"}
@@ -0,0 +1,35 @@
1
+ import type { Peer } from "./network.js";
2
+ /** Thrown by `advance` and `settle` when a peer lost a synced key; see `checkLostKeys`. */
3
+ export declare class LostKeyError extends Error {
4
+ readonly seed: number;
5
+ /** The peer's name. */
6
+ readonly peer: string;
7
+ readonly key: string;
8
+ /** Virtual time of the checkpoint that found the loss, in milliseconds. */
9
+ readonly at: number;
10
+ constructor(message: string, details: {
11
+ seed: number;
12
+ peer: string;
13
+ key: string;
14
+ at: number;
15
+ });
16
+ }
17
+ /**
18
+ * Checks that no peer's store and document both lose a synced top-level key that no peer
19
+ * deleted. The network calls it as its documents change and around its deliveries.
20
+ */
21
+ export interface LostKeyCheck {
22
+ /** Start tracking a peer, before it links and connects. */
23
+ addPeer: (peer: Peer) => void;
24
+ /** The peer's document changed; `outside` when no delivery is running. */
25
+ changed: (peer: Peer, outside: boolean) => void;
26
+ /**
27
+ * Before deliveries: what changed since the last checkpoint was done by the test, so a key
28
+ * that left a store or document is deleted.
29
+ */
30
+ beforeDeliveries: (peers: readonly Peer[]) => void;
31
+ /** After a delivery to `peer` from `from`, or after the last one; throws a `LostKeyError`. */
32
+ afterDelivery: (peer: Peer, at: number, from?: Peer) => void;
33
+ }
34
+ export declare function createLostKeyCheck(seed: number): LostKeyCheck;
35
+ //# sourceMappingURL=lost-keys.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"lost-keys.d.ts","sourceRoot":"","sources":["../src/lost-keys.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,cAAc,CAAC;AAEzC,2FAA2F;AAC3F,qBAAa,YAAa,SAAQ,KAAK;IACrC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,uBAAuB;IACvB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAC;IACrB,2EAA2E;IAC3E,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;gBAGlB,OAAO,EAAE,MAAM,EACf,OAAO,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,GAAG,EAAE,MAAM,CAAC;QAAC,EAAE,EAAE,MAAM,CAAA;KAAE;CASnE;AAgBD;;;GAGG;AACH,MAAM,WAAW,YAAY;IAC3B,2DAA2D;IAC3D,OAAO,EAAE,CAAC,IAAI,EAAE,IAAI,KAAK,IAAI,CAAC;IAC9B,0EAA0E;IAC1E,OAAO,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,OAAO,EAAE,OAAO,KAAK,IAAI,CAAC;IAChD;;;OAGG;IACH,gBAAgB,EAAE,CAAC,KAAK,EAAE,SAAS,IAAI,EAAE,KAAK,IAAI,CAAC;IACnD,8FAA8F;IAC9F,aAAa,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,EAAE,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,IAAI,KAAK,IAAI,CAAC;CAC9D;AAiBD,wBAAgB,kBAAkB,CAAC,IAAI,EAAE,MAAM,GAAG,YAAY,CA2F7D"}
@@ -0,0 +1,139 @@
1
+ import { formatPath } from "./compare.js";
2
+ /** Thrown by `advance` and `settle` when a peer lost a synced key; see `checkLostKeys`. */
3
+ export class LostKeyError extends Error {
4
+ constructor(message, details) {
5
+ super(message);
6
+ Object.defineProperty(this, "seed", {
7
+ enumerable: true,
8
+ configurable: true,
9
+ writable: true,
10
+ value: void 0
11
+ });
12
+ /** The peer's name. */
13
+ Object.defineProperty(this, "peer", {
14
+ enumerable: true,
15
+ configurable: true,
16
+ writable: true,
17
+ value: void 0
18
+ });
19
+ Object.defineProperty(this, "key", {
20
+ enumerable: true,
21
+ configurable: true,
22
+ writable: true,
23
+ value: void 0
24
+ });
25
+ /** Virtual time of the checkpoint that found the loss, in milliseconds. */
26
+ Object.defineProperty(this, "at", {
27
+ enumerable: true,
28
+ configurable: true,
29
+ writable: true,
30
+ value: void 0
31
+ });
32
+ this.name = "LostKeyError";
33
+ this.seed = details.seed;
34
+ this.peer = details.peer;
35
+ this.key = details.key;
36
+ this.at = details.at;
37
+ }
38
+ }
39
+ /** The keys of `state` that JSON keeps, as `peer.syncedState()` has them. */
40
+ const jsonKeys = (state) => {
41
+ const keys = new Set();
42
+ for (const [key, value] of Object.entries(state))
43
+ if (value !== undefined &&
44
+ typeof value !== "function" &&
45
+ typeof value !== "symbol")
46
+ keys.add(key);
47
+ return keys;
48
+ };
49
+ const formatTime = (ms) => String(Number(ms.toFixed(2)));
50
+ export function createLostKeyCheck(seed) {
51
+ const tracked = new Map();
52
+ /** Keys a checkpoint outside a delivery saw leave a store or document; never cleared. */
53
+ const deleted = new Set();
54
+ const storeKeys = (peer) => jsonKeys(peer.engine.getSyncedState());
55
+ const docKeys = (peer, state) => {
56
+ if (!state.doc) {
57
+ state.doc = jsonKeys(peer.backend.read());
58
+ // Only the test's own changes delete; a delivery that removes a key may be a loss.
59
+ if (state.changedOutside && state.lastDoc)
60
+ for (const key of state.lastDoc)
61
+ if (!state.doc.has(key))
62
+ deleted.add(key);
63
+ state.lastDoc = state.doc;
64
+ state.changedOutside = false;
65
+ }
66
+ return state.doc;
67
+ };
68
+ /** Mark the keys that left the store since its last checkpoint as deleted. */
69
+ const noteRemovals = (state, store) => {
70
+ for (const key of state.store)
71
+ if (!store.has(key))
72
+ deleted.add(key);
73
+ state.store = store;
74
+ };
75
+ const hold = (peer, state, keys) => {
76
+ for (const key of keys) {
77
+ if (state.held.has(key) || deleted.has(key))
78
+ continue;
79
+ if (docKeys(peer, state).has(key))
80
+ state.held.add(key);
81
+ }
82
+ };
83
+ const track = (peer) => {
84
+ const state = tracked.get(peer);
85
+ if (!state)
86
+ throw new Error(`${peer.name} is not on the network.`);
87
+ return state;
88
+ };
89
+ return {
90
+ addPeer: (peer) => {
91
+ tracked.set(peer, {
92
+ store: storeKeys(peer),
93
+ doc: undefined,
94
+ lastDoc: undefined,
95
+ changedOutside: false,
96
+ held: new Set(),
97
+ });
98
+ },
99
+ changed: (peer, outside) => {
100
+ const state = track(peer);
101
+ state.doc = undefined;
102
+ if (!outside)
103
+ return;
104
+ state.changedOutside = true;
105
+ // A key added and removed again before the next delivery still counts as deleted.
106
+ noteRemovals(state, storeKeys(peer));
107
+ },
108
+ beforeDeliveries: (peers) => {
109
+ for (const peer of peers) {
110
+ const state = track(peer);
111
+ const store = storeKeys(peer);
112
+ noteRemovals(state, store);
113
+ if (state.changedOutside)
114
+ docKeys(peer, state);
115
+ hold(peer, state, store);
116
+ }
117
+ },
118
+ afterDelivery: (peer, at, from) => {
119
+ const state = track(peer);
120
+ const store = storeKeys(peer);
121
+ for (const key of state.held) {
122
+ if (store.has(key))
123
+ continue;
124
+ if (deleted.has(key)) {
125
+ state.held.delete(key);
126
+ continue;
127
+ }
128
+ if (docKeys(peer, state).has(key))
129
+ continue;
130
+ const source = from ? `, on an update from ${from.name}` : "";
131
+ throw new LostKeyError(`${peer.name} lost the synced key ${formatPath([key])} at ${formatTime(at)} ms (seed ${seed})${source}: its store and document both lack it, and no peer deleted it.`, { seed, peer: peer.name, key, at });
132
+ }
133
+ const appeared = [...store].filter((key) => !state.store.has(key));
134
+ state.store = store;
135
+ hold(peer, state, appeared);
136
+ },
137
+ };
138
+ }
139
+ //# sourceMappingURL=lost-keys.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"lost-keys.js","sourceRoot":"","sources":["../src/lost-keys.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,cAAc,CAAC;AAG1C,2FAA2F;AAC3F,MAAM,OAAO,YAAa,SAAQ,KAAK;IAQrC,YACE,OAAe,EACf,OAAgE;QAEhE,KAAK,CAAC,OAAO,CAAC,CAAC;QAXR;;;;;WAAa;QACtB,uBAAuB;QACd;;;;;WAAa;QACb;;;;;WAAY;QACrB,2EAA2E;QAClE;;;;;WAAW;QAOlB,IAAI,CAAC,IAAI,GAAG,cAAc,CAAC;QAC3B,IAAI,CAAC,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC;QACzB,IAAI,CAAC,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC;QACzB,IAAI,CAAC,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC;QACvB,IAAI,CAAC,EAAE,GAAG,OAAO,CAAC,EAAE,CAAC;IACvB,CAAC;CACF;AAkCD,6EAA6E;AAC7E,MAAM,QAAQ,GAAG,CAAC,KAAa,EAAe,EAAE;IAC9C,MAAM,IAAI,GAAG,IAAI,GAAG,EAAU,CAAC;IAC/B,KAAK,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC;QAC9C,IACE,KAAK,KAAK,SAAS;YACnB,OAAO,KAAK,KAAK,UAAU;YAC3B,OAAO,KAAK,KAAK,QAAQ;YAEzB,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IAClB,OAAO,IAAI,CAAC;AACd,CAAC,CAAC;AAEF,MAAM,UAAU,GAAG,CAAC,EAAU,EAAU,EAAE,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AAEzE,MAAM,UAAU,kBAAkB,CAAC,IAAY;IAC7C,MAAM,OAAO,GAAG,IAAI,GAAG,EAAiB,CAAC;IACzC,yFAAyF;IACzF,MAAM,OAAO,GAAG,IAAI,GAAG,EAAU,CAAC;IAElC,MAAM,SAAS,GAAG,CAAC,IAAU,EAAe,EAAE,CAC5C,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,cAAc,EAAE,CAAC,CAAC;IAEzC,MAAM,OAAO,GAAG,CAAC,IAAU,EAAE,KAAc,EAAe,EAAE;QAC1D,IAAI,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC;YACf,KAAK,CAAC,GAAG,GAAG,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC;YAC1C,mFAAmF;YACnF,IAAI,KAAK,CAAC,cAAc,IAAI,KAAK,CAAC,OAAO;gBACvC,KAAK,MAAM,GAAG,IAAI,KAAK,CAAC,OAAO;oBAC7B,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC;wBAAE,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;YAC9C,KAAK,CAAC,OAAO,GAAG,KAAK,CAAC,GAAG,CAAC;YAC1B,KAAK,CAAC,cAAc,GAAG,KAAK,CAAC;QAC/B,CAAC;QACD,OAAO,KAAK,CAAC,GAAG,CAAC;IACnB,CAAC,CAAC;IAEF,8EAA8E;IAC9E,MAAM,YAAY,GAAG,CAAC,KAAc,EAAE,KAAkB,EAAQ,EAAE;QAChE,KAAK,MAAM,GAAG,IAAI,KAAK,CAAC,KAAK;YAAE,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC;gBAAE,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QACrE,KAAK,CAAC,KAAK,GAAG,KAAK,CAAC;IACtB,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,CAAC,IAAU,EAAE,KAAc,EAAE,IAAsB,EAAQ,EAAE;QACxE,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;YACvB,IAAI,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC;gBAAE,SAAS;YACtD,IAAI,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC;gBAAE,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QACzD,CAAC;IACH,CAAC,CAAC;IAEF,MAAM,KAAK,GAAG,CAAC,IAAU,EAAW,EAAE;QACpC,MAAM,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAChC,IAAI,CAAC,KAAK;YAAE,MAAM,IAAI,KAAK,CAAC,GAAG,IAAI,CAAC,IAAI,yBAAyB,CAAC,CAAC;QACnE,OAAO,KAAK,CAAC;IACf,CAAC,CAAC;IAEF,OAAO;QACL,OAAO,EAAE,CAAC,IAAI,EAAE,EAAE;YAChB,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE;gBAChB,KAAK,EAAE,SAAS,CAAC,IAAI,CAAC;gBACtB,GAAG,EAAE,SAAS;gBACd,OAAO,EAAE,SAAS;gBAClB,cAAc,EAAE,KAAK;gBACrB,IAAI,EAAE,IAAI,GAAG,EAAE;aAChB,CAAC,CAAC;QACL,CAAC;QAED,OAAO,EAAE,CAAC,IAAI,EAAE,OAAO,EAAE,EAAE;YACzB,MAAM,KAAK,GAAG,KAAK,CAAC,IAAI,CAAC,CAAC;YAC1B,KAAK,CAAC,GAAG,GAAG,SAAS,CAAC;YACtB,IAAI,CAAC,OAAO;gBAAE,OAAO;YACrB,KAAK,CAAC,cAAc,GAAG,IAAI,CAAC;YAC5B,kFAAkF;YAClF,YAAY,CAAC,KAAK,EAAE,SAAS,CAAC,IAAI,CAAC,CAAC,CAAC;QACvC,CAAC;QAED,gBAAgB,EAAE,CAAC,KAAK,EAAE,EAAE;YAC1B,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;gBACzB,MAAM,KAAK,GAAG,KAAK,CAAC,IAAI,CAAC,CAAC;gBAC1B,MAAM,KAAK,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;gBAC9B,YAAY,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;gBAC3B,IAAI,KAAK,CAAC,cAAc;oBAAE,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;gBAC/C,IAAI,CAAC,IAAI,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC;YAC3B,CAAC;QACH,CAAC;QAED,aAAa,EAAE,CAAC,IAAI,EAAE,EAAE,EAAE,IAAI,EAAE,EAAE;YAChC,MAAM,KAAK,GAAG,KAAK,CAAC,IAAI,CAAC,CAAC;YAC1B,MAAM,KAAK,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;YAC9B,KAAK,MAAM,GAAG,IAAI,KAAK,CAAC,IAAI,EAAE,CAAC;gBAC7B,IAAI,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC;oBAAE,SAAS;gBAC7B,IAAI,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,CAAC;oBACrB,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;oBACvB,SAAS;gBACX,CAAC;gBACD,IAAI,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC;oBAAE,SAAS;gBAC5C,MAAM,MAAM,GAAG,IAAI,CAAC,CAAC,CAAC,uBAAuB,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;gBAC9D,MAAM,IAAI,YAAY,CACpB,GAAG,IAAI,CAAC,IAAI,wBAAwB,UAAU,CAAC,CAAC,GAAG,CAAC,CAAC,OAAO,UAAU,CAAC,EAAE,CAAC,aAAa,IAAI,IAAI,MAAM,gEAAgE,EACrK,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,GAAG,EAAE,EAAE,EAAE,CACnC,CAAC;YACJ,CAAC;YACD,MAAM,QAAQ,GAAG,CAAC,GAAG,KAAK,CAAC,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC;YACnE,KAAK,CAAC,KAAK,GAAG,KAAK,CAAC;YACpB,IAAI,CAAC,IAAI,EAAE,KAAK,EAAE,QAAQ,CAAC,CAAC;QAC9B,CAAC;KACF,CAAC;AACJ,CAAC"}
package/dist/network.d.ts CHANGED
@@ -54,6 +54,17 @@ export interface NetworkOptions {
54
54
  * as two peers writing in a loop. Defaults to 100 000.
55
55
  */
56
56
  maxDeliveries?: number;
57
+ /**
58
+ * Whether `advance` and `settle` reject with a `LostKeyError` when a peer's store and
59
+ * document both lose a synced top-level key that no peer deleted. Defaults to `true`.
60
+ *
61
+ * A peer holds a key once a checkpoint sees it in both its store's synced state and its
62
+ * document. A key is deleted for good once it leaves a store or document outside a delivery:
63
+ * in a step, or in a test's own change, between `advance` and `settle` calls. Checkpoints
64
+ * come after each delivery and when `advance` or `settle` returns. A key deleted during a
65
+ * delivery, such as by a store subscriber or a timer, is reported as lost: pass `false`.
66
+ */
67
+ checkLostKeys?: boolean;
57
68
  }
58
69
  export interface NetworkStats {
59
70
  /** Messages handed to the network, updates and the documents exchanged on connecting. */
@@ -71,6 +82,9 @@ export interface NetworkStats {
71
82
  * document goes to every peer it is linked to, after the latency, unless it is dropped or its
72
83
  * link is cut first. Linking two peers, as `connect` and `heal` do, makes them exchange their
73
84
  * whole documents, as providers do on reconnecting, so what was lost on the way arrives.
85
+ *
86
+ * Unless `checkLostKeys` is `false`, `advance` and `settle` also check after each delivery
87
+ * that no peer's store and document both lost a synced key that no peer deleted.
74
88
  */
75
89
  export interface Network {
76
90
  /** The seed of this network's random choices. */
@@ -100,11 +114,21 @@ export interface Network {
100
114
  /**
101
115
  * Link every pair of peers and have each exchange documents, as if every peer reconnected,
102
116
  * so that updates lost to drops and cut links arrive too.
117
+ *
118
+ * With a `dropRate`, the updates stores write in answer to that exchange, such as a key one
119
+ * derives from synced state, can be dropped too. Repeat `heal()` and `settle()` until
120
+ * `stats().dropped` stops growing before checking convergence.
103
121
  */
104
122
  heal: () => void;
105
- /** Deliver the messages due within the next `ms` of virtual time, and move time on. */
123
+ /**
124
+ * Deliver the messages due within the next `ms` of virtual time, and move time on. Rejects
125
+ * with a `LostKeyError` when a peer lost a synced key; see `checkLostKeys`.
126
+ */
106
127
  advance: (ms: number) => Promise<void>;
107
- /** Deliver every message on its way, and those they lead to, until none are left. */
128
+ /**
129
+ * Deliver every message on its way, and those they lead to, until none are left. Rejects
130
+ * with a `LostKeyError` when a peer lost a synced key; see `checkLostKeys`.
131
+ */
108
132
  settle: () => Promise<void>;
109
133
  stats: () => NetworkStats;
110
134
  /**
@@ -113,7 +137,10 @@ export interface Network {
113
137
  * document. Empty once the peers have converged.
114
138
  */
115
139
  differences: () => string[];
116
- /** Throws a `ConvergenceError` unless `differences()` is empty. */
140
+ /**
141
+ * Throws a `ConvergenceError` unless `differences()` is empty. Its message also says how many
142
+ * messages are still on their way, and how many updates were dropped since the last `heal()`.
143
+ */
117
144
  expectConverged: () => void;
118
145
  /** Disconnect every engine, stop listening to every document and drop what is on its way. */
119
146
  destroy: () => void;
@@ -1 +1 @@
1
- {"version":3,"file":"network.d.ts","sourceRoot":"","sources":["../src/network.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,KAAK,WAAW,EAChB,KAAK,cAAc,EACnB,KAAK,YAAY,EACjB,KAAK,UAAU,EACf,KAAK,gBAAgB,EAEtB,MAAM,kBAAkB,CAAC;AAI1B;;;;GAIG;AAGH,MAAM,WAAW,SAAS,CAAC,CAAC,SAAS,MAAM,GAAG,GAAG;IAC/C,OAAO,EAAE,YAAY,CAAC,CAAC,CAAC,CAAC;IACzB,OAAO,EAAE,WAAW,CAAC;IACrB,GAAG,EAAE,cAAc,CAAC;IACpB;;;OAGG;IACH,MAAM,CAAC,EAAE,OAAO,CAAC,gBAAgB,CAAC,CAAC,CAAC,CAAC,CAAC;IACtC,wEAAwE;IACxE,IAAI,CAAC,EAAE,MAAM,CAAC;IACd;;;;OAIG;IACH,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED,kGAAkG;AAElG,MAAM,WAAW,IAAI,CAAC,CAAC,SAAS,MAAM,GAAG,GAAG;IAC1C,mCAAmC;IACnC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,OAAO,EAAE,YAAY,CAAC,CAAC,CAAC,CAAC;IAClC,QAAQ,CAAC,OAAO,EAAE,WAAW,CAAC;IAC9B,QAAQ,CAAC,GAAG,EAAE,cAAc,CAAC;IAC7B,QAAQ,CAAC,MAAM,EAAE,UAAU,CAAC;IAC5B,wCAAwC;IACxC,WAAW,EAAE,MAAM,OAAO,CAAC;CAC5B;AAED,MAAM,WAAW,cAAc;IAC7B,yFAAyF;IACzF,IAAI,CAAC,EAAE,MAAM,CAAC;IACd;;;OAGG;IACH,OAAO,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC7C,oEAAoE;IACpE,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB;;;OAGG;IACH,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB;;;OAGG;IACH,aAAa,CAAC,EAAE,MAAM,CAAC;CACxB;AAED,MAAM,WAAW,YAAY;IAC3B,yFAAyF;IACzF,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,MAAM,CAAC;IAClB,kCAAkC;IAClC,OAAO,EAAE,MAAM,CAAC;IAChB,6EAA6E;IAC7E,GAAG,EAAE,MAAM,CAAC;IACZ,6BAA6B;IAC7B,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED;;;;;GAKG;AACH,MAAM,WAAW,OAAO;IACtB,iDAAiD;IACjD,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,4CAA4C;IAC5C,QAAQ,CAAC,KAAK,EAAE,SAAS,IAAI,EAAE,CAAC;IAChC,qCAAqC;IACrC,GAAG,EAAE,MAAM,MAAM,CAAC;IAClB;;;OAGG;IACH,OAAO,EAAE,CAAC,CAAC,SAAS,MAAM,EAAE,KAAK,EAAE,SAAS,CAAC,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC;IAC5D,oCAAoC;IACpC,MAAM,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,IAAI,KAAK,OAAO,CAAC;IACtC,8DAA8D;IAC9D,OAAO,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,IAAI,KAAK,IAAI,CAAC;IACpC,mFAAmF;IACnF,UAAU,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,IAAI,KAAK,IAAI,CAAC;IACvC,8DAA8D;IAC9D,OAAO,EAAE,CAAC,IAAI,EAAE,IAAI,KAAK,IAAI,CAAC;IAC9B;;;OAGG;IACH,SAAS,EAAE,CAAC,GAAG,MAAM,EAAE,IAAI,EAAE,EAAE,KAAK,IAAI,CAAC;IACzC;;;OAGG;IACH,IAAI,EAAE,MAAM,IAAI,CAAC;IACjB,uFAAuF;IACvF,OAAO,EAAE,CAAC,EAAE,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IACvC,qFAAqF;IACrF,MAAM,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAC5B,KAAK,EAAE,MAAM,YAAY,CAAC;IAC1B;;;;OAIG;IACH,WAAW,EAAE,MAAM,MAAM,EAAE,CAAC;IAC5B,mEAAmE;IACnE,eAAe,EAAE,MAAM,IAAI,CAAC;IAC5B,6FAA6F;IAC7F,OAAO,EAAE,MAAM,IAAI,CAAC;CACrB;AAED,sEAAsE;AACtE,qBAAa,gBAAiB,SAAQ,KAAK;IACzC,QAAQ,CAAC,WAAW,EAAE,MAAM,EAAE,CAAC;IAC/B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;gBAEV,OAAO,EAAE,MAAM,EAAE,WAAW,EAAE,MAAM,EAAE,EAAE,IAAI,EAAE,MAAM;CAMjE;AAqBD;;;;;;;;;;;;;;;;;;;;;;GAsBG;AACH,wBAAgB,aAAa,CAAC,OAAO,GAAE,cAAmB,GAAG,OAAO,CAgPnE"}
1
+ {"version":3,"file":"network.d.ts","sourceRoot":"","sources":["../src/network.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,KAAK,WAAW,EAChB,KAAK,cAAc,EACnB,KAAK,YAAY,EACjB,KAAK,UAAU,EACf,KAAK,gBAAgB,EAEtB,MAAM,kBAAkB,CAAC;AAK1B;;;;GAIG;AAGH,MAAM,WAAW,SAAS,CAAC,CAAC,SAAS,MAAM,GAAG,GAAG;IAC/C,OAAO,EAAE,YAAY,CAAC,CAAC,CAAC,CAAC;IACzB,OAAO,EAAE,WAAW,CAAC;IACrB,GAAG,EAAE,cAAc,CAAC;IACpB;;;OAGG;IACH,MAAM,CAAC,EAAE,OAAO,CAAC,gBAAgB,CAAC,CAAC,CAAC,CAAC,CAAC;IACtC,wEAAwE;IACxE,IAAI,CAAC,EAAE,MAAM,CAAC;IACd;;;;OAIG;IACH,OAAO,CAAC,EAAE,OAAO,CAAC;CACnB;AAED,kGAAkG;AAElG,MAAM,WAAW,IAAI,CAAC,CAAC,SAAS,MAAM,GAAG,GAAG;IAC1C,mCAAmC;IACnC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,OAAO,EAAE,YAAY,CAAC,CAAC,CAAC,CAAC;IAClC,QAAQ,CAAC,OAAO,EAAE,WAAW,CAAC;IAC9B,QAAQ,CAAC,GAAG,EAAE,cAAc,CAAC;IAC7B,QAAQ,CAAC,MAAM,EAAE,UAAU,CAAC;IAC5B,wCAAwC;IACxC,WAAW,EAAE,MAAM,OAAO,CAAC;CAC5B;AAED,MAAM,WAAW,cAAc;IAC7B,yFAAyF;IACzF,IAAI,CAAC,EAAE,MAAM,CAAC;IACd;;;OAGG;IACH,OAAO,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC7C,oEAAoE;IACpE,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB;;;OAGG;IACH,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB;;;OAGG;IACH,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB;;;;;;;;;OASG;IACH,aAAa,CAAC,EAAE,OAAO,CAAC;CACzB;AAED,MAAM,WAAW,YAAY;IAC3B,yFAAyF;IACzF,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,MAAM,CAAC;IAClB,kCAAkC;IAClC,OAAO,EAAE,MAAM,CAAC;IAChB,6EAA6E;IAC7E,GAAG,EAAE,MAAM,CAAC;IACZ,6BAA6B;IAC7B,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED;;;;;;;;GAQG;AACH,MAAM,WAAW,OAAO;IACtB,iDAAiD;IACjD,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,4CAA4C;IAC5C,QAAQ,CAAC,KAAK,EAAE,SAAS,IAAI,EAAE,CAAC;IAChC,qCAAqC;IACrC,GAAG,EAAE,MAAM,MAAM,CAAC;IAClB;;;OAGG;IACH,OAAO,EAAE,CAAC,CAAC,SAAS,MAAM,EAAE,KAAK,EAAE,SAAS,CAAC,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC;IAC5D,oCAAoC;IACpC,MAAM,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,IAAI,KAAK,OAAO,CAAC;IACtC,8DAA8D;IAC9D,OAAO,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,IAAI,KAAK,IAAI,CAAC;IACpC,mFAAmF;IACnF,UAAU,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,IAAI,KAAK,IAAI,CAAC;IACvC,8DAA8D;IAC9D,OAAO,EAAE,CAAC,IAAI,EAAE,IAAI,KAAK,IAAI,CAAC;IAC9B;;;OAGG;IACH,SAAS,EAAE,CAAC,GAAG,MAAM,EAAE,IAAI,EAAE,EAAE,KAAK,IAAI,CAAC;IACzC;;;;;;;OAOG;IACH,IAAI,EAAE,MAAM,IAAI,CAAC;IACjB;;;OAGG;IACH,OAAO,EAAE,CAAC,EAAE,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IACvC;;;OAGG;IACH,MAAM,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC;IAC5B,KAAK,EAAE,MAAM,YAAY,CAAC;IAC1B;;;;OAIG;IACH,WAAW,EAAE,MAAM,MAAM,EAAE,CAAC;IAC5B;;;OAGG;IACH,eAAe,EAAE,MAAM,IAAI,CAAC;IAC5B,6FAA6F;IAC7F,OAAO,EAAE,MAAM,IAAI,CAAC;CACrB;AAED,sEAAsE;AACtE,qBAAa,gBAAiB,SAAQ,KAAK;IACzC,QAAQ,CAAC,WAAW,EAAE,MAAM,EAAE,CAAC;IAC/B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;gBAEV,OAAO,EAAE,MAAM,EAAE,WAAW,EAAE,MAAM,EAAE,EAAE,IAAI,EAAE,MAAM;CAMjE;AAqBD;;;;;;;;;;;;;;;;;;;;;;GAsBG;AACH,wBAAgB,aAAa,CAAC,OAAO,GAAE,cAAmB,GAAG,OAAO,CAkRnE"}
package/dist/network.js CHANGED
@@ -1,5 +1,6 @@
1
1
  import { createSyncEngine, } from "@homeostate/core";
2
2
  import { asJson, firstDifference, formatPath, formatValue } from "./compare.js";
3
+ import { createLostKeyCheck } from "./lost-keys.js";
3
4
  import { createRandom, randomSeed } from "./random.js";
4
5
  /** Thrown by `expectConverged`; the message lists the differences. */
5
6
  export class ConvergenceError extends Error {
@@ -51,8 +52,11 @@ const linkKey = (a, b) => a.index < b.index ? `${a.index}:${b.index}` : `${b.ind
51
52
  * ```
52
53
  */
53
54
  export function createNetwork(options = {}) {
54
- const { seed = randomSeed(), latency = 0, dropRate = 0, reorder = false, maxDeliveries = 100000, } = options;
55
+ const { seed = randomSeed(), latency = 0, dropRate = 0, reorder = false, maxDeliveries = 100000, checkLostKeys = true, } = options;
55
56
  const random = createRandom(seed);
57
+ const lostKeys = checkLostKeys ? createLostKeyCheck(seed) : undefined;
58
+ /** Whether `advance` or `settle` is delivering, so a document's change is not the test's. */
59
+ let delivering = false;
56
60
  const peers = [];
57
61
  const unsubscribers = [];
58
62
  /** Pairs of peers whose link is cut; every other pair is linked. */
@@ -63,6 +67,8 @@ export function createNetwork(options = {}) {
63
67
  let now = 0;
64
68
  let order = 0;
65
69
  const stats = { sent: 0, delivered: 0, dropped: 0, cut: 0 };
70
+ /** `stats.dropped` at the last `heal()`, or `undefined` before the first. */
71
+ let droppedAtHeal;
66
72
  const delay = () => {
67
73
  if (typeof latency === "number")
68
74
  return latency;
@@ -119,10 +125,11 @@ export function createNetwork(options = {}) {
119
125
  const deliver = (message) => {
120
126
  now = Math.max(now, message.at);
121
127
  stats.delivered++;
128
+ lostKeys?.changed(message.to, false);
122
129
  message.to.doc.apply(message.update);
123
130
  };
124
131
  /** Deliver messages due by `until`, letting each delivery's work finish before the next. */
125
- const run = async (until) => {
132
+ const deliverUntil = async (until) => {
126
133
  let deliveries = 0;
127
134
  for (;;) {
128
135
  const next = queue[0];
@@ -132,6 +139,7 @@ export function createNetwork(options = {}) {
132
139
  queue.shift();
133
140
  deliver(next);
134
141
  await Promise.resolve();
142
+ lostKeys?.afterDelivery(next.to, now, next.from);
135
143
  continue;
136
144
  }
137
145
  // Asynchronous stores may still send; only stop once a full turn sends nothing.
@@ -141,6 +149,19 @@ export function createNetwork(options = {}) {
141
149
  return;
142
150
  }
143
151
  };
152
+ /** `deliverUntil`, with the lost key check's checkpoints before and after. */
153
+ const run = async (until) => {
154
+ lostKeys?.beforeDeliveries(peers);
155
+ delivering = true;
156
+ try {
157
+ await deliverUntil(until);
158
+ }
159
+ finally {
160
+ delivering = false;
161
+ }
162
+ for (const peer of peers)
163
+ lostKeys?.afterDelivery(peer, now);
164
+ };
144
165
  const view = (peer) => ({
145
166
  store: peer.syncedState(),
146
167
  document: asJson(peer.backend.read()),
@@ -154,7 +175,7 @@ export function createNetwork(options = {}) {
154
175
  };
155
176
  const views = peers.map(view);
156
177
  peers.forEach((peer, i) => {
157
- if (peer.engine.isConnected())
178
+ if (peer.engine.getStatus().connected)
158
179
  describe(`${peer.name}'s store differs from its document`, views[i].store, views[i].document);
159
180
  });
160
181
  const [first] = peers;
@@ -170,7 +191,11 @@ export function createNetwork(options = {}) {
170
191
  now: () => now,
171
192
  addPeer: (setup) => {
172
193
  const index = peers.length;
173
- const engine = createSyncEngine(setup.backend, setup.adapter, setup.config);
194
+ const engine = createSyncEngine({
195
+ ...setup.config,
196
+ backend: setup.backend,
197
+ adapter: setup.adapter,
198
+ });
174
199
  const peer = {
175
200
  index,
176
201
  name: setup.name ?? `peer-${index}`,
@@ -181,7 +206,11 @@ export function createNetwork(options = {}) {
181
206
  syncedState: () => asJson(engine.getSyncedState()),
182
207
  };
183
208
  peers.push(peer);
184
- unsubscribers.push(setup.doc.subscribe((update) => broadcast(peer, update)));
209
+ lostKeys?.addPeer(peer);
210
+ unsubscribers.push(setup.doc.subscribe((update) => {
211
+ lostKeys?.changed(peer, !delivering);
212
+ broadcast(peer, update);
213
+ }));
185
214
  for (const other of peers)
186
215
  link(peer, other);
187
216
  if (setup.connect !== false)
@@ -209,6 +238,7 @@ export function createNetwork(options = {}) {
209
238
  }
210
239
  },
211
240
  heal: () => {
241
+ droppedAtHeal = stats.dropped;
212
242
  for (const a of peers)
213
243
  for (const b of peers)
214
244
  if (a.index < b.index)
@@ -226,12 +256,18 @@ export function createNetwork(options = {}) {
226
256
  const lines = differences();
227
257
  if (lines.length === 0)
228
258
  return;
229
- const inFlight = queue.length > 0
230
- ? `\n${queue.length} messages are still on their way: settle() delivers them.`
259
+ const inFlight = queue.length === 1
260
+ ? "\n1 message is still on its way: settle() delivers it."
261
+ : queue.length > 1
262
+ ? `\n${queue.length} messages are still on their way: settle() delivers them.`
263
+ : "";
264
+ const dropped = stats.dropped - (droppedAtHeal ?? 0);
265
+ const droppedHint = dropped > 0
266
+ ? `\n${dropped === 1 ? "1 update was" : `${dropped} updates were`} dropped${droppedAtHeal === undefined ? "" : " since the last heal()"}: heal() and settle() until stats().dropped stops growing.`
231
267
  : "";
232
268
  throw new ConvergenceError(`${peers.length} peers did not converge (seed ${seed}):\n${lines
233
269
  .map((line) => `- ${line}`)
234
- .join("\n")}${inFlight}`, lines, seed);
270
+ .join("\n")}${inFlight}${droppedHint}`, lines, seed);
235
271
  },
236
272
  destroy: () => {
237
273
  for (const peer of peers)
@@ -1 +1 @@
1
- {"version":3,"file":"network.js","sourceRoot":"","sources":["../src/network.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,gBAAgB,GAOjB,MAAM,kBAAkB,CAAC;AAC1B,OAAO,EAAE,MAAM,EAAE,eAAe,EAAE,UAAU,EAAE,WAAW,EAAE,MAAM,cAAc,CAAC;AAChF,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAiIvD,sEAAsE;AACtE,MAAM,OAAO,gBAAiB,SAAQ,KAAK;IAIzC,YAAY,OAAe,EAAE,WAAqB,EAAE,IAAY;QAC9D,KAAK,CAAC,OAAO,CAAC,CAAC;QAJR;;;;;WAAsB;QACtB;;;;;WAAa;QAIpB,IAAI,CAAC,IAAI,GAAG,kBAAkB,CAAC;QAC/B,IAAI,CAAC,WAAW,GAAG,WAAW,CAAC;QAC/B,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;IACnB,CAAC;CACF;AAWD,2FAA2F;AAC3F,MAAM,cAAc,GAAG,UAAU,CAAC,UAAU,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;AAE9D,0FAA0F;AAC1F,MAAM,gBAAgB,GAAG,GAAkB,EAAE,CAC3C,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,cAAc,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC;AAEvD,MAAM,OAAO,GAAG,CAAC,CAAO,EAAE,CAAO,EAAU,EAAE,CAC3C,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC;AAExE;;;;;;;;;;;;;;;;;;;;;;GAsBG;AACH,MAAM,UAAU,aAAa,CAAC,UAA0B,EAAE;IACxD,MAAM,EACJ,IAAI,GAAG,UAAU,EAAE,EACnB,OAAO,GAAG,CAAC,EACX,QAAQ,GAAG,CAAC,EACZ,OAAO,GAAG,KAAK,EACf,aAAa,GAAG,MAAO,GACxB,GAAG,OAAO,CAAC;IACZ,MAAM,MAAM,GAAG,YAAY,CAAC,IAAI,CAAC,CAAC;IAClC,MAAM,KAAK,GAAW,EAAE,CAAC;IACzB,MAAM,aAAa,GAAkB,EAAE,CAAC;IACxC,oEAAoE;IACpE,MAAM,GAAG,GAAG,IAAI,GAAG,EAAU,CAAC;IAC9B,qFAAqF;IACrF,MAAM,OAAO,GAAG,IAAI,GAAG,EAAkB,CAAC;IAC1C,IAAI,KAAK,GAAc,EAAE,CAAC;IAC1B,IAAI,GAAG,GAAG,CAAC,CAAC;IACZ,IAAI,KAAK,GAAG,CAAC,CAAC;IACd,MAAM,KAAK,GAAG,EAAE,IAAI,EAAE,CAAC,EAAE,SAAS,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,GAAG,EAAE,CAAC,EAAE,CAAC;IAE5D,MAAM,KAAK,GAAG,GAAW,EAAE;QACzB,IAAI,OAAO,OAAO,KAAK,QAAQ;YAAE,OAAO,OAAO,CAAC;QAChD,MAAM,CAAC,GAAG,EAAE,GAAG,CAAC,GAAG,OAAO,CAAC;QAC3B,OAAO,GAAG,GAAG,MAAM,CAAC,IAAI,EAAE,GAAG,CAAC,GAAG,GAAG,GAAG,CAAC,CAAC;IAC3C,CAAC,CAAC;IAEF,MAAM,QAAQ,GAAG,CAAC,IAAU,EAAE,EAAQ,EAAE,MAAkB,EAAQ,EAAE;QAClE,KAAK,CAAC,IAAI,EAAE,CAAC;QACb,IAAI,EAAE,GAAG,GAAG,GAAG,KAAK,EAAE,CAAC;QACvB,IAAI,CAAC,OAAO,EAAE,CAAC;YACb,MAAM,GAAG,GAAG,GAAG,IAAI,CAAC,KAAK,IAAI,EAAE,CAAC,KAAK,EAAE,CAAC;YACxC,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC;YACzC,OAAO,CAAC,GAAG,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;QACvB,CAAC;QACD,MAAM,OAAO,GAAY,EAAE,IAAI,EAAE,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,KAAK,EAAE,KAAK,EAAE,EAAE,CAAC;QAClE,4DAA4D;QAC5D,IAAI,KAAK,GAAG,KAAK,CAAC,MAAM,CAAC;QACzB,OACE,KAAK,GAAG,CAAC;YACT,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,EAAE,GAAG,EAAE;gBACvB,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC;YAEzE,KAAK,EAAE,CAAC;QACV,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC,EAAE,OAAO,CAAC,CAAC;IAClC,CAAC,CAAC;IAEF,MAAM,MAAM,GAAG,CAAC,CAAO,EAAE,CAAO,EAAW,EAAE,CAC3C,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;IAErC,MAAM,SAAS,GAAG,CAAC,IAAU,EAAE,MAAkB,EAAQ,EAAE;QACzD,KAAK,MAAM,EAAE,IAAI,KAAK,EAAE,CAAC;YACvB,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;gBAAE,SAAS;YAChC,IAAI,MAAM,CAAC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC;gBAC5B,KAAK,CAAC,IAAI,EAAE,CAAC;gBACb,KAAK,CAAC,OAAO,EAAE,CAAC;gBAChB,SAAS;YACX,CAAC;YACD,yDAAyD;YACzD,QAAQ,CAAC,IAAI,EAAE,EAAE,EAAE,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC;QACrC,CAAC;IACH,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,CAAC,CAAO,EAAE,CAAO,EAAQ,EAAE;QACtC,IAAI,CAAC,KAAK,CAAC;YAAE,OAAO;QACpB,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;QAC1B,4DAA4D;QAC5D,QAAQ,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,MAAM,EAAE,CAAC,CAAC;QAC/B,QAAQ,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,MAAM,EAAE,CAAC,CAAC;IACjC,CAAC,CAAC;IAEF,MAAM,MAAM,GAAG,CAAC,CAAO,EAAE,CAAO,EAAQ,EAAE;QACxC,IAAI,CAAC,KAAK,CAAC;YAAE,OAAO;QACpB,GAAG,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;QACvB,MAAM,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC;QAC5B,KAAK,GAAG,KAAK,CAAC,MAAM,CAClB,CAAC,EAAE,IAAI,EAAE,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,IAAI,CAAC,IAAI,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,CAC1E,CAAC;QACF,KAAK,CAAC,GAAG,IAAI,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC;IACrC,CAAC,CAAC;IAEF,MAAM,OAAO,GAAG,CAAC,OAAgB,EAAQ,EAAE;QACzC,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,CAAC,CAAC;QAChC,KAAK,CAAC,SAAS,EAAE,CAAC;QAClB,OAAO,CAAC,EAAE,CAAC,GAAG,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC;IACvC,CAAC,CAAC;IAEF,4FAA4F;IAC5F,MAAM,GAAG,GAAG,KAAK,EAAE,KAAa,EAAiB,EAAE;QACjD,IAAI,UAAU,GAAG,CAAC,CAAC;QACnB,SAAS,CAAC;YACR,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;YACtB,IAAI,IAAI,IAAI,IAAI,CAAC,EAAE,IAAI,KAAK,EAAE,CAAC;gBAC7B,IAAI,EAAE,UAAU,GAAG,aAAa;oBAC9B,MAAM,IAAI,KAAK,CACb,oCAAoC,aAAa,2CAA2C,CAC7F,CAAC;gBACJ,KAAK,CAAC,KAAK,EAAE,CAAC;gBACd,OAAO,CAAC,IAAI,CAAC,CAAC;gBACd,MAAM,OAAO,CAAC,OAAO,EAAE,CAAC;gBACxB,SAAS;YACX,CAAC;YACD,gFAAgF;YAChF,MAAM,gBAAgB,EAAE,CAAC;YACzB,MAAM,GAAG,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;YACrB,IAAI,CAAC,GAAG,IAAI,GAAG,CAAC,EAAE,GAAG,KAAK;gBAAE,OAAO;QACrC,CAAC;IACH,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,CAAC,IAAU,EAAE,EAAE,CAAC,CAAC;QAC5B,KAAK,EAAE,IAAI,CAAC,WAAW,EAAE;QACzB,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;KACtC,CAAC,CAAC;IAEH,MAAM,WAAW,GAAG,GAAa,EAAE;QACjC,MAAM,KAAK,GAAa,EAAE,CAAC;QAC3B,MAAM,QAAQ,GAAG,CAAC,OAAe,EAAE,IAAa,EAAE,KAAc,EAAQ,EAAE;YACxE,MAAM,UAAU,GAAG,eAAe,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;YAChD,IAAI,UAAU;gBACZ,KAAK,CAAC,IAAI,CACR,GAAG,OAAO,OAAO,UAAU,CAAC,UAAU,CAAC,IAAI,CAAC,KAAK,WAAW,CAAC,UAAU,CAAC,IAAI,CAAC,OAAO,WAAW,CAAC,UAAU,CAAC,KAAK,CAAC,EAAE,CACpH,CAAC;QACN,CAAC,CAAC;QAEF,MAAM,KAAK,GAAG,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAC9B,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,IAAI,CAAC,MAAM,CAAC,WAAW,EAAE;gBAC3B,QAAQ,CACN,GAAG,IAAI,CAAC,IAAI,oCAAoC,EAChD,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,EACd,KAAK,CAAC,CAAC,CAAC,CAAC,QAAQ,CAClB,CAAC;QACN,CAAC,CAAC,CAAC;QACH,MAAM,CAAC,KAAK,CAAC,GAAG,KAAK,CAAC;QACtB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACtC,QAAQ,CACN,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,4BAA4B,KAAK,CAAC,IAAI,IAAI,EAC1D,KAAK,CAAC,CAAC,CAAC,CAAC,QAAQ,EACjB,KAAK,CAAC,CAAC,CAAC,CAAC,QAAQ,CAClB,CAAC;YACF,QAAQ,CACN,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,yBAAyB,KAAK,CAAC,IAAI,IAAI,EACvD,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,EACd,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CACf,CAAC;QACJ,CAAC;QACD,OAAO,KAAK,CAAC;IACf,CAAC,CAAC;IAEF,OAAO;QACL,IAAI;QACJ,KAAK;QACL,GAAG,EAAE,GAAG,EAAE,CAAC,GAAG;QAEd,OAAO,EAAE,CAAmB,KAAmB,EAAW,EAAE;YAC1D,MAAM,KAAK,GAAG,KAAK,CAAC,MAAM,CAAC;YAC3B,MAAM,MAAM,GAAG,gBAAgB,CAC7B,KAAK,CAAC,OAAO,EACb,KAAK,CAAC,OAAO,EACb,KAAK,CAAC,MAAM,CACb,CAAC;YACF,MAAM,IAAI,GAAY;gBACpB,KAAK;gBACL,IAAI,EAAE,KAAK,CAAC,IAAI,IAAI,QAAQ,KAAK,EAAE;gBACnC,OAAO,EAAE,KAAK,CAAC,OAAO;gBACtB,OAAO,EAAE,KAAK,CAAC,OAAO;gBACtB,GAAG,EAAE,KAAK,CAAC,GAAG;gBACd,MAAM;gBACN,WAAW,EAAE,GAAG,EAAE,CAAC,MAAM,CAAC,MAAM,CAAC,cAAc,EAAE,CAAC;aACnD,CAAC;YACF,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YACjB,aAAa,CAAC,IAAI,CAChB,KAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,SAAS,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC,CACzD,CAAC;YACF,KAAK,MAAM,KAAK,IAAI,KAAK;gBAAE,IAAI,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;YAC7C,IAAI,KAAK,CAAC,OAAO,KAAK,KAAK;gBAAE,MAAM,CAAC,OAAO,EAAE,CAAC;YAC9C,OAAO,IAAI,CAAC;QACd,CAAC;QAED,MAAM;QAEN,OAAO,EAAE,IAAI;QAEb,UAAU,EAAE,MAAM;QAElB,OAAO,EAAE,CAAC,IAAI,EAAE,EAAE;YAChB,KAAK,MAAM,KAAK,IAAI,KAAK;gBAAE,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;QACjD,CAAC;QAED,SAAS,EAAE,CAAC,GAAG,MAAM,EAAE,EAAE;YACvB,MAAM,OAAO,GAAG,CAAC,IAAU,EAAU,EAAE,CACrC,MAAM,CAAC,SAAS,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC;YACpD,KAAK,MAAM,CAAC,IAAI,KAAK;gBACnB,KAAK,MAAM,CAAC,IAAI,KAAK,EAAE,CAAC;oBACtB,IAAI,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,KAAK;wBAAE,SAAS;oBACjC,MAAM,QAAQ,GAAG,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,IAAI,OAAO,CAAC,CAAC,CAAC,KAAK,OAAO,CAAC,CAAC,CAAC,CAAC;oBAChE,IAAI,CAAC,QAAQ;wBAAE,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;yBACvB,IAAI,CAAC,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC;wBAAE,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;gBACrC,CAAC;QACL,CAAC;QAED,IAAI,EAAE,GAAG,EAAE;YACT,KAAK,MAAM,CAAC,IAAI,KAAK;gBACnB,KAAK,MAAM,CAAC,IAAI,KAAK;oBAAE,IAAI,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK;wBAAE,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;QAC7D,CAAC;QAED,OAAO,EAAE,KAAK,EAAE,EAAE,EAAE,EAAE;YACpB,MAAM,KAAK,GAAG,GAAG,GAAG,EAAE,CAAC;YACvB,MAAM,GAAG,CAAC,KAAK,CAAC,CAAC;YACjB,GAAG,GAAG,KAAK,CAAC;QACd,CAAC;QAED,MAAM,EAAE,GAAG,EAAE,CAAC,GAAG,CAAC,QAAQ,CAAC;QAE3B,KAAK,EAAE,GAAG,EAAE,CAAC,CAAC,EAAE,GAAG,KAAK,EAAE,QAAQ,EAAE,KAAK,CAAC,MAAM,EAAE,CAAC;QAEnD,WAAW;QAEX,eAAe,EAAE,GAAG,EAAE;YACpB,MAAM,KAAK,GAAG,WAAW,EAAE,CAAC;YAC5B,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;gBAAE,OAAO;YAC/B,MAAM,QAAQ,GACZ,KAAK,CAAC,MAAM,GAAG,CAAC;gBACd,CAAC,CAAC,KAAK,KAAK,CAAC,MAAM,2DAA2D;gBAC9E,CAAC,CAAC,EAAE,CAAC;YACT,MAAM,IAAI,gBAAgB,CACxB,GAAG,KAAK,CAAC,MAAM,iCAAiC,IAAI,OAAO,KAAK;iBAC7D,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,KAAK,IAAI,EAAE,CAAC;iBAC1B,IAAI,CAAC,IAAI,CAAC,GAAG,QAAQ,EAAE,EAC1B,KAAK,EACL,IAAI,CACL,CAAC;QACJ,CAAC;QAED,OAAO,EAAE,GAAG,EAAE;YACZ,KAAK,MAAM,IAAI,IAAI,KAAK;gBAAE,IAAI,CAAC,MAAM,CAAC,UAAU,EAAE,CAAC;YACnD,aAAa,CAAC,OAAO,CAAC,CAAC,WAAW,EAAE,EAAE,CAAC,WAAW,EAAE,CAAC,CAAC;YACtD,aAAa,CAAC,MAAM,GAAG,CAAC,CAAC;YACzB,KAAK,GAAG,EAAE,CAAC;QACb,CAAC;KACF,CAAC;AACJ,CAAC"}
1
+ {"version":3,"file":"network.js","sourceRoot":"","sources":["../src/network.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,gBAAgB,GAOjB,MAAM,kBAAkB,CAAC;AAC1B,OAAO,EAAE,MAAM,EAAE,eAAe,EAAE,UAAU,EAAE,WAAW,EAAE,MAAM,cAAc,CAAC;AAChF,OAAO,EAAE,kBAAkB,EAAE,MAAM,gBAAgB,CAAC;AACpD,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AA4JvD,sEAAsE;AACtE,MAAM,OAAO,gBAAiB,SAAQ,KAAK;IAIzC,YAAY,OAAe,EAAE,WAAqB,EAAE,IAAY;QAC9D,KAAK,CAAC,OAAO,CAAC,CAAC;QAJR;;;;;WAAsB;QACtB;;;;;WAAa;QAIpB,IAAI,CAAC,IAAI,GAAG,kBAAkB,CAAC;QAC/B,IAAI,CAAC,WAAW,GAAG,WAAW,CAAC;QAC/B,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;IACnB,CAAC;CACF;AAWD,2FAA2F;AAC3F,MAAM,cAAc,GAAG,UAAU,CAAC,UAAU,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;AAE9D,0FAA0F;AAC1F,MAAM,gBAAgB,GAAG,GAAkB,EAAE,CAC3C,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,cAAc,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC;AAEvD,MAAM,OAAO,GAAG,CAAC,CAAO,EAAE,CAAO,EAAU,EAAE,CAC3C,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC;AAExE;;;;;;;;;;;;;;;;;;;;;;GAsBG;AACH,MAAM,UAAU,aAAa,CAAC,UAA0B,EAAE;IACxD,MAAM,EACJ,IAAI,GAAG,UAAU,EAAE,EACnB,OAAO,GAAG,CAAC,EACX,QAAQ,GAAG,CAAC,EACZ,OAAO,GAAG,KAAK,EACf,aAAa,GAAG,MAAO,EACvB,aAAa,GAAG,IAAI,GACrB,GAAG,OAAO,CAAC;IACZ,MAAM,MAAM,GAAG,YAAY,CAAC,IAAI,CAAC,CAAC;IAClC,MAAM,QAAQ,GAAG,aAAa,CAAC,CAAC,CAAC,kBAAkB,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IACtE,6FAA6F;IAC7F,IAAI,UAAU,GAAG,KAAK,CAAC;IACvB,MAAM,KAAK,GAAW,EAAE,CAAC;IACzB,MAAM,aAAa,GAAkB,EAAE,CAAC;IACxC,oEAAoE;IACpE,MAAM,GAAG,GAAG,IAAI,GAAG,EAAU,CAAC;IAC9B,qFAAqF;IACrF,MAAM,OAAO,GAAG,IAAI,GAAG,EAAkB,CAAC;IAC1C,IAAI,KAAK,GAAc,EAAE,CAAC;IAC1B,IAAI,GAAG,GAAG,CAAC,CAAC;IACZ,IAAI,KAAK,GAAG,CAAC,CAAC;IACd,MAAM,KAAK,GAAG,EAAE,IAAI,EAAE,CAAC,EAAE,SAAS,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,GAAG,EAAE,CAAC,EAAE,CAAC;IAC5D,6EAA6E;IAC7E,IAAI,aAAiC,CAAC;IAEtC,MAAM,KAAK,GAAG,GAAW,EAAE;QACzB,IAAI,OAAO,OAAO,KAAK,QAAQ;YAAE,OAAO,OAAO,CAAC;QAChD,MAAM,CAAC,GAAG,EAAE,GAAG,CAAC,GAAG,OAAO,CAAC;QAC3B,OAAO,GAAG,GAAG,MAAM,CAAC,IAAI,EAAE,GAAG,CAAC,GAAG,GAAG,GAAG,CAAC,CAAC;IAC3C,CAAC,CAAC;IAEF,MAAM,QAAQ,GAAG,CAAC,IAAU,EAAE,EAAQ,EAAE,MAAkB,EAAQ,EAAE;QAClE,KAAK,CAAC,IAAI,EAAE,CAAC;QACb,IAAI,EAAE,GAAG,GAAG,GAAG,KAAK,EAAE,CAAC;QACvB,IAAI,CAAC,OAAO,EAAE,CAAC;YACb,MAAM,GAAG,GAAG,GAAG,IAAI,CAAC,KAAK,IAAI,EAAE,CAAC,KAAK,EAAE,CAAC;YACxC,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC;YACzC,OAAO,CAAC,GAAG,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;QACvB,CAAC;QACD,MAAM,OAAO,GAAY,EAAE,IAAI,EAAE,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,KAAK,EAAE,KAAK,EAAE,EAAE,CAAC;QAClE,4DAA4D;QAC5D,IAAI,KAAK,GAAG,KAAK,CAAC,MAAM,CAAC;QACzB,OACE,KAAK,GAAG,CAAC;YACT,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,EAAE,GAAG,EAAE;gBACvB,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC;YAEzE,KAAK,EAAE,CAAC;QACV,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC,EAAE,OAAO,CAAC,CAAC;IAClC,CAAC,CAAC;IAEF,MAAM,MAAM,GAAG,CAAC,CAAO,EAAE,CAAO,EAAW,EAAE,CAC3C,CAAC,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;IAErC,MAAM,SAAS,GAAG,CAAC,IAAU,EAAE,MAAkB,EAAQ,EAAE;QACzD,KAAK,MAAM,EAAE,IAAI,KAAK,EAAE,CAAC;YACvB,IAAI,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;gBAAE,SAAS;YAChC,IAAI,MAAM,CAAC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC;gBAC5B,KAAK,CAAC,IAAI,EAAE,CAAC;gBACb,KAAK,CAAC,OAAO,EAAE,CAAC;gBAChB,SAAS;YACX,CAAC;YACD,yDAAyD;YACzD,QAAQ,CAAC,IAAI,EAAE,EAAE,EAAE,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC;QACrC,CAAC;IACH,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,CAAC,CAAO,EAAE,CAAO,EAAQ,EAAE;QACtC,IAAI,CAAC,KAAK,CAAC;YAAE,OAAO;QACpB,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;QAC1B,4DAA4D;QAC5D,QAAQ,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,MAAM,EAAE,CAAC,CAAC;QAC/B,QAAQ,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,MAAM,EAAE,CAAC,CAAC;IACjC,CAAC,CAAC;IAEF,MAAM,MAAM,GAAG,CAAC,CAAO,EAAE,CAAO,EAAQ,EAAE;QACxC,IAAI,CAAC,KAAK,CAAC;YAAE,OAAO;QACpB,GAAG,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;QACvB,MAAM,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC;QAC5B,KAAK,GAAG,KAAK,CAAC,MAAM,CAClB,CAAC,EAAE,IAAI,EAAE,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,IAAI,CAAC,IAAI,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,CAC1E,CAAC;QACF,KAAK,CAAC,GAAG,IAAI,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC;IACrC,CAAC,CAAC;IAEF,MAAM,OAAO,GAAG,CAAC,OAAgB,EAAQ,EAAE;QACzC,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,CAAC,CAAC;QAChC,KAAK,CAAC,SAAS,EAAE,CAAC;QAClB,QAAQ,EAAE,OAAO,CAAC,OAAO,CAAC,EAAE,EAAE,KAAK,CAAC,CAAC;QACrC,OAAO,CAAC,EAAE,CAAC,GAAG,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC;IACvC,CAAC,CAAC;IAEF,4FAA4F;IAC5F,MAAM,YAAY,GAAG,KAAK,EAAE,KAAa,EAAiB,EAAE;QAC1D,IAAI,UAAU,GAAG,CAAC,CAAC;QACnB,SAAS,CAAC;YACR,MAAM,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;YACtB,IAAI,IAAI,IAAI,IAAI,CAAC,EAAE,IAAI,KAAK,EAAE,CAAC;gBAC7B,IAAI,EAAE,UAAU,GAAG,aAAa;oBAC9B,MAAM,IAAI,KAAK,CACb,oCAAoC,aAAa,2CAA2C,CAC7F,CAAC;gBACJ,KAAK,CAAC,KAAK,EAAE,CAAC;gBACd,OAAO,CAAC,IAAI,CAAC,CAAC;gBACd,MAAM,OAAO,CAAC,OAAO,EAAE,CAAC;gBACxB,QAAQ,EAAE,aAAa,CAAC,IAAI,CAAC,EAAE,EAAE,GAAG,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC;gBACjD,SAAS;YACX,CAAC;YACD,gFAAgF;YAChF,MAAM,gBAAgB,EAAE,CAAC;YACzB,MAAM,GAAG,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;YACrB,IAAI,CAAC,GAAG,IAAI,GAAG,CAAC,EAAE,GAAG,KAAK;gBAAE,OAAO;QACrC,CAAC;IACH,CAAC,CAAC;IAEF,8EAA8E;IAC9E,MAAM,GAAG,GAAG,KAAK,EAAE,KAAa,EAAiB,EAAE;QACjD,QAAQ,EAAE,gBAAgB,CAAC,KAAK,CAAC,CAAC;QAClC,UAAU,GAAG,IAAI,CAAC;QAClB,IAAI,CAAC;YACH,MAAM,YAAY,CAAC,KAAK,CAAC,CAAC;QAC5B,CAAC;gBAAS,CAAC;YACT,UAAU,GAAG,KAAK,CAAC;QACrB,CAAC;QACD,KAAK,MAAM,IAAI,IAAI,KAAK;YAAE,QAAQ,EAAE,aAAa,CAAC,IAAI,EAAE,GAAG,CAAC,CAAC;IAC/D,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,CAAC,IAAU,EAAE,EAAE,CAAC,CAAC;QAC5B,KAAK,EAAE,IAAI,CAAC,WAAW,EAAE;QACzB,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;KACtC,CAAC,CAAC;IAEH,MAAM,WAAW,GAAG,GAAa,EAAE;QACjC,MAAM,KAAK,GAAa,EAAE,CAAC;QAC3B,MAAM,QAAQ,GAAG,CAAC,OAAe,EAAE,IAAa,EAAE,KAAc,EAAQ,EAAE;YACxE,MAAM,UAAU,GAAG,eAAe,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;YAChD,IAAI,UAAU;gBACZ,KAAK,CAAC,IAAI,CACR,GAAG,OAAO,OAAO,UAAU,CAAC,UAAU,CAAC,IAAI,CAAC,KAAK,WAAW,CAAC,UAAU,CAAC,IAAI,CAAC,OAAO,WAAW,CAAC,UAAU,CAAC,KAAK,CAAC,EAAE,CACpH,CAAC;QACN,CAAC,CAAC;QAEF,MAAM,KAAK,GAAG,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAC9B,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE;YACxB,IAAI,IAAI,CAAC,MAAM,CAAC,SAAS,EAAE,CAAC,SAAS;gBACnC,QAAQ,CACN,GAAG,IAAI,CAAC,IAAI,oCAAoC,EAChD,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,EACd,KAAK,CAAC,CAAC,CAAC,CAAC,QAAQ,CAClB,CAAC;QACN,CAAC,CAAC,CAAC;QACH,MAAM,CAAC,KAAK,CAAC,GAAG,KAAK,CAAC;QACtB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACtC,QAAQ,CACN,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,4BAA4B,KAAK,CAAC,IAAI,IAAI,EAC1D,KAAK,CAAC,CAAC,CAAC,CAAC,QAAQ,EACjB,KAAK,CAAC,CAAC,CAAC,CAAC,QAAQ,CAClB,CAAC;YACF,QAAQ,CACN,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,yBAAyB,KAAK,CAAC,IAAI,IAAI,EACvD,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,EACd,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CACf,CAAC;QACJ,CAAC;QACD,OAAO,KAAK,CAAC;IACf,CAAC,CAAC;IAEF,OAAO;QACL,IAAI;QACJ,KAAK;QACL,GAAG,EAAE,GAAG,EAAE,CAAC,GAAG;QAEd,OAAO,EAAE,CAAmB,KAAmB,EAAW,EAAE;YAC1D,MAAM,KAAK,GAAG,KAAK,CAAC,MAAM,CAAC;YAC3B,MAAM,MAAM,GAAG,gBAAgB,CAAC;gBAC9B,GAAG,KAAK,CAAC,MAAM;gBACf,OAAO,EAAE,KAAK,CAAC,OAAO;gBACtB,OAAO,EAAE,KAAK,CAAC,OAAO;aACvB,CAAC,CAAC;YACH,MAAM,IAAI,GAAY;gBACpB,KAAK;gBACL,IAAI,EAAE,KAAK,CAAC,IAAI,IAAI,QAAQ,KAAK,EAAE;gBACnC,OAAO,EAAE,KAAK,CAAC,OAAO;gBACtB,OAAO,EAAE,KAAK,CAAC,OAAO;gBACtB,GAAG,EAAE,KAAK,CAAC,GAAG;gBACd,MAAM;gBACN,WAAW,EAAE,GAAG,EAAE,CAAC,MAAM,CAAC,MAAM,CAAC,cAAc,EAAE,CAAC;aACnD,CAAC;YACF,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YACjB,QAAQ,EAAE,OAAO,CAAC,IAAI,CAAC,CAAC;YACxB,aAAa,CAAC,IAAI,CAChB,KAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,MAAM,EAAE,EAAE;gBAC7B,QAAQ,EAAE,OAAO,CAAC,IAAI,EAAE,CAAC,UAAU,CAAC,CAAC;gBACrC,SAAS,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC;YAC1B,CAAC,CAAC,CACH,CAAC;YACF,KAAK,MAAM,KAAK,IAAI,KAAK;gBAAE,IAAI,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;YAC7C,IAAI,KAAK,CAAC,OAAO,KAAK,KAAK;gBAAE,MAAM,CAAC,OAAO,EAAE,CAAC;YAC9C,OAAO,IAAI,CAAC;QACd,CAAC;QAED,MAAM;QAEN,OAAO,EAAE,IAAI;QAEb,UAAU,EAAE,MAAM;QAElB,OAAO,EAAE,CAAC,IAAI,EAAE,EAAE;YAChB,KAAK,MAAM,KAAK,IAAI,KAAK;gBAAE,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;QACjD,CAAC;QAED,SAAS,EAAE,CAAC,GAAG,MAAM,EAAE,EAAE;YACvB,MAAM,OAAO,GAAG,CAAC,IAAU,EAAU,EAAE,CACrC,MAAM,CAAC,SAAS,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC;YACpD,KAAK,MAAM,CAAC,IAAI,KAAK;gBACnB,KAAK,MAAM,CAAC,IAAI,KAAK,EAAE,CAAC;oBACtB,IAAI,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,KAAK;wBAAE,SAAS;oBACjC,MAAM,QAAQ,GAAG,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,IAAI,OAAO,CAAC,CAAC,CAAC,KAAK,OAAO,CAAC,CAAC,CAAC,CAAC;oBAChE,IAAI,CAAC,QAAQ;wBAAE,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;yBACvB,IAAI,CAAC,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC;wBAAE,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;gBACrC,CAAC;QACL,CAAC;QAED,IAAI,EAAE,GAAG,EAAE;YACT,aAAa,GAAG,KAAK,CAAC,OAAO,CAAC;YAC9B,KAAK,MAAM,CAAC,IAAI,KAAK;gBACnB,KAAK,MAAM,CAAC,IAAI,KAAK;oBAAE,IAAI,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK;wBAAE,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;QAC7D,CAAC;QAED,OAAO,EAAE,KAAK,EAAE,EAAE,EAAE,EAAE;YACpB,MAAM,KAAK,GAAG,GAAG,GAAG,EAAE,CAAC;YACvB,MAAM,GAAG,CAAC,KAAK,CAAC,CAAC;YACjB,GAAG,GAAG,KAAK,CAAC;QACd,CAAC;QAED,MAAM,EAAE,GAAG,EAAE,CAAC,GAAG,CAAC,QAAQ,CAAC;QAE3B,KAAK,EAAE,GAAG,EAAE,CAAC,CAAC,EAAE,GAAG,KAAK,EAAE,QAAQ,EAAE,KAAK,CAAC,MAAM,EAAE,CAAC;QAEnD,WAAW;QAEX,eAAe,EAAE,GAAG,EAAE;YACpB,MAAM,KAAK,GAAG,WAAW,EAAE,CAAC;YAC5B,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;gBAAE,OAAO;YAC/B,MAAM,QAAQ,GACZ,KAAK,CAAC,MAAM,KAAK,CAAC;gBAChB,CAAC,CAAC,wDAAwD;gBAC1D,CAAC,CAAC,KAAK,CAAC,MAAM,GAAG,CAAC;oBAChB,CAAC,CAAC,KAAK,KAAK,CAAC,MAAM,2DAA2D;oBAC9E,CAAC,CAAC,EAAE,CAAC;YACX,MAAM,OAAO,GAAG,KAAK,CAAC,OAAO,GAAG,CAAC,aAAa,IAAI,CAAC,CAAC,CAAC;YACrD,MAAM,WAAW,GACf,OAAO,GAAG,CAAC;gBACT,CAAC,CAAC,KAAK,OAAO,KAAK,CAAC,CAAC,CAAC,CAAC,cAAc,CAAC,CAAC,CAAC,GAAG,OAAO,eAAe,WAC7D,aAAa,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,wBACrC,4DAA4D;gBAC9D,CAAC,CAAC,EAAE,CAAC;YACT,MAAM,IAAI,gBAAgB,CACxB,GAAG,KAAK,CAAC,MAAM,iCAAiC,IAAI,OAAO,KAAK;iBAC7D,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,KAAK,IAAI,EAAE,CAAC;iBAC1B,IAAI,CAAC,IAAI,CAAC,GAAG,QAAQ,GAAG,WAAW,EAAE,EACxC,KAAK,EACL,IAAI,CACL,CAAC;QACJ,CAAC;QAED,OAAO,EAAE,GAAG,EAAE;YACZ,KAAK,MAAM,IAAI,IAAI,KAAK;gBAAE,IAAI,CAAC,MAAM,CAAC,UAAU,EAAE,CAAC;YACnD,aAAa,CAAC,OAAO,CAAC,CAAC,WAAW,EAAE,EAAE,CAAC,WAAW,EAAE,CAAC,CAAC;YACtD,aAAa,CAAC,MAAM,GAAG,CAAC,CAAC;YACzB,KAAK,GAAG,EAAE,CAAC;QACb,CAAC;KACF,CAAC;AACJ,CAAC"}
@@ -13,7 +13,7 @@ export interface RandomizedOptions<S extends object = any> {
13
13
  steps?: number;
14
14
  /** What a step may do to a random peer's store, one picked at random. Defaults to `[randomEdit]`. */
15
15
  actions?: readonly RandomAction<S>[];
16
- /** Latency, drops and reordering, as for `createNetwork`. */
16
+ /** Latency, drops, reordering and the lost key check, as for `createNetwork`. */
17
17
  network?: Omit<NetworkOptions, "seed">;
18
18
  /** Chance that a step cuts or restores a random link instead of changing a store. Defaults to 0.1. */
19
19
  linkChangeRate?: number;
@@ -24,10 +24,16 @@ export interface RandomizedOptions<S extends object = any> {
24
24
  * Runs random steps against a fresh network, then heals it, settles it and checks that every
25
25
  * peer converged. Each step changes a random peer's store with a random action, or cuts or
26
26
  * restores a random link, and lets some virtual time pass, so changes cross each other on
27
- * their way.
27
+ * their way. Throughout, the network checks that no peer's store and document both lose a
28
+ * synced key that no peer deleted, unless `network.checkLostKeys` is `false`.
28
29
  *
29
- * Rejects with an error that names the seed and step when a step throws or the peers do not
30
- * converge; pass that `seed` to replay the same run. Give each document a fixed ID for runs
30
+ * With a `dropRate`, the updates stores write in answer to the healing exchange can be dropped
31
+ * too, so the run heals and settles again until a round drops nothing, at most 10 rounds, as a
32
+ * provider that resyncs would.
33
+ *
34
+ * Rejects with an error that names the seed and step when a step throws, a peer loses a synced
35
+ * key or the peers do not converge; pass that `seed` to replay the same run. When the last of
36
+ * those rounds still dropped updates, the error says so. Give each document a fixed ID for runs
31
37
  * that replay exactly, such as `doc.clientID = index + 1` for Yjs.
32
38
  */
33
39
  export declare function runRandomized<S extends object>(options: RandomizedOptions<S>): Promise<void>;
@@ -1 +1 @@
1
- {"version":3,"file":"randomized.d.ts","sourceRoot":"","sources":["../src/randomized.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,KAAK,cAAc,EACnB,KAAK,IAAI,EACT,KAAK,SAAS,EACf,MAAM,cAAc,CAAC;AAEtB,OAAO,EAA4B,KAAK,MAAM,EAAE,MAAM,aAAa,CAAC;AAEpE,mEAAmE;AAEnE,MAAM,MAAM,YAAY,CAAC,CAAC,SAAS,MAAM,GAAG,GAAG,IAAI,CACjD,IAAI,EAAE,IAAI,CAAC,CAAC,CAAC,EACb,MAAM,EAAE,MAAM,KACX,IAAI,CAAC;AAGV,MAAM,WAAW,iBAAiB,CAAC,CAAC,SAAS,MAAM,GAAG,GAAG;IACvD,kEAAkE;IAClE,UAAU,EAAE,CAAC,KAAK,EAAE,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,CAAC;IAC5C,iGAAiG;IACjG,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,qBAAqB;IACrB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,uBAAuB;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,qGAAqG;IACrG,OAAO,CAAC,EAAE,SAAS,YAAY,CAAC,CAAC,CAAC,EAAE,CAAC;IACrC,6DAA6D;IAC7D,OAAO,CAAC,EAAE,IAAI,CAAC,cAAc,EAAE,MAAM,CAAC,CAAC;IACvC,sGAAsG;IACtG,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,wFAAwF;IACxF,YAAY,CAAC,EAAE,MAAM,CAAC;CACvB;AAED;;;;;;;;;GASG;AACH,wBAAsB,aAAa,CAAC,CAAC,SAAS,MAAM,EAClD,OAAO,EAAE,iBAAiB,CAAC,CAAC,CAAC,GAC5B,OAAO,CAAC,IAAI,CAAC,CAiDf;AA+ED;;;;;;;;GAQG;AACH,eAAO,MAAM,UAAU,EAAE,YAyBxB,CAAC"}
1
+ {"version":3,"file":"randomized.d.ts","sourceRoot":"","sources":["../src/randomized.ts"],"names":[],"mappings":"AAAA,OAAO,EAEL,KAAK,cAAc,EACnB,KAAK,IAAI,EACT,KAAK,SAAS,EACf,MAAM,cAAc,CAAC;AAEtB,OAAO,EAA4B,KAAK,MAAM,EAAE,MAAM,aAAa,CAAC;AAEpE,mEAAmE;AAEnE,MAAM,MAAM,YAAY,CAAC,CAAC,SAAS,MAAM,GAAG,GAAG,IAAI,CACjD,IAAI,EAAE,IAAI,CAAC,CAAC,CAAC,EACb,MAAM,EAAE,MAAM,KACX,IAAI,CAAC;AAMV,MAAM,WAAW,iBAAiB,CAAC,CAAC,SAAS,MAAM,GAAG,GAAG;IACvD,kEAAkE;IAClE,UAAU,EAAE,CAAC,KAAK,EAAE,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,CAAC;IAC5C,iGAAiG;IACjG,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,qBAAqB;IACrB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,uBAAuB;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,qGAAqG;IACrG,OAAO,CAAC,EAAE,SAAS,YAAY,CAAC,CAAC,CAAC,EAAE,CAAC;IACrC,iFAAiF;IACjF,OAAO,CAAC,EAAE,IAAI,CAAC,cAAc,EAAE,MAAM,CAAC,CAAC;IACvC,sGAAsG;IACtG,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,wFAAwF;IACxF,YAAY,CAAC,EAAE,MAAM,CAAC;CACvB;AAED;;;;;;;;;;;;;;;GAeG;AACH,wBAAsB,aAAa,CAAC,CAAC,SAAS,MAAM,EAClD,OAAO,EAAE,iBAAiB,CAAC,CAAC,CAAC,GAC5B,OAAO,CAAC,IAAI,CAAC,CA8Df;AA+ED;;;;;;;;GAQG;AACH,eAAO,MAAM,UAAU,EAAE,YAyBxB,CAAC"}
@@ -1,13 +1,21 @@
1
1
  import { createNetwork, } from "./network.js";
2
2
  import { createRandom, randomSeed } from "./random.js";
3
+ /** Most rounds of `heal()` and `settle()` at the end of a run. */
4
+ const HEAL_ROUNDS = 10;
3
5
  /**
4
6
  * Runs random steps against a fresh network, then heals it, settles it and checks that every
5
7
  * peer converged. Each step changes a random peer's store with a random action, or cuts or
6
8
  * restores a random link, and lets some virtual time pass, so changes cross each other on
7
- * their way.
9
+ * their way. Throughout, the network checks that no peer's store and document both lose a
10
+ * synced key that no peer deleted, unless `network.checkLostKeys` is `false`.
8
11
  *
9
- * Rejects with an error that names the seed and step when a step throws or the peers do not
10
- * converge; pass that `seed` to replay the same run. Give each document a fixed ID for runs
12
+ * With a `dropRate`, the updates stores write in answer to the healing exchange can be dropped
13
+ * too, so the run heals and settles again until a round drops nothing, at most 10 rounds, as a
14
+ * provider that resyncs would.
15
+ *
16
+ * Rejects with an error that names the seed and step when a step throws, a peer loses a synced
17
+ * key or the peers do not converge; pass that `seed` to replay the same run. When the last of
18
+ * those rounds still dropped updates, the error says so. Give each document a fixed ID for runs
11
19
  * that replay exactly, such as `doc.clientID = index + 1` for Yjs.
12
20
  */
13
21
  export async function runRandomized(options) {
@@ -16,6 +24,8 @@ export async function runRandomized(options) {
16
24
  const random = createRandom(seed ^ 0x5bd1e995);
17
25
  const network = createNetwork({ ...networkOptions, seed });
18
26
  let step = 0;
27
+ /** Whether every round of healing dropped updates, the last included. */
28
+ let stillDropping = false;
19
29
  try {
20
30
  const peers = Array.from({ length: peerCount }, (_, index) => network.addPeer(createPeer(index)));
21
31
  for (; step < steps; step++) {
@@ -32,14 +42,25 @@ export async function runRandomized(options) {
32
42
  }
33
43
  await network.advance(random.int(stepInterval + 1));
34
44
  }
35
- network.heal();
36
- await network.settle();
45
+ // Stores may write in answer to the documents a heal exchanges, and those updates can be
46
+ // dropped too: heal again until a round drops nothing, as a provider that resyncs would.
47
+ let dropping = true;
48
+ for (let round = 0; dropping && round < HEAL_ROUNDS; round++) {
49
+ const dropped = network.stats().dropped;
50
+ network.heal();
51
+ await network.settle();
52
+ dropping = network.stats().dropped !== dropped;
53
+ }
54
+ stillDropping = dropping;
37
55
  network.expectConverged();
38
56
  }
39
57
  catch (error) {
40
58
  const when = step < steps ? `at step ${step + 1} of ${steps}` : `after ${steps} steps`;
41
59
  const message = error instanceof Error ? error.message : String(error);
42
- throw Object.assign(new Error(`Randomized run failed ${when}; pass \`seed: ${seed}\` to replay it.\n${message}`), { cause: error });
60
+ const dropping = stillDropping
61
+ ? `\nUpdates were still being dropped after ${HEAL_ROUNDS} rounds of heal() and settle(): does a store write in answer to every exchange?`
62
+ : "";
63
+ throw Object.assign(new Error(`Randomized run failed ${when}; pass \`seed: ${seed}\` to replay it.\n${message}${dropping}`), { cause: error });
43
64
  }
44
65
  finally {
45
66
  network.destroy();
@@ -1 +1 @@
1
- {"version":3,"file":"randomized.js","sourceRoot":"","sources":["../src/randomized.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,aAAa,GAId,MAAM,cAAc,CAAC;AAEtB,OAAO,EAAE,YAAY,EAAE,UAAU,EAAe,MAAM,aAAa,CAAC;AA6BpE;;;;;;;;;GASG;AACH,MAAM,CAAC,KAAK,UAAU,aAAa,CACjC,OAA6B;IAE7B,MAAM,EACJ,UAAU,EACV,IAAI,GAAG,UAAU,EAAE,EACnB,KAAK,EAAE,SAAS,GAAG,CAAC,EACpB,KAAK,GAAG,GAAG,EACX,OAAO,GAAG,CAAC,UAAU,CAAC,EACtB,OAAO,EAAE,cAAc,EACvB,cAAc,GAAG,GAAG,EACpB,YAAY,GAAG,EAAE,GAClB,GAAG,OAAO,CAAC;IACZ,+EAA+E;IAC/E,MAAM,MAAM,GAAG,YAAY,CAAC,IAAI,GAAG,UAAU,CAAC,CAAC;IAC/C,MAAM,OAAO,GAAG,aAAa,CAAC,EAAE,GAAG,cAAc,EAAE,IAAI,EAAE,CAAC,CAAC;IAC3D,IAAI,IAAI,GAAG,CAAC,CAAC;IAEb,IAAI,CAAC;QACH,MAAM,KAAK,GAAG,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,SAAS,EAAE,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,EAAE,CAC3D,OAAO,CAAC,OAAO,CAAC,UAAU,CAAC,KAAK,CAAC,CAAC,CACnC,CAAC;QAEF,OAAO,IAAI,GAAG,KAAK,EAAE,IAAI,EAAE,EAAE,CAAC;YAC5B,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,cAAc,CAAC,EAAE,CAAC;gBACtD,MAAM,CAAC,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;gBAC7B,MAAM,CAAC,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC;gBAC1D,IAAI,OAAO,CAAC,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC;oBAAE,OAAO,CAAC,UAAU,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;;oBAC9C,OAAO,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;YAC7B,CAAC;iBAAM,CAAC;gBACN,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC,CAAC;YACnD,CAAC;YACD,MAAM,OAAO,CAAC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,YAAY,GAAG,CAAC,CAAC,CAAC,CAAC;QACtD,CAAC;QAED,OAAO,CAAC,IAAI,EAAE,CAAC;QACf,MAAM,OAAO,CAAC,MAAM,EAAE,CAAC;QACvB,OAAO,CAAC,eAAe,EAAE,CAAC;IAC5B,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,IAAI,GACR,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,WAAW,IAAI,GAAG,CAAC,OAAO,KAAK,EAAE,CAAC,CAAC,CAAC,SAAS,KAAK,QAAQ,CAAC;QAC5E,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACvE,MAAM,MAAM,CAAC,MAAM,CACjB,IAAI,KAAK,CACP,yBAAyB,IAAI,kBAAkB,IAAI,qBAAqB,OAAO,EAAE,CAClF,EACD,EAAE,KAAK,EAAE,KAAK,EAAE,CACjB,CAAC;IACJ,CAAC;YAAS,CAAC;QACT,OAAO,CAAC,OAAO,EAAE,CAAC;IACpB,CAAC;AACH,CAAC;AAID,MAAM,QAAQ,GAAG,CAAC,KAAc,EAAkB,EAAE,CAClD,KAAK,KAAK,IAAI,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;AAEvE,0DAA0D;AAC1D,MAAM,IAAI,GAAG,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC,CAAC;AAClC,qFAAqF;AACrF,MAAM,UAAU,GAAG,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,CAAC,CAAC;AAE9C,MAAM,UAAU,GAAG,CAAC,MAAc,EAAU,EAAE,CAC5C,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,GAAG,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;AAEhF,MAAM,WAAW,GAAG,CAAC,MAAc,EAAE,KAAK,GAAG,CAAC,EAAW,EAAE;IACzD,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,EAAE,CAAC;IAC3B,IAAI,KAAK,GAAG,CAAC,IAAI,IAAI,GAAG,IAAI;QAC1B,OAAO,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,GAAG,EAAE,CAChD,WAAW,CAAC,MAAM,EAAE,KAAK,GAAG,CAAC,CAAC,CAC/B,CAAC;IACJ,IAAI,KAAK,GAAG,CAAC,IAAI,IAAI,GAAG,GAAG;QACzB,OAAO,MAAM,CAAC,WAAW,CACvB,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,GAAG,EAAE,CAAC;YAC1C,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC;YACjB,WAAW,CAAC,MAAM,EAAE,KAAK,GAAG,CAAC,CAAC;SAC/B,CAAC,CACH,CAAC;IACJ,IAAI,IAAI,GAAG,IAAI;QAAE,OAAO,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IACxC,IAAI,IAAI,GAAG,GAAG;QAAE,OAAO,UAAU,CAAC,MAAM,CAAC,CAAC;IAC1C,IAAI,IAAI,GAAG,IAAI;QAAE,OAAO,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;IAC3C,OAAO,IAAI,CAAC;AACd,CAAC,CAAC;AAEF,yFAAyF;AACzF,MAAM,QAAQ,GAAG,CACf,IAAa,EACb,IAAU,EACV,IAAiC,EACxB,EAAE;IACX,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC,IAAI,CAAC,CAAC;IACzC,MAAM,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,GAAG,IAAI,CAAC;IAC5B,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,EAAE,CAAC;QAC1B,IAAI,CAAC,GAAa,CAAC,GAAG,QAAQ,CAAC,IAAI,CAAC,GAAa,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC;QAChE,OAAO,IAAI,CAAC;IACd,CAAC;IACD,OAAO;QACL,GAAI,IAAc;QAClB,CAAC,GAAG,CAAC,EAAE,QAAQ,CAAE,IAAc,CAAC,GAAG,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC;KAClD,CAAC;AACJ,CAAC,CAAC;AAEF,oGAAoG;AACpG,MAAM,SAAS,GAAG,CAAC,KAAc,EAAE,MAAc,EAAW,EAAE;IAC5D,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;QAC/C,MAAM,IAAI,GAAG,KAAK,CAAC,KAAK,EAAE,CAAC;QAC3B,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC;YACvC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;;YACrC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,EAAE,CAAC,EAAE,WAAW,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC,CAAC;QACzE,OAAO,IAAI,CAAC;IACd,CAAC;IACD,IAAI,QAAQ,CAAC,KAAK,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;QAC1C,MAAM,IAAI,GAAG,EAAE,GAAG,KAAK,EAAE,CAAC;QAC1B,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QAC/B,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC;YAAE,OAAO,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC;;YACrE,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,GAAG,WAAW,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;QACtD,OAAO,IAAI,CAAC;IACd,CAAC;IACD,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;QACpD,MAAM,UAAU,GAAG,CAAC,GAAG,KAAK,CAAC,CAAC;QAC9B,MAAM,EAAE,GAAG,MAAM,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;QAC7C,IAAI,UAAU,CAAC,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC;YAAE,UAAU,CAAC,MAAM,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC;;YACrE,UAAU,CAAC,MAAM,CAAC,EAAE,EAAE,CAAC,EAAE,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC;QACvD,OAAO,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IAC7B,CAAC;IACD,OAAO,WAAW,CAAC,MAAM,CAAC,CAAC;AAC7B,CAAC,CAAC;AAEF;;;;;;;;GAQG;AACH,MAAM,CAAC,MAAM,UAAU,GAAiB,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE;IACvD,MAAM,MAAM,GAAG,IAAI,CAAC,WAAW,EAAE,CAAC;IAClC,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC;QAAE,OAAO;IAC9B,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IACjC,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO;IAE9B,MAAM,IAAI,GAAS,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC;IACvC,IAAI,IAAI,GAAY,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC;IACpC,OAAO,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;QAC1B,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC3C,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACtC,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACjB,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC;QACrB,CAAC;aAAM,IAAI,QAAQ,CAAC,IAAI,CAAC,IAAI,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC1D,MAAM,GAAG,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC;YAC3C,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YACf,IAAI,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC;QACnB,CAAC;;YAAM,MAAM;IACf,CAAC;IAED,IAAI,CAAC,OAAO,CAAC,QAAQ,CACnB,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,QAAQ,EAAE,EAAE,IAAI,EAAE,CAAC,KAAK,EAAE,EAAE,CAChD,SAAS,CAAC,KAAK,EAAE,MAAM,CAAC,CACf,CACZ,CAAC;AACJ,CAAC,CAAC"}
1
+ {"version":3,"file":"randomized.js","sourceRoot":"","sources":["../src/randomized.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,aAAa,GAId,MAAM,cAAc,CAAC;AAEtB,OAAO,EAAE,YAAY,EAAE,UAAU,EAAe,MAAM,aAAa,CAAC;AASpE,kEAAkE;AAClE,MAAM,WAAW,GAAG,EAAE,CAAC;AAsBvB;;;;;;;;;;;;;;;GAeG;AACH,MAAM,CAAC,KAAK,UAAU,aAAa,CACjC,OAA6B;IAE7B,MAAM,EACJ,UAAU,EACV,IAAI,GAAG,UAAU,EAAE,EACnB,KAAK,EAAE,SAAS,GAAG,CAAC,EACpB,KAAK,GAAG,GAAG,EACX,OAAO,GAAG,CAAC,UAAU,CAAC,EACtB,OAAO,EAAE,cAAc,EACvB,cAAc,GAAG,GAAG,EACpB,YAAY,GAAG,EAAE,GAClB,GAAG,OAAO,CAAC;IACZ,+EAA+E;IAC/E,MAAM,MAAM,GAAG,YAAY,CAAC,IAAI,GAAG,UAAU,CAAC,CAAC;IAC/C,MAAM,OAAO,GAAG,aAAa,CAAC,EAAE,GAAG,cAAc,EAAE,IAAI,EAAE,CAAC,CAAC;IAC3D,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,yEAAyE;IACzE,IAAI,aAAa,GAAG,KAAK,CAAC;IAE1B,IAAI,CAAC;QACH,MAAM,KAAK,GAAG,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,SAAS,EAAE,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,EAAE,CAC3D,OAAO,CAAC,OAAO,CAAC,UAAU,CAAC,KAAK,CAAC,CAAC,CACnC,CAAC;QAEF,OAAO,IAAI,GAAG,KAAK,EAAE,IAAI,EAAE,EAAE,CAAC;YAC5B,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,cAAc,CAAC,EAAE,CAAC;gBACtD,MAAM,CAAC,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;gBAC7B,MAAM,CAAC,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC;gBAC1D,IAAI,OAAO,CAAC,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC;oBAAE,OAAO,CAAC,UAAU,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;;oBAC9C,OAAO,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;YAC7B,CAAC;iBAAM,CAAC;gBACN,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC,CAAC;YACnD,CAAC;YACD,MAAM,OAAO,CAAC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,YAAY,GAAG,CAAC,CAAC,CAAC,CAAC;QACtD,CAAC;QAED,yFAAyF;QACzF,yFAAyF;QACzF,IAAI,QAAQ,GAAG,IAAI,CAAC;QACpB,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,QAAQ,IAAI,KAAK,GAAG,WAAW,EAAE,KAAK,EAAE,EAAE,CAAC;YAC7D,MAAM,OAAO,GAAG,OAAO,CAAC,KAAK,EAAE,CAAC,OAAO,CAAC;YACxC,OAAO,CAAC,IAAI,EAAE,CAAC;YACf,MAAM,OAAO,CAAC,MAAM,EAAE,CAAC;YACvB,QAAQ,GAAG,OAAO,CAAC,KAAK,EAAE,CAAC,OAAO,KAAK,OAAO,CAAC;QACjD,CAAC;QACD,aAAa,GAAG,QAAQ,CAAC;QACzB,OAAO,CAAC,eAAe,EAAE,CAAC;IAC5B,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,IAAI,GACR,IAAI,GAAG,KAAK,CAAC,CAAC,CAAC,WAAW,IAAI,GAAG,CAAC,OAAO,KAAK,EAAE,CAAC,CAAC,CAAC,SAAS,KAAK,QAAQ,CAAC;QAC5E,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACvE,MAAM,QAAQ,GAAG,aAAa;YAC5B,CAAC,CAAC,4CAA4C,WAAW,iFAAiF;YAC1I,CAAC,CAAC,EAAE,CAAC;QACP,MAAM,MAAM,CAAC,MAAM,CACjB,IAAI,KAAK,CACP,yBAAyB,IAAI,kBAAkB,IAAI,qBAAqB,OAAO,GAAG,QAAQ,EAAE,CAC7F,EACD,EAAE,KAAK,EAAE,KAAK,EAAE,CACjB,CAAC;IACJ,CAAC;YAAS,CAAC;QACT,OAAO,CAAC,OAAO,EAAE,CAAC;IACpB,CAAC;AACH,CAAC;AAID,MAAM,QAAQ,GAAG,CAAC,KAAc,EAAkB,EAAE,CAClD,KAAK,KAAK,IAAI,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;AAEvE,0DAA0D;AAC1D,MAAM,IAAI,GAAG,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC,CAAC;AAClC,qFAAqF;AACrF,MAAM,UAAU,GAAG,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,CAAC,CAAC;AAE9C,MAAM,UAAU,GAAG,CAAC,MAAc,EAAU,EAAE,CAC5C,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,GAAG,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;AAEhF,MAAM,WAAW,GAAG,CAAC,MAAc,EAAE,KAAK,GAAG,CAAC,EAAW,EAAE;IACzD,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,EAAE,CAAC;IAC3B,IAAI,KAAK,GAAG,CAAC,IAAI,IAAI,GAAG,IAAI;QAC1B,OAAO,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,GAAG,EAAE,CAChD,WAAW,CAAC,MAAM,EAAE,KAAK,GAAG,CAAC,CAAC,CAC/B,CAAC;IACJ,IAAI,KAAK,GAAG,CAAC,IAAI,IAAI,GAAG,GAAG;QACzB,OAAO,MAAM,CAAC,WAAW,CACvB,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,GAAG,EAAE,CAAC;YAC1C,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC;YACjB,WAAW,CAAC,MAAM,EAAE,KAAK,GAAG,CAAC,CAAC;SAC/B,CAAC,CACH,CAAC;IACJ,IAAI,IAAI,GAAG,IAAI;QAAE,OAAO,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IACxC,IAAI,IAAI,GAAG,GAAG;QAAE,OAAO,UAAU,CAAC,MAAM,CAAC,CAAC;IAC1C,IAAI,IAAI,GAAG,IAAI;QAAE,OAAO,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;IAC3C,OAAO,IAAI,CAAC;AACd,CAAC,CAAC;AAEF,yFAAyF;AACzF,MAAM,QAAQ,GAAG,CACf,IAAa,EACb,IAAU,EACV,IAAiC,EACxB,EAAE;IACX,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC,IAAI,CAAC,CAAC;IACzC,MAAM,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,GAAG,IAAI,CAAC;IAC5B,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,EAAE,CAAC;QAC1B,IAAI,CAAC,GAAa,CAAC,GAAG,QAAQ,CAAC,IAAI,CAAC,GAAa,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC;QAChE,OAAO,IAAI,CAAC;IACd,CAAC;IACD,OAAO;QACL,GAAI,IAAc;QAClB,CAAC,GAAG,CAAC,EAAE,QAAQ,CAAE,IAAc,CAAC,GAAG,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC;KAClD,CAAC;AACJ,CAAC,CAAC;AAEF,oGAAoG;AACpG,MAAM,SAAS,GAAG,CAAC,KAAc,EAAE,MAAc,EAAW,EAAE;IAC5D,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;QAC/C,MAAM,IAAI,GAAG,KAAK,CAAC,KAAK,EAAE,CAAC;QAC3B,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC;YACvC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;;YACrC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,EAAE,CAAC,EAAE,WAAW,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC,CAAC;QACzE,OAAO,IAAI,CAAC;IACd,CAAC;IACD,IAAI,QAAQ,CAAC,KAAK,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;QAC1C,MAAM,IAAI,GAAG,EAAE,GAAG,KAAK,EAAE,CAAC;QAC1B,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QAC/B,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC;YAAE,OAAO,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC;;YACrE,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,GAAG,WAAW,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC;QACtD,OAAO,IAAI,CAAC;IACd,CAAC;IACD,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;QACpD,MAAM,UAAU,GAAG,CAAC,GAAG,KAAK,CAAC,CAAC;QAC9B,MAAM,EAAE,GAAG,MAAM,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;QAC7C,IAAI,UAAU,CAAC,MAAM,GAAG,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC;YAAE,UAAU,CAAC,MAAM,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC;;YACrE,UAAU,CAAC,MAAM,CAAC,EAAE,EAAE,CAAC,EAAE,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC;QACvD,OAAO,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IAC7B,CAAC;IACD,OAAO,WAAW,CAAC,MAAM,CAAC,CAAC;AAC7B,CAAC,CAAC;AAEF;;;;;;;;GAQG;AACH,MAAM,CAAC,MAAM,UAAU,GAAiB,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE;IACvD,MAAM,MAAM,GAAG,IAAI,CAAC,WAAW,EAAE,CAAC;IAClC,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC;QAAE,OAAO;IAC9B,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IACjC,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO;IAE9B,MAAM,IAAI,GAAS,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC;IACvC,IAAI,IAAI,GAAY,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC;IACpC,OAAO,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC;QAC1B,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC3C,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACtC,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACjB,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC;QACrB,CAAC;aAAM,IAAI,QAAQ,CAAC,IAAI,CAAC,IAAI,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC1D,MAAM,GAAG,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC;YAC3C,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YACf,IAAI,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC;QACnB,CAAC;;YAAM,MAAM;IACf,CAAC;IAED,IAAI,CAAC,OAAO,CAAC,QAAQ,CACnB,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,QAAQ,EAAE,EAAE,IAAI,EAAE,CAAC,KAAK,EAAE,EAAE,CAChD,SAAS,CAAC,KAAK,EAAE,MAAM,CAAC,CACf,CACZ,CAAC;AACJ,CAAC,CAAC"}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@homeostate/tool-simulator",
3
- "version": "0.2.0",
3
+ "version": "0.3.0",
4
4
  "description": "Test homeostate peers on a simulated network with latency, drops, reordering and partitions, and check that they converge",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -43,17 +43,17 @@
43
43
  "access": "public"
44
44
  },
45
45
  "dependencies": {
46
- "@homeostate/core": "^0.4.0"
46
+ "@homeostate/core": "^0.5.0"
47
47
  },
48
48
  "devDependencies": {
49
49
  "@automerge/automerge": "^3.4.1",
50
- "loro-crdt": "^1.16.1",
50
+ "loro-crdt": "^1.17.1",
51
51
  "yjs": "^13.6.23",
52
52
  "zustand": "^4.5.2",
53
- "@homeostate/crdt-loro": "^0.3.0",
54
- "@homeostate/store-zustand": "^0.3.0",
55
- "@homeostate/crdt-automerge": "^0.3.0",
56
- "@homeostate/crdt-yjs": "^0.3.0"
53
+ "@homeostate/crdt-automerge": "^0.3.1",
54
+ "@homeostate/store-zustand": "^0.3.1",
55
+ "@homeostate/crdt-yjs": "^0.4.0",
56
+ "@homeostate/crdt-loro": "^0.4.0"
57
57
  },
58
58
  "scripts": {
59
59
  "build": "tsc -b"
package/src/index.ts CHANGED
@@ -1,4 +1,5 @@
1
1
  export { ConvergenceError, createNetwork } from "./network.js";
2
+ export { LostKeyError } from "./lost-keys.js";
2
3
  export type {
3
4
  Network,
4
5
  NetworkOptions,
@@ -0,0 +1,164 @@
1
+ import { formatPath } from "./compare.js";
2
+ import type { Peer } from "./network.js";
3
+
4
+ /** Thrown by `advance` and `settle` when a peer lost a synced key; see `checkLostKeys`. */
5
+ export class LostKeyError extends Error {
6
+ readonly seed: number;
7
+ /** The peer's name. */
8
+ readonly peer: string;
9
+ readonly key: string;
10
+ /** Virtual time of the checkpoint that found the loss, in milliseconds. */
11
+ readonly at: number;
12
+
13
+ constructor(
14
+ message: string,
15
+ details: { seed: number; peer: string; key: string; at: number },
16
+ ) {
17
+ super(message);
18
+ this.name = "LostKeyError";
19
+ this.seed = details.seed;
20
+ this.peer = details.peer;
21
+ this.key = details.key;
22
+ this.at = details.at;
23
+ }
24
+ }
25
+
26
+ /** What the check knows of one peer. */
27
+ interface Tracked {
28
+ /** The store's synced keys at the peer's last checkpoint. */
29
+ store: Set<string>;
30
+ /** The document's keys, while they cannot have changed since they were read. */
31
+ doc: Set<string> | undefined;
32
+ /** The document's keys when it was last read, for telling which ones a change removed. */
33
+ lastDoc: Set<string> | undefined;
34
+ /** Whether the document changed outside a delivery since it was last read. */
35
+ changedOutside: boolean;
36
+ /** Keys a checkpoint saw both in the store and in the document. */
37
+ held: Set<string>;
38
+ }
39
+
40
+ /**
41
+ * Checks that no peer's store and document both lose a synced top-level key that no peer
42
+ * deleted. The network calls it as its documents change and around its deliveries.
43
+ */
44
+ export interface LostKeyCheck {
45
+ /** Start tracking a peer, before it links and connects. */
46
+ addPeer: (peer: Peer) => void;
47
+ /** The peer's document changed; `outside` when no delivery is running. */
48
+ changed: (peer: Peer, outside: boolean) => void;
49
+ /**
50
+ * Before deliveries: what changed since the last checkpoint was done by the test, so a key
51
+ * that left a store or document is deleted.
52
+ */
53
+ beforeDeliveries: (peers: readonly Peer[]) => void;
54
+ /** After a delivery to `peer` from `from`, or after the last one; throws a `LostKeyError`. */
55
+ afterDelivery: (peer: Peer, at: number, from?: Peer) => void;
56
+ }
57
+
58
+ /** The keys of `state` that JSON keeps, as `peer.syncedState()` has them. */
59
+ const jsonKeys = (state: object): Set<string> => {
60
+ const keys = new Set<string>();
61
+ for (const [key, value] of Object.entries(state))
62
+ if (
63
+ value !== undefined &&
64
+ typeof value !== "function" &&
65
+ typeof value !== "symbol"
66
+ )
67
+ keys.add(key);
68
+ return keys;
69
+ };
70
+
71
+ const formatTime = (ms: number): string => String(Number(ms.toFixed(2)));
72
+
73
+ export function createLostKeyCheck(seed: number): LostKeyCheck {
74
+ const tracked = new Map<Peer, Tracked>();
75
+ /** Keys a checkpoint outside a delivery saw leave a store or document; never cleared. */
76
+ const deleted = new Set<string>();
77
+
78
+ const storeKeys = (peer: Peer): Set<string> =>
79
+ jsonKeys(peer.engine.getSyncedState());
80
+
81
+ const docKeys = (peer: Peer, state: Tracked): Set<string> => {
82
+ if (!state.doc) {
83
+ state.doc = jsonKeys(peer.backend.read());
84
+ // Only the test's own changes delete; a delivery that removes a key may be a loss.
85
+ if (state.changedOutside && state.lastDoc)
86
+ for (const key of state.lastDoc)
87
+ if (!state.doc.has(key)) deleted.add(key);
88
+ state.lastDoc = state.doc;
89
+ state.changedOutside = false;
90
+ }
91
+ return state.doc;
92
+ };
93
+
94
+ /** Mark the keys that left the store since its last checkpoint as deleted. */
95
+ const noteRemovals = (state: Tracked, store: Set<string>): void => {
96
+ for (const key of state.store) if (!store.has(key)) deleted.add(key);
97
+ state.store = store;
98
+ };
99
+
100
+ const hold = (peer: Peer, state: Tracked, keys: Iterable<string>): void => {
101
+ for (const key of keys) {
102
+ if (state.held.has(key) || deleted.has(key)) continue;
103
+ if (docKeys(peer, state).has(key)) state.held.add(key);
104
+ }
105
+ };
106
+
107
+ const track = (peer: Peer): Tracked => {
108
+ const state = tracked.get(peer);
109
+ if (!state) throw new Error(`${peer.name} is not on the network.`);
110
+ return state;
111
+ };
112
+
113
+ return {
114
+ addPeer: (peer) => {
115
+ tracked.set(peer, {
116
+ store: storeKeys(peer),
117
+ doc: undefined,
118
+ lastDoc: undefined,
119
+ changedOutside: false,
120
+ held: new Set(),
121
+ });
122
+ },
123
+
124
+ changed: (peer, outside) => {
125
+ const state = track(peer);
126
+ state.doc = undefined;
127
+ if (!outside) return;
128
+ state.changedOutside = true;
129
+ // A key added and removed again before the next delivery still counts as deleted.
130
+ noteRemovals(state, storeKeys(peer));
131
+ },
132
+
133
+ beforeDeliveries: (peers) => {
134
+ for (const peer of peers) {
135
+ const state = track(peer);
136
+ const store = storeKeys(peer);
137
+ noteRemovals(state, store);
138
+ if (state.changedOutside) docKeys(peer, state);
139
+ hold(peer, state, store);
140
+ }
141
+ },
142
+
143
+ afterDelivery: (peer, at, from) => {
144
+ const state = track(peer);
145
+ const store = storeKeys(peer);
146
+ for (const key of state.held) {
147
+ if (store.has(key)) continue;
148
+ if (deleted.has(key)) {
149
+ state.held.delete(key);
150
+ continue;
151
+ }
152
+ if (docKeys(peer, state).has(key)) continue;
153
+ const source = from ? `, on an update from ${from.name}` : "";
154
+ throw new LostKeyError(
155
+ `${peer.name} lost the synced key ${formatPath([key])} at ${formatTime(at)} ms (seed ${seed})${source}: its store and document both lack it, and no peer deleted it.`,
156
+ { seed, peer: peer.name, key, at },
157
+ );
158
+ }
159
+ const appeared = [...store].filter((key) => !state.store.has(key));
160
+ state.store = store;
161
+ hold(peer, state, appeared);
162
+ },
163
+ };
164
+ }
package/src/network.ts CHANGED
@@ -8,6 +8,7 @@ import {
8
8
  type Unsubscribe,
9
9
  } from "@homeostate/core";
10
10
  import { asJson, firstDifference, formatPath, formatValue } from "./compare.js";
11
+ import { createLostKeyCheck } from "./lost-keys.js";
11
12
  import { createRandom, randomSeed } from "./random.js";
12
13
 
13
14
  /**
@@ -70,6 +71,17 @@ export interface NetworkOptions {
70
71
  * as two peers writing in a loop. Defaults to 100 000.
71
72
  */
72
73
  maxDeliveries?: number;
74
+ /**
75
+ * Whether `advance` and `settle` reject with a `LostKeyError` when a peer's store and
76
+ * document both lose a synced top-level key that no peer deleted. Defaults to `true`.
77
+ *
78
+ * A peer holds a key once a checkpoint sees it in both its store's synced state and its
79
+ * document. A key is deleted for good once it leaves a store or document outside a delivery:
80
+ * in a step, or in a test's own change, between `advance` and `settle` calls. Checkpoints
81
+ * come after each delivery and when `advance` or `settle` returns. A key deleted during a
82
+ * delivery, such as by a store subscriber or a timer, is reported as lost: pass `false`.
83
+ */
84
+ checkLostKeys?: boolean;
73
85
  }
74
86
 
75
87
  export interface NetworkStats {
@@ -89,6 +101,9 @@ export interface NetworkStats {
89
101
  * document goes to every peer it is linked to, after the latency, unless it is dropped or its
90
102
  * link is cut first. Linking two peers, as `connect` and `heal` do, makes them exchange their
91
103
  * whole documents, as providers do on reconnecting, so what was lost on the way arrives.
104
+ *
105
+ * Unless `checkLostKeys` is `false`, `advance` and `settle` also check after each delivery
106
+ * that no peer's store and document both lost a synced key that no peer deleted.
92
107
  */
93
108
  export interface Network {
94
109
  /** The seed of this network's random choices. */
@@ -118,11 +133,21 @@ export interface Network {
118
133
  /**
119
134
  * Link every pair of peers and have each exchange documents, as if every peer reconnected,
120
135
  * so that updates lost to drops and cut links arrive too.
136
+ *
137
+ * With a `dropRate`, the updates stores write in answer to that exchange, such as a key one
138
+ * derives from synced state, can be dropped too. Repeat `heal()` and `settle()` until
139
+ * `stats().dropped` stops growing before checking convergence.
121
140
  */
122
141
  heal: () => void;
123
- /** Deliver the messages due within the next `ms` of virtual time, and move time on. */
142
+ /**
143
+ * Deliver the messages due within the next `ms` of virtual time, and move time on. Rejects
144
+ * with a `LostKeyError` when a peer lost a synced key; see `checkLostKeys`.
145
+ */
124
146
  advance: (ms: number) => Promise<void>;
125
- /** Deliver every message on its way, and those they lead to, until none are left. */
147
+ /**
148
+ * Deliver every message on its way, and those they lead to, until none are left. Rejects
149
+ * with a `LostKeyError` when a peer lost a synced key; see `checkLostKeys`.
150
+ */
126
151
  settle: () => Promise<void>;
127
152
  stats: () => NetworkStats;
128
153
  /**
@@ -131,7 +156,10 @@ export interface Network {
131
156
  * document. Empty once the peers have converged.
132
157
  */
133
158
  differences: () => string[];
134
- /** Throws a `ConvergenceError` unless `differences()` is empty. */
159
+ /**
160
+ * Throws a `ConvergenceError` unless `differences()` is empty. Its message also says how many
161
+ * messages are still on their way, and how many updates were dropped since the last `heal()`.
162
+ */
135
163
  expectConverged: () => void;
136
164
  /** Disconnect every engine, stop listening to every document and drop what is on its way. */
137
165
  destroy: () => void;
@@ -199,8 +227,12 @@ export function createNetwork(options: NetworkOptions = {}): Network {
199
227
  dropRate = 0,
200
228
  reorder = false,
201
229
  maxDeliveries = 100_000,
230
+ checkLostKeys = true,
202
231
  } = options;
203
232
  const random = createRandom(seed);
233
+ const lostKeys = checkLostKeys ? createLostKeyCheck(seed) : undefined;
234
+ /** Whether `advance` or `settle` is delivering, so a document's change is not the test's. */
235
+ let delivering = false;
204
236
  const peers: Peer[] = [];
205
237
  const unsubscribers: Unsubscribe[] = [];
206
238
  /** Pairs of peers whose link is cut; every other pair is linked. */
@@ -211,6 +243,8 @@ export function createNetwork(options: NetworkOptions = {}): Network {
211
243
  let now = 0;
212
244
  let order = 0;
213
245
  const stats = { sent: 0, delivered: 0, dropped: 0, cut: 0 };
246
+ /** `stats.dropped` at the last `heal()`, or `undefined` before the first. */
247
+ let droppedAtHeal: number | undefined;
214
248
 
215
249
  const delay = (): number => {
216
250
  if (typeof latency === "number") return latency;
@@ -275,11 +309,12 @@ export function createNetwork(options: NetworkOptions = {}): Network {
275
309
  const deliver = (message: Message): void => {
276
310
  now = Math.max(now, message.at);
277
311
  stats.delivered++;
312
+ lostKeys?.changed(message.to, false);
278
313
  message.to.doc.apply(message.update);
279
314
  };
280
315
 
281
316
  /** Deliver messages due by `until`, letting each delivery's work finish before the next. */
282
- const run = async (until: number): Promise<void> => {
317
+ const deliverUntil = async (until: number): Promise<void> => {
283
318
  let deliveries = 0;
284
319
  for (;;) {
285
320
  const next = queue[0];
@@ -291,6 +326,7 @@ export function createNetwork(options: NetworkOptions = {}): Network {
291
326
  queue.shift();
292
327
  deliver(next);
293
328
  await Promise.resolve();
329
+ lostKeys?.afterDelivery(next.to, now, next.from);
294
330
  continue;
295
331
  }
296
332
  // Asynchronous stores may still send; only stop once a full turn sends nothing.
@@ -300,6 +336,18 @@ export function createNetwork(options: NetworkOptions = {}): Network {
300
336
  }
301
337
  };
302
338
 
339
+ /** `deliverUntil`, with the lost key check's checkpoints before and after. */
340
+ const run = async (until: number): Promise<void> => {
341
+ lostKeys?.beforeDeliveries(peers);
342
+ delivering = true;
343
+ try {
344
+ await deliverUntil(until);
345
+ } finally {
346
+ delivering = false;
347
+ }
348
+ for (const peer of peers) lostKeys?.afterDelivery(peer, now);
349
+ };
350
+
303
351
  const view = (peer: Peer) => ({
304
352
  store: peer.syncedState(),
305
353
  document: asJson(peer.backend.read()),
@@ -317,7 +365,7 @@ export function createNetwork(options: NetworkOptions = {}): Network {
317
365
 
318
366
  const views = peers.map(view);
319
367
  peers.forEach((peer, i) => {
320
- if (peer.engine.isConnected())
368
+ if (peer.engine.getStatus().connected)
321
369
  describe(
322
370
  `${peer.name}'s store differs from its document`,
323
371
  views[i].store,
@@ -347,11 +395,11 @@ export function createNetwork(options: NetworkOptions = {}): Network {
347
395
 
348
396
  addPeer: <S extends object>(setup: PeerSetup<S>): Peer<S> => {
349
397
  const index = peers.length;
350
- const engine = createSyncEngine(
351
- setup.backend,
352
- setup.adapter,
353
- setup.config,
354
- );
398
+ const engine = createSyncEngine({
399
+ ...setup.config,
400
+ backend: setup.backend,
401
+ adapter: setup.adapter,
402
+ });
355
403
  const peer: Peer<S> = {
356
404
  index,
357
405
  name: setup.name ?? `peer-${index}`,
@@ -362,8 +410,12 @@ export function createNetwork(options: NetworkOptions = {}): Network {
362
410
  syncedState: () => asJson(engine.getSyncedState()),
363
411
  };
364
412
  peers.push(peer);
413
+ lostKeys?.addPeer(peer);
365
414
  unsubscribers.push(
366
- setup.doc.subscribe((update) => broadcast(peer, update)),
415
+ setup.doc.subscribe((update) => {
416
+ lostKeys?.changed(peer, !delivering);
417
+ broadcast(peer, update);
418
+ }),
367
419
  );
368
420
  for (const other of peers) link(peer, other);
369
421
  if (setup.connect !== false) engine.connect();
@@ -393,6 +445,7 @@ export function createNetwork(options: NetworkOptions = {}): Network {
393
445
  },
394
446
 
395
447
  heal: () => {
448
+ droppedAtHeal = stats.dropped;
396
449
  for (const a of peers)
397
450
  for (const b of peers) if (a.index < b.index) link(a, b);
398
451
  },
@@ -413,13 +466,22 @@ export function createNetwork(options: NetworkOptions = {}): Network {
413
466
  const lines = differences();
414
467
  if (lines.length === 0) return;
415
468
  const inFlight =
416
- queue.length > 0
417
- ? `\n${queue.length} messages are still on their way: settle() delivers them.`
469
+ queue.length === 1
470
+ ? "\n1 message is still on its way: settle() delivers it."
471
+ : queue.length > 1
472
+ ? `\n${queue.length} messages are still on their way: settle() delivers them.`
473
+ : "";
474
+ const dropped = stats.dropped - (droppedAtHeal ?? 0);
475
+ const droppedHint =
476
+ dropped > 0
477
+ ? `\n${dropped === 1 ? "1 update was" : `${dropped} updates were`} dropped${
478
+ droppedAtHeal === undefined ? "" : " since the last heal()"
479
+ }: heal() and settle() until stats().dropped stops growing.`
418
480
  : "";
419
481
  throw new ConvergenceError(
420
482
  `${peers.length} peers did not converge (seed ${seed}):\n${lines
421
483
  .map((line) => `- ${line}`)
422
- .join("\n")}${inFlight}`,
484
+ .join("\n")}${inFlight}${droppedHint}`,
423
485
  lines,
424
486
  seed,
425
487
  );
package/src/randomized.ts CHANGED
@@ -14,6 +14,9 @@ export type RandomAction<S extends object = any> = (
14
14
  random: Random,
15
15
  ) => void;
16
16
 
17
+ /** Most rounds of `heal()` and `settle()` at the end of a run. */
18
+ const HEAL_ROUNDS = 10;
19
+
17
20
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
18
21
  export interface RandomizedOptions<S extends object = any> {
19
22
  /** Makes peer `index`; called once per peer as the run starts. */
@@ -26,7 +29,7 @@ export interface RandomizedOptions<S extends object = any> {
26
29
  steps?: number;
27
30
  /** What a step may do to a random peer's store, one picked at random. Defaults to `[randomEdit]`. */
28
31
  actions?: readonly RandomAction<S>[];
29
- /** Latency, drops and reordering, as for `createNetwork`. */
32
+ /** Latency, drops, reordering and the lost key check, as for `createNetwork`. */
30
33
  network?: Omit<NetworkOptions, "seed">;
31
34
  /** Chance that a step cuts or restores a random link instead of changing a store. Defaults to 0.1. */
32
35
  linkChangeRate?: number;
@@ -38,10 +41,16 @@ export interface RandomizedOptions<S extends object = any> {
38
41
  * Runs random steps against a fresh network, then heals it, settles it and checks that every
39
42
  * peer converged. Each step changes a random peer's store with a random action, or cuts or
40
43
  * restores a random link, and lets some virtual time pass, so changes cross each other on
41
- * their way.
44
+ * their way. Throughout, the network checks that no peer's store and document both lose a
45
+ * synced key that no peer deleted, unless `network.checkLostKeys` is `false`.
46
+ *
47
+ * With a `dropRate`, the updates stores write in answer to the healing exchange can be dropped
48
+ * too, so the run heals and settles again until a round drops nothing, at most 10 rounds, as a
49
+ * provider that resyncs would.
42
50
  *
43
- * Rejects with an error that names the seed and step when a step throws or the peers do not
44
- * converge; pass that `seed` to replay the same run. Give each document a fixed ID for runs
51
+ * Rejects with an error that names the seed and step when a step throws, a peer loses a synced
52
+ * key or the peers do not converge; pass that `seed` to replay the same run. When the last of
53
+ * those rounds still dropped updates, the error says so. Give each document a fixed ID for runs
45
54
  * that replay exactly, such as `doc.clientID = index + 1` for Yjs.
46
55
  */
47
56
  export async function runRandomized<S extends object>(
@@ -61,6 +70,8 @@ export async function runRandomized<S extends object>(
61
70
  const random = createRandom(seed ^ 0x5bd1e995);
62
71
  const network = createNetwork({ ...networkOptions, seed });
63
72
  let step = 0;
73
+ /** Whether every round of healing dropped updates, the last included. */
74
+ let stillDropping = false;
64
75
 
65
76
  try {
66
77
  const peers = Array.from({ length: peerCount }, (_, index) =>
@@ -79,16 +90,27 @@ export async function runRandomized<S extends object>(
79
90
  await network.advance(random.int(stepInterval + 1));
80
91
  }
81
92
 
82
- network.heal();
83
- await network.settle();
93
+ // Stores may write in answer to the documents a heal exchanges, and those updates can be
94
+ // dropped too: heal again until a round drops nothing, as a provider that resyncs would.
95
+ let dropping = true;
96
+ for (let round = 0; dropping && round < HEAL_ROUNDS; round++) {
97
+ const dropped = network.stats().dropped;
98
+ network.heal();
99
+ await network.settle();
100
+ dropping = network.stats().dropped !== dropped;
101
+ }
102
+ stillDropping = dropping;
84
103
  network.expectConverged();
85
104
  } catch (error) {
86
105
  const when =
87
106
  step < steps ? `at step ${step + 1} of ${steps}` : `after ${steps} steps`;
88
107
  const message = error instanceof Error ? error.message : String(error);
108
+ const dropping = stillDropping
109
+ ? `\nUpdates were still being dropped after ${HEAL_ROUNDS} rounds of heal() and settle(): does a store write in answer to every exchange?`
110
+ : "";
89
111
  throw Object.assign(
90
112
  new Error(
91
- `Randomized run failed ${when}; pass \`seed: ${seed}\` to replay it.\n${message}`,
113
+ `Randomized run failed ${when}; pass \`seed: ${seed}\` to replay it.\n${message}${dropping}`,
92
114
  ),
93
115
  { cause: error },
94
116
  );