@optimystic/db-p2p 0.22.0 → 0.24.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (194) hide show
  1. package/dist/src/cluster/client.d.ts +10 -0
  2. package/dist/src/cluster/client.d.ts.map +1 -1
  3. package/dist/src/cluster/client.js +30 -1
  4. package/dist/src/cluster/client.js.map +1 -1
  5. package/dist/src/cluster/cluster-repo.d.ts +39 -14
  6. package/dist/src/cluster/cluster-repo.d.ts.map +1 -1
  7. package/dist/src/cluster/cluster-repo.js +226 -119
  8. package/dist/src/cluster/cluster-repo.js.map +1 -1
  9. package/dist/src/cluster/service.d.ts +32 -1
  10. package/dist/src/cluster/service.d.ts.map +1 -1
  11. package/dist/src/cluster/service.js +43 -2
  12. package/dist/src/cluster/service.js.map +1 -1
  13. package/dist/src/cohort-topic/host.js +34 -11
  14. package/dist/src/cohort-topic/host.js.map +1 -1
  15. package/dist/src/cohort-topic/stream-util.d.ts +37 -7
  16. package/dist/src/cohort-topic/stream-util.d.ts.map +1 -1
  17. package/dist/src/cohort-topic/stream-util.js +77 -19
  18. package/dist/src/cohort-topic/stream-util.js.map +1 -1
  19. package/dist/src/dispute/dispute-service.d.ts.map +1 -1
  20. package/dist/src/dispute/dispute-service.js +9 -3
  21. package/dist/src/dispute/dispute-service.js.map +1 -1
  22. package/dist/src/index.d.ts +3 -0
  23. package/dist/src/index.d.ts.map +1 -1
  24. package/dist/src/index.js +3 -0
  25. package/dist/src/index.js.map +1 -1
  26. package/dist/src/libp2p-key-network.d.ts +88 -2
  27. package/dist/src/libp2p-key-network.d.ts.map +1 -1
  28. package/dist/src/libp2p-key-network.js +134 -28
  29. package/dist/src/libp2p-key-network.js.map +1 -1
  30. package/dist/src/libp2p-node-base.d.ts.map +1 -1
  31. package/dist/src/libp2p-node-base.js +25 -1
  32. package/dist/src/libp2p-node-base.js.map +1 -1
  33. package/dist/src/logger.d.ts +17 -1
  34. package/dist/src/logger.d.ts.map +1 -1
  35. package/dist/src/logger.js +19 -2
  36. package/dist/src/logger.js.map +1 -1
  37. package/dist/src/matchmaking/query-transport.js +3 -3
  38. package/dist/src/matchmaking/query-transport.js.map +1 -1
  39. package/dist/src/owned-block-seed.d.ts +6 -3
  40. package/dist/src/owned-block-seed.d.ts.map +1 -1
  41. package/dist/src/owned-block-seed.js +16 -3
  42. package/dist/src/owned-block-seed.js.map +1 -1
  43. package/dist/src/peer-address-book.d.ts +72 -0
  44. package/dist/src/peer-address-book.d.ts.map +1 -0
  45. package/dist/src/peer-address-book.js +123 -0
  46. package/dist/src/peer-address-book.js.map +1 -0
  47. package/dist/src/reactivity/notify-transport.d.ts +4 -4
  48. package/dist/src/reactivity/notify-transport.js +6 -6
  49. package/dist/src/reactivity/notify-transport.js.map +1 -1
  50. package/dist/src/reactivity/push-state-gossip.js +2 -2
  51. package/dist/src/reactivity/push-state-gossip.js.map +1 -1
  52. package/dist/src/reactivity/recover-transport.d.ts +6 -2
  53. package/dist/src/reactivity/recover-transport.d.ts.map +1 -1
  54. package/dist/src/reactivity/recover-transport.js +7 -3
  55. package/dist/src/reactivity/recover-transport.js.map +1 -1
  56. package/dist/src/repo/client.d.ts.map +1 -1
  57. package/dist/src/repo/client.js +11 -2
  58. package/dist/src/repo/client.js.map +1 -1
  59. package/dist/src/repo/cluster-coordinator.d.ts +30 -0
  60. package/dist/src/repo/cluster-coordinator.d.ts.map +1 -1
  61. package/dist/src/repo/cluster-coordinator.js +95 -3
  62. package/dist/src/repo/cluster-coordinator.js.map +1 -1
  63. package/dist/src/repo/coordinator-repo.d.ts +62 -9
  64. package/dist/src/repo/coordinator-repo.d.ts.map +1 -1
  65. package/dist/src/repo/coordinator-repo.js +242 -73
  66. package/dist/src/repo/coordinator-repo.js.map +1 -1
  67. package/dist/src/rn.d.ts +3 -0
  68. package/dist/src/rn.d.ts.map +1 -1
  69. package/dist/src/rn.js +3 -0
  70. package/dist/src/rn.js.map +1 -1
  71. package/dist/src/storage/cached-raw-storage.d.ts +83 -0
  72. package/dist/src/storage/cached-raw-storage.d.ts.map +1 -0
  73. package/dist/src/storage/cached-raw-storage.js +152 -0
  74. package/dist/src/storage/cached-raw-storage.js.map +1 -0
  75. package/dist/src/storage/cached-store-driver.d.ts +186 -0
  76. package/dist/src/storage/cached-store-driver.d.ts.map +1 -0
  77. package/dist/src/storage/cached-store-driver.js +775 -0
  78. package/dist/src/storage/cached-store-driver.js.map +1 -0
  79. package/dist/src/storage/i-raw-storage.d.ts +12 -5
  80. package/dist/src/storage/i-raw-storage.d.ts.map +1 -1
  81. package/dist/src/storage/shared-cache-pool.d.ts +234 -0
  82. package/dist/src/storage/shared-cache-pool.d.ts.map +1 -0
  83. package/dist/src/storage/shared-cache-pool.js +354 -0
  84. package/dist/src/storage/shared-cache-pool.js.map +1 -0
  85. package/dist/src/testing/cohort-topic-mesh-harness.d.ts +13 -6
  86. package/dist/src/testing/cohort-topic-mesh-harness.d.ts.map +1 -1
  87. package/dist/src/testing/cohort-topic-mesh-harness.js +15 -6
  88. package/dist/src/testing/cohort-topic-mesh-harness.js.map +1 -1
  89. package/dist/src/testing/raw-storage-conformance.d.ts +2 -1
  90. package/dist/src/testing/raw-storage-conformance.d.ts.map +1 -1
  91. package/dist/src/testing/raw-storage-conformance.js +35 -2
  92. package/dist/src/testing/raw-storage-conformance.js.map +1 -1
  93. package/package.json +3 -3
  94. package/readme.md +668 -668
  95. package/src/cluster/block-transfer.ts +424 -424
  96. package/src/cluster/client.ts +119 -88
  97. package/src/cluster/cluster-error.ts +64 -64
  98. package/src/cluster/cluster-policy.ts +203 -203
  99. package/src/cluster/cluster-repo.ts +242 -122
  100. package/src/cluster/cluster-size-coupling.ts +45 -45
  101. package/src/cluster/commit-cert.ts +139 -139
  102. package/src/cluster/i-transaction-state-store.ts +43 -43
  103. package/src/cluster/memory-transaction-state-store.ts +56 -56
  104. package/src/cluster/peer-key-binding.ts +37 -37
  105. package/src/cluster/persistent-transaction-state-store.ts +92 -92
  106. package/src/cluster/quorum-restore.ts +223 -223
  107. package/src/cluster/reconcile-block.ts +203 -203
  108. package/src/cluster/service.ts +293 -241
  109. package/src/cluster/supermajority-coupling.ts +37 -37
  110. package/src/cohort-topic/bootstrap-evidence-builder.ts +122 -122
  111. package/src/cohort-topic/bootstrap-evidence-verifiers.ts +132 -132
  112. package/src/cohort-topic/bootstrap-parent-reference.ts +159 -159
  113. package/src/cohort-topic/change-bridge.ts +109 -109
  114. package/src/cohort-topic/cohort-gossip-driver.ts +231 -231
  115. package/src/cohort-topic/cohort-gossip-transport.ts +84 -84
  116. package/src/cohort-topic/fret-trust-anchor.ts +153 -153
  117. package/src/cohort-topic/host.ts +42 -11
  118. package/src/cohort-topic/index.ts +13 -13
  119. package/src/cohort-topic/membership-publish-sink.ts +20 -20
  120. package/src/cohort-topic/membership-source.ts +68 -68
  121. package/src/cohort-topic/peer-codec.ts +31 -31
  122. package/src/cohort-topic/peer-sig.ts +86 -86
  123. package/src/cohort-topic/protocols.ts +71 -71
  124. package/src/cohort-topic/reactivity-membership-gate.ts +77 -77
  125. package/src/cohort-topic/size-estimator.ts +16 -16
  126. package/src/cohort-topic/stream-util.ts +79 -19
  127. package/src/cohort-topic/threshold-crypto.ts +239 -239
  128. package/src/cohort-topic/topic-router.ts +77 -77
  129. package/src/dispute/arbitrator-selection.ts +138 -138
  130. package/src/dispute/cascade.ts +524 -524
  131. package/src/dispute/dispute-service.ts +11 -5
  132. package/src/dispute/invalidation.ts +625 -625
  133. package/src/inbound-authorization.ts +190 -190
  134. package/src/index.ts +52 -49
  135. package/src/libp2p-key-network.ts +1120 -990
  136. package/src/libp2p-node-base.ts +1675 -1651
  137. package/src/libp2p-node-rn.ts +30 -30
  138. package/src/libp2p-node.ts +36 -36
  139. package/src/logger.ts +19 -2
  140. package/src/matchmaking/aggregate-counts.ts +104 -104
  141. package/src/matchmaking/index.ts +20 -20
  142. package/src/matchmaking/module.ts +363 -363
  143. package/src/matchmaking/protocols.ts +51 -51
  144. package/src/matchmaking/provider-manager.ts +95 -95
  145. package/src/matchmaking/query-handler.ts +88 -88
  146. package/src/matchmaking/query-transport.ts +3 -3
  147. package/src/matchmaking/seeker-manager.ts +64 -64
  148. package/src/matchmaking/seeker-walk-client.ts +293 -293
  149. package/src/matchmaking/traffic-validation.ts +195 -195
  150. package/src/optimystic-node.ts +36 -36
  151. package/src/owned-block-seed.ts +53 -40
  152. package/src/peer-address-book.ts +149 -0
  153. package/src/protocol-limits.ts +33 -33
  154. package/src/reactivity/forwarder-host.ts +438 -438
  155. package/src/reactivity/index.ts +19 -19
  156. package/src/reactivity/notify-transport.ts +144 -144
  157. package/src/reactivity/origination-manager.ts +192 -192
  158. package/src/reactivity/protocols.ts +61 -61
  159. package/src/reactivity/push-state-gossip.ts +291 -291
  160. package/src/reactivity/recover-transport.ts +7 -3
  161. package/src/reactivity/rotation-rereg-scheduler.ts +256 -256
  162. package/src/reactivity/subscriber-registry.ts +96 -96
  163. package/src/reactivity/subscription-manager.ts +450 -450
  164. package/src/reactivity/topic-bytes.ts +37 -37
  165. package/src/repo/client.ts +12 -2
  166. package/src/repo/cluster-coordinator.ts +99 -3
  167. package/src/repo/coordinator-repo.ts +281 -74
  168. package/src/repo/types.ts +7 -7
  169. package/src/rn.ts +39 -36
  170. package/src/rpc-deadline.ts +45 -45
  171. package/src/storage/arachnode-partition.ts +74 -74
  172. package/src/storage/cached-raw-storage.ts +180 -0
  173. package/src/storage/cached-store-driver.ts +859 -0
  174. package/src/storage/i-kv-store.ts +8 -8
  175. package/src/storage/i-raw-storage.ts +12 -5
  176. package/src/storage/kv-raw-storage.ts +135 -135
  177. package/src/storage/memory-kv-store.ts +28 -28
  178. package/src/storage/memory-storage.ts +25 -25
  179. package/src/storage/memory-store-driver.ts +157 -157
  180. package/src/storage/raw-store-codec.ts +42 -42
  181. package/src/storage/raw-store-driver.ts +80 -80
  182. package/src/storage/ring-selector.ts +317 -317
  183. package/src/storage/ring-shift-coordinator.ts +271 -271
  184. package/src/storage/shared-cache-pool.ts +452 -0
  185. package/src/storage/storage-repo.ts +1014 -1014
  186. package/src/testing/cohort-topic-mesh-harness.ts +673 -663
  187. package/src/testing/index.ts +8 -8
  188. package/src/testing/matchmaking-mesh-harness.ts +475 -475
  189. package/src/testing/raw-storage-conformance.ts +453 -417
  190. package/src/testing/reactivity-mesh-harness.ts +922 -922
  191. package/dist/src/storage/restoration-coordinator-v2.d.ts +0 -67
  192. package/dist/src/storage/restoration-coordinator-v2.d.ts.map +0 -1
  193. package/dist/src/storage/restoration-coordinator-v2.js +0 -172
  194. package/dist/src/storage/restoration-coordinator-v2.js.map +0 -1
package/readme.md CHANGED
@@ -1,668 +1,668 @@
1
- # Optimystic DB-P2P
2
-
3
- A distributed peer-to-peer database system that provides concrete implementations of the Optimystic database abstractions using pluggable storage backends and libp2p networking. This package transforms the interfaces and abstractions from `@optimystic/db-core` into a fully operational distributed database system.
4
-
5
- > **Storage is pluggable.** `db-p2p` owns the distributed layer (repo/cluster/coordination) and the storage *interfaces* (`IRawStorage`, `IBlockStorage`, `IKVStore`) plus an in-memory backend. Concrete persistent backends live in separate packages — filesystem persistence is [`@optimystic/db-p2p-storage-fs`](../db-p2p-storage-fs); see also `-storage-ns` (NativeScript/SQLite), `-storage-rn` (React Native/LevelDB), and `-storage-web` (IndexedDB).
6
-
7
- ## Overview
8
-
9
- The `@optimystic/db-core` package provides abstractions and interfaces for database operations—immutable blocks, versioned transactions, repository interfaces, and collection data structures. However, it doesn't provide concrete implementations for storage or distribution. `db-p2p` provides concrete implementations of the core abstractions using:
10
-
11
- - **Versioned block storage** (`StorageRepo` / `BlockStorage`) layered over a pluggable `IRawStorage` backend — an in-memory backend ships here; filesystem persistence lives in `@optimystic/db-p2p-storage-fs`
12
- - **libp2p networking** for decentralized peer-to-peer communication
13
- - **Distributed consensus** using 2-phase commit protocols for consistency
14
- - **Fault-tolerant coordination** with automatic recovery and data restoration
15
-
16
- The result is a fully operational distributed database system that implements the interfaces defined in the core abstractions to provide a complete distributed database system.
17
-
18
- ## Architecture Overview
19
-
20
- The system provides three distinct communication interfaces, each serving different roles in the distributed architecture:
21
-
22
- ```mermaid
23
- graph TD
24
- subgraph "External Client Process"
25
- A[Application Code]
26
- B[RepoClient]
27
- A --> B
28
- end
29
-
30
- subgraph "Coordinator Node Process<br/>(Orchestrates Transactions)"
31
- C[RepoService]
32
- D[CoordinatorRepo]
33
- E[ClusterClient]
34
- F[StorageRepo]
35
- subgraph "Local Storage"
36
- F1[BlockStorage]
37
- F2[IRawStorage backend]
38
- F3[Backend store]
39
- end
40
- C --> D
41
- D --> E
42
- D --> F
43
- F --> F1
44
- F1 --> F2
45
- F2 --> F3
46
- end
47
-
48
- subgraph "Cluster Peer 1 Process<br/>(Participates in Consensus)"
49
- G1[ClusterService]
50
- H1[ClusterMember]
51
- I1[StorageRepo]
52
- subgraph "Local Storage 1"
53
- I1A[BlockStorage]
54
- I1B[IRawStorage backend]
55
- I1C[Backend store]
56
- end
57
- G1 --> H1
58
- H1 --> I1
59
- I1 --> I1A
60
- I1A --> I1B
61
- I1B --> I1C
62
- end
63
-
64
- subgraph "Cluster Peer 2 Process<br/>(Participates in Consensus)"
65
- G2[ClusterService]
66
- H2[ClusterMember]
67
- I2[StorageRepo]
68
- subgraph "Local Storage 2"
69
- I2A[BlockStorage]
70
- I2B[IRawStorage backend]
71
- I2C[Backend store]
72
- end
73
- G2 --> H2
74
- H2 --> I2
75
- I2 --> I2A
76
- I2A --> I2B
77
- I2B --> I2C
78
- end
79
-
80
- subgraph "Cluster Peer N Process<br/>(Participates in Consensus)"
81
- G3[ClusterService]
82
- H3[ClusterMember]
83
- I3[StorageRepo]
84
- subgraph "Local Storage N"
85
- I3A[BlockStorage]
86
- I3B[IRawStorage backend]
87
- I3C[Backend store]
88
- end
89
- G3 --> H3
90
- H3 --> I3
91
- I3 --> I3A
92
- I3A --> I3B
93
- I3B --> I3C
94
- end
95
-
96
- %% Network communication
97
- B -.->|"/optimystic/&lt;network&gt;/repo/1.0.0"| C
98
- E -.->|"/optimystic/&lt;network&gt;/cluster/1.0.0"| G1
99
- E -.->|"/optimystic/&lt;network&gt;/cluster/1.0.0"| G2
100
- E -.->|"/optimystic/&lt;network&gt;/cluster/1.0.0"| G3
101
- ```
102
-
103
- ### Repo Layer: External Client Interface
104
-
105
- The repo layer provides the interface for external clients to communicate with the distributed database:
106
-
107
- - **RepoClient**: Allows external clients to connect to any coordinator node
108
- - **RepoService**: Handles incoming requests from external clients
109
- - **CoordinatorRepo**: Orchestrates distributed transactions by coordinating with cluster peers
110
- - **Network transparency**: Same `IRepo` interface whether local or distributed
111
-
112
- ### Cluster Layer: Peer-to-Peer Coordination Interface
113
-
114
- The cluster layer provides the interface for coordinators to communicate with other peers in the cluster:
115
-
116
- - **ClusterClient**: Allows coordinators to send consensus requests to cluster peers
117
- - **ClusterService**: Handles incoming consensus requests from coordinators
118
- - **ClusterMember**: Implements 2-phase commit protocol for distributed consensus
119
- - **Byzantine fault tolerance**: Ensures consistency even with peer failures
120
-
121
- ### Storage Layer: Persistent Data Implementation
122
-
123
- The storage layer implements the core database abstractions over a pluggable raw-storage backend:
124
-
125
- - **StorageRepo**: Implements the `IRepo` interface on top of `BlockStorage` — backend-agnostic
126
- - **BlockStorage**: Provides versioned block storage with conflict resolution, reading/writing through an injected `IRawStorage`
127
- - **`IRawStorage`**: The raw persistence interface. An in-memory backend ships in this package; the filesystem backend (`FileRawStorage`) lives in `@optimystic/db-p2p-storage-fs`
128
- - **Data restoration**: Pluggable restoration for missing data from network peers
129
-
130
- ## Detailed Component Architecture
131
-
132
- ### Repo Layer Components
133
-
134
- #### `RepoClient`
135
- Allows external clients to connect to coordinator nodes:
136
-
137
- ```typescript
138
- class RepoClient extends ProtocolClient implements IRepo {
139
- static create(peerId: PeerId, peerNetwork: IPeerNetwork): RepoClient
140
- // Implements same IRepo interface as local storage
141
- }
142
- ```
143
-
144
- **Key Features:**
145
- - Network-transparent database operations for external clients
146
- - Automatic protocol handling and message serialization
147
- - Error handling for network failures and timeouts
148
- - Connection pooling and optimization
149
-
150
- #### `RepoService`
151
- Handles incoming repository protocol messages from external clients:
152
-
153
- ```typescript
154
- class RepoService implements Startable {
155
- // Protocol: /optimystic/<network>/repo/1.0.0 (see docs/repo.md § Protocol id conventions)
156
- // Transport: Length-prefixed JSON over libp2p streams
157
- async start(): Promise<void>
158
- async stop(): Promise<void>
159
- }
160
- ```
161
-
162
- **Key Features:**
163
- - Registers and handles the repository protocol
164
- - Routes operations from external clients to coordinator implementation
165
- - Manages concurrent connections and stream processing
166
- - Provides comprehensive error handling and logging
167
-
168
- #### `CoordinatorRepo`
169
- Orchestrates distributed transactions by coordinating with cluster peers:
170
-
171
- ```typescript
172
- class CoordinatorRepo implements IRepo {
173
- constructor(
174
- keyNetwork: IKeyNetwork,
175
- createClusterClient: (peerId: PeerId) => ClusterClient,
176
- storageRepo: IRepo
177
- )
178
- }
179
- ```
180
-
181
- **Key Features:**
182
- - Manages distributed transactions using cluster consensus
183
- - Integrates with key network for peer discovery
184
- - Uses cluster layer to coordinate with other peers
185
- - Maintains local storage directly while coordinating remote operations
186
-
187
- ### Cluster Layer Components
188
-
189
- #### `ClusterClient`
190
- Allows coordinators to send consensus requests to cluster peers:
191
-
192
- ```typescript
193
- class ClusterClient extends ProtocolClient implements ICluster {
194
- async update(record: ClusterRecord): Promise<ClusterRecord>
195
- }
196
- ```
197
-
198
- **Key Features:**
199
- - Network communication for consensus operations
200
- - Handles cluster record updates and responses
201
- - Manages timeouts and retries for consensus requests
202
-
203
- #### `ClusterService`
204
- Handles incoming consensus requests from coordinators:
205
-
206
- ```typescript
207
- class ClusterService implements Startable {
208
- // Protocol: /optimystic/<network>/cluster/1.0.0 (built from protocolPrefix)
209
- // Handles ClusterRecord updates for consensus
210
- }
211
- ```
212
-
213
- **Key Features:**
214
- - Registers and handles the cluster consensus protocol
215
- - Routes consensus requests to local cluster member
216
- - Manages concurrent consensus operations
217
-
218
- #### `ClusterMember`
219
- Implements 2-phase commit protocol for distributed consensus:
220
-
221
- ```typescript
222
- class ClusterMember implements ICluster {
223
- async update(record: ClusterRecord): Promise<ClusterRecord>
224
- // Handles: Promise collection → Majority consensus → Commit execution
225
- }
226
- ```
227
-
228
- **Key Features:**
229
- - Complete 2-phase commit protocol implementation
230
- - Conflict detection and resolution
231
- - Cryptographic signature verification
232
- - Automatic timeout and cleanup management
233
- - Uses storage layer to execute operations when consensus is reached
234
-
235
- ### Storage Layer Components
236
-
237
- #### `StorageRepo`
238
- Implements the `IRepo` interface over `BlockStorage` — independent of any particular storage backend:
239
-
240
- ```typescript
241
- class StorageRepo implements IRepo {
242
- async get(blockGets: BlockGets): Promise<GetBlockResults>
243
- async pend(request: PendRequest): Promise<PendResult>
244
- async commit(request: CommitRequest): Promise<CommitResult>
245
- async cancel(actionRef: ActionBlocks): Promise<void>
246
- }
247
- ```
248
-
249
- **Key Features:**
250
- - Concrete implementation of core database abstractions
251
- - Orchestrates transactions across multiple blocks
252
- - Handles revision conflicts and missing transaction detection
253
- - Provides atomic commit operations with proper locking
254
-
255
- #### `BlockStorage`
256
- Provides versioned block storage with conflict resolution, reading and writing through an injected `IRawStorage` backend:
257
-
258
- ```typescript
259
- class BlockStorage implements IBlockStorage {
260
- constructor(blockId: BlockId, storage: IRawStorage, /* … */)
261
- async getBlock(rev?: number): Promise<{ block: IBlock, actionRev: ActionRev }>
262
- async savePendingAction(actionId: ActionId, transform: Transform): Promise<void>
263
- async promotePendingAction(actionId: ActionId): Promise<void>
264
- async ensureRevision(rev: number): Promise<void>
265
- }
266
- ```
267
-
268
- **Key Features:**
269
- - Maintains complete revision history for each block
270
- - Reconstructs blocks by applying transforms to base versions
271
- - Integrates with restoration callbacks for missing data
272
- - Uses latches for thread-safe concurrent access
273
-
274
- #### `IRawStorage` backends
275
- `BlockStorage` persists through the `IRawStorage` interface, so the backing store is pluggable:
276
-
277
- - **In-memory** (`MemoryRawStorage`) — ships in this package; used for tests and ephemeral nodes.
278
- - **Filesystem** (`FileRawStorage`) — lives in the separate [`@optimystic/db-p2p-storage-fs`](../db-p2p-storage-fs) package; JSON files with atomic writes, organized per block:
279
-
280
- ```typescript
281
- // Filesystem backend layout (@optimystic/db-p2p-storage-fs)
282
- {basePath}/
283
- ├── {blockId}/
284
- │ ├── meta.json # Block metadata and revision ranges
285
- │ ├── revs/{rev}.json # Revision → ActionId mappings
286
- │ ├── pend/{actionId}.json # Pending actions
287
- │ ├── actions/{actionId}.json # Committed actions
288
- │ └── blocks/{actionId}.json # Materialized blocks
289
- ```
290
-
291
- - **Other backends** — SQLite (`-storage-ns`), LevelDB (`-storage-rn`), IndexedDB (`-storage-web`), each in its own package.
292
-
293
- ## Integration Patterns
294
-
295
- ### Relationship to `@optimystic/db-core`
296
-
297
- The `db-p2p` package provides concrete implementations of the core database abstractions:
298
-
299
- ```typescript
300
- // Core provides interfaces and abstractions
301
- import { IRepo, IBlock, Transform, IBlockStorage } from '@optimystic/db-core';
302
-
303
- // P2P provides concrete implementations
304
- class StorageRepo implements IRepo { /* backend-agnostic versioned repo */ }
305
- class BlockStorage implements IBlockStorage { /* versioned block storage over IRawStorage */ }
306
- class RepoClient implements IRepo { /* network-transparent repo */ }
307
- class CoordinatorRepo implements IRepo { /* distributed consensus repo */ }
308
- ```
309
-
310
- **Implementation Points:**
311
- - **Storage Layer**: Implements the core storage interfaces over a pluggable `IRawStorage` backend (in-memory here; filesystem in `@optimystic/db-p2p-storage-fs`)
312
- - **Network Layer**: Implements peer-to-peer communication using libp2p
313
- - **Consensus Layer**: Adds distributed consensus while maintaining core interfaces
314
- - **API Compatibility**: Same interfaces as core abstractions for seamless integration
315
-
316
- ### Libp2p Integration
317
-
318
- The package integrates deeply with libp2p for networking:
319
-
320
- ```typescript
321
- // Node creation with integrated services
322
- const node = await createLibp2pNode({
323
- services: {
324
- repo: repoService({ protocolPrefix: '/optimystic/<network>' }),
325
- cluster: clusterService({ protocolPrefix: '/optimystic/<network>' })
326
- }
327
- });
328
-
329
- // Key network integration — use the node's OWN key network, do not construct a second one.
330
- // `createLibp2pNode` returns an `OptimysticNode`, which declares this handle (and
331
- // `coordinatedRepo`, `storageRepo`, `blockChangeNotifier`, `reputation`, `peerPrivateKey`).
332
- // The attached instance carries the node's resolved cluster size and its network-namespaced
333
- // protocol prefix; a separately constructed one selects a different-width cohort and has the
334
- // "does this peer serve my network?" filter switched off.
335
- const keyNetwork = node.keyNetwork;
336
- ```
337
-
338
- **Integration Features:**
339
- - **Protocol Management**: Registers custom protocols for repo and cluster operations
340
- - **Stream Handling**: Uses libp2p streams for reliable message transport
341
- - **Peer Discovery**: Integrates with libp2p DHT for peer discovery
342
- - **Security**: Leverages libp2p's cryptographic peer identity
343
-
344
- **Inbound stream authorization (optional).** The `repo`, `cluster`, `sync` and `block-transfer`
345
- protocols are open to any peer that can connect. An embedder whose database is private supplies one
346
- node-level, fail-closed predicate, consulted once per inbound stream before any decoding or execution:
347
-
348
- ```typescript
349
- const node = await createLibp2pNode({
350
- networkName: 'my-net',
351
- bootstrapNodes: [],
352
- // remotePeerId is the dialing peer's PeerId.toString(); only a literal `true` allows.
353
- // false / throw / rejection / timeout all deny and abort the stream.
354
- authorizeInboundStream: (remotePeerId, protocol) => memberSet.has(remotePeerId)
355
- });
356
- ```
357
-
358
- Omitting it is the default and leaves behavior unchanged. See
359
- [internals](../../docs/internals.md) § Inbound Stream Authorization for the full contract.
360
-
361
- **Announcing a different address than you bind (optional).** A node behind a NAT, port forward,
362
- reverse proxy or DNS front binds one address but is reachable at another. `announceAddrs` advertises
363
- the reachable address *instead of* whatever the transports report; `appendAnnounceAddrs` advertises
364
- it *in addition to* them:
365
-
366
- ```typescript
367
- const node = await createLibp2pNode({
368
- networkName: 'my-net',
369
- bootstrapNodes: ['...'],
370
- listenAddrs: ['/ip4/0.0.0.0/tcp/4001'], // what the process binds
371
- announceAddrs: ['/dns4/mynode.example.com/tcp/4001'] // what peers are told
372
- });
373
- ```
374
-
375
- `announceAddrs` replaces the advertised set **entirely** — observed and relayed addresses are
376
- dropped from it, and `appendAnnounceAddrs` is ignored while it is non-empty. Use
377
- `appendAnnounceAddrs` alone when the automatically discovered addresses are still worth advertising.
378
- Both are straight passthroughs to libp2p's `addresses.announce` / `addresses.appendAnnounce`; an
379
- empty array means "unset".
380
-
381
- ## Usage Examples
382
-
383
- ### Setting Up a Coordinator Node
384
-
385
- `createLibp2pNode` assembles the whole stack and **starts** the node: it builds the storage layer,
386
- the consensus coordinator and the key network from one resolved config, and hands them back on the
387
- returned `OptimysticNode`. Read them off the node — do not construct your own, or the two copies
388
- disagree about the same key's cohort.
389
-
390
- ```typescript
391
- import { createLibp2pNode } from '@optimystic/db-p2p';
392
- import { FileRawStorage } from '@optimystic/db-p2p-storage-fs';
393
-
394
- // Create (and start) a libp2p node. The repo and cluster protocol services are registered by the
395
- // factory itself — there is no `services` option to pass them in.
396
- const node = await createLibp2pNode({
397
- networkName: 'my-net',
398
- bootstrapNodes: ['...'],
399
- // In-memory when omitted; a raw-storage backend makes it durable
400
- storage: new FileRawStorage('/var/lib/optimystic'),
401
- });
402
-
403
- // The handles the node already owns (see `OptimysticNodeAttachments`)
404
- const storageRepo = node.storageRepo; // Local block storage
405
- const coordinatedRepo = node.coordinatedRepo; // Cluster-consensus writes and reads
406
- const keyNetwork = node.keyNetwork; // Peer/coordinator discovery
407
-
408
- // ... use them, then:
409
- await node.stop();
410
- ```
411
-
412
- `createLibp2pNode` is all-or-nothing: the node is started early and then wired, so if any wiring step
413
- fails the factory stops the node before rejecting and surfaces the original error. A rejection
414
- therefore never leaves a running node behind — which matters because the caller gets no handle to
415
- stop one. Retrying on the same port is safe.
416
-
417
- Constructing a `StorageRepo`, `CoordinatorRepo` or `Libp2pKeyPeerNetwork` yourself is only for a
418
- host assembling a stack *without* `createLibp2pNode`. A standalone `CoordinatorRepo` or
419
- `Libp2pKeyPeerNetwork` must be told its cluster size; pass `DEFAULT_CLUSTER_SIZE` (exported from
420
- this package) to match what an unconfigured node resolves to.
421
-
422
- ### External Client Operations
423
-
424
- ```typescript
425
- // External clients connect to coordinator nodes
426
- const client = RepoClient.create(coordinatorPeerId, peerNetwork);
427
-
428
- // Operations work the same as core database interfaces
429
- const blocks = await client.get({
430
- blockIds: ['block1', 'block2'],
431
- context: { rev: 10 }
432
- });
433
-
434
- // But automatically coordinate across the cluster
435
- const pendResult = await client.pend({
436
- actionId: 'tx1',
437
- transforms: { block1: [/* operations */] },
438
- rev: 11
439
- });
440
-
441
- const commitResult = await client.commit({
442
- actionId: 'tx1',
443
- blockIds: ['block1'],
444
- rev: 11
445
- });
446
- ```
447
-
448
- ### How Coordination Works
449
-
450
- ```mermaid
451
- sequenceDiagram
452
- participant Client as External Client
453
- participant RC as RepoClient
454
- participant CR as CoordinatorRepo
455
- participant CC as ClusterClient
456
- participant CS as ClusterService
457
- participant CM as ClusterMember
458
- participant SR as StorageRepo
459
-
460
- Client->>RC: pend(transaction)
461
- RC->>CR: pend(transaction)
462
- CR->>CC: consensus request
463
- CC->>CS: cluster protocol
464
- CS->>CM: update(ClusterRecord)
465
- CM->>CM: 2-phase commit
466
- CM->>SR: execute operation
467
- SR-->>CM: result
468
- CM-->>CS: consensus result
469
- CS-->>CC: response
470
- CC-->>CR: consensus complete
471
- CR-->>RC: transaction result
472
- RC-->>Client: pend result
473
- ```
474
-
475
- ## Documentation
476
-
477
- For detailed information about specific components:
478
-
479
- - **[Storage System](./docs/storage.md)**: Versioned block storage and persistence
480
- - **[Repo Interface](./docs/repo.md)**: Distributed database operations
481
- - **[Cluster Consensus](./docs/cluster.md)**: 2-phase commit and distributed consensus
482
-
483
- ## Block Restoration & Arachnode Integration
484
-
485
- The db-p2p package includes **dynamic Arachnode ring discovery** for automatic block restoration across storage tiers.
486
-
487
- ### How Block Restoration Works
488
-
489
- When a node is missing a block or revision, the restoration system:
490
-
491
- 1. **Determines storage rings** - Nodes self-select ring depth based on capacity
492
- 2. **Discovers peers via FRET** - Ring membership propagates via existing neighbor exchange
493
- 3. **Queries rings intelligently** - Tries transaction ring first, then inner storage rings
494
- 4. **Filters by partition** - Only queries peers responsible for the block's keyspace
495
-
496
- ### Ring Selection
497
-
498
- Nodes calculate their appropriate ring depth using:
499
-
500
- ```
501
- ringDepth = ceil(-log2(available_capacity / estimated_demand))
502
- ```
503
-
504
- **Examples:**
505
- - 100% coverage → Ring 0 (full keyspace)
506
- - 50% coverage → Ring 1 (2 partitions)
507
- - 1% coverage → Ring 7 (128 partitions)
508
-
509
- ### Architecture
510
-
511
- ```
512
- FRET (Pure DHT)
513
- - Generic metadata transport
514
- - Peer discovery & routing
515
-
516
- ArachnodeFretAdapter (Plugin Layer)
517
- - Arachnode-specific semantics
518
- - Ring discovery methods
519
-
520
- Restoration Components
521
- - RingSelector: Capacity-based ring selection
522
- - RestorationCoordinator: Multi-ring queries
523
- - StorageMonitor: Capacity tracking
524
- - SyncService/Client: Block request protocol
525
- ```
526
-
527
- ### Configuration
528
-
529
- ```typescript
530
- const node = await createLibp2pNode({
531
- port: 9000,
532
- networkName: 'mynet',
533
- bootstrapNodes: ['...'],
534
- clusterSize: 10, // Cluster size for peer discovery
535
- arachnode: {
536
- enableRingZulu: true // Default: enabled
537
- }
538
- });
539
- ```
540
-
541
- #### Solo / bootstrap / mobile nodes (no remote peers)
542
-
543
- A node can operate fully on its own — no bootstrap peers, no listen addresses, no
544
- connectivity to other peers. This is the common shape for a mobile app on first
545
- launch or for a local-only sanity test.
546
-
547
- ```typescript
548
- const soloNode = await createLibp2pNode({
549
- networkName: 'mynet',
550
- bootstrapNodes: [], // no bootstrap
551
- listenAddrs: [], // no incoming connections
552
- transports: [webSockets()], // transports still required
553
- clusterSize: 1,
554
- });
555
- ```
556
-
557
- In solo mode:
558
-
559
- - The node is its own coordinator. `CoordinatorRepo` short-circuits to local
560
- storage when the cluster size is ≤ 1, so `pend`/`commit` do not require
561
- cluster consensus and do not attempt to dial self.
562
- - Block restoration is skipped when the only discovered peer is self.
563
- `RestorationCoordinator` will return `undefined` immediately rather than
564
- attempt to dial self (which would hang on a node with no listen addrs).
565
- - If an operation fails for some other reason, `findCoordinator` surfaces a
566
- distinguishable `FindCoordinatorError` with `code ===
567
- FIND_COORDINATOR_ERROR_CODES.SELF_COORDINATION_EXHAUSTED` rather than the
568
- generic "all candidates excluded". The aggregate error thrown by
569
- `NetworkTransactor` preserves the ORIGINAL first-attempt cause so callers can
570
- diagnose the real problem.
571
-
572
- Once remote peers become reachable, the node transitions out of solo mode
573
- automatically as FRET discovers them. Persisted network state (HWM, FRET
574
- table) carries across restarts when an `options.persistence` is supplied.
575
-
576
- #### Custom transports (including React Native)
577
-
578
- By default, `createLibp2pNode()` uses TCP + circuit-relay transport.
579
-
580
- To use non-default transports, pass `transports` (and typically `listenAddrs`):
581
-
582
- ```typescript
583
- import { webSockets } from '@libp2p/websockets';
584
- import { circuitRelayTransport } from '@libp2p/circuit-relay-v2';
585
- import { createLibp2pNode } from '@optimystic/db-p2p';
586
-
587
- const node = await createLibp2pNode({
588
- networkName: 'mynet',
589
- bootstrapNodes: ['/dns4/relay.example.com/tcp/443/wss/p2p/12D3...'],
590
- transports: [webSockets(), circuitRelayTransport()],
591
- listenAddrs: [], // client-only; no incoming connections
592
- });
593
- ```
594
-
595
- #### React Native
596
-
597
- For React Native, import from `@optimystic/db-p2p/rn` instead of the root entrypoint.
598
- The `/rn` entrypoint does **not** import `@libp2p/tcp`, so Metro/Hermes won't try to
599
- bundle Node-only native modules. It requires `options.transports` explicitly:
600
-
601
- ```typescript
602
- import { webSockets } from '@libp2p/websockets';
603
- import { circuitRelayTransport } from '@libp2p/circuit-relay-v2';
604
- import { createLibp2pNode } from '@optimystic/db-p2p/rn';
605
-
606
- const node = await createLibp2pNode({
607
- networkName: 'mynet',
608
- bootstrapNodes: ['/dns4/relay.example.com/tcp/443/wss/p2p/12D3...'],
609
- transports: [webSockets(), circuitRelayTransport()],
610
- });
611
- ```
612
-
613
- RN environments (Hermes) need polyfills for globals that libp2p and its
614
- dependencies expect. Install these early (e.g. in your app's entry file
615
- before any other imports).
616
-
617
- **Global polyfills** (patch `globalThis` before any library code):
618
-
619
- | API | Required by | Notes |
620
- |-----|-------------|-------|
621
- | `crypto.getRandomValues()` | @noble/hashes, @libp2p/crypto | RN 0.76+ New Architecture provides natively; fallback via e.g. `react-native-get-random-values` |
622
- | `crypto.subtle.digest()` | multiformats/hashes/sha2-browser | Async SHA-256/SHA-512 — implement via @noble/hashes |
623
- | `structuredClone()` | @optimystic/db-core | JSON round-trip is sufficient |
624
- | `Promise.withResolvers()` | @libp2p/utils, @chainsafe/libp2p-yamux, it-queue | ES2024 — simple shim |
625
- | `AbortSignal.prototype.throwIfAborted()` | libp2p, @libp2p/circuit-relay-v2, it-pushable | DOM spec addition — simple shim |
626
- | Timer `.ref()` / `.unref()` | @optimystic/db-p2p, undici | Wrap Hermes numeric timer IDs in objects with no-op `.ref()`/`.unref()` methods; patch `clearTimeout`/`clearInterval` to unwrap |
627
- | `Event`, `CustomEvent`, `EventTarget` | libp2p, @libp2p/interface | Custom shim or npm `event-target-polyfill` |
628
- | `Intl.PluralRules` | moat-maker | English-only ordinal/cardinal shim is sufficient |
629
-
630
- **Node.js built-in module shims** (via Metro `extraNodeModules` or bundler aliases):
631
-
632
- | Module | Required by | Recommended shim |
633
- |--------|-------------|------------------|
634
- | `os` / `node:os` | @libp2p/utils | Custom shim: `networkInterfaces()` → `{}`, `platform()` → `Platform.OS` |
635
- | `crypto` / `node:crypto` | multiformats/hashes/sha2 | Custom shim: `createHash()` via @noble/hashes |
636
- | `stream` / `node:stream` | libp2p | `readable-stream` (npm) |
637
- | `buffer` / `node:buffer` | libp2p, multiformats | `buffer` (npm) |
638
-
639
- **Built-in (no polyfill needed):**
640
- - `TextEncoder` — built-in to Hermes
641
- - `TextDecoder` — built-in to Expo SDK 52+ (UTF-8 only)
642
- - `BigInt` — built-in to Hermes since RN 0.70
643
-
644
- See the [Sereus reference-app-rn](https://github.com/gotchoices/sereus/tree/master/packages/reference-app-rn/polyfills) for working polyfill implementations.
645
-
646
- ### Ring Transitions
647
-
648
- Nodes automatically transition between rings based on capacity thresholds:
649
- - **Move OUT** (to more granular ring) when > 85% capacity used
650
- - **Move IN** (to broader ring) when < 40% capacity used
651
-
652
- ## Related Packages
653
-
654
- - **[@optimystic/db-core](../db-core)**: Core database interfaces and local operations
655
- - **[@optimystic/db-p2p-storage-fs](../db-p2p-storage-fs)**: Filesystem `IRawStorage` backend (Node)
656
- - **[@optimystic/db-p2p-storage-ns](../db-p2p-storage-ns)**: NativeScript SQLite storage backend
657
- - **[@optimystic/db-p2p-storage-rn](../db-p2p-storage-rn)**: React Native LevelDB storage backend
658
- - **[@optimystic/db-p2p-storage-web](../db-p2p-storage-web)**: Browser IndexedDB storage backend
659
- - **[@optimystic/db-quereus](../db-quereus)**: Query engine and data access patterns
660
- - **[p2p-fret](../fret)**: DHT implementation for peer discovery
661
-
662
- ### Internal Architecture
663
-
664
- For developers and AI agents working on internals, see [Internals Guide](../../docs/internals.md) for:
665
- - Data flow diagrams (read/write/commit paths)
666
- - Mutation contracts (which functions mutate vs clone)
667
- - Key invariants and common pitfalls
668
- - Type glossary and debugging tips
1
+ # Optimystic DB-P2P
2
+
3
+ A distributed peer-to-peer database system that provides concrete implementations of the Optimystic database abstractions using pluggable storage backends and libp2p networking. This package transforms the interfaces and abstractions from `@optimystic/db-core` into a fully operational distributed database system.
4
+
5
+ > **Storage is pluggable.** `db-p2p` owns the distributed layer (repo/cluster/coordination) and the storage *interfaces* (`IRawStorage`, `IBlockStorage`, `IKVStore`) plus an in-memory backend. Concrete persistent backends live in separate packages — filesystem persistence is [`@optimystic/db-p2p-storage-fs`](../db-p2p-storage-fs); see also `-storage-ns` (NativeScript/SQLite), `-storage-rn` (React Native/LevelDB), and `-storage-web` (IndexedDB).
6
+
7
+ ## Overview
8
+
9
+ The `@optimystic/db-core` package provides abstractions and interfaces for database operations—immutable blocks, versioned transactions, repository interfaces, and collection data structures. However, it doesn't provide concrete implementations for storage or distribution. `db-p2p` provides concrete implementations of the core abstractions using:
10
+
11
+ - **Versioned block storage** (`StorageRepo` / `BlockStorage`) layered over a pluggable `IRawStorage` backend — an in-memory backend ships here; filesystem persistence lives in `@optimystic/db-p2p-storage-fs`
12
+ - **libp2p networking** for decentralized peer-to-peer communication
13
+ - **Distributed consensus** using 2-phase commit protocols for consistency
14
+ - **Fault-tolerant coordination** with automatic recovery and data restoration
15
+
16
+ The result is a fully operational distributed database system that implements the interfaces defined in the core abstractions to provide a complete distributed database system.
17
+
18
+ ## Architecture Overview
19
+
20
+ The system provides three distinct communication interfaces, each serving different roles in the distributed architecture:
21
+
22
+ ```mermaid
23
+ graph TD
24
+ subgraph "External Client Process"
25
+ A[Application Code]
26
+ B[RepoClient]
27
+ A --> B
28
+ end
29
+
30
+ subgraph "Coordinator Node Process<br/>(Orchestrates Transactions)"
31
+ C[RepoService]
32
+ D[CoordinatorRepo]
33
+ E[ClusterClient]
34
+ F[StorageRepo]
35
+ subgraph "Local Storage"
36
+ F1[BlockStorage]
37
+ F2[IRawStorage backend]
38
+ F3[Backend store]
39
+ end
40
+ C --> D
41
+ D --> E
42
+ D --> F
43
+ F --> F1
44
+ F1 --> F2
45
+ F2 --> F3
46
+ end
47
+
48
+ subgraph "Cluster Peer 1 Process<br/>(Participates in Consensus)"
49
+ G1[ClusterService]
50
+ H1[ClusterMember]
51
+ I1[StorageRepo]
52
+ subgraph "Local Storage 1"
53
+ I1A[BlockStorage]
54
+ I1B[IRawStorage backend]
55
+ I1C[Backend store]
56
+ end
57
+ G1 --> H1
58
+ H1 --> I1
59
+ I1 --> I1A
60
+ I1A --> I1B
61
+ I1B --> I1C
62
+ end
63
+
64
+ subgraph "Cluster Peer 2 Process<br/>(Participates in Consensus)"
65
+ G2[ClusterService]
66
+ H2[ClusterMember]
67
+ I2[StorageRepo]
68
+ subgraph "Local Storage 2"
69
+ I2A[BlockStorage]
70
+ I2B[IRawStorage backend]
71
+ I2C[Backend store]
72
+ end
73
+ G2 --> H2
74
+ H2 --> I2
75
+ I2 --> I2A
76
+ I2A --> I2B
77
+ I2B --> I2C
78
+ end
79
+
80
+ subgraph "Cluster Peer N Process<br/>(Participates in Consensus)"
81
+ G3[ClusterService]
82
+ H3[ClusterMember]
83
+ I3[StorageRepo]
84
+ subgraph "Local Storage N"
85
+ I3A[BlockStorage]
86
+ I3B[IRawStorage backend]
87
+ I3C[Backend store]
88
+ end
89
+ G3 --> H3
90
+ H3 --> I3
91
+ I3 --> I3A
92
+ I3A --> I3B
93
+ I3B --> I3C
94
+ end
95
+
96
+ %% Network communication
97
+ B -.->|"/optimystic/&lt;network&gt;/repo/1.0.0"| C
98
+ E -.->|"/optimystic/&lt;network&gt;/cluster/1.0.0"| G1
99
+ E -.->|"/optimystic/&lt;network&gt;/cluster/1.0.0"| G2
100
+ E -.->|"/optimystic/&lt;network&gt;/cluster/1.0.0"| G3
101
+ ```
102
+
103
+ ### Repo Layer: External Client Interface
104
+
105
+ The repo layer provides the interface for external clients to communicate with the distributed database:
106
+
107
+ - **RepoClient**: Allows external clients to connect to any coordinator node
108
+ - **RepoService**: Handles incoming requests from external clients
109
+ - **CoordinatorRepo**: Orchestrates distributed transactions by coordinating with cluster peers
110
+ - **Network transparency**: Same `IRepo` interface whether local or distributed
111
+
112
+ ### Cluster Layer: Peer-to-Peer Coordination Interface
113
+
114
+ The cluster layer provides the interface for coordinators to communicate with other peers in the cluster:
115
+
116
+ - **ClusterClient**: Allows coordinators to send consensus requests to cluster peers
117
+ - **ClusterService**: Handles incoming consensus requests from coordinators
118
+ - **ClusterMember**: Implements 2-phase commit protocol for distributed consensus
119
+ - **Byzantine fault tolerance**: Ensures consistency even with peer failures
120
+
121
+ ### Storage Layer: Persistent Data Implementation
122
+
123
+ The storage layer implements the core database abstractions over a pluggable raw-storage backend:
124
+
125
+ - **StorageRepo**: Implements the `IRepo` interface on top of `BlockStorage` — backend-agnostic
126
+ - **BlockStorage**: Provides versioned block storage with conflict resolution, reading/writing through an injected `IRawStorage`
127
+ - **`IRawStorage`**: The raw persistence interface. An in-memory backend ships in this package; the filesystem backend (`FileRawStorage`) lives in `@optimystic/db-p2p-storage-fs`
128
+ - **Data restoration**: Pluggable restoration for missing data from network peers
129
+
130
+ ## Detailed Component Architecture
131
+
132
+ ### Repo Layer Components
133
+
134
+ #### `RepoClient`
135
+ Allows external clients to connect to coordinator nodes:
136
+
137
+ ```typescript
138
+ class RepoClient extends ProtocolClient implements IRepo {
139
+ static create(peerId: PeerId, peerNetwork: IPeerNetwork): RepoClient
140
+ // Implements same IRepo interface as local storage
141
+ }
142
+ ```
143
+
144
+ **Key Features:**
145
+ - Network-transparent database operations for external clients
146
+ - Automatic protocol handling and message serialization
147
+ - Error handling for network failures and timeouts
148
+ - Connection pooling and optimization
149
+
150
+ #### `RepoService`
151
+ Handles incoming repository protocol messages from external clients:
152
+
153
+ ```typescript
154
+ class RepoService implements Startable {
155
+ // Protocol: /optimystic/<network>/repo/1.0.0 (see docs/repo.md § Protocol id conventions)
156
+ // Transport: Length-prefixed JSON over libp2p streams
157
+ async start(): Promise<void>
158
+ async stop(): Promise<void>
159
+ }
160
+ ```
161
+
162
+ **Key Features:**
163
+ - Registers and handles the repository protocol
164
+ - Routes operations from external clients to coordinator implementation
165
+ - Manages concurrent connections and stream processing
166
+ - Provides comprehensive error handling and logging
167
+
168
+ #### `CoordinatorRepo`
169
+ Orchestrates distributed transactions by coordinating with cluster peers:
170
+
171
+ ```typescript
172
+ class CoordinatorRepo implements IRepo {
173
+ constructor(
174
+ keyNetwork: IKeyNetwork,
175
+ createClusterClient: (peerId: PeerId) => ClusterClient,
176
+ storageRepo: IRepo
177
+ )
178
+ }
179
+ ```
180
+
181
+ **Key Features:**
182
+ - Manages distributed transactions using cluster consensus
183
+ - Integrates with key network for peer discovery
184
+ - Uses cluster layer to coordinate with other peers
185
+ - Maintains local storage directly while coordinating remote operations
186
+
187
+ ### Cluster Layer Components
188
+
189
+ #### `ClusterClient`
190
+ Allows coordinators to send consensus requests to cluster peers:
191
+
192
+ ```typescript
193
+ class ClusterClient extends ProtocolClient implements ICluster {
194
+ async update(record: ClusterRecord): Promise<ClusterRecord>
195
+ }
196
+ ```
197
+
198
+ **Key Features:**
199
+ - Network communication for consensus operations
200
+ - Handles cluster record updates and responses
201
+ - Manages timeouts and retries for consensus requests
202
+
203
+ #### `ClusterService`
204
+ Handles incoming consensus requests from coordinators:
205
+
206
+ ```typescript
207
+ class ClusterService implements Startable {
208
+ // Protocol: /optimystic/<network>/cluster/1.0.0 (built from protocolPrefix)
209
+ // Handles ClusterRecord updates for consensus
210
+ }
211
+ ```
212
+
213
+ **Key Features:**
214
+ - Registers and handles the cluster consensus protocol
215
+ - Routes consensus requests to local cluster member
216
+ - Manages concurrent consensus operations
217
+
218
+ #### `ClusterMember`
219
+ Implements 2-phase commit protocol for distributed consensus:
220
+
221
+ ```typescript
222
+ class ClusterMember implements ICluster {
223
+ async update(record: ClusterRecord): Promise<ClusterRecord>
224
+ // Handles: Promise collection → Majority consensus → Commit execution
225
+ }
226
+ ```
227
+
228
+ **Key Features:**
229
+ - Complete 2-phase commit protocol implementation
230
+ - Conflict detection and resolution
231
+ - Cryptographic signature verification
232
+ - Automatic timeout and cleanup management
233
+ - Uses storage layer to execute operations when consensus is reached
234
+
235
+ ### Storage Layer Components
236
+
237
+ #### `StorageRepo`
238
+ Implements the `IRepo` interface over `BlockStorage` — independent of any particular storage backend:
239
+
240
+ ```typescript
241
+ class StorageRepo implements IRepo {
242
+ async get(blockGets: BlockGets): Promise<GetBlockResults>
243
+ async pend(request: PendRequest): Promise<PendResult>
244
+ async commit(request: CommitRequest): Promise<CommitResult>
245
+ async cancel(actionRef: ActionBlocks): Promise<void>
246
+ }
247
+ ```
248
+
249
+ **Key Features:**
250
+ - Concrete implementation of core database abstractions
251
+ - Orchestrates transactions across multiple blocks
252
+ - Handles revision conflicts and missing transaction detection
253
+ - Provides atomic commit operations with proper locking
254
+
255
+ #### `BlockStorage`
256
+ Provides versioned block storage with conflict resolution, reading and writing through an injected `IRawStorage` backend:
257
+
258
+ ```typescript
259
+ class BlockStorage implements IBlockStorage {
260
+ constructor(blockId: BlockId, storage: IRawStorage, /* … */)
261
+ async getBlock(rev?: number): Promise<{ block: IBlock, actionRev: ActionRev }>
262
+ async savePendingAction(actionId: ActionId, transform: Transform): Promise<void>
263
+ async promotePendingAction(actionId: ActionId): Promise<void>
264
+ async ensureRevision(rev: number): Promise<void>
265
+ }
266
+ ```
267
+
268
+ **Key Features:**
269
+ - Maintains complete revision history for each block
270
+ - Reconstructs blocks by applying transforms to base versions
271
+ - Integrates with restoration callbacks for missing data
272
+ - Uses latches for thread-safe concurrent access
273
+
274
+ #### `IRawStorage` backends
275
+ `BlockStorage` persists through the `IRawStorage` interface, so the backing store is pluggable:
276
+
277
+ - **In-memory** (`MemoryRawStorage`) — ships in this package; used for tests and ephemeral nodes.
278
+ - **Filesystem** (`FileRawStorage`) — lives in the separate [`@optimystic/db-p2p-storage-fs`](../db-p2p-storage-fs) package; JSON files with atomic writes, organized per block:
279
+
280
+ ```typescript
281
+ // Filesystem backend layout (@optimystic/db-p2p-storage-fs)
282
+ {basePath}/
283
+ ├── {blockId}/
284
+ │ ├── meta.json # Block metadata and revision ranges
285
+ │ ├── revs/{rev}.json # Revision → ActionId mappings
286
+ │ ├── pend/{actionId}.json # Pending actions
287
+ │ ├── actions/{actionId}.json # Committed actions
288
+ │ └── blocks/{actionId}.json # Materialized blocks
289
+ ```
290
+
291
+ - **Other backends** — SQLite (`-storage-ns`), LevelDB (`-storage-rn`), IndexedDB (`-storage-web`), each in its own package.
292
+
293
+ ## Integration Patterns
294
+
295
+ ### Relationship to `@optimystic/db-core`
296
+
297
+ The `db-p2p` package provides concrete implementations of the core database abstractions:
298
+
299
+ ```typescript
300
+ // Core provides interfaces and abstractions
301
+ import { IRepo, IBlock, Transform, IBlockStorage } from '@optimystic/db-core';
302
+
303
+ // P2P provides concrete implementations
304
+ class StorageRepo implements IRepo { /* backend-agnostic versioned repo */ }
305
+ class BlockStorage implements IBlockStorage { /* versioned block storage over IRawStorage */ }
306
+ class RepoClient implements IRepo { /* network-transparent repo */ }
307
+ class CoordinatorRepo implements IRepo { /* distributed consensus repo */ }
308
+ ```
309
+
310
+ **Implementation Points:**
311
+ - **Storage Layer**: Implements the core storage interfaces over a pluggable `IRawStorage` backend (in-memory here; filesystem in `@optimystic/db-p2p-storage-fs`)
312
+ - **Network Layer**: Implements peer-to-peer communication using libp2p
313
+ - **Consensus Layer**: Adds distributed consensus while maintaining core interfaces
314
+ - **API Compatibility**: Same interfaces as core abstractions for seamless integration
315
+
316
+ ### Libp2p Integration
317
+
318
+ The package integrates deeply with libp2p for networking:
319
+
320
+ ```typescript
321
+ // Node creation with integrated services
322
+ const node = await createLibp2pNode({
323
+ services: {
324
+ repo: repoService({ protocolPrefix: '/optimystic/<network>' }),
325
+ cluster: clusterService({ protocolPrefix: '/optimystic/<network>' })
326
+ }
327
+ });
328
+
329
+ // Key network integration — use the node's OWN key network, do not construct a second one.
330
+ // `createLibp2pNode` returns an `OptimysticNode`, which declares this handle (and
331
+ // `coordinatedRepo`, `storageRepo`, `blockChangeNotifier`, `reputation`, `peerPrivateKey`).
332
+ // The attached instance carries the node's resolved cluster size and its network-namespaced
333
+ // protocol prefix; a separately constructed one selects a different-width cohort and has the
334
+ // "does this peer serve my network?" filter switched off.
335
+ const keyNetwork = node.keyNetwork;
336
+ ```
337
+
338
+ **Integration Features:**
339
+ - **Protocol Management**: Registers custom protocols for repo and cluster operations
340
+ - **Stream Handling**: Uses libp2p streams for reliable message transport
341
+ - **Peer Discovery**: Integrates with libp2p DHT for peer discovery
342
+ - **Security**: Leverages libp2p's cryptographic peer identity
343
+
344
+ **Inbound stream authorization (optional).** The `repo`, `cluster`, `sync` and `block-transfer`
345
+ protocols are open to any peer that can connect. An embedder whose database is private supplies one
346
+ node-level, fail-closed predicate, consulted once per inbound stream before any decoding or execution:
347
+
348
+ ```typescript
349
+ const node = await createLibp2pNode({
350
+ networkName: 'my-net',
351
+ bootstrapNodes: [],
352
+ // remotePeerId is the dialing peer's PeerId.toString(); only a literal `true` allows.
353
+ // false / throw / rejection / timeout all deny and abort the stream.
354
+ authorizeInboundStream: (remotePeerId, protocol) => memberSet.has(remotePeerId)
355
+ });
356
+ ```
357
+
358
+ Omitting it is the default and leaves behavior unchanged. See
359
+ [internals](../../docs/internals.md) § Inbound Stream Authorization for the full contract.
360
+
361
+ **Announcing a different address than you bind (optional).** A node behind a NAT, port forward,
362
+ reverse proxy or DNS front binds one address but is reachable at another. `announceAddrs` advertises
363
+ the reachable address *instead of* whatever the transports report; `appendAnnounceAddrs` advertises
364
+ it *in addition to* them:
365
+
366
+ ```typescript
367
+ const node = await createLibp2pNode({
368
+ networkName: 'my-net',
369
+ bootstrapNodes: ['...'],
370
+ listenAddrs: ['/ip4/0.0.0.0/tcp/4001'], // what the process binds
371
+ announceAddrs: ['/dns4/mynode.example.com/tcp/4001'] // what peers are told
372
+ });
373
+ ```
374
+
375
+ `announceAddrs` replaces the advertised set **entirely** — observed and relayed addresses are
376
+ dropped from it, and `appendAnnounceAddrs` is ignored while it is non-empty. Use
377
+ `appendAnnounceAddrs` alone when the automatically discovered addresses are still worth advertising.
378
+ Both are straight passthroughs to libp2p's `addresses.announce` / `addresses.appendAnnounce`; an
379
+ empty array means "unset".
380
+
381
+ ## Usage Examples
382
+
383
+ ### Setting Up a Coordinator Node
384
+
385
+ `createLibp2pNode` assembles the whole stack and **starts** the node: it builds the storage layer,
386
+ the consensus coordinator and the key network from one resolved config, and hands them back on the
387
+ returned `OptimysticNode`. Read them off the node — do not construct your own, or the two copies
388
+ disagree about the same key's cohort.
389
+
390
+ ```typescript
391
+ import { createLibp2pNode } from '@optimystic/db-p2p';
392
+ import { FileRawStorage } from '@optimystic/db-p2p-storage-fs';
393
+
394
+ // Create (and start) a libp2p node. The repo and cluster protocol services are registered by the
395
+ // factory itself — there is no `services` option to pass them in.
396
+ const node = await createLibp2pNode({
397
+ networkName: 'my-net',
398
+ bootstrapNodes: ['...'],
399
+ // In-memory when omitted; a raw-storage backend makes it durable
400
+ storage: new FileRawStorage('/var/lib/optimystic'),
401
+ });
402
+
403
+ // The handles the node already owns (see `OptimysticNodeAttachments`)
404
+ const storageRepo = node.storageRepo; // Local block storage
405
+ const coordinatedRepo = node.coordinatedRepo; // Cluster-consensus writes and reads
406
+ const keyNetwork = node.keyNetwork; // Peer/coordinator discovery
407
+
408
+ // ... use them, then:
409
+ await node.stop();
410
+ ```
411
+
412
+ `createLibp2pNode` is all-or-nothing: the node is started early and then wired, so if any wiring step
413
+ fails the factory stops the node before rejecting and surfaces the original error. A rejection
414
+ therefore never leaves a running node behind — which matters because the caller gets no handle to
415
+ stop one. Retrying on the same port is safe.
416
+
417
+ Constructing a `StorageRepo`, `CoordinatorRepo` or `Libp2pKeyPeerNetwork` yourself is only for a
418
+ host assembling a stack *without* `createLibp2pNode`. A standalone `CoordinatorRepo` or
419
+ `Libp2pKeyPeerNetwork` must be told its cluster size; pass `DEFAULT_CLUSTER_SIZE` (exported from
420
+ this package) to match what an unconfigured node resolves to.
421
+
422
+ ### External Client Operations
423
+
424
+ ```typescript
425
+ // External clients connect to coordinator nodes
426
+ const client = RepoClient.create(coordinatorPeerId, peerNetwork);
427
+
428
+ // Operations work the same as core database interfaces
429
+ const blocks = await client.get({
430
+ blockIds: ['block1', 'block2'],
431
+ context: { rev: 10 }
432
+ });
433
+
434
+ // But automatically coordinate across the cluster
435
+ const pendResult = await client.pend({
436
+ actionId: 'tx1',
437
+ transforms: { block1: [/* operations */] },
438
+ rev: 11
439
+ });
440
+
441
+ const commitResult = await client.commit({
442
+ actionId: 'tx1',
443
+ blockIds: ['block1'],
444
+ rev: 11
445
+ });
446
+ ```
447
+
448
+ ### How Coordination Works
449
+
450
+ ```mermaid
451
+ sequenceDiagram
452
+ participant Client as External Client
453
+ participant RC as RepoClient
454
+ participant CR as CoordinatorRepo
455
+ participant CC as ClusterClient
456
+ participant CS as ClusterService
457
+ participant CM as ClusterMember
458
+ participant SR as StorageRepo
459
+
460
+ Client->>RC: pend(transaction)
461
+ RC->>CR: pend(transaction)
462
+ CR->>CC: consensus request
463
+ CC->>CS: cluster protocol
464
+ CS->>CM: update(ClusterRecord)
465
+ CM->>CM: 2-phase commit
466
+ CM->>SR: execute operation
467
+ SR-->>CM: result
468
+ CM-->>CS: consensus result
469
+ CS-->>CC: response
470
+ CC-->>CR: consensus complete
471
+ CR-->>RC: transaction result
472
+ RC-->>Client: pend result
473
+ ```
474
+
475
+ ## Documentation
476
+
477
+ For detailed information about specific components:
478
+
479
+ - **[Storage System](./docs/storage.md)**: Versioned block storage and persistence
480
+ - **[Repo Interface](./docs/repo.md)**: Distributed database operations
481
+ - **[Cluster Consensus](./docs/cluster.md)**: 2-phase commit and distributed consensus
482
+
483
+ ## Block Restoration & Arachnode Integration
484
+
485
+ The db-p2p package includes **dynamic Arachnode ring discovery** for automatic block restoration across storage tiers.
486
+
487
+ ### How Block Restoration Works
488
+
489
+ When a node is missing a block or revision, the restoration system:
490
+
491
+ 1. **Determines storage rings** - Nodes self-select ring depth based on capacity
492
+ 2. **Discovers peers via FRET** - Ring membership propagates via existing neighbor exchange
493
+ 3. **Queries rings intelligently** - Tries transaction ring first, then inner storage rings
494
+ 4. **Filters by partition** - Only queries peers responsible for the block's keyspace
495
+
496
+ ### Ring Selection
497
+
498
+ Nodes calculate their appropriate ring depth using:
499
+
500
+ ```
501
+ ringDepth = ceil(-log2(available_capacity / estimated_demand))
502
+ ```
503
+
504
+ **Examples:**
505
+ - 100% coverage → Ring 0 (full keyspace)
506
+ - 50% coverage → Ring 1 (2 partitions)
507
+ - 1% coverage → Ring 7 (128 partitions)
508
+
509
+ ### Architecture
510
+
511
+ ```
512
+ FRET (Pure DHT)
513
+ - Generic metadata transport
514
+ - Peer discovery & routing
515
+
516
+ ArachnodeFretAdapter (Plugin Layer)
517
+ - Arachnode-specific semantics
518
+ - Ring discovery methods
519
+
520
+ Restoration Components
521
+ - RingSelector: Capacity-based ring selection
522
+ - RestorationCoordinator: Multi-ring queries
523
+ - StorageMonitor: Capacity tracking
524
+ - SyncService/Client: Block request protocol
525
+ ```
526
+
527
+ ### Configuration
528
+
529
+ ```typescript
530
+ const node = await createLibp2pNode({
531
+ port: 9000,
532
+ networkName: 'mynet',
533
+ bootstrapNodes: ['...'],
534
+ clusterSize: 10, // Cluster size for peer discovery
535
+ arachnode: {
536
+ enableRingZulu: true // Default: enabled
537
+ }
538
+ });
539
+ ```
540
+
541
+ #### Solo / bootstrap / mobile nodes (no remote peers)
542
+
543
+ A node can operate fully on its own — no bootstrap peers, no listen addresses, no
544
+ connectivity to other peers. This is the common shape for a mobile app on first
545
+ launch or for a local-only sanity test.
546
+
547
+ ```typescript
548
+ const soloNode = await createLibp2pNode({
549
+ networkName: 'mynet',
550
+ bootstrapNodes: [], // no bootstrap
551
+ listenAddrs: [], // no incoming connections
552
+ transports: [webSockets()], // transports still required
553
+ clusterSize: 1,
554
+ });
555
+ ```
556
+
557
+ In solo mode:
558
+
559
+ - The node is its own coordinator. `CoordinatorRepo` short-circuits to local
560
+ storage when the cluster size is ≤ 1, so `pend`/`commit` do not require
561
+ cluster consensus and do not attempt to dial self.
562
+ - Block restoration is skipped when the only discovered peer is self.
563
+ `RestorationCoordinator` will return `undefined` immediately rather than
564
+ attempt to dial self (which would hang on a node with no listen addrs).
565
+ - If an operation fails for some other reason, `findCoordinator` surfaces a
566
+ distinguishable `FindCoordinatorError` with `code ===
567
+ FIND_COORDINATOR_ERROR_CODES.SELF_COORDINATION_EXHAUSTED` rather than the
568
+ generic "all candidates excluded". The aggregate error thrown by
569
+ `NetworkTransactor` preserves the ORIGINAL first-attempt cause so callers can
570
+ diagnose the real problem.
571
+
572
+ Once remote peers become reachable, the node transitions out of solo mode
573
+ automatically as FRET discovers them. Persisted network state (HWM, FRET
574
+ table) carries across restarts when an `options.persistence` is supplied.
575
+
576
+ #### Custom transports (including React Native)
577
+
578
+ By default, `createLibp2pNode()` uses TCP + circuit-relay transport.
579
+
580
+ To use non-default transports, pass `transports` (and typically `listenAddrs`):
581
+
582
+ ```typescript
583
+ import { webSockets } from '@libp2p/websockets';
584
+ import { circuitRelayTransport } from '@libp2p/circuit-relay-v2';
585
+ import { createLibp2pNode } from '@optimystic/db-p2p';
586
+
587
+ const node = await createLibp2pNode({
588
+ networkName: 'mynet',
589
+ bootstrapNodes: ['/dns4/relay.example.com/tcp/443/wss/p2p/12D3...'],
590
+ transports: [webSockets(), circuitRelayTransport()],
591
+ listenAddrs: [], // client-only; no incoming connections
592
+ });
593
+ ```
594
+
595
+ #### React Native
596
+
597
+ For React Native, import from `@optimystic/db-p2p/rn` instead of the root entrypoint.
598
+ The `/rn` entrypoint does **not** import `@libp2p/tcp`, so Metro/Hermes won't try to
599
+ bundle Node-only native modules. It requires `options.transports` explicitly:
600
+
601
+ ```typescript
602
+ import { webSockets } from '@libp2p/websockets';
603
+ import { circuitRelayTransport } from '@libp2p/circuit-relay-v2';
604
+ import { createLibp2pNode } from '@optimystic/db-p2p/rn';
605
+
606
+ const node = await createLibp2pNode({
607
+ networkName: 'mynet',
608
+ bootstrapNodes: ['/dns4/relay.example.com/tcp/443/wss/p2p/12D3...'],
609
+ transports: [webSockets(), circuitRelayTransport()],
610
+ });
611
+ ```
612
+
613
+ RN environments (Hermes) need polyfills for globals that libp2p and its
614
+ dependencies expect. Install these early (e.g. in your app's entry file
615
+ before any other imports).
616
+
617
+ **Global polyfills** (patch `globalThis` before any library code):
618
+
619
+ | API | Required by | Notes |
620
+ |-----|-------------|-------|
621
+ | `crypto.getRandomValues()` | @noble/hashes, @libp2p/crypto | RN 0.76+ New Architecture provides natively; fallback via e.g. `react-native-get-random-values` |
622
+ | `crypto.subtle.digest()` | multiformats/hashes/sha2-browser | Async SHA-256/SHA-512 — implement via @noble/hashes |
623
+ | `structuredClone()` | @optimystic/db-core | JSON round-trip is sufficient |
624
+ | `Promise.withResolvers()` | @libp2p/utils, @chainsafe/libp2p-yamux, it-queue | ES2024 — simple shim |
625
+ | `AbortSignal.prototype.throwIfAborted()` | libp2p, @libp2p/circuit-relay-v2, it-pushable | DOM spec addition — simple shim |
626
+ | Timer `.ref()` / `.unref()` | @optimystic/db-p2p, undici | Wrap Hermes numeric timer IDs in objects with no-op `.ref()`/`.unref()` methods; patch `clearTimeout`/`clearInterval` to unwrap |
627
+ | `Event`, `CustomEvent`, `EventTarget` | libp2p, @libp2p/interface | Custom shim or npm `event-target-polyfill` |
628
+ | `Intl.PluralRules` | moat-maker | English-only ordinal/cardinal shim is sufficient |
629
+
630
+ **Node.js built-in module shims** (via Metro `extraNodeModules` or bundler aliases):
631
+
632
+ | Module | Required by | Recommended shim |
633
+ |--------|-------------|------------------|
634
+ | `os` / `node:os` | @libp2p/utils | Custom shim: `networkInterfaces()` → `{}`, `platform()` → `Platform.OS` |
635
+ | `crypto` / `node:crypto` | multiformats/hashes/sha2 | Custom shim: `createHash()` via @noble/hashes |
636
+ | `stream` / `node:stream` | libp2p | `readable-stream` (npm) |
637
+ | `buffer` / `node:buffer` | libp2p, multiformats | `buffer` (npm) |
638
+
639
+ **Built-in (no polyfill needed):**
640
+ - `TextEncoder` — built-in to Hermes
641
+ - `TextDecoder` — built-in to Expo SDK 52+ (UTF-8 only)
642
+ - `BigInt` — built-in to Hermes since RN 0.70
643
+
644
+ See the [Sereus reference-app-rn](https://github.com/gotchoices/sereus/tree/master/packages/reference-app-rn/polyfills) for working polyfill implementations.
645
+
646
+ ### Ring Transitions
647
+
648
+ Nodes automatically transition between rings based on capacity thresholds:
649
+ - **Move OUT** (to more granular ring) when > 85% capacity used
650
+ - **Move IN** (to broader ring) when < 40% capacity used
651
+
652
+ ## Related Packages
653
+
654
+ - **[@optimystic/db-core](../db-core)**: Core database interfaces and local operations
655
+ - **[@optimystic/db-p2p-storage-fs](../db-p2p-storage-fs)**: Filesystem `IRawStorage` backend (Node)
656
+ - **[@optimystic/db-p2p-storage-ns](../db-p2p-storage-ns)**: NativeScript SQLite storage backend
657
+ - **[@optimystic/db-p2p-storage-rn](../db-p2p-storage-rn)**: React Native LevelDB storage backend
658
+ - **[@optimystic/db-p2p-storage-web](../db-p2p-storage-web)**: Browser IndexedDB storage backend
659
+ - **[@optimystic/db-quereus](../db-quereus)**: Query engine and data access patterns
660
+ - **[p2p-fret](../fret)**: DHT implementation for peer discovery
661
+
662
+ ### Internal Architecture
663
+
664
+ For developers and AI agents working on internals, see [Internals Guide](../../docs/internals.md) for:
665
+ - Data flow diagrams (read/write/commit paths)
666
+ - Mutation contracts (which functions mutate vs clone)
667
+ - Key invariants and common pitfalls
668
+ - Type glossary and debugging tips