loro-repo 0.21.2 → 0.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/README.md +570 -2
  2. package/dist/doc-updates.cjs +73 -0
  3. package/dist/doc-updates.cjs.map +1 -0
  4. package/dist/doc-updates.js +49 -0
  5. package/dist/doc-updates.js.map +1 -0
  6. package/dist/effect/streams.cjs +463 -0
  7. package/dist/effect/streams.cjs.map +1 -0
  8. package/dist/effect/streams.d.cts +52 -0
  9. package/dist/effect/streams.d.ts +52 -0
  10. package/dist/effect/streams.js +444 -0
  11. package/dist/effect/streams.js.map +1 -0
  12. package/dist/effect.cjs +3567 -0
  13. package/dist/effect.cjs.map +1 -0
  14. package/dist/effect.d.cts +120 -0
  15. package/dist/effect.d.ts +120 -0
  16. package/dist/effect.js +3542 -0
  17. package/dist/effect.js.map +1 -0
  18. package/dist/flock-persistence-journal.cjs +2 -1
  19. package/dist/flock-persistence-journal.cjs.map +1 -1
  20. package/dist/flock-persistence-journal.js +2 -1
  21. package/dist/flock-persistence-journal.js.map +1 -1
  22. package/dist/flock-snapshot.cjs +0 -190
  23. package/dist/flock-snapshot.cjs.map +1 -1
  24. package/dist/flock-snapshot.js +1 -166
  25. package/dist/flock-snapshot.js.map +1 -1
  26. package/dist/index.cjs +14 -13
  27. package/dist/index.cjs.map +1 -1
  28. package/dist/index.d.cts +2 -3
  29. package/dist/index.d.ts +2 -3
  30. package/dist/index.js +2 -2
  31. package/dist/index.js.map +1 -1
  32. package/dist/meta-snapshot.cjs +350 -0
  33. package/dist/meta-snapshot.cjs.map +1 -0
  34. package/dist/meta-snapshot.js +283 -0
  35. package/dist/meta-snapshot.js.map +1 -0
  36. package/dist/replica-checkpoints.cjs.map +1 -1
  37. package/dist/replica-checkpoints.js.map +1 -1
  38. package/dist/repo.d.cts +391 -0
  39. package/dist/repo.d.ts +391 -0
  40. package/dist/room-state.cjs +195 -0
  41. package/dist/room-state.cjs.map +1 -0
  42. package/dist/room-state.js +122 -0
  43. package/dist/room-state.js.map +1 -0
  44. package/dist/storage/filesystem.cjs +207 -13
  45. package/dist/storage/filesystem.cjs.map +1 -1
  46. package/dist/storage/filesystem.d.cts +28 -1
  47. package/dist/storage/filesystem.d.ts +28 -1
  48. package/dist/storage/filesystem.js +204 -10
  49. package/dist/storage/filesystem.js.map +1 -1
  50. package/dist/storage/indexeddb.cjs +192 -21
  51. package/dist/storage/indexeddb.cjs.map +1 -1
  52. package/dist/storage/indexeddb.d.cts +43 -2
  53. package/dist/storage/indexeddb.d.ts +43 -2
  54. package/dist/storage/indexeddb.js +180 -9
  55. package/dist/storage/indexeddb.js.map +1 -1
  56. package/dist/storage/sqlite.cjs +169 -22
  57. package/dist/storage/sqlite.cjs.map +1 -1
  58. package/dist/storage/sqlite.d.cts +14 -2
  59. package/dist/storage/sqlite.d.ts +14 -2
  60. package/dist/storage/sqlite.js +168 -21
  61. package/dist/storage/sqlite.js.map +1 -1
  62. package/dist/streams-error-context.cjs +306 -0
  63. package/dist/streams-error-context.cjs.map +1 -0
  64. package/dist/streams-error-context.js +275 -0
  65. package/dist/streams-error-context.js.map +1 -0
  66. package/dist/streams.d.cts +525 -0
  67. package/dist/streams.d.ts +525 -0
  68. package/dist/transport/broadcast-channel.d.cts +1 -1
  69. package/dist/transport/broadcast-channel.d.ts +1 -1
  70. package/dist/transport/streams.cjs +22 -253
  71. package/dist/transport/streams.cjs.map +1 -1
  72. package/dist/transport/streams.d.cts +3 -503
  73. package/dist/transport/streams.d.ts +3 -503
  74. package/dist/transport/streams.js +9 -241
  75. package/dist/transport/streams.js.map +1 -1
  76. package/dist/transport/websocket.cjs +2 -2
  77. package/dist/transport/websocket.d.cts +1 -1
  78. package/dist/transport/websocket.d.ts +2 -2
  79. package/dist/transport/websocket.js +1 -1
  80. package/dist/types.d.cts +97 -14
  81. package/dist/types.d.ts +97 -14
  82. package/dist/writer-recovery.cjs +237 -0
  83. package/dist/writer-recovery.cjs.map +1 -0
  84. package/dist/writer-recovery.js +203 -0
  85. package/dist/writer-recovery.js.map +1 -0
  86. package/package.json +27 -7
  87. package/dist/types.cjs +0 -138
  88. package/dist/types.cjs.map +0 -1
  89. package/dist/types.js +0 -102
  90. package/dist/types.js.map +0 -1
package/README.md CHANGED
@@ -209,6 +209,29 @@ adaptor deliberately does **not** reopen it on its own, because a reopen usually
209
209
  fails for the same reason and silent self-healing makes it harder for an
210
210
  application to fail closed. That recovery is the caller's decision.
211
211
 
212
+ ## Optional Streams upload transport
213
+
214
+ `StreamsTransportAdapter.uploadTransport` forwards the SDK-owned durable-upload
215
+ capability to Meta, Loro document and named Flock sessions, including replacements.
216
+ The installed `StreamsCrdt` must advertise `uploadTransportVersion === 1`;
217
+ otherwise construction fails before I/O instead of silently ignoring the option.
218
+ The public type is derived from the installed SDK and is `never` on legacy peers.
219
+
220
+ The optional peer range retains ordinary fetch support for 0.16.x and 0.18.x
221
+ and admits the 0.19.x feature release. The development dependency and committed
222
+ lock select official SDK 0.19.0, whose package depends on Client 0.12.0. CI therefore
223
+ executes the capability-specific loopback cases against installed release packages.
224
+ A version number alone does not certify platform upload progress: callers still
225
+ provide an actual-transmission capability and the SDK marker is checked at runtime.
226
+
227
+ The caller owns the platform transport. Only observations from actual transmission
228
+ may renew the SDK idle deadline; body reads and buffering are not progress.
229
+ Authentication, retry identity, frozen bytes, cancellation, response interpretation
230
+ and finite hard caps remain SDK-owned. Repo does not create a progress adapter or
231
+ change timeout defaults. Native Repo `/effect` remains a separate capability port
232
+ without a concrete Streams adapter; this option belongs to the Promise adapter.
233
+ See the [integration decision](../../docs/plans/2026-10-15-streams-upload-transport.md).
234
+
212
235
  ## Per-room Streams payload protection
213
236
 
214
237
  `StreamsTransportAdapter` can select one streams-crdt provider config for each
@@ -290,10 +313,21 @@ CRDT import and before `beforeRemoteCursorSave`; the adapter keeps the existing
290
313
  "persist CRDT state, then advance cursor" boundary. Each room owns an independent
291
314
  instance, so one room's failure does not stop other rooms.
292
315
 
293
- This API requires `@loro-dev/streams-crdt@^0.15.0`, specifically its common
316
+ This API requires `@loro-dev/streams-crdt@^0.16.2`, specifically its common
294
317
  protected-payload boundary and `payloadProtectionRequired` guard. No 0.12.x
295
318
  encryption compatibility shim is retained; the adapter passes the published
296
- 0.15 opaque `streamUrl` transport target directly.
319
+ opaque `streamUrl` transport target directly.
320
+
321
+ A trusted snapshot-admission host can import
322
+ `encodeStreamsRoomAdditionalData(namespace, room, envelopeAdditionalData)` from
323
+ `loro-repo/transport/streams` to reconstruct the exact AAD supplied to the provider.
324
+ Use authenticated application namespace and logical room, not uploaded identity
325
+ claims or routing URLs. This helper changes no wire bytes and does not perform
326
+ signature verification or grant publication permission.
327
+
328
+ Structured errors retain a whitelisted `payloadProtectionReason`, including
329
+ `missing_read_key` versus `decrypt_failed`, on `RepoTransportError`. Provider
330
+ messages and causes remain sanitized. See the [compatibility decision](../../docs/plans/2026-10-10-streams-protection-host.md).
297
331
 
298
332
  ## Replica-bound Streams checkpoints (IndexedDB)
299
333
 
@@ -477,6 +511,7 @@ flock_docs (flock_doc_id PK, snapshot, updated_at, snapshot_digest)
477
511
  flock_doc_updates (id, flock_doc_id, update_data, created_at)
478
512
  meta_snapshot (id=1, snapshot, updated_at)
479
513
  meta_updates (id, update_data, created_at)
514
+ doc_cursor_urls (doc_id + stream_url PK; device-local Doc cursor ownership)
480
515
  remote_cursors (stream_url PK, next_offset, server_lower_bound_version, updated_at_ms)
481
516
  replica_checkpoints (resource_key PK, generation, cursors, updated_at_ms)
482
517
  ```
@@ -612,3 +647,536 @@ Streams transport errors expose `streamsCode`, `streamsContext`, `failureKind`
612
647
  and the upstream `retryable` policy on `RepoTransportError`. Read the
613
648
  [consumer contract](../../docs/streams-error-context.md) for trusted fields,
614
649
  single/multiple transport reports, join/wait errors and diagnostic logging.
650
+
651
+ ## Native Effect repository
652
+
653
+ Install the optional `effect` peer to use `loro-repo/effect`. Existing Promise
654
+ consumers do not need Effect and keep their original writer and runtime behavior.
655
+ The entry manages scoped replicas, metadata catalogs, incremental/automatic
656
+ persistence, room leases and native transport composition. Attach native transport
657
+ capabilities after construction; Promise `TransportAdapter` options are rejected.
658
+ Native WebSocket, BroadcastChannel and MessagePort transports are exported by this entry;
659
+ the original Promise transports retain their own implementations.
660
+
661
+ The native ESM entry uses the Flock SDK's WebAssembly import. On Node 18, run
662
+ with `--experimental-wasm-modules`; bundlers must support WASM modules. The
663
+ existing CommonJS entry retains its normal Node 18 loading behavior.
664
+
665
+ ```ts
666
+ import { Effect } from 'effect';
667
+ import * as Repo from 'loro-repo/effect';
668
+
669
+ const program = Effect.scoped(
670
+ Effect.gen(function* () {
671
+ const repo = yield* Repo.make<{ title: string }>({ storageAdapter });
672
+ yield* repo.withCreateDoc('note:one', { title: 'One' }, (doc) =>
673
+ Effect.sync(() => {
674
+ doc.getText('content').insert(0, 'Hello');
675
+ })
676
+ );
677
+ // Last reference automatically commits pending changes, persists and evicts
678
+ // memory. Stored content remains available for the next acquisition.
679
+ yield* repo.withOpenDoc('note:one', (doc) =>
680
+ Effect.sync(() => console.log(doc.getText('content').toString()))
681
+ );
682
+ })
683
+ );
684
+ ```
685
+
686
+ `createDoc(id, metadata)` registers a new local identity and returns a scoped lease
687
+ for application initialization. It rejects existing identities, tombstones and
688
+ unregistered stored/edited bodies with `RepoDocAlreadyExists`. Even an empty created
689
+ body is saved on flush/release. Concurrent creation is serialized within one repo;
690
+ use fresh application-generated ids rather than treating this as distributed uniqueness.
691
+ `withCreateDoc` runs initialization in a short Scope and saves before success.
692
+ Schema, first-message and other domain initialization remain application-owned;
693
+ callback failure does not roll back authored CRDT writes or identity registration.
694
+
695
+ `openDoc(id)` returns a scoped `DocLease | undefined`: an unregistered or deleted
696
+ local identity returns `undefined` without loading a body or acquiring a lease.
697
+ Absence from this local registry does not prove remote NotFound.
698
+ A registered identity may open a new local replica awaiting synchronization.
699
+ `DocLease.source` is `storage` or `new-local`, describing that shared replica's
700
+ materialization origin rather than logical creation, current contents or sync readiness.
701
+ `withOpenDoc` returns `A | undefined`, skips the callback on absence and provides
702
+ the same short-scope and typed-release boundary. Storage and close failures stay
703
+ in the typed error channel; they are never converted into `undefined`.
704
+
705
+ `acquireDoc` and `withDoc` retain low-level load-or-create-local-replica semantics
706
+ for synchronization. Storage errors propagate as `RepoIOError`, never as empty
707
+ fallbacks. Loading has no automatic read retry or acquisition deadline; a later
708
+ acquisition recovers the failed cache, and callers may bound their own wait.
709
+
710
+ All acquisitions require the consumer's Scope and share one replica.
711
+ Do not retain the returned CRDT object beyond that Scope. `withDoc` and
712
+ `withFlockDoc` provide short scoped uses without manual release or unload.
713
+ `withDoc`/`withFlockDoc` report failed last-release saves through the typed
714
+ `RepoReleaseFailed` error, including full Cause and `retry`. Raw acquisition
715
+ Scope finalizers and repo owner teardown follow Effect's defect semantics; their
716
+ Cause retains that same recovery object.
717
+
718
+ `flush` persists at least the watermarks captured at call time, including required
719
+ existence metadata. Later writes do not hold that barrier open. It does not wait
720
+ for remote acknowledgement or make all resources one atomic transaction.
721
+
722
+ Storage is owned by the repo. Close stops new acquisition and waits for acquiring
723
+ Scopes to end before retiring replicas or closing storage. A bounded close failure
724
+ keeps live replicas subscribed; their remaining edits are saved when borrower
725
+ Scopes end. End those Scopes before retrying close. Calling close inside a Scope
726
+ that still owns its own lease reaches the deadline; short `withDoc`/`withFlockDoc`
727
+ scopes avoid this ordering problem.
728
+
729
+ The release observer deadline defaults to 30 seconds (`releaseTimeoutMs`); saves make
730
+ at most three attempts per observation (`persistenceAttempts`). A failed release
731
+ keeps its replica and original I/O, fences replacement, and is retried on subsequent
732
+ acquisition/flush. Scope shutdown failures retain complete Cause and a recovery
733
+ Effect in `RepoReleaseFailed.retry`; timeout never implies raw I/O has stopped.
734
+ Without a storage adapter, released content is ephemeral and there is no durability
735
+ guarantee. The library does not create a permanent in-memory backup after RC reaches zero.
736
+
737
+ Catalog consumers use `listDoc`, active-only `listDocIds`, `getDocMetaMany`,
738
+ `deleteDoc` and `restoreDoc`. `observeDocs(query, filter, capacity)` acquires a
739
+ snapshot, watermark and subscribed event Stream at one synchronous boundary. An
740
+ overflow fails the Stream with `RepoIOError` and requires a new snapshot. A query
741
+ also filters document events in the tail; its limit bounds only the initial list.
742
+ Observers terminate on repo close. Deletion writes a durable identity tombstone;
743
+ RC eviction only releases memory.
744
+
745
+ Doc/Flock leases expose `persist` and a replica-bound `checkpoint` capability;
746
+ `persistMeta` covers metadata. These barriers require storage and reject stale
747
+ leases. Checkpoint saves persist the covered replica first. No checkpoint store
748
+ means typed capability failure, never a successful no-op. Doc cursor persistence
749
+ is injected with `documentRemoteCursorStore`; meta/Flock stores come from the
750
+ storage adapter's hydrated replica.
751
+
752
+ Remote document input must use `lease.importUpdates(bytes)`, which returns
753
+ `{ complete }` and retains raw input that cannot yet be represented by a snapshot.
754
+ Ordered import is not atomic: accepted input is retained even if a later input
755
+ is rejected. A cursor cannot advance while imports have missing dependencies. Direct raw
756
+ `doc.import` bypasses this tracking and is for applications that independently
757
+ own import durability. SQLite, IndexedDB and filesystem hydration keep incomplete
758
+ update journals instead of compacting them away.
759
+
760
+ Automatic saves default to `autoPersistMs: 100`; set zero to disable them. A
761
+ native per-resource dirty queue coalesces changes with bounded exponential retry
762
+ backoff and reports errors through `persistenceFailures`. Each save has the same
763
+ finite attempt policy. Subsequent doc saves use incremental updates; meta/Flock
764
+ changed keys use journals with full-file fallback and optional backend compaction
765
+ at `compactionByteThreshold` (default 100 KiB; zero disables it). Doc hydration
766
+ consolidates complete journals. No automatic task keeps an idle document leased.
767
+
768
+ ```ts
769
+ const network = yield* Repo.makeMemoryNetwork(); // scoped, in-process CRDT transport
770
+ const repo = yield* Repo.make({ storageAdapter });
771
+ const transport = yield* repo.attachTransport(network.transport('local'));
772
+ const room = yield* repo.acquireRoom({ kind: 'doc', id: 'note:one' });
773
+ const report = yield* room.sync('local');
774
+ // A protocol barrier reports confirmed, skipped (no route/offline), or partial.
775
+ const initialReady = room.awaitSynced('local');
776
+ const roomStates = room.status('local');
777
+ const connectionStates = transport.status;
778
+ ```
779
+
780
+ `acquireRoom` requires Scope and shares one internal replica lease across room
781
+ borrowers. Targets are `{kind:'meta'}`, `{kind:'doc',id}` and `{kind:'flock',id}`.
782
+ Without a routed transport a durable room stays pending. `syncOnce(target, id?)`
783
+ uses a temporary room Scope. All attempted transports failing produces typed
784
+ `RepoSyncFailed` with the full report; partial success returns a partial report.
785
+ An empty/offline report never counts as remote confirmation. `awaitSynced` observes
786
+ initial room readiness; use `sync` for a new protocol barrier.
787
+
788
+ `attachTransport` requires Scope; ending that Scope closes memberships and
789
+ disconnects only that transport. Its lease exposes `refreshRoutes`, `reconnect`,
790
+ `rooms`, `status` and `statusSnapshot`. Room status is stable per target/transport,
791
+ independent from connection health. Select the required local/cloud transport
792
+ explicitly. Last room release closes memberships before ending its replica Scope.
793
+ Late joins and failed closes fence replacement and retain recovery ownership.
794
+ Repo close drains external room borrowers before the internal replica borrowers.
795
+
796
+ `acquireEphemeralRoom(transportId, id)` requires an attached native ephemeral
797
+ capability and Scope. It exposes bounded message delivery and publishing without
798
+ storage; there is no fallback when the capability is missing. `makeMemoryNetwork`
799
+ is an in-process implementation using real Doc/Flock merges and scoped ephemeral
800
+ queues; it provides no production-network, auth, encryption or persistence protocol.
801
+ The production [native Streams adapter](#native-effect-streams) is a separate entry.
802
+ The generic native MessagePort/duplex bridge is available; mapping Lody-specific IPC DTOs remains downstream. The new entry does not wrap
803
+ the old SyncRunner or Streams Promise workflow and does not migrate Lody itself.
804
+
805
+ Use `Repo.layer(MyRepoService, options)` for application DI with a metadata-specific
806
+ `Context.Service`. It uses the same scoped factory, not a separate runtime.
807
+ The Promise API retains its existing explicit document lifetime, sync and
808
+ persistence behavior. Its convergence onto a native facade is a later unit.
809
+ See [ownership and verification](../../docs/plans/2026-10-13-effect-replica-lifecycle.md)
810
+ and [Lody consumer coverage and adapter dependencies](../../docs/plans/2026-10-13-effect-lody-coverage.md).
811
+
812
+
813
+ ### Catalog, maintenance and full synchronization
814
+
815
+ Flock identities use the existing `ef`/`efx` metadata format. `getFlockDoc`,
816
+ `listFlockDoc`, `deleteFlockDoc`, `restoreFlockDoc` and `extendFlockDoc(id, expiresAtMs)`
817
+ cover discovery, tombstones and expiration. Renewal only extends an absolute TTL;
818
+ it never restores a deleted identity. `observeFlockDocs` captures snapshot/watermark/tail
819
+ atomically and includes expiration changes. Custom filters without `kinds` retain
820
+ the Flock-only default. `watch` exposes the scoped catalog tail.
821
+
822
+ `purgeDoc`, `purgeFlockDoc` and `purgeExpiredFlockDocs(nowMs?)` physically delete idle
823
+ resources. Live leases return `RepoResourceBusy` before room/cursor/body changes.
824
+ There is no forced revocation of raw documents. Once admitted, purge fences new
825
+ acquisitions through actual I/O settlement, clears remote cursors before deleting
826
+ the body, and removes catalog fields last. Failure retains `RepoReleaseFailed.retry`;
827
+ canceling a waiter does not abandon deletion. Expired sweeps report purged, busy and
828
+ failed ids separately. Storage without delete capability fails explicitly. Doc cursor
829
+ URLs are recorded in the storage adapter's device-local `docCursorIndex` before
830
+ checkpoint access/advancement. They never enter replicated metadata. SQLite, IndexedDB
831
+ and filesystem provide this index; custom adapters must implement it to use durable
832
+ Doc checkpoints. Filesystem cursor entries are immutable; duplicate publication
833
+ contention is accepted only after identical target bytes are verified and unpublished
834
+ temporary files are removed. Purge retains the index until cursors, body and catalog cleanup succeed,
835
+ including after restart without an attached transport. Doc purge also clears its
836
+ metadata, frontier (`f`), link (`ld`) and timestamp (`ts`) fields.
837
+
838
+ `openDetachedDoc` / `openDetachedFlockDoc` return independent current local snapshots,
839
+ or `undefined` for a missing body. Cached unpersisted edits and unresolved Doc input
840
+ are preserved; editing a detached value does not modify the managed replica. These
841
+ reads do not register identities or create missing bodies.
842
+
843
+ `sync({ scope: 'full', concurrency: 4, timeoutMs: 30000 })` first synchronizes metadata,
844
+ then discovers active Doc and unexpired Flock bodies from the updated catalog. The
845
+ default scope remains metadata-only. Explicit `docIds` / `flockDocIds` form an allow-list;
846
+ `includeDeleted` enables tombstone recovery for discovery. `requireTransports` makes
847
+ missing, unrouted or unconfirmed required transports a failure, including offline results. Each resource carries its complete Exit/report,
848
+ and the aggregate distinguishes `confirmed`, `skipped`, `partial` and `failed`.
849
+ Timeout bounds observations; shared transport/persistence work remains owned. Bodies
850
+ loaded for the sweep follow their normal scoped RC lifetime. `transportIds`, `health`,
851
+ `removeTransport`, `reconnect` and `hasStorage` provide Repo-level operational access;
852
+ reconnect reports an Exit per transport instead of discarding sibling failures.
853
+
854
+ ### Native local and WebSocket transports
855
+
856
+ ```ts
857
+ const transport = yield* Repo.makeWebSocketTransport({ url: 'wss://example.test/sync' });
858
+ const attachment = yield* repo.attachTransport(transport);
859
+ const room = yield* repo.acquireRoom({ kind: 'doc', id: 'note' });
860
+ yield* room.sync(attachment.id);
861
+ ```
862
+
863
+ `makeWebSocketTransport` uses the existing `loro-protocol` server wire format. Each
864
+ membership owns its socket, frame reader and ACK receipts. It supports Doc/Flock/meta,
865
+ fragmentation, ephemeral bytes, native auth effects, explicit reconnect and automatic
866
+ Effect Clock backoff (customize `reconnectDelays`; `[]` disables automatic retries).
867
+ Read-only synchronization requests a fresh server version and waits for actual imports;
868
+ local edits without write permission fail. Reconnect preserves the issued replica.
869
+ Connection health is separate from protocol readiness. Close also owns a reconnect
870
+ that is still opening its socket. The bundled SimpleServer 0.6.2 can reject large
871
+ bootstrap frames; native fragmentation does not remove that server-side limit.
872
+
873
+ The common protocol sends incremental Doc/Flock updates after a version baseline,
874
+ including version-based join catch-up; it does not export a full snapshot on every edit.
875
+ Reconnect attempts every owned room and presence stream, aggregates failures afterwards,
876
+ and preserves healthy sibling recovery.
877
+
878
+ The WebSocket adapter currently opens **one socket per active room**, including metadata
879
+ and ephemeral rooms; it does not multiplex rooms over a shared socket. Full-sync concurrency
880
+ bounds simultaneous acquisitions, while long-held rooms retain their individual sockets.
881
+ Account for this connection cost when choosing a large-room deployment. Shared-socket
882
+ multiplexing and browser/server connection-scale acceptance remain separate work.
883
+
884
+ `makeBroadcastChannelTransport({ namespace })` uses a separate `loro-repo-effect`
885
+ namespace and acknowledged peer frames, rather than claiming readiness after a timer.
886
+ It confirms application at a reachable peer; it cannot certify all browser tabs or
887
+ provide server durability. Its wire format intentionally differs from the old Promise
888
+ Broadcast adapter; migrate communicating peers together. No peer means no confirmation.
889
+
890
+ `makeMessagePortTransport({ id, openPort })` accepts owned browser/worker MessagePorts.
891
+ `makeMessageTransport({ id, open })` accepts a native duplex byte port. Ports only do
892
+ I/O; the common Effect worker owns joins, validation, controlled imports, ACK barriers
893
+ and release. Port implementations must retain raw I/O until settlement and make close
894
+ recoverable. Applications must authorize and isolate these links. Lody's existing
895
+ workspace/peer routing, chunking and IPC DTOs still require their own downstream adapter;
896
+ this package does not pretend that a generic byte port speaks that protocol.
897
+
898
+
899
+ The scoped EphemeralStore bridge coalesces pending local updates by key, retaining
900
+ latest values and deletion tombstones while a publish waits for ACK. A transient publish
901
+ failure fails its observers and appears on `failures`; the next update or `sync` can retry
902
+ without reacquiring the store. Concurrent `sync` observers are bounded; closing the lease
903
+ settles every waiting observer with `RepoClosed` before stopping workers. Remote-input
904
+ failure is terminal for that input stream and remains explicit.
905
+
906
+ ## WorkspaceData and host capabilities
907
+
908
+ Configure authority explicitly. `Repo.make({ authority: { mode: 'local' } })`
909
+ means local data is authoritative. Remote mode names allowed server routes:
910
+ `{ mode: 'remote', transports: ['streams'] }`. An omitted policy remains
911
+ `unconfigured`; losing a server route never turns a replica into local authority.
912
+ `NativeTransport.authority` is a trusted composition setting (`server` or `peer`),
913
+ not a claim accepted from incoming frames. Undeclared custom transports are peers. Custom ports must provide structured
914
+ state for ongoing authority readiness; a status-only port remains unproven.
915
+ `requireTransports` accepts only actual confirmed server routes. Sync reports
916
+ also label authority as `local`, `server`, `peer` or `unconfirmed`; a confirmed
917
+ peer exchange is not a server receipt or an authoritative absence proof.
918
+
919
+ `room.state(id)` / `stateSnapshot(id)` expose connection, sticky baseline,
920
+ current catch-up, uploadPending and epoch separately. Epoch advances once after a
921
+ rejoin catches up, never for a local edit or upload. Replacing a transport id clears
922
+ baseline/current evidence while preserving the room's monotone epoch counter. A historical baseline alone
923
+ cannot prove current absence. `room.authority` combines the configured authoritative
924
+ routes; `status` and `awaitSynced` remain compatibility projections.
925
+
926
+ ```ts
927
+ const receipt = yield* room.beginUpload('streams', { route: 'wait', timeoutMs: 5_000 });
928
+ // The caller's lease Scope may now end. Repo retains an independent room lease.
929
+ const result = yield* receipt.await; // outcome: acked | not-attempted | not-confirmed
930
+ // Optional observation of cleanup; ACK delivery does not wait for downlink teardown.
931
+ yield* receipt.released;
932
+ // room.awaitUploaded(id, options) is the begin + await convenience operation.
933
+ ```
934
+
935
+ Upload requests capture the local version at execution and accept only server
936
+ ACKs. `require` returns without waiting for a missing route; `wait` waits within
937
+ its finite deadline. Later edits do not extend a captured target. Cancellation
938
+ detaches observers; producers and actual adapter I/O remain owned. Streams needs
939
+ the SDK's **native `uploadThrough` / `readStatus` / `readEpoch` capability**,
940
+ released in CRDT 0.19.0 with Client 0.12.0 after [SDK #425](https://github.com/loro-dev/loro-streams/pull/425).
941
+ The development lock selects those official packages, and normal `pnpm check`
942
+ enables their upload acceptance. Older SDKs fail upload requests explicitly,
943
+ with no sync-as-upload fallback. For a focused run use
944
+ `LORO_TEST_UPLOAD_SDK=1 pnpm --filter loro-repo exec vitest run test/effect-streams.test.ts`.
945
+ Keep one shared Loro WASM peer instance in the consuming graph.
946
+
947
+ `repo.acquirePresence(transportId, id, { ttlMs, heartbeatMs, releaseTimeoutMs, clockSkewMs })`
948
+ provides scoped `set`, `remove`, `snapshot`, `changes`, `sync` and recoverable `close`.
949
+ Cleanup ownership is registered before its workers start. Cancelling acquisition
950
+ releases initialized workers and the room, while an unfinished raw publish remains
951
+ owned behind the bounded close/retry receipt.
952
+ Each lease has a unique wire identity; application/user identity belongs in its
953
+ JSON values. The separate `owned-presence-v2-*` channel publishes whole owned
954
+ snapshots in one batch, resends on rejoin, renews by default every `ttlMs / 3`
955
+ using Effect Clock, and actively publishes a final clear. TTL starts at local receipt,
956
+ so sender clock offsets do not shorten it. Lease/sequence fences reject duplicate
957
+ and cleared generations. `clockSkewMs` defaults to five minutes: timestamps only
958
+ reject frames outside that generous age/skew bound. Retain generation fences for
959
+ the receive TTL plus another TTL and twice this skew allowance before collection;
960
+ frames old enough to outlive that retention are rejected on arrival. This is bounded
961
+ replay protection under the configured clock-skew assumption, not protection against
962
+ arbitrary clock changes or an authenticated malicious sender. Unknown leases trigger one rate-limited snapshot
963
+ response; failed publishes retry without another edit. Set `heartbeatMs: 0` when
964
+ expiry without renewal is intended. A crash uses TTL. Keys/bytes/generations are
965
+ bounded; input failures are explicit through `failures` and `sync`. A close timeout
966
+ retains publish and room ownership in `RepoReleaseFailed.retry`. The raw
967
+ `acquireEphemeralStore` wire/API is unchanged.
968
+
969
+ MessagePort/message transports support `permission: target => 'read' | 'write'`;
970
+ updates and fragment headers from a read-only peer are rejected with nonzero ACKs
971
+ before import. Use isolated ports for peers with different trust/permissions.
972
+ BroadcastChannel uses a distinct versioned namespace and addressed peer/session
973
+ envelopes; each peer owns its version cursor, bounded ACK window and retry. Slow
974
+ peers coalesce pending edits and rejoin from their versions after an ACK deadline,
975
+ while healthy peers continue. Discovery heartbeats run every 5 seconds; three minutes
976
+ without any traffic retires a peer and frees its slot. This accommodates minute-level
977
+ background timer throttling; a suspended tab can still be retired after that window
978
+ and will rejoin on resume. Presence fan-out isolates peer failures. MessagePort and
979
+ WebSocket ACK deadlines start after the final fragment is handed to the I/O port:
980
+ 5 seconds plus 10 milliseconds per KiB for queued transfer/import (about 210 seconds
981
+ for 20 MiB). `ackTimeoutMs` overrides the base. Receipts retain their room-owned
982
+ deadline even if observers cancel. Very slow routes can raise the base explicitly. Its optional permission callback applies to all
983
+ peers of each room. Broadcast IDs are routing labels, not authenticated identities.
984
+ Outbound Message/Broadcast permissions start unknown. A peer's JoinRequest does
985
+ not grant upload permission; only its JoinResponseOk does. Read-only peers can
986
+ bootstrap, refresh and reconnect without sending forbidden updates; local edits
987
+ outside the remote version fail synchronization rather than uploading.
988
+ WebSocket still uses one socket per room.
989
+
990
+ `observeFlock(lease, prefix, capacity)` (also `lease.observe`) retains that replica
991
+ in the observation Scope and returns an atomic row snapshot plus ordered commit
992
+ batches, including tombstones. Its overflow and catalog overflow fail with
993
+ `RepoResnapshotRequired`, rather than silently dropping changes or requiring
994
+ message-string matching.
995
+
996
+ For a single host writer use
997
+ `Repo.make({ storageAdapter, storageOwnership: 'exclusive' })`. Filesystem,
998
+ IndexedDB and SQLite adapters expose exclusive leases and complete raw inventory.
999
+ Shared mode remains the default; all updated writers still participate in the
1000
+ owner fence. IndexedDB transactions include the owner record store, so writes to
1001
+ otherwise independent stores share its ordering; ordinary Doc durability stays
1002
+ unchanged. A current exclusive owner fences competing new writes in updated adapters. A filesystem ownership handoff
1003
+ requires shared/older writers to be stopped and drained first; exclusive
1004
+ participants acquire the lease before any data access. SQLite/IndexedDB check the
1005
+ owner within write transactions. Filesystem uses an atomic lock directory with
1006
+ a process-identified owner token. Locks are **never stolen by timeout**.
1007
+ Filesystem publishes a prepared, nonempty lock directory atomically. Immutable owner
1008
+ filenames keep concurrent recovery from unlinking a replacement owner; old empty
1009
+ acquisition/recovery directories can be cleared explicitly after draining old writers.
1010
+ `await storage.recoverStaleWriter()` proves retirement using stable machine identity,
1011
+ boot identity and process start identity, without hostname checks. Linux reads boot UUID
1012
+ and `/proc` start ticks (within the same PID namespace); macOS uses hardware UUID,
1013
+ `kern.bootsessionuuid` and UTC/C-locale `ps` start time through asynchronous subprocesses.
1014
+ Different boots on the same machine or a reused PID with a different start are safe;
1015
+ Windows uses the registry MachineGuid, CIM LastBootUpTime and the process StartTime
1016
+ (FILETIME) through bounded asynchronous PowerShell probes. Successful acquisition
1017
+ identity probes are cached for this process, including shared in-flight probes;
1018
+ subsequent leases reuse the identity with a fresh fencing nonce. This avoids
1019
+ repeated PowerShell startup within a process, but the first process acquisition
1020
+ still pays the probe cost. Failed/unavailable identities are not cached, so later
1021
+ acquisitions may recover when OS access becomes available. Recovery checks of
1022
+ another process still refresh that process's start identity. Linux also tries
1023
+ `/var/lib/dbus/machine-id` when `/etc/machine-id` is absent. If any OS identity probe
1024
+ fails (minimal containers, denied subprocesses, missing tools), acquisition still
1025
+ succeeds with an opaque random fencing token. That token cannot prove retirement:
1026
+ a later recovery reports `RepoWriterLocked.recoverable === undefined` and requires
1027
+ an operator-controlled handoff after all writers stop; it is never stolen.
1028
+ EPERM alone is no proof. Live, unknown, foreign-machine/namespace and legacy opaque
1029
+ owners are refused. Other platforms retain fencing but cannot automatically prove
1030
+ stale ownership. `Repo.make` surfaces `RepoWriterLocked` as a typed failure:
1031
+ `recoverable` is true (proven stale), false (live), or undefined (unproven).
1032
+ SQLite supports `await SqliteRepoStore.open({ path, recoverStaleWriter: true })`
1033
+ on Linux/macOS/Windows. Its synchronous constructor/instance recovery remains available on
1034
+ Linux; use async `open` on macOS/Windows. IndexedDB exclusive leases require `navigator.locks`
1035
+ (or an explicit `lockManager` provider). Shared writes hold the same named lock in
1036
+ shared mode through transaction completion, clearing a stale browser token only
1037
+ while that lock proves there is no exclusive owner. No exclusive reopen is needed
1038
+ after a tab closes or crashes. Exclusive acquisition queues behind shared transactions,
1039
+ with an abortable `writerLockTimeoutMs` budget (default 5000 ms); expiry cancels the
1040
+ request and reports `RepoWriterLocked` with unknown recoverability and a TimeoutError
1041
+ cause. A known held exclusive owner fails immediately as live. While an exclusive
1042
+ request is queued, new shared `ifAvailable` writes also fail immediately with
1043
+ `RepoWriterLocked(false)`, even if that exclusive request later times out. Here
1044
+ `false` means browser arbitration currently blocks the write; it does not prove
1045
+ a persistent exclusive owner exists. The adapter does not retry those writes.
1046
+ Coordinate ownership handoff with shared writers, or retry the failed operation
1047
+ after the handoff/cancellation under an application-owned policy. A custom lock provider
1048
+ must honor queued `request({ signal })` cancellation; optional `query()` is only a
1049
+ fast refusal, never proof of acquisition. Older adapter versions and arbitrary raw
1050
+ backend writes do not participate in this contract. Normal close releases the
1051
+ lease only after actual producer/persistence work settles.
1052
+
1053
+ `migrateStorage(from, to)` requires stopped producer repos and exclusive/inventory
1054
+ capabilities. It freezes both stores, requires an empty target (including local
1055
+ checkpoints), copies Doc snapshots and unresolved journals, checks target version
1056
+ coverage and retained pending input, and verifies complete Flock records/clocks/
1057
+ tombstones. Catalog publication comes last; reports cover deleted identities and
1058
+ orphan bodies. The source is never deleted. `RepoMigrationFailed.report` exposes
1059
+ partial verification. Migration writes a persistent incomplete marker before the
1060
+ first copy and clears it only after all verification; `Repo.make` refuses such a
1061
+ target with `RepoMigrationIncomplete`, including across adapter reopen. Custom
1062
+ migration targets must implement `migrationState`.
1063
+ To retry a failed target, explicitly call `yield* discardIncompleteMigration(target)`:
1064
+ it obtains the exclusive lease, verifies the incomplete marker, clears bodies,
1065
+ catalog and local indexes/checkpoints, then removes the marker last. Cleanup failure
1066
+ retains the marker for retry. A completed store or a live owner is refused; the
1067
+ source is never touched. Custom adapters also need the matching-marker
1068
+ `discardIncompleteMigration` storage capability for this operation. Cursors are reset; an external Doc cursor without a target-local association is
1069
+ cleared before its association is recorded or its offset trusted. An adapter
1070
+ without deletion fails explicitly when such a stale cursor exists. There is no
1071
+ unsafe checkpoint-copy mode. For a Promise-to-Effect **in-place upgrade of the same
1072
+ store**, choose `documentCursorPolicy: 'reuse-existing'` to retain existing external
1073
+ Doc cursors while populating their local index. Never use that option for a new
1074
+ migration target. CI tests parent-driven forced termination and recovery for
1075
+ filesystem/SQLite on Linux, macOS and Windows. Browser storage deployment acceptance
1076
+ remains separate. To measure per-transaction Web Lock overhead with the actual IDB
1077
+ adapter, build then run `node bench/indexeddb-locks.mjs` from this package and open
1078
+ the localhost URL. It rotates five passes of 200 sequential writes for 512-byte
1079
+ and 32-KiB text snapshots, comparing an isolated uncoordinated baseline, native shared
1080
+ locks and a held exclusive lease. The baseline is for measurement only and cannot
1081
+ fence writers. Setup/warmup/lease acquisition are excluded; the final body is read
1082
+ back. The page reports medians and p95 without correctness thresholds. Optional
1083
+ headless automation accepts `--playwright /absolute/path/to/playwright/index.mjs`
1084
+ and `BENCH_CHROMIUM=/path/to/chromium`; no new runtime dependency is required.
1085
+
1086
+ On the published 0.18 SDK, Streams rooms omit the upload-only capability;
1087
+ `beginUpload` returns `not-attempted` with a typed `RepoUnsupported` cause. The
1088
+ `upload-sdk` CI job checks the installed capability before running actual upload
1089
+ acceptance. The committed dev dependency/lock now use official 0.19.0, and normal
1090
+ tests also enable this acceptance. The job still rejects unsupported older SDKs.
1091
+ No cross-repository credentials or unpublished source overrides are used. `route: 'wait'` observes room as well as transport changes and
1092
+ retries declared transient errors with bounded backoff; hub close cancels these
1093
+ waiters before draining their internal leases. Attachment replacement resets prior
1094
+ baseline/current evidence, while epoch stays monotone when the transport id is reused.
1095
+
1096
+ ## Native Effect Streams
1097
+
1098
+ Use `loro-repo/effect/streams` with **streams-crdt 0.19.x** (streams-client 0.12.0)
1099
+ and Effect 4.0.2. Reuse the same `StreamsAuth` service/Layer instance for this adapter
1100
+ and other same-process native Streams clients (for example application RPC). Token
1101
+ refresh and credential ownership belong to that shared service; Repo does not create
1102
+ a separate refresh runtime. This separate entry is ESM-only, matching the upstream SDK;
1103
+ the SDK is an optional peer, so the base Repo install does not pull in Effect
1104
+ through Streams. Install the SDK explicitly to use either Streams entry. Promise users retain the
1105
+ existing adapter and their compatible SDK peer range.
1106
+
1107
+ ```ts
1108
+ import { Effect } from 'effect';
1109
+ import * as Repo from 'loro-repo/effect';
1110
+ import { makeStreamsTransport, StreamsAuth } from 'loro-repo/effect/streams';
1111
+
1112
+ const program = Effect.scoped(Effect.gen(function* () {
1113
+ const repo = yield* Repo.make({ storageAdapter, documentRemoteCursorStore });
1114
+ const streams = yield* makeStreamsTransport({
1115
+ bucketId: 'workspace',
1116
+ // Native StreamsAuth is captured from the caller's context.
1117
+ // Use a total per-room protection resolver + stable namespace for E2EE.
1118
+ });
1119
+ yield* repo.attachTransport(streams);
1120
+ // Local identity/schema initialization stays application-owned.
1121
+ yield* repo.withCreateDoc('note:one', { title: 'New note' }, (doc) =>
1122
+ Effect.sync(() => { doc.getText('text').insert(0, 'hello'); })
1123
+ );
1124
+ yield* streams.createStream({ kind: 'doc', id: 'note:one' }); // explicit PUT only
1125
+ yield* Effect.scoped(Effect.gen(function* () {
1126
+ const room = yield* repo.acquireRoom({ kind: 'doc', id: 'note:one' });
1127
+ yield* room.awaitSynced('streams'); // background initial protocol barrier
1128
+ const report = yield* room.sync('streams'); // new read-to-tail / append ACK
1129
+ }));
1130
+ }));
1131
+ // Provide your native auth Layer. For an anonymous test server only:
1132
+ // Effect.runPromise(program.pipe(Effect.provide(StreamsAuth.anonymous)));
1133
+ ```
1134
+
1135
+ Meta/Doc/Flock targets share the same room kernel. Default `persistence: 'replica'`
1136
+ requires the exact lease's checkpoint capability and saves its data before cursor
1137
+ advancement. Doc/Flock remote imports use their lease-owned journals, preserving accepted bytes before later overwrites.
1138
+ Doc needs `documentRemoteCursorStore`; SQLite/IndexedDB Meta/Flock use
1139
+ replica-bound stores from hydration. Missing durability fails. Explicit
1140
+ `persistence: 'volatile'` keeps cursors in memory and bootstraps again after reopen.
1141
+
1142
+ Remote creation defaults off. `createStream` never opens local replicas or claims
1143
+ readiness; `createStreamIfMissing: true` explicitly opts into automatic provisioning.
1144
+ `uploadSnapshot(target)` requires a live acquired room. Native auth refresh,
1145
+ reconnect policy, shard URLs, snapshot codec and named-Flock TTL are supported.
1146
+ Protected rooms use the same namespace/logical-room AAD encoder and strict encrypted
1147
+ policies as the existing Streams adapter. Providers/vaults remain caller-owned.
1148
+
1149
+ Native upload policy is an optional newer SDK capability. `uploadTransport`,
1150
+ explicit `reconnectConfig.authTimeoutMs`, `uploadTimeoutMs`, `uploadIdleTimeoutMs`,
1151
+ and `snapshotUpload.onDiagnostic` require the native SDK's
1152
+ `durableUploadTransportVersion === 1`. Unsupported configurations fail before
1153
+ auth or SDK resource acquisition; omitted/undefined fields retain the ordinary
1154
+ published 0.18.0 path. `StreamsUploadTransport` derives its contract from the
1155
+ installed native SDK, so legacy declarations expose `never`. The same room
1156
+ configuration serves Doc/Meta/Flock joins and explicit provisioning.
1157
+
1158
+ The SDK owns independent auth, request and upload deadlines, qualified progress,
1159
+ frozen retry data and I/O settlement. A progress transport must preserve request
1160
+ semantics and actual ownership through cancellation; body consumption is not
1161
+ transmission, EOF is not durability, and the hard cap never renews. No platform
1162
+ progress adapter is supplied here. Shared error projection retains only validated
1163
+ closed enums and finite scalars; full Causes remain private recovery evidence.
1164
+ The official SDK release must combine captured-upload/read-health and native
1165
+ upload policy capabilities, followed by a minimum peer/dev upgrade. Exact local
1166
+ artifacts prove the integration, without bypassing the committed release gate.
1167
+ See the [native upload integration note](../../docs/plans/2026-10-15-native-upload-transport.md).
1168
+
1169
+ `health(target)` returns a safe SDK room-health projection. Full error Causes and
1170
+ retry owners are private recovery data; do not serialize or log them. Closing the
1171
+ room/attachment retains failed raw work, blocks replacement, and exposes recovery
1172
+ through `RepoReleaseFailed.retry`. Local storage remains independently owned by Repo.
1173
+ Cloud presence is opt-in through `ephemeral: { auth }`, where `auth` is a native
1174
+ `EphemeralAuth` service implementation. `acquireEphemeralStore('streams', id)` exposes
1175
+ a scoped Loro `EphemeralStore`, a local publish `sync` barrier and a failure Stream.
1176
+ The SDK owns POST/SSE producers; `sync` waits for a real POST ACK, not peer delivery.
1177
+ Ephemeral has no durable fallback or E2EE in the released SDK. When durable payload
1178
+ protection is configured, enabling it additionally requires `allowPlaintext: true`;
1179
+ otherwise construction fails rather than silently downgrading protected presence.
1180
+ Application RPC stays outside Repo. Production Lody IPC and
1181
+ CLI/renderer migration remain downstream work. See the [ownership and acceptance
1182
+ note](../../docs/plans/2026-10-14-native-effect-streams.md).