@vrtmrz/livesync-commonlib 0.3.0 → 0.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (73) hide show
  1. package/README.md +27 -4
  2. package/dist/API/DirectFileManipulatorV2.d.ts +1 -0
  3. package/dist/API/DirectFileManipulatorV2.js +6 -12
  4. package/dist/API/processSetting.d.ts +1 -0
  5. package/dist/common/models/setting.const.defaults.js +1 -0
  6. package/dist/common/models/setting.const.qr.js +1 -0
  7. package/dist/common/models/setting.policy.d.ts +4 -0
  8. package/dist/common/models/setting.policy.js +1 -0
  9. package/dist/common/models/setting.type.d.ts +2 -0
  10. package/dist/common/models/tweak.definition.d.ts +1 -0
  11. package/dist/common/pathNamespace.d.ts +4 -0
  12. package/dist/common/pathNamespace.js +10 -0
  13. package/dist/common/settingConstants.d.ts +1 -0
  14. package/dist/common/settingConstants.js +4 -0
  15. package/dist/platform/AmbiguousNormalisedPathError.d.ts +16 -0
  16. package/dist/platform/AmbiguousNormalisedPathError.js +21 -0
  17. package/dist/platform/StoragePathResolver.d.ts +15 -0
  18. package/dist/platform/StoragePathResolver.js +113 -0
  19. package/dist/platform/browser/UnicodeResolvingFileSystemAccessStorageAdapter.d.ts +74 -0
  20. package/dist/platform/browser/UnicodeResolvingFileSystemAccessStorageAdapter.js +169 -0
  21. package/dist/platform/browser/index.d.ts +1 -0
  22. package/dist/platform/browser/index.js +2 -0
  23. package/dist/platform/browser/storage.d.ts +23 -0
  24. package/dist/platform/browser/storage.js +138 -0
  25. package/dist/platform/node/UnicodeResolvingNodeStorageAdapter.d.ts +79 -0
  26. package/dist/platform/node/UnicodeResolvingNodeStorageAdapter.js +218 -0
  27. package/dist/platform/node/index.d.ts +1 -0
  28. package/dist/platform/node/index.js +2 -0
  29. package/dist/platform/node/storage.d.ts +30 -1
  30. package/dist/platform/node/storage.js +88 -0
  31. package/dist/platform/pathComparison.d.ts +53 -0
  32. package/dist/platform/pathComparison.js +39 -0
  33. package/dist/platform/pathTypes.d.ts +154 -0
  34. package/dist/platform/pathTypes.js +0 -0
  35. package/dist/platform/storagePathResolution.d.ts +13 -0
  36. package/dist/platform/storagePathResolution.js +9 -0
  37. package/dist/replication/SyncParamsHandler.d.ts +1 -0
  38. package/dist/replication/SyncParamsHandler.js +5 -1
  39. package/dist/replication/couchdb/LiveSyncReplicator.js +5 -3
  40. package/dist/replication/journal/JournalReceiveFailure.d.ts +7 -0
  41. package/dist/replication/journal/JournalReceiveFailure.js +16 -0
  42. package/dist/replication/journal/JournalSyncCore.d.ts +22 -3
  43. package/dist/replication/journal/JournalSyncCore.js +288 -166
  44. package/dist/replication/journal/JournalSyncTypes.d.ts +13 -0
  45. package/dist/replication/journal/LiveSyncJournalReplicator.d.ts +6 -1
  46. package/dist/replication/journal/LiveSyncJournalReplicator.js +31 -44
  47. package/dist/replication/journal/journalCompatibility.d.ts +9 -0
  48. package/dist/replication/journal/journalCompatibility.js +58 -0
  49. package/dist/replication/journal/journalReceive.d.ts +23 -0
  50. package/dist/replication/journal/journalReceive.js +44 -0
  51. package/dist/replication/journal/journalRecords.d.ts +10 -0
  52. package/dist/replication/journal/journalRecords.js +96 -0
  53. package/dist/replication/trystero/TrysteroReplicator.d.ts +2 -0
  54. package/dist/serviceModules/Rebuilder.js +2 -1
  55. package/dist/services/base/FileProcessingService.d.ts +2 -2
  56. package/dist/services/base/RemoteConnection.d.ts +2 -0
  57. package/dist/services/base/RemoteService.d.ts +1 -1
  58. package/dist/services/base/RemoteService.js +13 -5
  59. package/dist/services/implements/injectable/InjectableVaultService.d.ts +1 -1
  60. package/dist/services/lib/couchdbHttpCache.d.ts +5 -0
  61. package/dist/services/lib/couchdbHttpCache.js +36 -0
  62. package/dist/storage.d.ts +4 -0
  63. package/dist/storage.js +19 -0
  64. package/dist/string_and_binary/path.js +4 -7
  65. package/dist/worker/bgWorker.js +1 -1
  66. package/docs/adr/2026_11_shared_storage_path_resolution.md +75 -0
  67. package/docs/design_docs/storage_path_resolution.md +306 -0
  68. package/docs/development.md +11 -1
  69. package/docs/journal-receive-recovery.md +66 -0
  70. package/docs/platform-storage.md +61 -0
  71. package/docs/releasing.md +25 -0
  72. package/docs/unicode-path-identity.md +49 -17
  73. package/package.json +9 -1
@@ -0,0 +1,75 @@
1
+ ---
2
+ date: 2026-11-03
3
+ commonlib-version: "0.3.0"
4
+ self-hosted-livesync-version: "1.0.44"
5
+ status: published
6
+ ---
7
+
8
+ # Architectural Decision Record: Shared Storage Path Resolution
9
+
10
+ ## Conclusion
11
+
12
+ Commonlib owns strict Unicode path comparison and resolution behind a focused
13
+ `/storage` entry. Node and File System Access provide thin storage integrations;
14
+ hosts retain filesystem handles, original paths, caches, watchers, and operation
15
+ lifecycle. The [design](../design_docs/storage_path_resolution.md) defines the
16
+ accepted contract and remaining adoption work.
17
+
18
+ Accepted on 2026-11-03 against the versions recorded above. The implementation
19
+ is published in [Commonlib 0.3.1](https://www.npmjs.com/package/@vrtmrz/livesync-commonlib/v/0.3.1).
20
+ The original date and reviewed versions retain the accepted baseline.
21
+ Publication does not by itself enable NFC in WebApp or Bridge.
22
+
23
+ ## Context
24
+
25
+ The Self-hosted LiveSync CLI already resolves NFC-equivalent names component by
26
+ component while preserving physical spelling. Repeating that logic in browser
27
+ and Deno hosts would duplicate ambiguity, missing-path, and rename rules.
28
+ Meanwhile, the legacy path union includes comparison keys, so removing a
29
+ namespace can appear to turn a lossy comparison value back into a file path.
30
+
31
+ ## Decisions
32
+
33
+ - Separate original document paths, unprefixed file paths, NFC refinements,
34
+ resolved storage paths, and comparison keys. Preserve the old public types;
35
+ new namespace helpers exclude their legacy comparison-key member.
36
+ - Inject only strict directory enumeration into the resolver. Return `null`
37
+ for absence, and propagate permission, non-directory, and other I/O failures.
38
+ Resolve ambiguity before mutation, including when an exact spelling exists.
39
+ - Copy comparison settings for one operation or scan. Retain no cross-operation
40
+ directory cache or persistent path mapping. Scan already resolved entries
41
+ once per directory, and re-resolve later lookups after external changes.
42
+ - Preserve ordinary writes' physical spelling. An explicit rename may request
43
+ another spelling of the same leaf. Mutation and replacement semantics remain
44
+ with the storage owner; resolution is not an atomic reservation.
45
+ - Keep string-taking storage APIs and Legacy behaviour. New physical-I/O ports
46
+ take `StoragePath`. Namespace routing and cryptographic document-ID derivation
47
+ remain separate from filesystem resolution.
48
+ - Adopt the shared implementation in the CLI and the shared comparison and
49
+ namespace helpers at applicable Obsidian boundaries. WebApp and Bridge NFC
50
+ enablement remains separate work, including admission, events, and caches.
51
+
52
+ ## Alternatives considered
53
+
54
+ - Duplicating the CLI resolver per host would duplicate the behavioural contract
55
+ and its failure handling.
56
+ - Renaming all physical files to NFC would alter user-owned storage and require
57
+ collision recovery beyond the accepted scope.
58
+ - A persistent alias map or shared directory cache would introduce invalidation,
59
+ restart, and lifecycle obligations which current directory reads avoid.
60
+ - Combining namespace, Unicode form, and case in a family of filename types
61
+ would conflate independent properties. Comparison keys instead name their
62
+ purpose and their storage or document domain.
63
+
64
+ ## Consequences and verification
65
+
66
+ Hosts supply trustworthy complete listings and preserve distinct names before
67
+ constructing keyed collections. Filesystems or APIs which hide aliases limit
68
+ what the resolver can observe. Host concurrency and permission checks remain
69
+ necessary after resolution.
70
+
71
+ Contract tests cover spelling, ambiguity, strict failures, snapshots, scans,
72
+ and renames. Packed-consumer tests check assignability and platform bundling.
73
+ The downstream CLI retains its behavioural suite against the exact packed
74
+ candidate. Memory-backed File System Access tests establish adapter behaviour;
75
+ real-browser permissions and Deno integration require their own host evidence.
@@ -0,0 +1,306 @@
1
+ ---
2
+ date: 2026-11-03
3
+ commonlib-version: "0.3.0"
4
+ self-hosted-livesync-version: "1.0.44"
5
+ status: published
6
+ ---
7
+
8
+ # Shared storage path resolution
9
+
10
+ This design moves the CLI's existing Unicode-aware path resolution into
11
+ Commonlib so Node, File System Access, and Bridge can apply the same naming
12
+ rules. The shared resolver, path types, and Node and File System Access
13
+ integrations are published in Commonlib 0.3.1. Installing the package does
14
+ not enable NFC in an existing host or migrate its database.
15
+
16
+ Accepted on 2026-11-03. The [decision record](../adr/2026_11_shared_storage_path_resolution.md)
17
+ records the ownership and compatibility choices. The [platform guide](../platform-storage.md#unicode-path-resolution)
18
+ provides consumer examples.
19
+
20
+ The baseline is Commonlib 0.3.0 and the CLI implementation reviewed in
21
+ [Self-hosted LiveSync #1309](https://github.com/vrtmrz/obsidian-livesync/pull/1309)
22
+ at `2716d68b3d21d493cdfdefa0857fad4c6a3c522a`. The existing
23
+ [Unicode identity contract](../unicode-path-identity.md) continues to govern
24
+ Metadata IDs, remote admission, defaults, and database migration.
25
+
26
+ ## Ownership
27
+
28
+ Commonlib owns path comparison, component-by-component resolution, ambiguity
29
+ checks, and thin platform-storage integrations. Hosts own root selection,
30
+ permissions, filesystem handles, file objects, watchers, caches, and operation
31
+ lifecycle. A resolver performs directory reads but no filesystem mutation.
32
+
33
+ Keep the existing Node and File System Access storage APIs and their Legacy
34
+ behaviour. Compose the resolver only for NFC mode. Its comparison policy is
35
+ an immutable snapshot for one operation or scan, copied at construction;
36
+ a settings change requires a new composition. Explicit case-sensitive and
37
+ case-insensitive lookups use
38
+ the corresponding snapshot, without mutating a shared resolver.
39
+
40
+ ## Path types
41
+
42
+ Separate namespace, Unicode form, comparison use, and storage resolution.
43
+ An actual filename can already be NFC, so these are not mutually exclusive
44
+ string categories.
45
+
46
+ | Type | Meaning | Producer |
47
+ | --- | --- | --- |
48
+ | `FilePath` | Original spelling without a Metadata namespace; no NFC or resolution guarantee. | Existing input and host boundaries. |
49
+ | `DocumentPath` | Original spelling with an optional recognised Metadata namespace; excludes comparison keys. | Document input and namespace helpers. |
50
+ | `NfcFilePath` | A `FilePath` normalised to NFC, without lower-casing. | The shared NFC normalisation function. |
51
+ | `StoragePath` | A root-relative storage target whose existing components use their resolved spelling. | Resolver or supported physical enumeration. |
52
+ | `PathComparisonKey<"storage">` | A comparison key for a path within one storage root. | The shared comparison helper. |
53
+ | `PathComparisonKey<"document">` | A comparison key which retains the document namespace. | The shared comparison helper. |
54
+
55
+ `StoragePath` is not proof that an entry exists, still exists, is a file,
56
+ or cannot change concurrently. A write target may contain missing components.
57
+ The root is the empty path; each operation retains its existing root rules.
58
+
59
+ Reuse the existing `FilePath` and OW `TaggedType` definitions. Add independent
60
+ symbols for refinements: nesting `TaggedType<FilePath, "StoragePath">` would
61
+ intersect incompatible values of OW's existing tag property.
62
+
63
+ ```ts
64
+ declare const unicodeForm: unique symbol;
65
+ declare const storagePath: unique symbol;
66
+
67
+ type DocumentPath = Exclude<FilePathWithPrefix, FilePathWithPrefixLC>;
68
+
69
+ type NfcFilePath = FilePath & {
70
+ readonly [unicodeForm]: "NFC";
71
+ };
72
+
73
+ type StoragePath = FilePath & {
74
+ readonly [storagePath]: true;
75
+ };
76
+
77
+ type PathComparisonKey<Domain extends "storage" | "document"> =
78
+ TaggedType<string, `PathComparisonKey:${Domain}`>;
79
+ ```
80
+
81
+ Use `NfcFilePath` within the normalisation boundary initially. Publish the
82
+ path and key types needed by consumers of the new focused API. Preserve the
83
+ existing definitions of `FilePathWithPrefix` and `FilePathWithPrefixLC` at
84
+ compatibility entry points. Do not alias either old type to a new key type:
85
+ that would change existing assignability.
86
+
87
+ The old `FilePathWithPrefix` union includes both `FilePath` and
88
+ `FilePathWithPrefixLC`. The new `DocumentPath` intentionally removes the latter.
89
+ The old scanner's `convertCase()` returns its `LC` type even when case-sensitive
90
+ handling retains letter case. New code therefore names its output by its
91
+ comparison purpose, not an assumed lower-case form.
92
+
93
+ Brand values at the boundary which establishes their meaning. `parseFilePath`
94
+ validates raw storage input using the existing path vocabulary without changing
95
+ its spelling. `parseDocumentPath` accepts original document-path strings;
96
+ namespace routing precedes storage validation. Neither input helper recovers
97
+ original spelling from a comparison key. NFC conversion and resolution establish their own refinements.
98
+ Keep casts inside those owning implementations; consumer code receives the
99
+ typed result. The existing string-taking APIs remain compatibility escape
100
+ routes, so this migration does not claim to make all old call sites type-safe.
101
+ Within the new integrations, physical-I/O ports accept `StoragePath`; NFC
102
+ normalisation or key generation alone cannot supply that argument.
103
+
104
+ ## Namespaces and comparison
105
+
106
+ Namespace helpers accept `DocumentPath`, return a separate namespace and
107
+ `FilePath`, and recognise the existing `i:`, `ix:`, and `ps:` rules. Split one
108
+ namespace only; further colons belong to the body. The caller handles the
109
+ namespace's meaning before submitting a filesystem path to the resolver.
110
+ Chunk prefixes and obfuscated IDs remain `DocumentID` concerns.
111
+
112
+ Document comparison preserves the namespace; storage comparison does not
113
+ receive one. A normal document and an internal document with the same body
114
+ remain distinct document identities. Neither comparison-key domain is accepted
115
+ by the resolver or by the new namespace helpers.
116
+
117
+ The comparison policy uses the existing `normalisePathForIdentity` operation,
118
+ then the existing `toLowerCase()` rule only when configured. NFC conversion
119
+ alone produces `NfcFilePath`; the complete comparison pipeline produces a key.
120
+ It does not infer the host filesystem's case rules or introduce a new Unicode
121
+ case-folding rule. Preserve original paths alongside keys; a key is never
122
+ converted back into an original path.
123
+
124
+ ```ts
125
+ type PathComparisonSettings = Readonly<{
126
+ pathNormalizationVersion: 0 | 1;
127
+ handleFilenameCaseSensitive: boolean;
128
+ }>;
129
+
130
+ type DocumentPathParts = Readonly<{
131
+ prefix: "" | "i:" | "ix:" | "ps:";
132
+ path: FilePath;
133
+ }>;
134
+
135
+ declare function splitDocumentPath(path: DocumentPath): DocumentPathParts;
136
+ declare function joinDocumentPath(parts: DocumentPathParts): DocumentPath;
137
+
138
+ declare function getStoragePathComparisonKey(
139
+ path: FilePath, settings: PathComparisonSettings
140
+ ): PathComparisonKey<"storage">;
141
+
142
+ declare function getDocumentPathComparisonKey(
143
+ path: DocumentPath, settings: PathComparisonSettings
144
+ ): PathComparisonKey<"document">;
145
+ ```
146
+
147
+ Namespace routing and cryptographic document-ID derivation remain outside the
148
+ resolver. The new comparison helpers must retain the existing key bytes for
149
+ each supported policy; extracting them does not rewrite IDs or stored paths.
150
+
151
+ ## Directory and resolver contract
152
+
153
+ The only injected I/O operation enumerates immediate children using their
154
+ actual names and kinds. Paths are relative to the selected root and use `/`.
155
+ `null` means the directory is absent. Permission failures, a non-directory
156
+ parent, and other I/O failures reject; they are not empty listings.
157
+
158
+ The existing storage `list()` methods can hide some failures as empty results.
159
+ Provide a strict enumeration path for the resolver while preserving the old
160
+ methods' published behaviour. Reuse each platform's containment and symbolic-link
161
+ checks. Keep native handles and platform stat types behind this boundary.
162
+
163
+ ```ts
164
+ type DirectoryEntry = Readonly<{
165
+ name: string;
166
+ kind: "file" | "directory";
167
+ }>;
168
+
169
+ type ReadDirectory = (
170
+ physicalDirectory: StoragePath
171
+ ) => Promise<readonly DirectoryEntry[] | null>;
172
+
173
+ type StorageEntry = Readonly<{
174
+ path: StoragePath;
175
+ kind: "file" | "directory";
176
+ }>;
177
+
178
+ type ResolvedRename = Readonly<{
179
+ source: StoragePath;
180
+ target: StoragePath;
181
+ }>;
182
+
183
+ interface StoragePathResolver {
184
+ resolveExisting(path: FilePath): Promise<StoragePath | null>;
185
+ resolveForWrite(path: FilePath): Promise<StoragePath>;
186
+ resolveRename(source: FilePath, target: FilePath): Promise<ResolvedRename | null>;
187
+ listResolvedDirectory(path: StoragePath): Promise<readonly StorageEntry[] | null>;
188
+ }
189
+
190
+ declare function createStoragePathResolver(options: Readonly<{
191
+ readDirectory: ReadDirectory;
192
+ policy: Readonly<{
193
+ pathNormalizationVersion: 1;
194
+ handleFilenameCaseSensitive: boolean;
195
+ }>;
196
+ }>): StoragePathResolver;
197
+ ```
198
+
199
+ `DirectoryEntry.name` is an immediate child name, never a path. Reject malformed
200
+ entries at the enumeration boundary. Add a separate filename brand only when a
201
+ consumer boundary needs it; do not publish a family of filename variants now.
202
+
203
+ The resolver uses complete sibling enumeration before deciding uniqueness;
204
+ a literal match does not override a second equivalent name. File/directory
205
+ collisions count as ambiguity. A unique file cannot be traversed as a parent.
206
+ An individual lookup checks candidates for each requested component; a full
207
+ scan checks every sibling group before returning that directory's entries.
208
+
209
+ | Operation | Result |
210
+ | --- | --- |
211
+ | `resolveExisting` | Resolve each existing component to its listed spelling; return `null` on absence. |
212
+ | `resolveForWrite` | Preserve listed names for existing components and the requested spelling for the missing suffix. |
213
+ | `resolveRename` | Resolve both ends. If the target resolves to the source, retain the resolved parent and the explicitly requested leaf spelling. Return `null` when the source is absent. |
214
+ | `listResolvedDirectory` | Read an already resolved directory, check all sibling identities, and return actual child paths for scanning and direct access. |
215
+
216
+ All operations reject ambiguity using the shared `AmbiguousNormalisedPathError`.
217
+ Include the requested path and conflicting actual names for diagnosis. Preserve
218
+ I/O failures separately from absence. Host catch blocks must propagate these
219
+ failures instead of converting them to a missing file or a successful partial
220
+ scan.
221
+
222
+ Normal writes preserve existing Unicode spelling. Explicit renames may change
223
+ it. Existing-destination replacement policy and the actual rename, write,
224
+ rollback, or deletion remain the storage owner's responsibility. Resolution is
225
+ not an atomic reservation; existing host operation ordering and platform
226
+ protections remain necessary. The guarantee is limited to names exposed by the
227
+ host's supported listing.
228
+
229
+ ## Composition and adoption
230
+
231
+ The `/storage` entry contains the host-neutral types, comparison and
232
+ namespace helpers, and resolver. It imports no Node, DOM, database, or Service
233
+ Hub runtime. `/node` and `/browser` expose their platform integrations through
234
+ the existing bundling boundaries.
235
+
236
+ Commonlib owns `UnicodeResolvingNodeStorageAdapter` and delegates
237
+ its resolution to the common implementation. The current CLI candidate adopts
238
+ the focused API while retaining Legacy delegation and thin compatibility
239
+ methods. File System Access uses the same resolver, including host paths which
240
+ operate directly on handles. Bridge supplies a Deno directory reader and uses
241
+ the same contract in an unreleased integration candidate.
242
+
243
+ Keep the resolver free of persistent mappings, cross-operation caches, watchers,
244
+ retry owners, and disposal requirements. A full scan descends through the
245
+ resolved entries and reads each directory once. A later lookup re-reads the
246
+ relevant directories, including after a file-object cache hit. OW Scope or
247
+ context can own host operation resources where needed; the resolver itself
248
+ requires no resource scope.
249
+
250
+ Hosts retain original physical paths alongside any comparison keys. Detect
251
+ aliases before collapsing them into a keyed collection. Cache invalidation,
252
+ queue ownership, and watch-event reconciliation must follow the same comparison
253
+ policy without losing distinct names. A deletion event may name an entry which
254
+ is already absent; preserve that event path rather than trying to recover its
255
+ spelling from a live directory lookup. Check rename events as an ordered change
256
+ of state, including hosts which report them as deletion and creation.
257
+
258
+ ## Current adoption and publication status
259
+
260
+ The shared `/storage` contract, Node and File System Access adapters, and
261
+ explicit `DirectFileManipulator` path-version option are published in
262
+ [Commonlib 0.3.1](https://www.npmjs.com/package/@vrtmrz/livesync-commonlib/v/0.3.1).
263
+ The original decision date and reviewed versions above retain the accepted
264
+ baseline. These additions are absent from the registry artefact `0.3.0`;
265
+ a source checkout carrying that version label is not the published artefact.
266
+
267
+ The CLI candidate uses the shared Node resolver, and the current Obsidian
268
+ candidate uses shared namespace and comparison helpers while retaining Vault
269
+ APIs and its alias guard. The matching Journal and P2P host adoption remains
270
+ unreleased in Self-hosted LiveSync [#1309](https://github.com/vrtmrz/obsidian-livesync/pull/1309).
271
+ The browser resolver is a general primitive, not WebApp NFC support: WebApp
272
+ remains Legacy-only and rejects an explicit version `1` at startup. WebPeer
273
+ has no Vault filesystem and is not an NFC filesystem host.
274
+
275
+ Bridge's Deno integration is also an unreleased candidate. NFC mode is an
276
+ explicit per-peer opt-in, and every peer in a group must use the same path
277
+ version. Its ordinary case-mismatch check remains enabled; the Bridge-wide
278
+ `disableCheckingConfigMismatch: true` override disables that case check only
279
+ and does not bypass path-version admission. Preserve the existing coordinated
280
+ Rebuild/Fetch migration and remote mismatch protection. This design adds no
281
+ automatic migration or translation between identity modes.
282
+
283
+ ## Verification
284
+
285
+ Check the public declarations against the actual published path types and OW
286
+ tag implementation. Positive cases cover source paths, namespace round trips,
287
+ and independent NFC/storage refinements. Negative cases cover raw strings,
288
+ prefix-bearing paths at the resolver, comparison keys at resolution and namespace
289
+ boundaries, and mixing the two key domains. Retain an assertion that the legacy
290
+ `LC` type is still assignable to the legacy union, but not to `DocumentPath`.
291
+
292
+ Implementation must preserve the existing CLI behavioural cases and apply one
293
+ resolver contract to Node and File System Access, with Bridge-specific checks:
294
+
295
+ - NFD files and mixed-spelling parents remain addressable without implicit renames.
296
+ - New components retain the requested spelling under the actual parent.
297
+ - File, directory, and case collisions reject before mutation, including exact-match requests.
298
+ - Permission and listing failures do not become absence, creation, deletion, or partial-scan success.
299
+ - Explicit spelling renames preserve contents and the host's replacement and recovery semantics.
300
+ - External additions, deletions, renames, and collisions are observed after earlier cache hits.
301
+ - Full scans inspect each directory once; Legacy behaviour and namespace separation remain unchanged.
302
+ - Watcher deletion/creation sequences, restart state, and cache keys preserve the intended identity.
303
+
304
+ Packed-package checks must verify the focused entry and platform bundling
305
+ boundaries. Real browser and Deno checks establish their host behaviour;
306
+ Commonlib's memory and Node tests alone do not establish those guarantees.
@@ -6,7 +6,7 @@ This document is for Commonlib developers. Package consumers should begin with t
6
6
 
7
7
  The source tree is larger than the supported package surface. Consumers may import only paths in the generated package export map:
8
8
 
9
- - `src/index.ts`, `src/context.ts`, `src/settings.ts`, `src/remoteConfigurations.ts`, `src/p2p/index.ts`, `src/platform/browser/index.ts`, and `src/platform/node/index.ts` define the focused entries;
9
+ - `src/index.ts`, `src/context.ts`, `src/storage.ts`, `src/settings.ts`, `src/remoteConfigurations.ts`, `src/p2p/index.ts`, `src/platform/browser/index.ts`, and `src/platform/node/index.ts` define the focused entries;
10
10
  - `src/conflicts.ts` defines the host-owned conflict-check queue contract; `src/concurrency/keyedPendingWork.ts` remains package-private;
11
11
  - `src/replication/index.ts` defines the focused provider-capability and interaction contract for hosts which compose replication;
12
12
  - `src/diagnostics/index.ts` defines read-only Usage and integrity inspection, report projection, and formatting; hosts own the run dialogue and output destination;
@@ -19,6 +19,16 @@ The source tree is larger than the supported package surface. Consumers may impo
19
19
 
20
20
  Files elsewhere under `src` are implementation details unless a focused entry re-exports them or the compatibility inventory names them. Do not use the source layout, TypeScript path aliases, or the presence of generated declarations as evidence of a supported consumer import.
21
21
 
22
+ ## Storage path boundary
23
+
24
+ The `/storage` entry, published in Commonlib 0.3.1, owns namespace helpers,
25
+ comparison keys, and strict component resolution. The [design](design_docs/storage_path_resolution.md)
26
+ and [decision record](adr/2026_11_shared_storage_path_resolution.md) separate
27
+ that contract from host handles, caches, watchers, and migration. See the
28
+ [platform guide](platform-storage.md#unicode-path-resolution) for composition.
29
+ The [Unicode identity contract](unicode-path-identity.md) retains its existing
30
+ Metadata, remote-admission, and database-migration rules.
31
+
22
32
  ## Module filenames
23
33
 
24
34
  For new or renamed TypeScript and JavaScript source modules, start the filename
@@ -0,0 +1,66 @@
1
+ ---
2
+ date: 2026-11-04
3
+ commonlib-version: "0.3.1"
4
+ self-hosted-livesync-version: "1.0.44"
5
+ status: unreleased
6
+ ---
7
+
8
+ # Journal receive failures and recovery
9
+
10
+ This document describes unreleased receive behaviour. Hosts choose when to
11
+ offer the manual skip operation and remain responsible for Vault reflection
12
+ and conflict handling.
13
+
14
+ ## Stop and retry by default
15
+
16
+ A per-Journal decode, record-processing, or write failure stops receipt. The
17
+ failed Journal and all later Journals remain unacknowledged. The next normal
18
+ receive attempt retries the failed Journal before proceeding to later Journals.
19
+ Partial progress does not change this rule.
20
+
21
+ Chunks are written before Metadata, so a Chunk failure prevents that Journal's
22
+ Metadata stage. Metadata retains its received revision ancestry through
23
+ `new_edits: false`, whose empty success result is valid. Successfully written
24
+ records remain in the database when another record fails. There is no rollback.
25
+
26
+ A retry avoids rewriting existing records and delivers accepted Metadata to
27
+ the host again, including revisions saved before an earlier partial failure.
28
+ The host must tolerate repeated delivery. Chunk-arrival events describe only
29
+ successful writes. The `knownIDs` set suppresses unnecessary
30
+ outbound transfer; it does not prove that a receive has completed locally.
31
+
32
+ Warnings identify the failed Journal key and the stage where receipt stopped,
33
+ without logging document contents. A stop request lets current I/O settle and
34
+ closes the receive operation; it is not a Journal failure.
35
+
36
+ ## Skip the stopped Journal and resume
37
+
38
+ When receipt stops on a Journal failure, a host may offer the explicit
39
+ 'Skip stopped Journal and resume' operation. It accepts the exact stopped
40
+ failure snapshot, marks only that Journal received, and resumes receipt of the
41
+ remaining Journals. If another Journal fails, receipt stops at that Journal
42
+ and requires a separate decision.
43
+
44
+ Skipping does not roll back partial writes. It can leave content or revision
45
+ history missing, and receipt of later Journals does not prove that every file
46
+ can be reconstructed or reflected to storage. Each skip acknowledges the
47
+ failed Journal despite its warning.
48
+
49
+ The stopped failure is runtime state, not a persistent skip mode or failure
50
+ queue. After a restart, normal receipt must encounter the failure again before
51
+ the host can offer a skip for it.
52
+
53
+ ## Replay and host boundaries
54
+
55
+ The existing 'Reset journal received history' operation clears `receivedFiles`
56
+ and `knownIDs`, while retaining the local database and `sentFiles`. After the
57
+ failure has been corrected, resetting this history lets receipt replay remote
58
+ Journals other than those sent by this device. Replay requires the affected
59
+ remote objects to remain available and readable; resetting history cannot
60
+ recreate missing remote bytes.
61
+
62
+ A host operation which replaces the local database, such as CLI Fetch, has
63
+ separate reconstruction and completion rules. Repeating Fetch is not the
64
+ same-database retry described above. Journal receipt also remains separate
65
+ from Vault reflection and conflict handling; hosts must preserve their own
66
+ failure and content-protection checks.
@@ -61,6 +61,67 @@ Writes create missing parent directories. Listing returns direct child paths rel
61
61
 
62
62
  The File System Access API does not expose portable creation times or permit setting file times. Consumers must not rely on identical timestamp fidelity across the two adapters.
63
63
 
64
+ ## Unicode path resolution
65
+
66
+ These APIs are published in [Commonlib 0.3.1](https://www.npmjs.com/package/@vrtmrz/livesync-commonlib/v/0.3.1).
67
+ Node and File System Access hosts can use the same NFC comparison and component
68
+ resolution while retaining the names listed by their filesystem. Hosts still
69
+ own the selected identity mode and its database migration.
70
+
71
+ ```ts
72
+ import { UnicodeResolvingNodeStorageAdapter } from '@vrtmrz/livesync-commonlib/node';
73
+ import { parseFilePath } from '@vrtmrz/livesync-commonlib/storage';
74
+
75
+ const storage = new UnicodeResolvingNodeStorageAdapter(
76
+ '/srv/livesync',
77
+ () => settings.pathNormalizationVersion,
78
+ () => !settings.handleFilenameCaseSensitive,
79
+ );
80
+ const existing = await storage.resolveLogicalPath(parseFilePath('Café.md'), undefined, false);
81
+ if (existing !== null) {
82
+ const content = await storage.readResolved(existing);
83
+ }
84
+ ```
85
+
86
+ The browser equivalent is `UnicodeResolvingFileSystemAccessStorageAdapter`
87
+ from `/browser`, constructed with an authorised root handle and the same
88
+ settings callbacks. Version `0` delegates to the existing storage behaviour;
89
+ version `1` resolves every component against complete sibling listings. Normal
90
+ writes preserve existing spelling. Node's explicit `rename` can change a leaf's
91
+ Unicode or letter-case spelling; the browser adapter retains its existing
92
+ operation set and does not add a rename primitive.
93
+
94
+ `AmbiguousNormalisedPathError` from `/storage` reports multiple equivalent
95
+ names, even when one is a literal match. Preserve that error and directory or
96
+ permission failures. Only absence produces `null`; a failure must not be used
97
+ as permission to create a file or as a successful partial scan.
98
+
99
+ For another host, compose `createStoragePathResolver({ readDirectory, policy })`
100
+ from `/storage`. `readDirectory` accepts a `StoragePath` and returns immediate
101
+ child names and kinds, or `null` for absence. Propagate other failures. The
102
+ resolver copies one NFC policy and retains no directory cache. For a full scan,
103
+ start at `resolveForWrite(parseFilePath(''))`, then recurse through
104
+ `listResolvedDirectory` entries using the same resolver. Each directory is read
105
+ once. Re-resolve later operations so external changes are visible.
106
+
107
+ `FilePath` retains original, unprefixed spelling. `StoragePath` records that
108
+ existing components have been resolved; it does not guarantee existence or
109
+ reserve a target. `parseDocumentPath` marks an original document-path input,
110
+ and `splitDocumentPath` separates one recognised namespace before host routing.
111
+ Use `joinDocumentPath` to restore it. `getStoragePathComparisonKey` and
112
+ `getDocumentPathComparisonKey` return distinct comparison-only types; the latter
113
+ preserves the namespace. Keep the original path beside each key. The old
114
+ `FilePathWithPrefix` and `FilePathWithPrefixLC` remain available for compatibility.
115
+
116
+ The [design](design_docs/storage_path_resolution.md) defines the detailed
117
+ contract. These primitives do not enable NFC automatically in a host
118
+ application. WebApp remains Legacy-only and rejects explicit version `1` at
119
+ startup. The Bridge Deno integration is an unreleased candidate; version `1`
120
+ is an explicit peer opt-in, and every peer in a group must use the same path
121
+ version. Its ordinary case-mismatch check remains separate from path-version
122
+ admission. See [Unicode path identity](unicode-path-identity.md#supported-hosts)
123
+ for the current publication and host-adoption boundaries.
124
+
64
125
  ## Bundling boundary
65
126
 
66
127
  Browser consumers import only `@vrtmrz/livesync-commonlib/browser`. Packed-package tests build that entry with browser conditions and verify that the Node implementation is absent. Node consumers import `@vrtmrz/livesync-commonlib/node`; the root and context entries do not load Node built-ins.
package/docs/releasing.md CHANGED
@@ -34,6 +34,31 @@ Inspect the lockfile diff before committing it. A dependency-only maintenance ch
34
34
 
35
35
  When a named host-specific risk is not established by the Commonlib owner checks, run the relevant downstream workflow against an exact Self-hosted LiveSync ref which consumes the package. It installs the tarball produced from the selected Commonlib commit and runs LiveSync type checks, unit tests, plug-in and application builds, and CLI E2E. Real Obsidian E2E remains local-only and is appropriate when the named risk concerns actual plug-in composition, storage, UI, or platform behaviour. These checks are conditional on the risk; LiveSync adoption is not a general release prerequisite.
36
36
 
37
+ ## Coordinating package and application releases
38
+
39
+ A downstream release which imports a new package entry point or depends on new
40
+ behaviour must use a published immutable Commonlib version. Prepare and test
41
+ consumer branches against an exact local pack while the package is unreleased,
42
+ but retain those results as local-candidate evidence.
43
+
44
+ 1. Complete the Commonlib owner checks and review the release source. For a
45
+ stable package, the exact reviewed release commit must be on `main` before
46
+ staging, as required by the trusted workflow below. An optional prerelease
47
+ can select a reviewed draft branch instead.
48
+ 2. Stage the exact package to `next`, validate the published registry artefact,
49
+ and complete any specifically required downstream checks before promoting
50
+ that same version to `latest`. These remain separate maintainer operations.
51
+ 3. Update each releasing consumer to the exact validated version and registry
52
+ integrity. Self-hosted LiveSync records one root dependency for its plug-in,
53
+ CLI, WebApp, and WebPeer; Bridge records its own Deno import map and lockfile.
54
+ Verify clean installation and the affected consumer boundaries before their
55
+ release workflows run.
56
+
57
+ Commonlib publication does not require every consumer branch to be merged.
58
+ Conversely, publishing Commonlib does not release an application or migrate an
59
+ existing database. A source checkout with the same version label as a registry
60
+ package is not a substitute for that immutable registry artefact.
61
+
37
62
  ## Version compatibility policy
38
63
 
39
64
  The Commonlib 0.2 policy begins with 0.2.0 and does not retroactively change compatibility expectations for published 0.1.x versions. Within 0.2.x, preserve published APIs, types, behaviour, supported persisted data, and exported `compat/*` paths. Intentional breaking changes belong in 0.3.0. Consumers may declare a range such as `^0.2.0`; lockfiles and `npm ci` remain the reproducible install path.