@vrtmrz/livesync-commonlib 0.3.0 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +27 -4
- package/dist/API/DirectFileManipulatorV2.d.ts +1 -0
- package/dist/API/DirectFileManipulatorV2.js +6 -12
- package/dist/API/processSetting.d.ts +1 -0
- package/dist/common/models/setting.const.defaults.js +1 -0
- package/dist/common/models/setting.const.qr.js +1 -0
- package/dist/common/models/setting.policy.d.ts +4 -0
- package/dist/common/models/setting.policy.js +1 -0
- package/dist/common/models/setting.type.d.ts +2 -0
- package/dist/common/models/tweak.definition.d.ts +1 -0
- package/dist/common/pathNamespace.d.ts +4 -0
- package/dist/common/pathNamespace.js +10 -0
- package/dist/common/settingConstants.d.ts +1 -0
- package/dist/common/settingConstants.js +4 -0
- package/dist/platform/AmbiguousNormalisedPathError.d.ts +16 -0
- package/dist/platform/AmbiguousNormalisedPathError.js +21 -0
- package/dist/platform/StoragePathResolver.d.ts +15 -0
- package/dist/platform/StoragePathResolver.js +113 -0
- package/dist/platform/browser/UnicodeResolvingFileSystemAccessStorageAdapter.d.ts +74 -0
- package/dist/platform/browser/UnicodeResolvingFileSystemAccessStorageAdapter.js +169 -0
- package/dist/platform/browser/index.d.ts +1 -0
- package/dist/platform/browser/index.js +2 -0
- package/dist/platform/browser/storage.d.ts +23 -0
- package/dist/platform/browser/storage.js +138 -0
- package/dist/platform/node/UnicodeResolvingNodeStorageAdapter.d.ts +79 -0
- package/dist/platform/node/UnicodeResolvingNodeStorageAdapter.js +218 -0
- package/dist/platform/node/index.d.ts +1 -0
- package/dist/platform/node/index.js +2 -0
- package/dist/platform/node/storage.d.ts +30 -1
- package/dist/platform/node/storage.js +88 -0
- package/dist/platform/pathComparison.d.ts +53 -0
- package/dist/platform/pathComparison.js +39 -0
- package/dist/platform/pathTypes.d.ts +154 -0
- package/dist/platform/pathTypes.js +0 -0
- package/dist/platform/storagePathResolution.d.ts +13 -0
- package/dist/platform/storagePathResolution.js +9 -0
- package/dist/replication/SyncParamsHandler.d.ts +1 -0
- package/dist/replication/SyncParamsHandler.js +5 -1
- package/dist/replication/couchdb/LiveSyncReplicator.js +5 -3
- package/dist/replication/journal/JournalReceiveFailure.d.ts +7 -0
- package/dist/replication/journal/JournalReceiveFailure.js +16 -0
- package/dist/replication/journal/JournalSyncCore.d.ts +22 -3
- package/dist/replication/journal/JournalSyncCore.js +288 -166
- package/dist/replication/journal/JournalSyncTypes.d.ts +13 -0
- package/dist/replication/journal/LiveSyncJournalReplicator.d.ts +6 -1
- package/dist/replication/journal/LiveSyncJournalReplicator.js +31 -44
- package/dist/replication/journal/journalCompatibility.d.ts +9 -0
- package/dist/replication/journal/journalCompatibility.js +58 -0
- package/dist/replication/journal/journalReceive.d.ts +23 -0
- package/dist/replication/journal/journalReceive.js +44 -0
- package/dist/replication/journal/journalRecords.d.ts +10 -0
- package/dist/replication/journal/journalRecords.js +96 -0
- package/dist/replication/trystero/TrysteroReplicator.d.ts +2 -0
- package/dist/serviceModules/Rebuilder.js +2 -1
- package/dist/services/base/FileProcessingService.d.ts +2 -2
- package/dist/services/base/RemoteConnection.d.ts +2 -0
- package/dist/services/base/RemoteService.d.ts +1 -1
- package/dist/services/base/RemoteService.js +13 -5
- package/dist/services/implements/injectable/InjectableVaultService.d.ts +1 -1
- package/dist/services/lib/couchdbHttpCache.d.ts +5 -0
- package/dist/services/lib/couchdbHttpCache.js +36 -0
- package/dist/storage.d.ts +4 -0
- package/dist/storage.js +19 -0
- package/dist/string_and_binary/path.js +4 -7
- package/dist/worker/bgWorker.js +1 -1
- package/docs/adr/2026_11_shared_storage_path_resolution.md +75 -0
- package/docs/design_docs/storage_path_resolution.md +306 -0
- package/docs/development.md +11 -1
- package/docs/journal-receive-recovery.md +66 -0
- package/docs/platform-storage.md +61 -0
- package/docs/releasing.md +25 -0
- package/docs/unicode-path-identity.md +49 -17
- package/package.json +9 -1
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
---
|
|
2
|
+
date: 2026-11-03
|
|
3
|
+
commonlib-version: "0.3.0"
|
|
4
|
+
self-hosted-livesync-version: "1.0.44"
|
|
5
|
+
status: published
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Architectural Decision Record: Shared Storage Path Resolution
|
|
9
|
+
|
|
10
|
+
## Conclusion
|
|
11
|
+
|
|
12
|
+
Commonlib owns strict Unicode path comparison and resolution behind a focused
|
|
13
|
+
`/storage` entry. Node and File System Access provide thin storage integrations;
|
|
14
|
+
hosts retain filesystem handles, original paths, caches, watchers, and operation
|
|
15
|
+
lifecycle. The [design](../design_docs/storage_path_resolution.md) defines the
|
|
16
|
+
accepted contract and remaining adoption work.
|
|
17
|
+
|
|
18
|
+
Accepted on 2026-11-03 against the versions recorded above. The implementation
|
|
19
|
+
is published in [Commonlib 0.3.1](https://www.npmjs.com/package/@vrtmrz/livesync-commonlib/v/0.3.1).
|
|
20
|
+
The original date and reviewed versions retain the accepted baseline.
|
|
21
|
+
Publication does not by itself enable NFC in WebApp or Bridge.
|
|
22
|
+
|
|
23
|
+
## Context
|
|
24
|
+
|
|
25
|
+
The Self-hosted LiveSync CLI already resolves NFC-equivalent names component by
|
|
26
|
+
component while preserving physical spelling. Repeating that logic in browser
|
|
27
|
+
and Deno hosts would duplicate ambiguity, missing-path, and rename rules.
|
|
28
|
+
Meanwhile, the legacy path union includes comparison keys, so removing a
|
|
29
|
+
namespace can appear to turn a lossy comparison value back into a file path.
|
|
30
|
+
|
|
31
|
+
## Decisions
|
|
32
|
+
|
|
33
|
+
- Separate original document paths, unprefixed file paths, NFC refinements,
|
|
34
|
+
resolved storage paths, and comparison keys. Preserve the old public types;
|
|
35
|
+
new namespace helpers exclude their legacy comparison-key member.
|
|
36
|
+
- Inject only strict directory enumeration into the resolver. Return `null`
|
|
37
|
+
for absence, and propagate permission, non-directory, and other I/O failures.
|
|
38
|
+
Resolve ambiguity before mutation, including when an exact spelling exists.
|
|
39
|
+
- Copy comparison settings for one operation or scan. Retain no cross-operation
|
|
40
|
+
directory cache or persistent path mapping. Scan already resolved entries
|
|
41
|
+
once per directory, and re-resolve later lookups after external changes.
|
|
42
|
+
- Preserve ordinary writes' physical spelling. An explicit rename may request
|
|
43
|
+
another spelling of the same leaf. Mutation and replacement semantics remain
|
|
44
|
+
with the storage owner; resolution is not an atomic reservation.
|
|
45
|
+
- Keep string-taking storage APIs and Legacy behaviour. New physical-I/O ports
|
|
46
|
+
take `StoragePath`. Namespace routing and cryptographic document-ID derivation
|
|
47
|
+
remain separate from filesystem resolution.
|
|
48
|
+
- Adopt the shared implementation in the CLI and the shared comparison and
|
|
49
|
+
namespace helpers at applicable Obsidian boundaries. WebApp and Bridge NFC
|
|
50
|
+
enablement remains separate work, including admission, events, and caches.
|
|
51
|
+
|
|
52
|
+
## Alternatives considered
|
|
53
|
+
|
|
54
|
+
- Duplicating the CLI resolver per host would duplicate the behavioural contract
|
|
55
|
+
and its failure handling.
|
|
56
|
+
- Renaming all physical files to NFC would alter user-owned storage and require
|
|
57
|
+
collision recovery beyond the accepted scope.
|
|
58
|
+
- A persistent alias map or shared directory cache would introduce invalidation,
|
|
59
|
+
restart, and lifecycle obligations which current directory reads avoid.
|
|
60
|
+
- Combining namespace, Unicode form, and case in a family of filename types
|
|
61
|
+
would conflate independent properties. Comparison keys instead name their
|
|
62
|
+
purpose and their storage or document domain.
|
|
63
|
+
|
|
64
|
+
## Consequences and verification
|
|
65
|
+
|
|
66
|
+
Hosts supply trustworthy complete listings and preserve distinct names before
|
|
67
|
+
constructing keyed collections. Filesystems or APIs which hide aliases limit
|
|
68
|
+
what the resolver can observe. Host concurrency and permission checks remain
|
|
69
|
+
necessary after resolution.
|
|
70
|
+
|
|
71
|
+
Contract tests cover spelling, ambiguity, strict failures, snapshots, scans,
|
|
72
|
+
and renames. Packed-consumer tests check assignability and platform bundling.
|
|
73
|
+
The downstream CLI retains its behavioural suite against the exact packed
|
|
74
|
+
candidate. Memory-backed File System Access tests establish adapter behaviour;
|
|
75
|
+
real-browser permissions and Deno integration require their own host evidence.
|
|
@@ -0,0 +1,306 @@
|
|
|
1
|
+
---
|
|
2
|
+
date: 2026-11-03
|
|
3
|
+
commonlib-version: "0.3.0"
|
|
4
|
+
self-hosted-livesync-version: "1.0.44"
|
|
5
|
+
status: published
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Shared storage path resolution
|
|
9
|
+
|
|
10
|
+
This design moves the CLI's existing Unicode-aware path resolution into
|
|
11
|
+
Commonlib so Node, File System Access, and Bridge can apply the same naming
|
|
12
|
+
rules. The shared resolver, path types, and Node and File System Access
|
|
13
|
+
integrations are published in Commonlib 0.3.1. Installing the package does
|
|
14
|
+
not enable NFC in an existing host or migrate its database.
|
|
15
|
+
|
|
16
|
+
Accepted on 2026-11-03. The [decision record](../adr/2026_11_shared_storage_path_resolution.md)
|
|
17
|
+
records the ownership and compatibility choices. The [platform guide](../platform-storage.md#unicode-path-resolution)
|
|
18
|
+
provides consumer examples.
|
|
19
|
+
|
|
20
|
+
The baseline is Commonlib 0.3.0 and the CLI implementation reviewed in
|
|
21
|
+
[Self-hosted LiveSync #1309](https://github.com/vrtmrz/obsidian-livesync/pull/1309)
|
|
22
|
+
at `2716d68b3d21d493cdfdefa0857fad4c6a3c522a`. The existing
|
|
23
|
+
[Unicode identity contract](../unicode-path-identity.md) continues to govern
|
|
24
|
+
Metadata IDs, remote admission, defaults, and database migration.
|
|
25
|
+
|
|
26
|
+
## Ownership
|
|
27
|
+
|
|
28
|
+
Commonlib owns path comparison, component-by-component resolution, ambiguity
|
|
29
|
+
checks, and thin platform-storage integrations. Hosts own root selection,
|
|
30
|
+
permissions, filesystem handles, file objects, watchers, caches, and operation
|
|
31
|
+
lifecycle. A resolver performs directory reads but no filesystem mutation.
|
|
32
|
+
|
|
33
|
+
Keep the existing Node and File System Access storage APIs and their Legacy
|
|
34
|
+
behaviour. Compose the resolver only for NFC mode. Its comparison policy is
|
|
35
|
+
an immutable snapshot for one operation or scan, copied at construction;
|
|
36
|
+
a settings change requires a new composition. Explicit case-sensitive and
|
|
37
|
+
case-insensitive lookups use
|
|
38
|
+
the corresponding snapshot, without mutating a shared resolver.
|
|
39
|
+
|
|
40
|
+
## Path types
|
|
41
|
+
|
|
42
|
+
Separate namespace, Unicode form, comparison use, and storage resolution.
|
|
43
|
+
An actual filename can already be NFC, so these are not mutually exclusive
|
|
44
|
+
string categories.
|
|
45
|
+
|
|
46
|
+
| Type | Meaning | Producer |
|
|
47
|
+
| --- | --- | --- |
|
|
48
|
+
| `FilePath` | Original spelling without a Metadata namespace; no NFC or resolution guarantee. | Existing input and host boundaries. |
|
|
49
|
+
| `DocumentPath` | Original spelling with an optional recognised Metadata namespace; excludes comparison keys. | Document input and namespace helpers. |
|
|
50
|
+
| `NfcFilePath` | A `FilePath` normalised to NFC, without lower-casing. | The shared NFC normalisation function. |
|
|
51
|
+
| `StoragePath` | A root-relative storage target whose existing components use their resolved spelling. | Resolver or supported physical enumeration. |
|
|
52
|
+
| `PathComparisonKey<"storage">` | A comparison key for a path within one storage root. | The shared comparison helper. |
|
|
53
|
+
| `PathComparisonKey<"document">` | A comparison key which retains the document namespace. | The shared comparison helper. |
|
|
54
|
+
|
|
55
|
+
`StoragePath` is not proof that an entry exists, still exists, is a file,
|
|
56
|
+
or cannot change concurrently. A write target may contain missing components.
|
|
57
|
+
The root is the empty path; each operation retains its existing root rules.
|
|
58
|
+
|
|
59
|
+
Reuse the existing `FilePath` and OW `TaggedType` definitions. Add independent
|
|
60
|
+
symbols for refinements: nesting `TaggedType<FilePath, "StoragePath">` would
|
|
61
|
+
intersect incompatible values of OW's existing tag property.
|
|
62
|
+
|
|
63
|
+
```ts
|
|
64
|
+
declare const unicodeForm: unique symbol;
|
|
65
|
+
declare const storagePath: unique symbol;
|
|
66
|
+
|
|
67
|
+
type DocumentPath = Exclude<FilePathWithPrefix, FilePathWithPrefixLC>;
|
|
68
|
+
|
|
69
|
+
type NfcFilePath = FilePath & {
|
|
70
|
+
readonly [unicodeForm]: "NFC";
|
|
71
|
+
};
|
|
72
|
+
|
|
73
|
+
type StoragePath = FilePath & {
|
|
74
|
+
readonly [storagePath]: true;
|
|
75
|
+
};
|
|
76
|
+
|
|
77
|
+
type PathComparisonKey<Domain extends "storage" | "document"> =
|
|
78
|
+
TaggedType<string, `PathComparisonKey:${Domain}`>;
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Use `NfcFilePath` within the normalisation boundary initially. Publish the
|
|
82
|
+
path and key types needed by consumers of the new focused API. Preserve the
|
|
83
|
+
existing definitions of `FilePathWithPrefix` and `FilePathWithPrefixLC` at
|
|
84
|
+
compatibility entry points. Do not alias either old type to a new key type:
|
|
85
|
+
that would change existing assignability.
|
|
86
|
+
|
|
87
|
+
The old `FilePathWithPrefix` union includes both `FilePath` and
|
|
88
|
+
`FilePathWithPrefixLC`. The new `DocumentPath` intentionally removes the latter.
|
|
89
|
+
The old scanner's `convertCase()` returns its `LC` type even when case-sensitive
|
|
90
|
+
handling retains letter case. New code therefore names its output by its
|
|
91
|
+
comparison purpose, not an assumed lower-case form.
|
|
92
|
+
|
|
93
|
+
Brand values at the boundary which establishes their meaning. `parseFilePath`
|
|
94
|
+
validates raw storage input using the existing path vocabulary without changing
|
|
95
|
+
its spelling. `parseDocumentPath` accepts original document-path strings;
|
|
96
|
+
namespace routing precedes storage validation. Neither input helper recovers
|
|
97
|
+
original spelling from a comparison key. NFC conversion and resolution establish their own refinements.
|
|
98
|
+
Keep casts inside those owning implementations; consumer code receives the
|
|
99
|
+
typed result. The existing string-taking APIs remain compatibility escape
|
|
100
|
+
routes, so this migration does not claim to make all old call sites type-safe.
|
|
101
|
+
Within the new integrations, physical-I/O ports accept `StoragePath`; NFC
|
|
102
|
+
normalisation or key generation alone cannot supply that argument.
|
|
103
|
+
|
|
104
|
+
## Namespaces and comparison
|
|
105
|
+
|
|
106
|
+
Namespace helpers accept `DocumentPath`, return a separate namespace and
|
|
107
|
+
`FilePath`, and recognise the existing `i:`, `ix:`, and `ps:` rules. Split one
|
|
108
|
+
namespace only; further colons belong to the body. The caller handles the
|
|
109
|
+
namespace's meaning before submitting a filesystem path to the resolver.
|
|
110
|
+
Chunk prefixes and obfuscated IDs remain `DocumentID` concerns.
|
|
111
|
+
|
|
112
|
+
Document comparison preserves the namespace; storage comparison does not
|
|
113
|
+
receive one. A normal document and an internal document with the same body
|
|
114
|
+
remain distinct document identities. Neither comparison-key domain is accepted
|
|
115
|
+
by the resolver or by the new namespace helpers.
|
|
116
|
+
|
|
117
|
+
The comparison policy uses the existing `normalisePathForIdentity` operation,
|
|
118
|
+
then the existing `toLowerCase()` rule only when configured. NFC conversion
|
|
119
|
+
alone produces `NfcFilePath`; the complete comparison pipeline produces a key.
|
|
120
|
+
It does not infer the host filesystem's case rules or introduce a new Unicode
|
|
121
|
+
case-folding rule. Preserve original paths alongside keys; a key is never
|
|
122
|
+
converted back into an original path.
|
|
123
|
+
|
|
124
|
+
```ts
|
|
125
|
+
type PathComparisonSettings = Readonly<{
|
|
126
|
+
pathNormalizationVersion: 0 | 1;
|
|
127
|
+
handleFilenameCaseSensitive: boolean;
|
|
128
|
+
}>;
|
|
129
|
+
|
|
130
|
+
type DocumentPathParts = Readonly<{
|
|
131
|
+
prefix: "" | "i:" | "ix:" | "ps:";
|
|
132
|
+
path: FilePath;
|
|
133
|
+
}>;
|
|
134
|
+
|
|
135
|
+
declare function splitDocumentPath(path: DocumentPath): DocumentPathParts;
|
|
136
|
+
declare function joinDocumentPath(parts: DocumentPathParts): DocumentPath;
|
|
137
|
+
|
|
138
|
+
declare function getStoragePathComparisonKey(
|
|
139
|
+
path: FilePath, settings: PathComparisonSettings
|
|
140
|
+
): PathComparisonKey<"storage">;
|
|
141
|
+
|
|
142
|
+
declare function getDocumentPathComparisonKey(
|
|
143
|
+
path: DocumentPath, settings: PathComparisonSettings
|
|
144
|
+
): PathComparisonKey<"document">;
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Namespace routing and cryptographic document-ID derivation remain outside the
|
|
148
|
+
resolver. The new comparison helpers must retain the existing key bytes for
|
|
149
|
+
each supported policy; extracting them does not rewrite IDs or stored paths.
|
|
150
|
+
|
|
151
|
+
## Directory and resolver contract
|
|
152
|
+
|
|
153
|
+
The only injected I/O operation enumerates immediate children using their
|
|
154
|
+
actual names and kinds. Paths are relative to the selected root and use `/`.
|
|
155
|
+
`null` means the directory is absent. Permission failures, a non-directory
|
|
156
|
+
parent, and other I/O failures reject; they are not empty listings.
|
|
157
|
+
|
|
158
|
+
The existing storage `list()` methods can hide some failures as empty results.
|
|
159
|
+
Provide a strict enumeration path for the resolver while preserving the old
|
|
160
|
+
methods' published behaviour. Reuse each platform's containment and symbolic-link
|
|
161
|
+
checks. Keep native handles and platform stat types behind this boundary.
|
|
162
|
+
|
|
163
|
+
```ts
|
|
164
|
+
type DirectoryEntry = Readonly<{
|
|
165
|
+
name: string;
|
|
166
|
+
kind: "file" | "directory";
|
|
167
|
+
}>;
|
|
168
|
+
|
|
169
|
+
type ReadDirectory = (
|
|
170
|
+
physicalDirectory: StoragePath
|
|
171
|
+
) => Promise<readonly DirectoryEntry[] | null>;
|
|
172
|
+
|
|
173
|
+
type StorageEntry = Readonly<{
|
|
174
|
+
path: StoragePath;
|
|
175
|
+
kind: "file" | "directory";
|
|
176
|
+
}>;
|
|
177
|
+
|
|
178
|
+
type ResolvedRename = Readonly<{
|
|
179
|
+
source: StoragePath;
|
|
180
|
+
target: StoragePath;
|
|
181
|
+
}>;
|
|
182
|
+
|
|
183
|
+
interface StoragePathResolver {
|
|
184
|
+
resolveExisting(path: FilePath): Promise<StoragePath | null>;
|
|
185
|
+
resolveForWrite(path: FilePath): Promise<StoragePath>;
|
|
186
|
+
resolveRename(source: FilePath, target: FilePath): Promise<ResolvedRename | null>;
|
|
187
|
+
listResolvedDirectory(path: StoragePath): Promise<readonly StorageEntry[] | null>;
|
|
188
|
+
}
|
|
189
|
+
|
|
190
|
+
declare function createStoragePathResolver(options: Readonly<{
|
|
191
|
+
readDirectory: ReadDirectory;
|
|
192
|
+
policy: Readonly<{
|
|
193
|
+
pathNormalizationVersion: 1;
|
|
194
|
+
handleFilenameCaseSensitive: boolean;
|
|
195
|
+
}>;
|
|
196
|
+
}>): StoragePathResolver;
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
`DirectoryEntry.name` is an immediate child name, never a path. Reject malformed
|
|
200
|
+
entries at the enumeration boundary. Add a separate filename brand only when a
|
|
201
|
+
consumer boundary needs it; do not publish a family of filename variants now.
|
|
202
|
+
|
|
203
|
+
The resolver uses complete sibling enumeration before deciding uniqueness;
|
|
204
|
+
a literal match does not override a second equivalent name. File/directory
|
|
205
|
+
collisions count as ambiguity. A unique file cannot be traversed as a parent.
|
|
206
|
+
An individual lookup checks candidates for each requested component; a full
|
|
207
|
+
scan checks every sibling group before returning that directory's entries.
|
|
208
|
+
|
|
209
|
+
| Operation | Result |
|
|
210
|
+
| --- | --- |
|
|
211
|
+
| `resolveExisting` | Resolve each existing component to its listed spelling; return `null` on absence. |
|
|
212
|
+
| `resolveForWrite` | Preserve listed names for existing components and the requested spelling for the missing suffix. |
|
|
213
|
+
| `resolveRename` | Resolve both ends. If the target resolves to the source, retain the resolved parent and the explicitly requested leaf spelling. Return `null` when the source is absent. |
|
|
214
|
+
| `listResolvedDirectory` | Read an already resolved directory, check all sibling identities, and return actual child paths for scanning and direct access. |
|
|
215
|
+
|
|
216
|
+
All operations reject ambiguity using the shared `AmbiguousNormalisedPathError`.
|
|
217
|
+
Include the requested path and conflicting actual names for diagnosis. Preserve
|
|
218
|
+
I/O failures separately from absence. Host catch blocks must propagate these
|
|
219
|
+
failures instead of converting them to a missing file or a successful partial
|
|
220
|
+
scan.
|
|
221
|
+
|
|
222
|
+
Normal writes preserve existing Unicode spelling. Explicit renames may change
|
|
223
|
+
it. Existing-destination replacement policy and the actual rename, write,
|
|
224
|
+
rollback, or deletion remain the storage owner's responsibility. Resolution is
|
|
225
|
+
not an atomic reservation; existing host operation ordering and platform
|
|
226
|
+
protections remain necessary. The guarantee is limited to names exposed by the
|
|
227
|
+
host's supported listing.
|
|
228
|
+
|
|
229
|
+
## Composition and adoption
|
|
230
|
+
|
|
231
|
+
The `/storage` entry contains the host-neutral types, comparison and
|
|
232
|
+
namespace helpers, and resolver. It imports no Node, DOM, database, or Service
|
|
233
|
+
Hub runtime. `/node` and `/browser` expose their platform integrations through
|
|
234
|
+
the existing bundling boundaries.
|
|
235
|
+
|
|
236
|
+
Commonlib owns `UnicodeResolvingNodeStorageAdapter` and delegates
|
|
237
|
+
its resolution to the common implementation. The current CLI candidate adopts
|
|
238
|
+
the focused API while retaining Legacy delegation and thin compatibility
|
|
239
|
+
methods. File System Access uses the same resolver, including host paths which
|
|
240
|
+
operate directly on handles. Bridge supplies a Deno directory reader and uses
|
|
241
|
+
the same contract in an unreleased integration candidate.
|
|
242
|
+
|
|
243
|
+
Keep the resolver free of persistent mappings, cross-operation caches, watchers,
|
|
244
|
+
retry owners, and disposal requirements. A full scan descends through the
|
|
245
|
+
resolved entries and reads each directory once. A later lookup re-reads the
|
|
246
|
+
relevant directories, including after a file-object cache hit. OW Scope or
|
|
247
|
+
context can own host operation resources where needed; the resolver itself
|
|
248
|
+
requires no resource scope.
|
|
249
|
+
|
|
250
|
+
Hosts retain original physical paths alongside any comparison keys. Detect
|
|
251
|
+
aliases before collapsing them into a keyed collection. Cache invalidation,
|
|
252
|
+
queue ownership, and watch-event reconciliation must follow the same comparison
|
|
253
|
+
policy without losing distinct names. A deletion event may name an entry which
|
|
254
|
+
is already absent; preserve that event path rather than trying to recover its
|
|
255
|
+
spelling from a live directory lookup. Check rename events as an ordered change
|
|
256
|
+
of state, including hosts which report them as deletion and creation.
|
|
257
|
+
|
|
258
|
+
## Current adoption and publication status
|
|
259
|
+
|
|
260
|
+
The shared `/storage` contract, Node and File System Access adapters, and
|
|
261
|
+
explicit `DirectFileManipulator` path-version option are published in
|
|
262
|
+
[Commonlib 0.3.1](https://www.npmjs.com/package/@vrtmrz/livesync-commonlib/v/0.3.1).
|
|
263
|
+
The original decision date and reviewed versions above retain the accepted
|
|
264
|
+
baseline. These additions are absent from the registry artefact `0.3.0`;
|
|
265
|
+
a source checkout carrying that version label is not the published artefact.
|
|
266
|
+
|
|
267
|
+
The CLI candidate uses the shared Node resolver, and the current Obsidian
|
|
268
|
+
candidate uses shared namespace and comparison helpers while retaining Vault
|
|
269
|
+
APIs and its alias guard. The matching Journal and P2P host adoption remains
|
|
270
|
+
unreleased in Self-hosted LiveSync [#1309](https://github.com/vrtmrz/obsidian-livesync/pull/1309).
|
|
271
|
+
The browser resolver is a general primitive, not WebApp NFC support: WebApp
|
|
272
|
+
remains Legacy-only and rejects an explicit version `1` at startup. WebPeer
|
|
273
|
+
has no Vault filesystem and is not an NFC filesystem host.
|
|
274
|
+
|
|
275
|
+
Bridge's Deno integration is also an unreleased candidate. NFC mode is an
|
|
276
|
+
explicit per-peer opt-in, and every peer in a group must use the same path
|
|
277
|
+
version. Its ordinary case-mismatch check remains enabled; the Bridge-wide
|
|
278
|
+
`disableCheckingConfigMismatch: true` override disables that case check only
|
|
279
|
+
and does not bypass path-version admission. Preserve the existing coordinated
|
|
280
|
+
Rebuild/Fetch migration and remote mismatch protection. This design adds no
|
|
281
|
+
automatic migration or translation between identity modes.
|
|
282
|
+
|
|
283
|
+
## Verification
|
|
284
|
+
|
|
285
|
+
Check the public declarations against the actual published path types and OW
|
|
286
|
+
tag implementation. Positive cases cover source paths, namespace round trips,
|
|
287
|
+
and independent NFC/storage refinements. Negative cases cover raw strings,
|
|
288
|
+
prefix-bearing paths at the resolver, comparison keys at resolution and namespace
|
|
289
|
+
boundaries, and mixing the two key domains. Retain an assertion that the legacy
|
|
290
|
+
`LC` type is still assignable to the legacy union, but not to `DocumentPath`.
|
|
291
|
+
|
|
292
|
+
Implementation must preserve the existing CLI behavioural cases and apply one
|
|
293
|
+
resolver contract to Node and File System Access, with Bridge-specific checks:
|
|
294
|
+
|
|
295
|
+
- NFD files and mixed-spelling parents remain addressable without implicit renames.
|
|
296
|
+
- New components retain the requested spelling under the actual parent.
|
|
297
|
+
- File, directory, and case collisions reject before mutation, including exact-match requests.
|
|
298
|
+
- Permission and listing failures do not become absence, creation, deletion, or partial-scan success.
|
|
299
|
+
- Explicit spelling renames preserve contents and the host's replacement and recovery semantics.
|
|
300
|
+
- External additions, deletions, renames, and collisions are observed after earlier cache hits.
|
|
301
|
+
- Full scans inspect each directory once; Legacy behaviour and namespace separation remain unchanged.
|
|
302
|
+
- Watcher deletion/creation sequences, restart state, and cache keys preserve the intended identity.
|
|
303
|
+
|
|
304
|
+
Packed-package checks must verify the focused entry and platform bundling
|
|
305
|
+
boundaries. Real browser and Deno checks establish their host behaviour;
|
|
306
|
+
Commonlib's memory and Node tests alone do not establish those guarantees.
|
package/docs/development.md
CHANGED
|
@@ -6,7 +6,7 @@ This document is for Commonlib developers. Package consumers should begin with t
|
|
|
6
6
|
|
|
7
7
|
The source tree is larger than the supported package surface. Consumers may import only paths in the generated package export map:
|
|
8
8
|
|
|
9
|
-
- `src/index.ts`, `src/context.ts`, `src/settings.ts`, `src/remoteConfigurations.ts`, `src/p2p/index.ts`, `src/platform/browser/index.ts`, and `src/platform/node/index.ts` define the focused entries;
|
|
9
|
+
- `src/index.ts`, `src/context.ts`, `src/storage.ts`, `src/settings.ts`, `src/remoteConfigurations.ts`, `src/p2p/index.ts`, `src/platform/browser/index.ts`, and `src/platform/node/index.ts` define the focused entries;
|
|
10
10
|
- `src/conflicts.ts` defines the host-owned conflict-check queue contract; `src/concurrency/keyedPendingWork.ts` remains package-private;
|
|
11
11
|
- `src/replication/index.ts` defines the focused provider-capability and interaction contract for hosts which compose replication;
|
|
12
12
|
- `src/diagnostics/index.ts` defines read-only Usage and integrity inspection, report projection, and formatting; hosts own the run dialogue and output destination;
|
|
@@ -19,6 +19,16 @@ The source tree is larger than the supported package surface. Consumers may impo
|
|
|
19
19
|
|
|
20
20
|
Files elsewhere under `src` are implementation details unless a focused entry re-exports them or the compatibility inventory names them. Do not use the source layout, TypeScript path aliases, or the presence of generated declarations as evidence of a supported consumer import.
|
|
21
21
|
|
|
22
|
+
## Storage path boundary
|
|
23
|
+
|
|
24
|
+
The `/storage` entry, published in Commonlib 0.3.1, owns namespace helpers,
|
|
25
|
+
comparison keys, and strict component resolution. The [design](design_docs/storage_path_resolution.md)
|
|
26
|
+
and [decision record](adr/2026_11_shared_storage_path_resolution.md) separate
|
|
27
|
+
that contract from host handles, caches, watchers, and migration. See the
|
|
28
|
+
[platform guide](platform-storage.md#unicode-path-resolution) for composition.
|
|
29
|
+
The [Unicode identity contract](unicode-path-identity.md) retains its existing
|
|
30
|
+
Metadata, remote-admission, and database-migration rules.
|
|
31
|
+
|
|
22
32
|
## Module filenames
|
|
23
33
|
|
|
24
34
|
For new or renamed TypeScript and JavaScript source modules, start the filename
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
---
|
|
2
|
+
date: 2026-11-04
|
|
3
|
+
commonlib-version: "0.3.1"
|
|
4
|
+
self-hosted-livesync-version: "1.0.44"
|
|
5
|
+
status: unreleased
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Journal receive failures and recovery
|
|
9
|
+
|
|
10
|
+
This document describes unreleased receive behaviour. Hosts choose when to
|
|
11
|
+
offer the manual skip operation and remain responsible for Vault reflection
|
|
12
|
+
and conflict handling.
|
|
13
|
+
|
|
14
|
+
## Stop and retry by default
|
|
15
|
+
|
|
16
|
+
A per-Journal decode, record-processing, or write failure stops receipt. The
|
|
17
|
+
failed Journal and all later Journals remain unacknowledged. The next normal
|
|
18
|
+
receive attempt retries the failed Journal before proceeding to later Journals.
|
|
19
|
+
Partial progress does not change this rule.
|
|
20
|
+
|
|
21
|
+
Chunks are written before Metadata, so a Chunk failure prevents that Journal's
|
|
22
|
+
Metadata stage. Metadata retains its received revision ancestry through
|
|
23
|
+
`new_edits: false`, whose empty success result is valid. Successfully written
|
|
24
|
+
records remain in the database when another record fails. There is no rollback.
|
|
25
|
+
|
|
26
|
+
A retry avoids rewriting existing records and delivers accepted Metadata to
|
|
27
|
+
the host again, including revisions saved before an earlier partial failure.
|
|
28
|
+
The host must tolerate repeated delivery. Chunk-arrival events describe only
|
|
29
|
+
successful writes. The `knownIDs` set suppresses unnecessary
|
|
30
|
+
outbound transfer; it does not prove that a receive has completed locally.
|
|
31
|
+
|
|
32
|
+
Warnings identify the failed Journal key and the stage where receipt stopped,
|
|
33
|
+
without logging document contents. A stop request lets current I/O settle and
|
|
34
|
+
closes the receive operation; it is not a Journal failure.
|
|
35
|
+
|
|
36
|
+
## Skip the stopped Journal and resume
|
|
37
|
+
|
|
38
|
+
When receipt stops on a Journal failure, a host may offer the explicit
|
|
39
|
+
'Skip stopped Journal and resume' operation. It accepts the exact stopped
|
|
40
|
+
failure snapshot, marks only that Journal received, and resumes receipt of the
|
|
41
|
+
remaining Journals. If another Journal fails, receipt stops at that Journal
|
|
42
|
+
and requires a separate decision.
|
|
43
|
+
|
|
44
|
+
Skipping does not roll back partial writes. It can leave content or revision
|
|
45
|
+
history missing, and receipt of later Journals does not prove that every file
|
|
46
|
+
can be reconstructed or reflected to storage. Each skip acknowledges the
|
|
47
|
+
failed Journal despite its warning.
|
|
48
|
+
|
|
49
|
+
The stopped failure is runtime state, not a persistent skip mode or failure
|
|
50
|
+
queue. After a restart, normal receipt must encounter the failure again before
|
|
51
|
+
the host can offer a skip for it.
|
|
52
|
+
|
|
53
|
+
## Replay and host boundaries
|
|
54
|
+
|
|
55
|
+
The existing 'Reset journal received history' operation clears `receivedFiles`
|
|
56
|
+
and `knownIDs`, while retaining the local database and `sentFiles`. After the
|
|
57
|
+
failure has been corrected, resetting this history lets receipt replay remote
|
|
58
|
+
Journals other than those sent by this device. Replay requires the affected
|
|
59
|
+
remote objects to remain available and readable; resetting history cannot
|
|
60
|
+
recreate missing remote bytes.
|
|
61
|
+
|
|
62
|
+
A host operation which replaces the local database, such as CLI Fetch, has
|
|
63
|
+
separate reconstruction and completion rules. Repeating Fetch is not the
|
|
64
|
+
same-database retry described above. Journal receipt also remains separate
|
|
65
|
+
from Vault reflection and conflict handling; hosts must preserve their own
|
|
66
|
+
failure and content-protection checks.
|
package/docs/platform-storage.md
CHANGED
|
@@ -61,6 +61,67 @@ Writes create missing parent directories. Listing returns direct child paths rel
|
|
|
61
61
|
|
|
62
62
|
The File System Access API does not expose portable creation times or permit setting file times. Consumers must not rely on identical timestamp fidelity across the two adapters.
|
|
63
63
|
|
|
64
|
+
## Unicode path resolution
|
|
65
|
+
|
|
66
|
+
These APIs are published in [Commonlib 0.3.1](https://www.npmjs.com/package/@vrtmrz/livesync-commonlib/v/0.3.1).
|
|
67
|
+
Node and File System Access hosts can use the same NFC comparison and component
|
|
68
|
+
resolution while retaining the names listed by their filesystem. Hosts still
|
|
69
|
+
own the selected identity mode and its database migration.
|
|
70
|
+
|
|
71
|
+
```ts
|
|
72
|
+
import { UnicodeResolvingNodeStorageAdapter } from '@vrtmrz/livesync-commonlib/node';
|
|
73
|
+
import { parseFilePath } from '@vrtmrz/livesync-commonlib/storage';
|
|
74
|
+
|
|
75
|
+
const storage = new UnicodeResolvingNodeStorageAdapter(
|
|
76
|
+
'/srv/livesync',
|
|
77
|
+
() => settings.pathNormalizationVersion,
|
|
78
|
+
() => !settings.handleFilenameCaseSensitive,
|
|
79
|
+
);
|
|
80
|
+
const existing = await storage.resolveLogicalPath(parseFilePath('Café.md'), undefined, false);
|
|
81
|
+
if (existing !== null) {
|
|
82
|
+
const content = await storage.readResolved(existing);
|
|
83
|
+
}
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
The browser equivalent is `UnicodeResolvingFileSystemAccessStorageAdapter`
|
|
87
|
+
from `/browser`, constructed with an authorised root handle and the same
|
|
88
|
+
settings callbacks. Version `0` delegates to the existing storage behaviour;
|
|
89
|
+
version `1` resolves every component against complete sibling listings. Normal
|
|
90
|
+
writes preserve existing spelling. Node's explicit `rename` can change a leaf's
|
|
91
|
+
Unicode or letter-case spelling; the browser adapter retains its existing
|
|
92
|
+
operation set and does not add a rename primitive.
|
|
93
|
+
|
|
94
|
+
`AmbiguousNormalisedPathError` from `/storage` reports multiple equivalent
|
|
95
|
+
names, even when one is a literal match. Preserve that error and directory or
|
|
96
|
+
permission failures. Only absence produces `null`; a failure must not be used
|
|
97
|
+
as permission to create a file or as a successful partial scan.
|
|
98
|
+
|
|
99
|
+
For another host, compose `createStoragePathResolver({ readDirectory, policy })`
|
|
100
|
+
from `/storage`. `readDirectory` accepts a `StoragePath` and returns immediate
|
|
101
|
+
child names and kinds, or `null` for absence. Propagate other failures. The
|
|
102
|
+
resolver copies one NFC policy and retains no directory cache. For a full scan,
|
|
103
|
+
start at `resolveForWrite(parseFilePath(''))`, then recurse through
|
|
104
|
+
`listResolvedDirectory` entries using the same resolver. Each directory is read
|
|
105
|
+
once. Re-resolve later operations so external changes are visible.
|
|
106
|
+
|
|
107
|
+
`FilePath` retains original, unprefixed spelling. `StoragePath` records that
|
|
108
|
+
existing components have been resolved; it does not guarantee existence or
|
|
109
|
+
reserve a target. `parseDocumentPath` marks an original document-path input,
|
|
110
|
+
and `splitDocumentPath` separates one recognised namespace before host routing.
|
|
111
|
+
Use `joinDocumentPath` to restore it. `getStoragePathComparisonKey` and
|
|
112
|
+
`getDocumentPathComparisonKey` return distinct comparison-only types; the latter
|
|
113
|
+
preserves the namespace. Keep the original path beside each key. The old
|
|
114
|
+
`FilePathWithPrefix` and `FilePathWithPrefixLC` remain available for compatibility.
|
|
115
|
+
|
|
116
|
+
The [design](design_docs/storage_path_resolution.md) defines the detailed
|
|
117
|
+
contract. These primitives do not enable NFC automatically in a host
|
|
118
|
+
application. WebApp remains Legacy-only and rejects explicit version `1` at
|
|
119
|
+
startup. The Bridge Deno integration is an unreleased candidate; version `1`
|
|
120
|
+
is an explicit peer opt-in, and every peer in a group must use the same path
|
|
121
|
+
version. Its ordinary case-mismatch check remains separate from path-version
|
|
122
|
+
admission. See [Unicode path identity](unicode-path-identity.md#supported-hosts)
|
|
123
|
+
for the current publication and host-adoption boundaries.
|
|
124
|
+
|
|
64
125
|
## Bundling boundary
|
|
65
126
|
|
|
66
127
|
Browser consumers import only `@vrtmrz/livesync-commonlib/browser`. Packed-package tests build that entry with browser conditions and verify that the Node implementation is absent. Node consumers import `@vrtmrz/livesync-commonlib/node`; the root and context entries do not load Node built-ins.
|
package/docs/releasing.md
CHANGED
|
@@ -34,6 +34,31 @@ Inspect the lockfile diff before committing it. A dependency-only maintenance ch
|
|
|
34
34
|
|
|
35
35
|
When a named host-specific risk is not established by the Commonlib owner checks, run the relevant downstream workflow against an exact Self-hosted LiveSync ref which consumes the package. It installs the tarball produced from the selected Commonlib commit and runs LiveSync type checks, unit tests, plug-in and application builds, and CLI E2E. Real Obsidian E2E remains local-only and is appropriate when the named risk concerns actual plug-in composition, storage, UI, or platform behaviour. These checks are conditional on the risk; LiveSync adoption is not a general release prerequisite.
|
|
36
36
|
|
|
37
|
+
## Coordinating package and application releases
|
|
38
|
+
|
|
39
|
+
A downstream release which imports a new package entry point or depends on new
|
|
40
|
+
behaviour must use a published immutable Commonlib version. Prepare and test
|
|
41
|
+
consumer branches against an exact local pack while the package is unreleased,
|
|
42
|
+
but retain those results as local-candidate evidence.
|
|
43
|
+
|
|
44
|
+
1. Complete the Commonlib owner checks and review the release source. For a
|
|
45
|
+
stable package, the exact reviewed release commit must be on `main` before
|
|
46
|
+
staging, as required by the trusted workflow below. An optional prerelease
|
|
47
|
+
can select a reviewed draft branch instead.
|
|
48
|
+
2. Stage the exact package to `next`, validate the published registry artefact,
|
|
49
|
+
and complete any specifically required downstream checks before promoting
|
|
50
|
+
that same version to `latest`. These remain separate maintainer operations.
|
|
51
|
+
3. Update each releasing consumer to the exact validated version and registry
|
|
52
|
+
integrity. Self-hosted LiveSync records one root dependency for its plug-in,
|
|
53
|
+
CLI, WebApp, and WebPeer; Bridge records its own Deno import map and lockfile.
|
|
54
|
+
Verify clean installation and the affected consumer boundaries before their
|
|
55
|
+
release workflows run.
|
|
56
|
+
|
|
57
|
+
Commonlib publication does not require every consumer branch to be merged.
|
|
58
|
+
Conversely, publishing Commonlib does not release an application or migrate an
|
|
59
|
+
existing database. A source checkout with the same version label as a registry
|
|
60
|
+
package is not a substitute for that immutable registry artefact.
|
|
61
|
+
|
|
37
62
|
## Version compatibility policy
|
|
38
63
|
|
|
39
64
|
The Commonlib 0.2 policy begins with 0.2.0 and does not retroactively change compatibility expectations for published 0.1.x versions. Within 0.2.x, preserve published APIs, types, behaviour, supported persisted data, and exported `compat/*` paths. Intentional breaking changes belong in 0.3.0. Consumers may declare a range such as `^0.2.0`; lockfiles and `npm ci` remain the reproducible install path.
|