@vrtmrz/livesync-commonlib 0.1.40 → 0.1.42

package/docs/conflict-resolution.md

@@ -18,6 +18,8 @@ A1
 
 Resolving a conflict writes the selected or merged result on one branch and deletes every losing live leaf which the resolver has observed. Deleted leaves remain part of the tree until compaction removes their bodies. A stale client can therefore receive the resolved branch and the tombstone for a branch whose old content is still present in its storage.
 
+Metadata reads which explicitly include deleted entries also return a current PouchDB `_deleted` winner. An ordinary read still excludes deleted entries, and a request for an unavailable specific revision does not fall back to the current winner. A deleted-winner lookup rechecks the winning revision after fetching its body. If it changes, the lookup follows the new winner for at most three attempts and returns no entry if it cannot establish a stable revision. Incoming file reflection uses the current winner and rechecks it before deleting storage, so an older deletion notification does not override a subsequent restoration. Existing conflict and unsynchronised-edit protection still applies.
+
 ## Chunk reachability during conflicts
 
 A host which collects unreferenced chunks must include the current winner, every other live conflict leaf, their available divergent revisions, and their nearest available shared ancestor in its reachability scan. This preserves the content and merge base required to review an unresolved conflict. Chunk identifiers are shared across documents, so one reachability set must cover the whole database.
@@ -28,17 +30,17 @@ After the conflict is resolved, the deleted losing branch and no-longer-needed m
 
 Commonlib follows these rules:
 
-- compare file content byte-for-byte; a path, size, modification time, or revision generation is not proof of identity;
+- never treat a path, size, modification time, or revision generation as proof of content identity;
 - use the nearest `available` revision shared by both leaves as the base for three-way merge;
 - retain a manual conflict when the common revision or a required body is missing or compacted;
-- recognise an unchanged file by comparing its bytes with its exact readable file-reflection provenance;
+- recognise an unchanged file through its exact readable file-reflection revision or the saved SHA-256 hash and byte length of that snapshot;
 - save a genuine edit as a child of that recorded revision, even when a newer database winner exists;
 - deduplicate unknown-origin content against current non-deleted leaves, preserving differing content as a fresh independent root under the same document ID; and
 - never select the newest modification time as a package-level default.
 
 Ordinary saving and incoming reflection share this classification. A historical byte match alone cannot distinguish an unchanged stale file from an intentional revert, and a prefix or suffix relationship does not prove which version the user edited. Unknown-origin content must not acquire an inferred parent from the current winner or its parent.
 
-After a resolution has propagated, an exact recorded revision can still identify unchanged content below a deleted losing leaf. The resolved winner may replace those bytes under the existing reflection policy. If the recorded body is unavailable, the file follows the conservative unknown-origin path. General history-query helpers remain available for callers with separate contracts, including existing deletion and rename recovery; ordinary save and overwrite protection do not scan all historical bodies.
+After a resolution has propagated, an exact recorded revision can still identify unchanged content below a deleted losing leaf. The resolved winner may replace those bytes under the existing reflection policy. If neither the recorded body nor valid matching content evidence is available, the file follows the conservative unknown-origin path. General history-query helpers remain available for callers with separate contracts, including existing deletion and rename recovery; ordinary save and overwrite protection do not scan all historical bodies.
 
 ## Resolution classes
 
@@ -49,10 +51,11 @@ After a resolution has propagated, an exact recorded revision can still identify
 | One side deletes content which the other leaves unchanged                       | Preserve the deletion.                                                      |
 | One side deletes content which the other modifies                               | Retain a manual conflict.                                                   |
 | A receiving file matches its exact readable recorded revision                   | Apply the propagated result under the existing conflict policy.             |
+| A receiving file matches its saved content hash and byte length                 | Apply the propagated result under the existing conflict policy, even after its recorded revision is pruned. |
 | A receiving file differs from its readable recorded revision                    | Preserve the edit as a child of that revision.                              |
 | Origin is unknown and no current non-deleted leaf matches                        | Preserve a fresh independent root for host conflict resolution.             |
 | Origin is unknown and current non-deleted leaves already contain the bytes       | Avoid duplicate storage; infer provenance only for a unique match.          |
-| A body or common ancestor is missing or compacted                               | Retain a manual conflict.                                                   |
+| A required merge body or common ancestor is missing or compacted                | Retain a manual conflict.                                                   |
 | Content is binary or otherwise not semantically mergeable                       | Leave the selection policy to the host; Commonlib cannot infer user intent. |
 
 ## Stale and concurrent resolutions
@@ -74,12 +77,20 @@ A host must not apply a dialogue result after either compared revision has cease
 The compatibility implementation accepts an injected, device-local `FileReflectionProvenance` capability owned by the database-to-storage composition rather than by a filesystem adapter. Maintained hosts persist:
 
 ```text
-path -> { revision, observedStorageMtime? }
+path -> { revision, observedStorageMtime?, contentHash?, contentSize? }
 ```
 
 `revision` is the exact database revision most recently saved from or reflected in this device's storage. It identifies the base for subsequent local edits, rather than certifying that the current file still contains those bytes. `observedStorageMtime` is the raw modification time of the saved snapshot or the file observed after reflection. It is neither rounded nor compared across devices, and it is diagnostic only: revision identity never comes from a timestamp, path, size, or content hash.
 
-The record is updated only after a successful database-to-storage reflection or storage-to-database write. A plain read does not change provenance. A record remains authoritative when a user edits the displayed file to bytes which happen to equal another branch; content equality must not silently change the branch being extended.
+The record is updated after a successful database-to-storage reflection or storage-to-database write, or after verifying an exact unchanged revision. A plain read does not change provenance. A record remains authoritative when a user edits the displayed file to bytes which happen to equal another branch; content equality must not silently change the branch being extended.
+
+`contentHash` is an algorithm-tagged SHA-256 digest (`sha256:<lowercase hexadecimal>`) of the exact immutable content saved or reflected; `contentSize` is its byte length. Together they let the handler recognise unchanged content after PouchDB prunes the recorded revision. They do not recreate missing revision ancestry or authorise attaching an edited file to an unavailable base. Unsupported or malformed evidence follows the legacy comparison path.
+
+Old records remain readable. An equal-time scan can add content evidence when the recorded revision is still the conflict-free current revision and its bytes match storage. An ordinary unchanged save can also upgrade its verified current record. These upgrades revalidate storage and provenance after hashing; an unrelated branch or an unreadable revision is never certified from the current file. A hashing failure leaves legacy revision evidence, so overwrite protection remains conservative.
+
+Hashing is serialised across files to bound simultaneous whole-file buffers. The public `computeFileContentHash(Blob)` helper includes reading the Blob and Web Crypto SHA-256; the handler reuses results only for the same immutable Blob. It still allocates a whole-file buffer and Web Crypto may copy it, so serialisation does not imply constant memory use.
+
+The [file-reflection content hash decision](file-reflection-content-hash.md) records why this evidence uses SHA-256, the measured comparison with xxHash64, and the conditions for reconsidering that choice.
 
 File-handler saves and reflections share a lock for the same Metadata document, held from snapshot acquisition through provenance recording. Different documents can proceed concurrently. An ordinary save keeps one captured body and its recorded base through the database write; a later local edit is processed as a subsequent change, without rereading the file to certify that it remained unchanged. The handler acquires the lock before loading a file body from storage. Rename operations acquire both document locks in a consistent order, and internal calls reuse the caller's lock rather than acquiring it again. Host conflict-check hooks run after the locks are released, allowing a hook to await another file-handler operation on the same document. Host queues may count document-lock waiters against their concurrency limits, so a burst for one document can temporarily delay unrelated files.
 
@@ -89,13 +100,13 @@ User-directed reconciliation may select any exact current live revision, includi
 
 `storeWithBaseRevision()` is the force-write primitive for deliberately preserving a branch below an exact revision, including a revision which is no longer a leaf. It must not be used when an operation means 'advance this revision only if it is still current'. For that compare-and-store boundary, `storeWithLiveBaseRevision()` uses PouchDB's ordinary revision check and returns `false` if the supplied base has already been advanced. Any current revision-tree leaf is eligible, including a non-winning conflict leaf or a logical-deletion leaf. The caller must refresh its state after refusal rather than silently falling back to the deterministic winner.
 
-`storeIndependentRevision()` explicitly preserves unknown-origin content through `putDBEntryAsIndependentRoot()`. Passing an undefined base to `storeWithBaseRevision()` retains its existing latest-revision behaviour and does not create an independent root. `findLiveContentRevisions()` compares only current non-deleted leaves using locally available chunks. Additional provenance comparisons also use local-only retrieval, so they do not introduce remote chunk requests or delivery waits; a missing body leads to preservation rather than an inferred base.
+`storeIndependentRevision()` explicitly preserves unknown-origin content through `putDBEntryAsIndependentRoot()`. Passing an undefined base to `storeWithBaseRevision()` retains its existing latest-revision behaviour and does not create an independent root. `findLiveContentRevisions()` compares only current non-deleted leaves using locally available chunks. Additional provenance comparisons also use local-only retrieval, so they do not introduce remote chunk requests or delivery waits; a missing body without matching saved content evidence leads to preservation rather than an inferred base.
 
 A successful non-deleted reconciliation records the exact selected or newly created revision as provenance. Applying a logical deletion removes the storage item and its provenance record. Deletion provenance is not retained indefinitely: an absent storage item and a current logical-deletion winner already agree and need no further reconciliation.
 
 Hosts may construct the namespaced store handle during service composition, before its backing database is open. Store operations begin only after the host's storage lifecycle is ready. They fail rather than wait when that lifecycle contract is violated, because an implicit readiness wait could hang after failed initialisation or wait on its own initialisation handler. Database reset is a transient unavailable boundary; the host must avoid file processing during it and reconstruct derived provenance after reopening.
 
-For ordinary saving and incoming reflection, an absent or unreadable recorded base permits reconstruction only from exactly one matching current non-deleted leaf. Multiple matches avoid duplicate storage without guessing branch identity. No current match creates a fresh independent root, even when an ancestor has the same bytes. Deletion and rename retain their existing provenance-recovery contracts.
+For ordinary saving and incoming reflection, an absent or unreadable recorded base without matching saved content evidence permits reconstruction only from exactly one matching current non-deleted leaf. Multiple matches avoid duplicate storage without guessing branch identity. No current match creates a fresh independent root, even when an ancestor has the same bytes. Deletion and rename retain their existing provenance-recovery contracts.
 
 An unchanged stale file is not saved below the winner. When the database has advanced without a conflict, the existing database-to-storage path reflects it; a conflict instead remains subject to the host's reflection and resolution policy. Independent roots retain the file's modification time and use the existing Metadata/Chunks format, with fresh revision identifiers and reusable chunks. They have no shared ancestor with unrelated existing branches, including after replication. This does not disable a host's explicit newer-file option.
 

package/docs/proven-in-use.md

@@ -8,6 +8,12 @@ The code paths below identify the maintained Self-hosted LiveSync composition po
 
 ## Obsidian plug-in
 
+Unreleased content-evidence tests replicate 99, 100, and 599 updates between real in-memory PouchDB databases, including a receiver with `revs_limit: 100`. They verify unchanged reflection after the recorded revision is pruned, including a recreated handler, and preserve same-size edits, intentional reverts, and records without valid content evidence. Separate tests verify legacy upgrades and changes during hashing. These are package-owned tests; they do not establish Windows, mobile, or remote CouchDB behaviour.
+
+The exact local packed candidate also passes Self-hosted LiveSync 1.0.37's checks, unit tests, application builds, and CLI E2E. Its focused real-Obsidian `stale-file-restart` scenario prunes the recorded revision before restarting, verifies unchanged reflection and legacy-record upgrades, and preserves an unknown-origin file. The Review Harness exercises the exported hash helper in desktop Obsidian. Maintainer-supplied Android and iOS reports additionally verify the helper and its benchmark on those devices; these timing measurements do not establish complete mobile synchronisation behaviour or peak memory use.
+
+Unreleased deletion tests exercise `deleteMetadataOfDeletedFiles` enabled and disabled with two in-memory PouchDB replicas and injected file storage. They verify deletion and rename reflection, preservation of genuine local edits, and restoration before or during a deleted-winner lookup, including the final revision check before deletion. They also verify that missing provenance or a compacted base without content evidence preserves local storage, and that continuously advancing deleted revisions do not return stale Metadata. These tests cover the Commonlib file handler and revision lookup, including the latest-revision check before deletion; they do not exercise Obsidian file watchers or a remote CouchDB server.
+
 Unreleased configuration-encryption tests cover changing the wrapper key without changing connection, E2EE, profile, or independent ID values. They also cover custom-key input before loading, failed writes, concurrent saves, cancellation, and unreadable-payload retention. Commonlib owns these service results. LiveSync owns the Apply controls, setting-tab registration, and database startup order; actual Apply and restart behaviour requires its focused real-Obsidian scenario.
 
 The CouchDB connection fix bypasses browser HTTP caching so encrypted replication can continue after a remote Rebuild preserves document IDs and revisions. Commonlib's regression tests cover Metadata and Chunks encrypted with a replacement Security Seed while an earlier response remains cached. Self-hosted LiveSync's `couchdb-http-cache` scenario additionally populates the real browser HTTP cache before rebuilding, confirms that the Seed and ciphertext change while the revision remains unchanged, and requires subsequent replication to succeed without clearing that cache. The released LiveSync 1.0.34 fails this scenario; a local checked Commonlib artefact passes in the maintained LiveSync 1.0.34 composition on Linux Obsidian 1.12.7. This establishes the tested build's behaviour, with the reporter's Windows and Obsidian 1.13.7 environment still unverified.
@@ -86,6 +92,8 @@ The CLI extends the same neutral context with its database path and injected `St
 
 The CLI filesystem composition receives a configured base directory and constructs the rooted Node storage adapter in `src/apps/cli/adapters/NodeFileSystemAdapter.ts`. Commonlib's shared storage result table checks the focused root contract; the consumer adds its file cache, Vault-shaped operations, case handling, and command policy.
 
+The unreleased [Unicode path identity contract](unicode-path-identity.md) adds opt-in NFC document IDs for CLI and Obsidian use with CouchDB. Commonlib tests exercise real in-memory Metadata lookup across different host normalisers. CLI temporary-filesystem tests cover resolution, updates, deletions, unchanged physical spellings, and ambiguous NFC paths. File-reflection tests preserve NFC-equivalent physical names while retaining case-only rename behaviour. With the packed candidate installed, the Linux CLI-to-real-Obsidian scenario verifies creation, remote updates, and daemon-applied deletions with and without E2EE and Path Obfuscation. A separate run migrates two legacy CLI databases with Rebuild and Fetch before the Obsidian round trip, preserving both NFD physical names and replacing the remote document ID. These checks do not establish native macOS or iOS behaviour.
+
 Unreleased Node timestamp tests verify that `stat` and `trystat` retain an available birth time when change time advances, fall back when birth time is zero, and return integer milliseconds. Packed-package checks exercise the shared `getNodeFileCreationTime` export. With the exact locally packed candidate installed, downstream CLI tests verify that an in-place edit retains creation time in local Metadata. The CLI-to-real-Obsidian scenario additionally verifies the edited content and creation time in received Metadata. These checks do not establish the reporter's ZFS behaviour or physical-file creation times in Obsidian.
 
 The maintained CLI checks include unit tests, Deno-driven CLI E2E, Compose-based compatibility runs, and the CLI-to-real-Obsidian synchronisation scenario in `test/e2e-obsidian/scripts/cli-to-obsidian-sync.ts`. The latter is the strongest current evidence that the CLI and plug-in compose compatible Commonlib behaviour across two hosts.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vrtmrz/livesync-commonlib",
-  "version": "0.1.40",
+  "version": "0.1.42",
   "description": "Shared data, storage, and replication primitives for Self-hosted LiveSync clients.",
   "type": "module",
   "types": "./dist/index.d.ts",