cry-synced-db-client 0.1.186 → 0.1.187

package/CHANGELOG.md

@@ -1,5 +1,173 @@
 # Versions
 
+## 0.1.187 (2026-05-16)
+
+### Fix: `setByPath` auto-creates plain-object intermediates (root cause of `sestevki` parent+child conflict)
+
+Production bug (klikvet vetnm, 2026-05-15, ~290 hits/day): obisk records
+stuck dirty with mongo upload errors —
+
+```
+[SyncEngine] Sync upload error [obiski] _id=…:
+  Updating the path 'sestevki.skupaj.prc' would create a conflict at 'sestevki'
+```
+
+The `0.1.185` `rebaseDirtyOnServerAdvance` fix targeted the case where
+server `_rev` advances past the local base (the `kritje + skupaj`
+two-write scenario). But a more general root cause was still latent in
+`setByPath`: when the function was asked to drill `skupaj.prc` into an
+existing `sestevki: {}` accumulated parent, it returned `false`
+because navigation hit `undefined` at the missing `skupaj`
+intermediate — and `mergeDirtyPath` Case 1's failure handler fell
+through to Case 3, writing the dot-path as a SIBLING key.
+
+`applyDiffLocally` had already worked around this asymmetrically by
+adding `materializePlainDotPath` in `0.1.185`, but that helper only
+ran inside `applyDiffLocally` — `mergeDirtyPath` continued to use the
+strict `setByPath` and produced the sibling-coexistence pattern in the
+dirty payload sent to the server.
+
+**This release lifts the fix into `setByPath` itself**, the single
+primitive used by both code paths. Default semantics change from
+"fail at any missing intermediate" to "auto-create empty plain-object
+intermediate and continue":
+
+```ts
+// Before (≤ 0.1.186): setByPath({}, "a.b", 1) → false
+// After  (≥ 0.1.187): setByPath({}, "a.b", 1) → true, target = {a: {b: 1}}
+```
+
+**Safety guards** added to keep the change strictly bug-fix
+direction:
+
+- **Path look-ahead**: refuse auto-create entirely when ANY segment in
+  the path is bracket-by-id (`[<id>]`) or numeric (`0`). Those shapes
+  imply an array intermediate which `setByPath` cannot synthesize
+  without external info (which `_id`? what other elements live there?).
+  The existing `materializeBracketPath` fallback in
+  `applyDiffLocally` keeps owning that case and was untouched.
+- **Host-type refusal**: when an intermediate exists but is a non-plain
+  host object (`Date`, `ObjectId`, `RegExp`, `Map`, …), refuse with
+  `false`. Drilling further would silently mutate the host instance
+  (e.g. `dateInstance.year = 2026`) — a data-corruption vector.
+  Arrays are EXEMPT from this refusal because bracket/numeric next
+  segments handle them correctly.
+- **Rollback on final-segment failure**: if `setByPath` auto-creates
+  intermediates and `setSegment` ultimately fails, remove every
+  intermediate it created. The caller's target sees no orphan `{}`
+  branches.
+- **Null treated as undefined**: a stored `null` at an intermediate
+  position is auto-created over (real production state included
+  `sestevki: null`).
+- **`opts.autoCreate = false` opt-out**: callers that explicitly need
+  the pre-0.1.187 strict fail-at-miss behavior pass `{ autoCreate: false }`.
+  No in-tree caller does, but the opt-out keeps the API
+  backwards-compatible for external consumers.
+
+```ts
+export function setByPath(
+  target: any,
+  path: string,
+  value: any,
+  opts?: { autoCreate?: boolean },  // default { autoCreate: true }
+): boolean { … }
+```
+
+### `mergeDirtyPath` Case 1 — promote-on-failure replaces fall-through
+
+The Case 1 fallback ("setByPath failed → fall through to Case 3
+orthogonal") used to be the SOURCE of the parent+child sibling
+payload. With auto-create on, `setByPath` returns `false` only for
+genuine type clashes (primitive parent + plain-key descendant, or path
+shape mismatch). In that case the right resolution is to drop the
+stale parent and promote the deep path:
+
+```ts
+// 1. setByPath(mutationTarget, relativePath, newValue) returns false.
+// 2. The two forms cannot coexist in a mongo $set.
+// 3. Per server-wins-rebase convention: newer write wins.
+delete accumulated[existingKey];
+accumulated[newPath] = newValue;
+return;
+```
+
+Also: when the existing parent value is `null` or `undefined`, Case 1
+now pre-materializes `accumulated[existingKey] = {}` BEFORE calling
+`setByPath` — mirrors the auto-create semantics for top-level keys
+that `setByPath` does for intermediates.
+
+### Simplification: `applyDiffLocally` drops `materializePlainDotPath`
+
+The `0.1.185` `materializePlainDotPath` private helper in
+`SyncedDb.applyDiffLocally` is now redundant — its job is subsumed by
+the new `setByPath` default. Removed (~60 LOC). Bracket-path
+materialization (`materializeBracketPath`) is unchanged and still
+owns the array-shape synthesis case.
+
+### Real-data regression test
+
+New fixture `test/fixtures/stuckSestevkiObiskiVetnm.json` (9 entries
+pulled from `https://vetnm.klik.vet/api/clients` 2026-05-16,
+heartbeat.dirtyContent.obiski). 5 entries exhibit the parent+child
+overlap pattern; 4 entries are stuck for unrelated reasons (no
+overlap). README documents the per-entry pattern breakdown and
+verifies (`mongosh`) that none of the 9 `_id`s exist on the server.
+
+New test `test/sestevkiRealProductionData.test.ts` (26 tests):
+
+- **Replay path** — simulates the write sequence that produces each
+  stuck pattern via `mergeDirtyPath`. Asserts post-merge has no
+  parent+child overlap, descendant values reachable via canonical path.
+- **Orthogonal merge non-destruction** — orthogonal field write on
+  legacy stuck dirty entry doesn't worsen the existing conflict
+  (recovery for those is via `repairExistingDirty` migration OR
+  app-side `preprocessDirtyItem` hook in klikvet `ocistiDirtyItem`).
+- **Batched-vs-sequential equivalence** — `mergeDirtyChanges(batch)`
+  produces same state as N individual `mergeDirtyPath` calls.
+- **Explicit variant coverage** — parent=`{}` + all-zero / non-zero
+  children, plus synthetic parent=null case (current fixture has no
+  null-parent overlap but the fix must handle it).
+- **computeDiff round-trip** — for each overlap fixture, synthesize a
+  base→target pair and assert `computeDiff(base, target)` produces no
+  parent+child overlap in its diff key set.
+
+Existing `test/computeDiff.test.ts` extended with 18 new unit tests
+in two describe blocks (`setByPath — auto-create plain-object
+intermediates (0.1.187)` and `mergeDirtyPath — sestevki regression`).
+
+### Backwards compatibility
+
+| Caller scenario                                          | Pre-0.1.187 | Post-0.1.187 |
+| -------------------------------------------------------- | ----------- | ------------ |
+| `setByPath({}, "a.b", v)`                                | `false`     | `true` (`{a:{b:v}}`) |
+| `setByPath({a:{c:1}}, "a.b", v)`                         | `true` (set b alongside c) | unchanged |
+| `setByPath({a:"str"}, "a.b", v)`                         | `false`     | unchanged (`false`)  |
+| `setByPath({a:Date}, "a.year", 2026)`                    | true (mutates Date instance) | `false` (NEW guard) |
+| `setByPath({arr:[]}, "arr.0.b", v)`                      | `false`     | unchanged (`false`)  |
+| `setByPath({arr:[]}, "arr[id]", [{_id:id,…}])`           | `true`      | unchanged            |
+| `setByPath(t, "...", v, { autoCreate: false })`          | (new param) | strict pre-0.1.187 mode |
+| `mergeDirtyPath` Case 1 with `{}` parent + dot-child     | siblings (BUG) | merges into parent |
+
+The Date instance mutation case is the only INTENTIONAL behavioral
+change beyond bug fix — that path was producing silent data corruption
+and is now correctly refused.
+
+### Migration path for pre-fix stuck entries
+
+The fix prevents NEW two-form payloads. Old stuck dirty entries
+already in IndexedDB `_dirty_changes` tables stay until either:
+
+1. The device makes a new write that overlaps the stuck path — Case 1
+   `setByPath` will then auto-create and absorb the dotted children
+   (or Case 1's fallback will drop+promote).
+2. The app provides a `preprocessDirtyItem` hook (`klikvet` ships one
+   in `code/klikvet/ocistiDirtyItem.ts` 0.20.123 — defensive
+   parent+child path conflict resolver that runs at upload time).
+
+A future `SyncedDb.repairExistingDirty()` migration could replay all
+`_dirty_changes` rows through the new `mergeDirtyChanges` for a
+one-shot batch cleanup; not included in this release.
+
 ## 0.1.186 (2026-05-15)
 
 ### `WakeSyncManager` silences expected wake-time `timeout` errors

package/dist/index.js

@@ -520,18 +520,51 @@ function tokenizePath(path) {
   if (buf) out.push(buf);
   return out;
 }
-function setByPath(target2, path, value) {
+function setByPath(target2, path, value, opts) {
   if (target2 === null || target2 === void 0) return false;
+  const autoCreate = (opts == null ? void 0 : opts.autoCreate) !== false;
   const parts = tokenizePath(path);
+  const pathHasArrayShape = parts.some(
+    (p) => p.startsWith("[") && p.endsWith("]") || /^\d+$/.test(p)
+  );
+  const canAutoCreate = autoCreate && !pathHasArrayShape;
   let current = target2;
+  const created = [];
   for (let i = 0; i < parts.length - 1; i++) {
     const part = parts[i];
-    const next = navigateSegment(current, part);
-    if (next === void 0) return false;
+    let next = navigateSegment(current, part);
+    const isMissing = next === void 0 || next === null;
+    if (isMissing && canAutoCreate) {
+      if (isPlainObjectContainer(current)) {
+        current[part] = {};
+        created.push({ container: current, key: part });
+        next = current[part];
+      } else {
+        return false;
+      }
+    } else if (isMissing) {
+      return false;
+    } else if (autoCreate && !pathHasArrayShape && !isPlainObjectContainer(next) && !Array.isArray(next)) {
+      return false;
+    }
     current = next;
   }
   const last = parts[parts.length - 1];
-  return setSegment(current, last, value);
+  const ok = setSegment(current, last, value);
+  if (!ok && created.length > 0) {
+    for (let i = created.length - 1; i >= 0; i--) {
+      const { container, key } = created[i];
+      delete container[key];
+    }
+  }
+  return ok;
+}
+function isPlainObjectContainer(value) {
+  if (value === null || value === void 0) return false;
+  if (typeof value !== "object") return false;
+  if (Array.isArray(value)) return false;
+  const proto = Object.getPrototypeOf(value);
+  return proto === Object.prototype || proto === null;
 }
 function navigateSegment(current, part) {
   if (current === null || current === void 0) return void 0;
@@ -646,6 +679,10 @@ function mergeDirtyPath(accumulated, newPath, newValue) {
       if (existingIsTerminal && Array.isArray(existingValue) && existingValue.length === 1) {
         mutationTarget = existingValue[0];
       }
+      if (!existingIsTerminal && (existingValue === null || existingValue === void 0)) {
+        accumulated[existingKey] = {};
+        mutationTarget = accumulated[existingKey];
+      }
       if (!existingIsTerminal && newPath[existingKey.length] === "[" && canExpandArrayToBrackets(existingValue)) {
         delete accumulated[existingKey];
         for (const el of existingValue) {
@@ -658,7 +695,9 @@ function mergeDirtyPath(accumulated, newPath, newValue) {
       const relativePath = sepChar === "[" ? newPath.substring(existingKey.length) : newPath.substring(existingKey.length + 1);
       const ok = setByPath(mutationTarget, relativePath, newValue);
       if (ok) return;
-      break;
+      delete accumulated[existingKey];
+      accumulated[newPath] = newValue;
+      return;
     }
   }
   const descendants = [];
@@ -6652,45 +6691,10 @@ var _SyncedDb = class _SyncedDb {
       }
       const ok = setByPath(seed, path, value);
       if (ok) continue;
-      if (!path.includes("[") && _SyncedDb.materializePlainDotPath(seed, path, value)) {
-        continue;
-      }
       _SyncedDb.materializeBracketPath(seed, path, value, collection, fallbackId);
     }
     return seed;
   }
-  /**
-   * Bracket-free dot-path fallback for `applyDiffLocally`. Walks the path,
-   * creating missing plain-object intermediates as needed, and sets the
-   * leaf. Refuses (returns `false`) if any segment is numeric (array
-   * index) or `[<id>]` (bracket selector) — those need shape knowledge
-   * outside the scope of plain-object materialization. Also refuses if
-   * an existing intermediate is non-plain (array, Date, primitive) —
-   * overwriting would risk silent data loss.
-   *
-   * Used after `setByPath` fails on a dot-only path (no `[` in `path`).
-   * Counterpart to `materializeBracketPath` for the bracket case.
-   */
-  static materializePlainDotPath(seed, path, value) {
-    const parts = tokenizePath(path);
-    if (parts.length < 2) return false;
-    for (const seg of parts) {
-      if (seg.startsWith("[") || /^\d+$/.test(seg)) return false;
-    }
-    let cur = seed;
-    for (let i = 0; i < parts.length - 1; i++) {
-      const seg = parts[i];
-      const next = cur[seg];
-      if (next == null) {
-        cur[seg] = {};
-      } else if (typeof next !== "object" || Array.isArray(next) || next instanceof Date) {
-        return false;
-      }
-      cur = cur[seg];
-    }
-    cur[parts[parts.length - 1]] = value;
-    return true;
-  }
   /**
    * Fallback for `setByPath` failures inside `applyDiffLocally`. Materializes
    * missing array containers AND missing intermediate plain objects so the

package/dist/src/db/SyncedDb.d.ts

@@ -519,19 +519,6 @@ export declare class SyncedDb implements I_SyncedDb {
      * the input `base` reference.
      */
     private static applyDiffLocally;
-    /**
-     * Bracket-free dot-path fallback for `applyDiffLocally`. Walks the path,
-     * creating missing plain-object intermediates as needed, and sets the
-     * leaf. Refuses (returns `false`) if any segment is numeric (array
-     * index) or `[<id>]` (bracket selector) — those need shape knowledge
-     * outside the scope of plain-object materialization. Also refuses if
-     * an existing intermediate is non-plain (array, Date, primitive) —
-     * overwriting would risk silent data loss.
-     *
-     * Used after `setByPath` fails on a dot-only path (no `[` in `path`).
-     * Counterpart to `materializeBracketPath` for the bracket case.
-     */
-    private static materializePlainDotPath;
     /**
      * Fallback for `setByPath` failures inside `applyDiffLocally`. Materializes
      * missing array containers AND missing intermediate plain objects so the

package/dist/src/utils/computeDiff.d.ts

@@ -55,9 +55,38 @@ export declare function tokenizePath(path: string): string[];
  *   - Bracket ("[<_id>]") → array element matched by `_id` field
  *   - Plain ("field") → object key
  *
- * @returns true if the value was successfully set, false if path traversal failed.
+ * Auto-creation policy (when `opts.autoCreate !== false`, the **default
+ * since 0.1.187**):
+ *   - PLAIN segment whose intermediate is `undefined` / `null` AND whose
+ *     parent is a plain object → materialize `{}` in place and continue.
+ *   - PLAIN segment whose intermediate is a primitive (string, number,
+ *     boolean, Date, ObjectId, …) → REFUSE (return false). Overwriting
+ *     would destroy data the caller didn't ask to delete.
+ *   - NUMERIC ("0", "1") segment → unchanged (cannot safely materialize an
+ *     array slot at a specific index without committing to the array
+ *     shape).
+ *   - BRACKET ("[<_id>]") segment → unchanged (cannot synthesize an array
+ *     element with the right `_id` without the full element value).
+ *
+ * Set `opts.autoCreate = false` to restore the pre-0.1.187 strict
+ * "fail at any missing intermediate" behavior.
+ *
+ * Why auto-create is default-on: callers like `mergeDirtyPath` Case 1 and
+ * `applyDiffLocally` always WANT the value to land inside the parent
+ * object. Pre-0.1.187 `setByPath` returned `false` on missing
+ * intermediates, the merge fell through to "add as sibling key", and the
+ * dirty payload ended up with both `sestevki: {}` AND
+ * `sestevki.skupaj.prc = 0` — which mongo `$set` rejects (production
+ * vetnm 2026-05-15, 290 hits/day).
+ *
+ * @returns true if the value was successfully set, false if path traversal
+ *          failed for a reason that cannot be auto-corrected (primitive
+ *          intermediate, missing array element, non-plain container at a
+ *          plain-segment site).
  */
-export declare function setByPath(target: any, path: string, value: any): boolean;
+export declare function setByPath(target: any, path: string, value: any, opts?: {
+    autoCreate?: boolean;
+}): boolean;
 /**
  * Delete the value at `path` within `target`. Sibling of `setByPath` —
  * navigates the same tokenized path forms (numeric, bracket-by-_id, plain),

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cry-synced-db-client",
-  "version": "0.1.186",
+  "version": "0.1.187",
   "type": "module",
   "main": "./dist/index.js",
   "module": "./dist/index.js",