@glance-apps/sync 1.10.0 → 2.0.0

package/CHANGELOG.md

@@ -7,6 +7,306 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [2.0.0] - 2026-08-30
+
+> ### ⚠️ ACTION REQUIRED for lastGLANCE and lifeGLANCE
+>
+> **Add `pullCursorCommit: 'per-page'` to your `createDbSyncEngine` config when
+> you take this release.** Without it you lose 1.10.0's mid-pagination resume
+> and silently regain the convergence bug it fixed: a large backlog on a flaky
+> connection re-downloads from the start on every failure instead of resuming
+> from the last good page.
+>
+> It will not look like a config gap. It will look like a network problem,
+> weeks after the bump, and nobody will connect the two. One line prevents it.
+
+**Should you take this release?** Yes, and read this entry before you do — it is
+the only release so far that changes what an existing caller gets **without that
+caller changing any code**.
+
+Since 1.10.0, `pullRemoteChanges` has persisted the pull cursor **per page**, on
+the assumption that a row is durable the moment `applyRemoteEntity` returns.
+That is true for a caller writing straight into its store. It is false for a
+caller applying into a per-cycle mirror it discards on failure
+(commit-only-on-success): those rows are consumed, the cursor advances past
+them, they are never committed, an incremental pull never re-lists them, and a
+glitch heal cannot reach them because they were never in a snapshot. That is
+permanent row loss, and it was demonstrated as such.
+
+The assumption was stated only in a source comment. The config contract
+documenting `applyRemoteEntity` said nothing about durability at all. **A
+caller cannot honour a contract it is never shown** — so it is now a declared,
+validated mode, and the default is the safe one.
+
+### Changed — BREAKING
+
+- **`pullCursorCommit` now controls when the pull cursor is persisted, and it
+  defaults to `'end-of-pull'`.** Previously the engine always behaved as
+  `'per-page'`, with no way to say otherwise.
+
+  | Mode | When the cursor advances | For |
+  |---|---|---|
+  | `'end-of-pull'` **(new default)** | Once, after a fully successful pagination loop | Any caller that has not thought about it |
+  | `'per-page'` | After each completed page — 1.10.0 behaviour | A caller whose apply is durable on return |
+  | `'caller'` | Never; the engine writes nothing | A commit-only-on-success composer |
+
+  Under `'end-of-pull'`, a mid-pagination failure re-pulls from where the run
+  started. Re-applying already-applied rows is idempotent under entity-grain
+  LWW, so the cost is bandwidth; the cost of the old default, for the wrong
+  caller, was rows.
+
+  In `'caller'` mode `pullRemoteChanges` returns `maxSeq` and the caller
+  commits it with `setHighWaterMark` once its own state is durable. No new API
+  was needed — `setHighWaterMark` was already public. `dbSyncCycle` commits for
+  itself on a fully successful cycle, so declaring `'caller'` and continuing to
+  use the cycle still converges.
+
+- **An unrecognised `pullCursorCommit` is REFUSED at construction**, not
+  defaulted. `'perPage'`, `'per_page'` or `'PER-PAGE'` throw with a message
+  naming every valid mode and the value actually passed. A typo silently
+  falling back to a default is precisely the failure this contract exists to
+  remove: it would quietly restore the unsafe behaviour on a caller that was
+  trying to opt out of it.
+
+### Why this is 2.0.0
+
+No API signature changed, and by the letter of this package's old versioning
+note — "breaking changes to the public API or the sync envelope schema" — this
+would have been a minor. That note was too narrow, and has been widened
+(see the README): **a behavioural default that changes what an existing caller
+gets without that caller changing code is breaking**, because the version
+number is the only signal that cannot be skipped. Under-signalling here costs a
+field bug that presents as a flaky connection; over-signalling costs a digit.
+
+### Why the default was flipped, and what argued against it
+
+The default is what the next composed caller inherits by saying nothing, and
+the next composed caller is an Obsidian plugin. Inheriting the unsafe path by
+saying nothing is exactly what put a data-loss bug in front of dayGLANCE.
+
+The alternative was real and was weighed: keep `'per-page'` as the default and
+require an explicit opt-out from callers needing the safe path. It costs the
+shipping consumers nothing and needs no major bump. It loses on the one point
+that matters — the unsafe path stays the thing you get by default — and the
+tiebreaker was sequencing rather than principle: all three consumers are
+touching engine config for 1.12.0 anyway, so the safe default costs them one
+extra line in a change they are already making. Had those landed far apart, the
+decision would have gone the other way.
+
+### Notes
+
+- **dayGLANCE's cursor rollback stays.** It covers two loss windows and
+  `'end-of-pull'` closes only the first. The second — the pull succeeded but a
+  later step (push, heal, commit) threw, so the mirror was discarded anyway —
+  is invisible to this package. Only `'caller'` mode closes both, and adopting
+  it is dayGLANCE's call on its own schedule.
+- **Per-page convergence does not come back for dayGLANCE under any mode.** It
+  needs per-page *durable* commits, which an all-or-nothing merge-aware commit
+  does not provide.
+- Not closed here: the cycle-only `syncing` re-entrancy guard (see 1.12.0).
+
+## [1.12.0] - 2026-08-30
+
+**Should you take this release?** Yes if anything in your app calls
+`pushDirtyRows`, `pullRemoteChanges` or `updateDeviceCursor` directly instead of
+`dbSyncCycle`. Until now the two tiers had unequal safety: the cycle carried
+backoff, over-quota suppression and the credential halt, and the primitives
+beneath it carried none. A caller composing its own cycle silently opted out of
+all three, with no signal that it had. This release moves enforcement down into
+the primitives, so every caller inherits it by construction. **If you only ever
+call `dbSyncCycle`, nothing about this release changes what your app does.**
+
+### Changed
+
+- **The three primitives now enforce their own protections.**
+  `pushDirtyRows` and `pullRemoteChanges` refuse to run while their direction's
+  backoff window is open, rejecting with a typed `SYNC_SUPPRESSED` error
+  *before* any network call, and they open, escalate and clear those windows
+  themselves. Both also refuse while the device is under the credential halt,
+  and both now SET the halt when a rejection reaches them — a guard over a halt
+  that nothing sets never fires, so it took both halves.
+- **`dbSyncCycle` delegates entirely downward.** It no longer checks a window
+  or records a failure; it distinguishes "my own gate refused this" from "the
+  server refused this" and reports as before. Enforcing in both places would
+  have **doubled the escalation** — `strikes` incrementing once in the
+  primitive and once in the cycle turns the ladder into 30s, 120s, 480s instead
+  of 30, 60, 120. The alternative shape (primitives no-op when called from
+  inside the cycle) was rejected: it makes correctness depend on a caller
+  passing a flag, which is the exact defect this release removes, relocated one
+  layer down.
+- **`updateDeviceCursor` is deliberately different**: halt-gated, but *not*
+  window-gated, never recording and never throwing. It reports a refusal as
+  `{ updated: false, suppressed: true }`. It cannot throw because callers await
+  it mid-cycle inside their own try blocks, and it is not window-gated because
+  the pull now records its own failure — so the window is already open by the
+  time the cursor report runs on the very cycle the pull was attempted, and
+  gating there would silently undo 1.10.0's decoupling ("the cursor report runs
+  whenever the pull was ATTEMPTED, success or failure"). Whether a cycle has
+  anything to report is a composition decision that stays with the caller.
+
+### Added
+
+- `SYNC_SUPPRESSED`: the typed rejection from the engine's own gate. It carries
+  `suppressed: true`, `direction` (`'push' | 'pull'`), `reason`, `causeCode`,
+  and `retryAt` / `retryInMs`.
+- `isSuppressedError(err)` — exported so a caller can tell a pause from a
+  failure **without string matching and without hardcoding a code literal**.
+
+### Notes for callers with their own retry ladder or circuit breaker
+
+- **Do not treat `SYNC_SUPPRESSED` as a failure.** This is the one integration
+  hazard in the release. An app that escalates its own cooldown on every thrown
+  error will escalate on a pause this engine already scheduled — two brakes
+  compounding, each correct in isolation, with the device waiting far longer
+  than either layer intends. Branch on `isSuppressedError(err)` (or
+  `err.suppressed`) and align to `err.retryAt` rather than adding a second
+  window on top of it.
+- **This cannot be made structurally impossible from inside the package.** The
+  app owns its breaker and reacts to what it catches; the only way to remove
+  the caller's judgement entirely would be for the package to own the whole
+  retry, which means adopting `dbSyncCycle` — exactly what composed callers
+  compose to avoid. So the signal is made as unmissable as we can: a typed
+  code, a redundant boolean, an exported predicate, and a retry deadline to
+  align to.
+- **Why a throw and not a skipped return value.** A return lets a caller that
+  ignores it treat a suppressed cycle as a successful one — committing state
+  built from a pull that never ran, and resetting its own breaker on a cycle
+  that did nothing. Throwing converts the worst failure mode from "silently
+  wrong" to "loudly refused". The argument the other way is real: a return
+  needs no `catch`, and a caller that forgets to special-case the code turns a
+  deliberate pause into what looks like an error. The `RATE_LIMITED` precedent
+  from 1.11.0 settled it — one shape for "the client refused this", not two.
+- **A direct call that fails now opens a window**, so an immediate retry of a
+  failed primitive is refused rather than re-sent. Retrying after `retryAt` is
+  the supported pattern; this is what the cycle has always done.
+
+### Notes
+
+- **Not closed by this release** (reported, deliberately unfixed): the
+  `syncing` re-entrancy guard is still cycle-only, so a composed caller can
+  overlap primitive calls. See the PR for what that does and why it is a
+  different shape of problem from the two this release addresses.
+- The cursor durability contract is Phase 4b and lands separately.
+
+## [1.11.0] - 2026-08-30
+
+**Should you take this release?** Yes if you use the GLANCEvault transport, and
+immediately if anything in your bundle writes to it on a timer. This release is
+the client-side answer to a 2026-08-30 incident in which an intent-cleanup loop
+in the dayGLANCE Obsidian plugin deleted rows that were already tombstones. The
+server re-tombstoned on every call — fresh seq, fresh SSE nudge — which put the
+row back above the caller's cursor and fed the loop its own output. It ran for
+hours and saturated the server's per-IP rate budget, and it was **invisible**:
+every delete was fire-and-forget behind a `.catch(() => {})`, so nothing failed
+loudly enough to notice. Two patches landed elsewhere (the plugin no longer
+deletes tombstones; the vault's `softDeleteRow` is now idempotent). This release
+adds the protection and the visibility **by construction**, so no caller has to
+remember them and no loop can be silent. No server changes: the wire contract is
+untouched.
+
+### Added
+
+- **The intents transport.** `intentsBatch(accountId, events)` and
+  `intentsList(accountId, { since, limit })` wrap the server's existing
+  `/intents/*` endpoints, so apps can drop their hand-rolled raw-fetch
+  transports and inherit this client's auth, error classification, brake and
+  meter. `intentsBatch` is insert-only server-side — a re-sent `eventId` is a
+  no-op — so retrying an ambiguous failure is safe by construction.
+  - **Pure transport**: the `envelope` is opaque here and is never decoded or
+    inspected. The codec, and the per-envelope key derivation it needs, stays
+    in `@glance-apps/intents`; this package carries the transport without
+    carrying the crypto.
+- **A device-wide request brake.** Any real 429 now pauses *every* vault
+  request in the bundle realm: 30s, doubling per burst to a 10-minute ceiling,
+  one arming per burst (a concurrent fan-out that all meets the limiter costs
+  one escalation step, not ten). While paused, calls fail fast — **before
+  touching the network** — with a `VaultError` carrying `status: 429`,
+  `code: 'RATE_LIMITED'` and `retryInMs`.
+  - It is at **module scope, not per client instance**, deliberately: the thing
+    it protects is shared by every client in the process (the server's per-IP
+    budget), so the state that models it belongs to the realm. A separately
+    bundled copy — the Obsidian plugin's, say — is its own realm, which is
+    correct: separate process, separate traffic.
+  - **Decay, never amnesty.** A 2xx releases the pause but only *halves* the
+    escalation memory (below the 30s base it drains to zero). This is the fix
+    for the earlier app-side brake, which reset escalation to zero on *any*
+    success: on a saturated shared budget an occasional cheap request slips
+    into a fresh limiter window and returns 200, and each lucky 200 wiped the
+    whole 30s→480s escalation and re-licensed full cadence. Halving keeps the
+    storm's level in memory — the next 429 re-arms at 2× what is left — while a
+    genuine recovery still drains to nothing in a few quiet successes.
+  - Reads: `isVaultRateLimited()` and `vaultBrakeStatus()`, for callers that
+    prefer to sit a whole cycle out pre-flight. Escape hatch:
+    `createVaultClient({ brake: false })` opts a client out of both the gate
+    and the arming.
+- **A budget meter.** Requests are counted per rolling minute and attributed by
+  method name. Crossing a soft threshold (default 300/min — half the server's
+  default per-IP budget, configurable) logs **one** line per window naming the
+  top contributors: `[vault] budget: 412 requests in the last minute (soft
+  limit 300) — top: intentsList 210, batch 130, deleteRow 72`. Visibility only;
+  it never throttles. Its whole argument is that it needs no theory about *why*
+  a loop is running, so the next storm is an attributed event on day one rather
+  than an afternoon of archaeology.
+- **A write-loop detector — the success-side signal.** Failure visibility
+  already existed; what was missing on the day was the signal for everything
+  "succeeding" pathologically. The client now keeps a bounded per-`entityId`
+  history across `batch` upserts and `deleteRow`, and warns loudly (once per id
+  per window) when four **redundant** writes to one id land inside ten minutes.
+  Diagnostic only — never a brake.
+  - A write is redundant when it cannot have changed the row's state from what
+    the previous write to that id left it. Three shapes qualify: a **polarity
+    flip** (the delete-and-resupply loop), a **repeat delete** (re-deleting a
+    row that is already a tombstone — the incident's own shape: same polarity
+    every time, no content at all), and an **identical-content rewrite**.
+  - **Fails open by construction**: the rule needs repeated events on the
+    *same* id, so a first sync, a large import, or a bulk cleanup deleting many
+    different ids — many ids, one event each — can never trigger it, however
+    large. A bounded retry of one failed delete does not reach the threshold
+    either.
+  - Both thresholds are tunable via `configureVaultDiagnostics`. Four redundant
+    writes means five events (upsert/delete/upsert/delete/upsert, or five
+    deletes of one tombstone), which no human edit pattern produces and every
+    delete/resupply or re-tombstone loop produces within seconds; ten minutes
+    is long enough to catch a loop running on a 60s sync cycle and short enough
+    that a week of ordinary edits never accumulates into it.
+- `getVaultStats()` returns all three in one read: brake status, the last
+  rolling minute of requests by method, and the current write-loop suspects.
+  `configureVaultDiagnostics(options)` tunes the visibility-only thresholds and
+  the log sink; `resetVaultDiagnostics()` clears everything (for tests).
+
+### Changed
+
+- Every `createVaultClient` method now passes through the brake and the meter.
+  The only behaviour change a healthy caller can observe is the new
+  `RATE_LIMITED` rejection while a window is open — and only after the server
+  has already answered a real 429.
+- A **429 arms the brake only when it is a rate-limiter hit**, never when it is
+  an over-quota rejection. The server uses 429 for both and they call for
+  opposite handling: a limiter hit means "stop asking for a while" (exactly
+  what the brake does), while a quota rejection means the account is full,
+  clears only when an operator acts, and already has its own self-resuming
+  window in the DB engine. Gating on it would hide `QUOTA_EXCEEDED` and its
+  descriptor behind `RATE_LIMITED` and leave apps unable to render "X of Y
+  used". The quota shape is recognised as narrowly as before, so anything that
+  is not unmistakably a quota body is treated as a limiter hit.
+
+### Notes
+
+- **Not gated**: `fetchVaultHealth` and `enrollVaultDevice`. Both are
+  pre-credential, user-initiated, one-shot calls — `/healthz` is the
+  unauthenticated probe you reach for *while* diagnosing a stuck client, and
+  refusing it during a braked window would take away the diagnostic just when
+  it is wanted.
+- **For lastGLANCE / lifeGLANCE**: safe to take at leisure (semver-minor).
+  During a braked window, client calls fail fast with `VaultError { status:
+  429, code: 'RATE_LIMITED' }` instead of hitting the wire; any ladder that
+  already treats a real 429 as transient — they all do — handles it unchanged.
+  The meter and the write-loop warnings come along for free.
+- An AES-GCM envelope carries a fresh random IV, so identical plaintext does
+  not produce an identical envelope: for encrypted payloads it is the polarity
+  flip that fires the detector, and the identical-content rule covers plaintext
+  and deterministic payloads.
+
 ## [1.10.0] - 2026-08-01
 
 **Should you take this release?** Yes, for any deployment using the database

package/README.md

@@ -158,6 +158,71 @@ Pass `encryptionEnabled: true` in the sync config object stored via `engine.setC
 | `createProviders(config)` | Creates cloud sync provider objects (`nextcloud`, `koofr`, `webdav`). |
 | `normalizeEtag(raw)` | Normalizes a raw ETag header value for If-Match use: strips a weak-validator prefix (`W/"abc"` → `"abc"`) and the content-coding suffixes some servers append inside the quotes (`"abc-gzip"` / `"abc-br"` → `"abc"`). Quotes are preserved; `null`/`undefined` pass through unchanged. |
 
+### GLANCEvault client
+
+| Export | Description |
+|--------|-------------|
+| `createVaultClient(config)` | HTTP client for a GLANCEvault server. `config` is `{ vaultUrl, vaultToken, fetchImpl?, brake? }`. Every method carries the Bearer token, classifies errors uniformly (`CREDENTIAL_INVALID`, `QUOTA_EXCEEDED`, `RATE_LIMITED`, else `VaultError` with `.status`), and passes through the module-scope brake and budget meter. |
+| `fetchVaultHealth(config)` | Unauthenticated `GET /healthz`. Returns `{ status, version, schemaVersion, authMode }`, with `authMode` normalized to `'shared'` on servers that predate the field. |
+| `enrollVaultDevice(config)` | Exchanges the admin-configured bootstrap secret for this device's own credential (`POST /enroll`). The credential is returned once and never again. |
+
+**`VaultClient` methods:**
+
+| Method | Description |
+|--------|-------------|
+| `batch(app, { accountId, rows })` | Upserts rows (`{ entityId, envelope, createdAt }`). Returns `{ written, maxSeq }`. |
+| `list(app, { accountId, since })` | One page of rows with `seq > since`. Returns `{ rows, hasMore }`. |
+| `getRow(app, entityId, accountId)` | Fetches one row; `null` on 404. |
+| `deleteRow(app, entityId, accountId, opts?)` | Soft-deletes a row. `opts.deletedAt` (epoch ms) stamps the tombstone for delete-vs-edit LWW. |
+| `device(app, { accountId, deviceId, lastSeenSeq })` | Updates this device's cursor. |
+| `getSalt(accountId)` / `putSalt(accountId, salt)` | Reads/registers the account root-key salt as a `Uint8Array` (first-write-wins). |
+| `intentsBatch(accountId, events)` | Appends intent events (`{ eventId, envelope, expiresAt }`) via `POST /intents/batch`. **Insert-only** — a re-sent `eventId` is a server no-op, so retrying an ambiguous failure is safe. Returns `{ written, maxSeq }`. |
+| `intentsList(accountId, { since, limit })` | One ascending page of non-expired intent events with `seq > since` via `GET /intents/list`. Returns `{ rows, hasMore }`; the server pages at 500. |
+
+The intents surface is **pure transport**: the `envelope` is opaque here and is
+never decoded or inspected. The codec (and the per-envelope key derivation it
+needs) stays in `@glance-apps/intents` — see `deriveKeyForSalt` above.
+
+### Request diagnostics
+
+One brake, one budget meter and one write-loop history per **bundle realm** —
+not per client instance. They model a resource every client in the process
+shares: the server's per-IP request budget. A separately bundled copy of this
+package (an Obsidian plugin, say) is its own realm, which is correct — it is a
+separate process with its own traffic.
+
+| Export | Description |
+|--------|-------------|
+| `isVaultRateLimited()` | `true` while the brake is engaged. For callers that would rather sit a whole cycle out pre-flight than fire a call and catch the rejection. |
+| `vaultBrakeStatus()` | `{ braked, until, memoryMs, retryInMs }`. |
+| `getVaultStats()` | The brake status, the last rolling minute of requests attributed by method, and the entityIds whose write history looks loop-shaped. |
+| `configureVaultDiagnostics(options?)` | Tunes the visibility-only thresholds (`softLimitPerMinute`, `loopTransitions`, `loopWindowMs`) and the log sink. The brake's curve is deliberately not configurable. |
+| `resetVaultDiagnostics()` | Clears all three and restores the defaults. For tests. |
+
+**The brake.** Any real 429 pauses *every* vault request in the realm: 30s,
+doubling per burst to a 10-minute ceiling, with one arming per burst. While
+paused, calls fail fast — before touching the network — with a `VaultError`
+carrying `status: 429`, `code: 'RATE_LIMITED'` and `retryInMs`, so retry
+ladders that already treat a real 429 as transient need no new code. A 2xx
+releases the pause but only **halves** the escalation memory, so the next 429
+re-arms at the storm's level; a genuine recovery drains to zero in a few quiet
+successes. An over-quota 429 (`QUOTA_EXCEEDED`) is not a limiter hit and does
+not arm it. `createVaultClient({ brake: false })` opts a client out of both the
+gate and the arming — an escape hatch for tests.
+
+**The budget meter and the write-loop detector** are visibility only; neither
+ever throttles. The meter logs one attributable line per minute when a realm
+crosses a soft threshold (default 300/min, half the server's default per-IP
+budget). The detector keeps a bounded per-`entityId` write history across
+`batch` upserts and `deleteRow` and warns once per id per window when four
+**redundant** writes to one id land inside ten minutes. A write is redundant
+when it cannot have changed the row's state from what the previous write to
+that id left it: a polarity flip, a **repeat delete** (re-deleting a row that
+is already a tombstone — the shape of the incident this was built for), or a
+rewrite with identical content. It fails open by construction: the rule needs
+repeated events on the *same* id, so a first sync, a large import, or a bulk
+cleanup deleting many different ids can never trigger it.
+
 ### Auto-backup
 
 | Export | Description |
@@ -173,6 +238,8 @@ TypeScript declarations for all of the above are in `types/index.d.ts`.
 
 This package follows [Semantic Versioning](https://semver.org/). Breaking changes to the public API or the sync envelope schema will result in a major version bump.
 
+A **change to a behavioural default** counts as breaking and gets a major bump too, even when no API signature changes. The test is whether a dependent that changes none of its own code can get different behaviour by taking the release: if it can, the version number has to say so, because the changelog is the thing people skip. 2.0.0 flipped the pull-cursor default from per-page to end-of-pull on exactly this basis.
+
 ## License
 
 MIT

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@glance-apps/sync",
-  "version": "1.10.0",
+  "version": "2.0.0",
   "license": "MIT",
   "type": "module",
   "main": "src/index.js",