@glance-apps/sync 1.6.1 → 1.10.0

package/CHANGELOG.md

@@ -7,6 +7,349 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [1.10.0] - 2026-08-01
+
+**Should you take this release?** Yes, for any deployment using the database
+(GLANCEvault) transport. Until now the DB sync engine had **no backoff of any
+kind**: on a persistent failure it re-sent the same request on every cycle,
+forever, at whatever rate the app polls. A server that was down, a device
+offline, a wrong token — all of them produced an unbounded request loop. This
+release adds the backoff the file tier has always had, and fixes two related
+defects found alongside it. The file-tier (WebDAV) engine is untouched.
+
+### Fixed
+
+- **The DB engine no longer hammers a failing server.** Both directions now
+  back off: 30s doubling to a 15-minute cap for pushes and a 5-minute cap for
+  pulls, with a flat one hour on an authentication failure — the file tier's
+  numbers, escalation curve and reset rule. Any success resets the window and
+  its strike count. A window only ever **delays**: the next ordinary cycle
+  after it expires probes again, with no user action and no restart, and
+  dirty rows are never dropped — backoff changes *when* a push is retried,
+  never *whether* it is.
+  - Unlike the file tier's, the DB engine's backoff is **self-enforcing**: the
+    engine skips its own work when called too soon. (The file tier's is
+    advisory — nothing reads `uploadBackoffUntil` except its getter, so
+    honouring it is the app's job. That is exactly why the hammering was live
+    in every published version.) `getUploadBackoffUntil()` and
+    `getDownloadBackoffUntil()` are exposed with the same names and meaning as
+    the file tier's so one scheduler can treat both tiers alike, but honouring
+    them is now an optimisation rather than a duty.
+- **A failed pull no longer aborts the cycle.** 1.9.0 contained push failures
+  but left pull failures throwing past the device-cursor report. A pull error
+  is now captured the same way, so the cursor still reports on a cycle whose
+  pull failed.
+- **The pull cursor is persisted per page.** It was written only after the
+  whole pagination loop, so a failure on page 3 discarded the progress of
+  pages 1 and 2 — a large backlog on a flaky connection could re-download the
+  same pages indefinitely and never converge. Every row in a page reaches a
+  terminal outcome (applied, quarantined, or skipped as engine-reserved)
+  before the page ends, so the cursor may safely advance past each completed
+  page.
+- **A rejected credential surfacing from the push or pull now halts.** Once
+  the key is verified in-session, `ensureRootKey` makes no network call, so a
+  mid-session revocation is first seen by the push or the pull — and 1.9.0's
+  error capture routed those away from the halt path. They now reach exactly
+  the same halt logic as before; the halt itself, its identity rule, and the
+  `invalid credential` body match are unchanged.
+
+### Changed
+
+- **Over-quota suppression is now one case of the backoff ladder**, not a
+  parallel mechanism. Its behaviour is unchanged — same curve, same
+  in-memory lifetime, still surfacing `QUOTA_EXCEEDED` with its descriptor on
+  every suppressed cycle so apps can render "X of Y used" — but
+  `getQuotaState()` and `isQuotaSuppressed()` are now derived from the push
+  window rather than a separate variable.
+- **A shared-mode 401 is now delayed rather than retried immediately.** It
+  opens the one-hour auth window instead of being re-sent every cycle. It is
+  still not terminal and still not the credential halt: it self-resumes.
+- New: `getBackoffState()` returns `{ push, pull }` windows with `until`,
+  `reason` (`'quota' | 'auth' | 'transport'`), `code`, `strikes` and `since`.
+  The `reason` is what lets an app distinguish a one-hour auth backoff from a
+  thirty-second transport one without inferring it from the timestamp.
+- The cycle result gains `pullFailed`, `pullSkipped` and `pullErrorCode`,
+  mirroring the push fields added in 1.9.0.
+
+### Notes
+
+- **Classes that are never delayed**, deliberately: `PASSPHRASE_REQUIRED` and
+  `ACCOUNT_ID_REQUIRED` fire before any network call and clear the instant the
+  app supplies the value, so a window would make the app feel broken for 30
+  seconds after a user types their passphrase; and `CREDENTIAL_INVALID` is
+  owned by the (terminal) credential halt, so stacking a window on it would be
+  meaningless at best. `QUOTA_EXCEEDED` opens exactly one window, its own.
+- **For app integration — do not render backoff from the last `onError`
+  message.** On a cycle suppressed by a quiet (transport or auth) window, the
+  cycle's usual `onError(null, null, false)` reset still fires and nothing is
+  re-surfaced, so an app that remembers the last error string will see it
+  *cleared* while `onStatusChange` reads `'error'`. This is deliberate (quiet
+  windows report once; repeating every cycle is the noise this release
+  removes). Render standing backoff from `getBackoffState()` — `reason` plus
+  `until` gives "retrying in Xs" — and treat `onError` as an event stream, not
+  a status store. Quota windows are the exception and keep re-surfacing
+  `QUOTA_EXCEEDED` with the descriptor each cycle.
+- The cycle result now reports a superseded stale instance truthfully:
+  `{ superseded: true }` rather than `{ halted: true }`, matching
+  `isSuperseded()` / `isCredentialHalted()` whichever path the credential
+  rejection arrived by.
+- A healthy client is unaffected: no window is ever opened, no added latency,
+  and an identical request pattern.
+
+## [1.9.0] - 2026-08-01
+
+**Should you take this release?** Yes if you run, or might run, a GLANCEvault
+server with a row cap configured (`GLANCEVAULT_QUOTA_*`, server Phase 3.2).
+Also yes for the sync-reliability fix below, which applies to **every**
+deployment including shared mode and unconfigured servers: before this
+release, a failed push also cost you the pull, so a device whose writes were
+failing silently stopped receiving other devices' changes too. Nothing here
+changes behavior against a server with no quota configured, which is and will
+remain the common case.
+
+### Fixed
+
+- **A failed push no longer prevents the pull.** The DB sync cycle threw
+  straight from a push failure to its error handler, so `pullRemoteChanges`
+  and the device-cursor report never ran. A device that could not write — for
+  *any* reason: a transport blip, a 500, a server-side cap — also stopped
+  **reading**, and silently fell behind its peers. The push error is now
+  captured, the pull and cursor run regardless, and the error is reported
+  after. Applies to every transport failure, not just the quota case that
+  exposed it. Behavior on a fully successful cycle is unchanged.
+  - A cycle whose push failed but whose pull succeeded now returns
+    `{ applied, skipped, skippedEntityIds, pushFailed: true, pushErrorCode,
+    quota }` — the pull's own numbers are reported as usual, and `onError`
+    fires with the push error's code and `isHardStop: false`. `lastSynced` is
+    deliberately **not** advanced (the device is not fully in step), and the
+    dirty rows are retained for the next cycle exactly as before.
+- **A quota rejection is no longer mislabelled "update your server".** The key
+  verifier's establishing write is a net-new entity, so a fresh device on an
+  account at the row cap gets a 413 there — and every error on that path was
+  relabelled `VERIFIER_UNSUPPORTED` ("Your GLANCEvault server needs to be
+  updated to support key verification"), sending the user to fix a server
+  that is working correctly. `QUOTA_EXCEEDED` now passes through that
+  classifier, the same way `CREDENTIAL_INVALID` and the client-readiness
+  codes already did.
+
+### Added
+
+- **Over-quota handling for the reachable dimension (server Phase 3.2 /
+  client Phase 3.3).** A 413 or 429 carrying the server's quota body
+  (`{ error: "quota exceeded", quota, limit, used, requested }`) is now typed
+  as `QUOTA_EXCEEDED` and carries the parsed descriptor, so consumers no
+  longer have to substring-match an error message to tell a quota rejection
+  from any other failure.
+  - **Never a halt.** A quota condition is the opposite of a rejected
+    credential: it clears when the operator raises the limit or reclaim runs
+    — with **no client action at all** — so the engine keeps running.
+    `isHardStop` is always `false`, nothing is persisted, and there is no
+    recovery entry point and nothing to clear.
+  - **Bounded, self-resuming suppression** instead of retrying a doomed write
+    every cycle: after a rejection the engine skips the *write* for 30s,
+    doubling on repeat to a 15-minute cap (the file tier's backoff shape),
+    while pulls and cursor reports continue untouched. The next ordinary
+    cycle after the window probes again; a write that gets through clears the
+    state by itself. Raising the server limit is picked up with no restart,
+    no re-enrollment, and no user action.
+  - New state queries: `getQuotaState()` (the descriptor plus `since` /
+    `retryAt`, in memory only) and `isQuotaSuppressed()`.
+  - **Defensive parse.** A body earns the typed classification only with the
+    exact wording, a non-empty `quota` string, and three finite numbers.
+    A missing field, a non-JSON body, a 413 with no quota fields, the legacy
+    SSE `{"error":"too many connections for account"}` 429, or an older
+    server that never sends this shape all degrade to the generic
+    `VAULT_ERROR` handled before — never a throw, never a mis-classification.
+    An **unrecognised** `quota` dimension from a newer server is passed
+    through verbatim and still treated as a quota condition, since the remedy
+    is the same; render an unfamiliar dimension generically.
+
+### Notes
+
+- **Scope: `rows` is the only quota dimension this package can reach.** The
+  server enforces `storage` and `concurrent-uploads` at the blob path, and
+  `intents` at the intents route; this package has no blob path (media is
+  stripped at the adapter boundary) and no intents path (that is
+  `@glance-apps/intents`), and no SSE client. So "text syncs while media
+  fails" is not something this package can exercise. Handling for those
+  dimensions belongs with the media transport work that would call them, and
+  was deliberately not written here.
+- **Known gap, not fixed here:** the DB engine has no backoff of any kind
+  outside the quota window above. On a persistent write failure the same
+  dirty rows are re-pushed on every cycle indefinitely, where the file tier
+  has exponential backoff with an auth-failure escalation. This is a real
+  defect with a live blast radius and it wants its own change rather than a
+  corner of this one.
+
+## [1.8.0] - 2026-07-31
+
+### Added
+
+- **Recovery from a halted credential (GLANCEvault Phase 2.2).** The 1.7.0
+  credential halt was built with no exit; this release adds the one exit:
+  **`recoverVaultSyncEngine(config)`**, user-initiated re-enrollment with the
+  bootstrap secret. Since server Phase 2.1, enrolling the same byte-exact
+  `(accountId, deviceId)` revokes every still-active predecessor credential
+  in the same transaction, so recovery is a rotation — the dead credential is
+  superseded, not accumulated alongside.
+  - **Halt-gated, structurally.** The call refuses with `NOT_HALTED` unless
+    the device is credential-halted, so it cannot be wired into a startup or
+    retry path as an on-demand rotator. Combined with the package holding no
+    secret (one must be supplied fresh, and it is confined to the call
+    exactly as in `connectVaultSyncEngine` — never stored, logged, or
+    retained), user initiation is structural, not procedural. **No code path
+    recovers automatically**, including transport failures, startup, or a
+    halt discovered at launch.
+  - **Per-account only, no fallback.** Unlike connect, recovery never falls
+    back on discovery failure: an unreachable `/healthz` is
+    `VAULT_UNREACHABLE` and a non-per-account mode is
+    `RECOVERY_UNSUPPORTED`; both leave the device halted and untouched.
+    Shared mode has no credentials, no halt, and no reachable recovery path.
+  - **Order of operations**, each failure leaving the device halted rather
+    than ambiguous: halt gate → deviceId resolution → mode guard → storage
+    canary (now restores the slot's prior contents, so a failed recovery
+    keeps the stale record byte-identical) → enroll → persist + read-back
+    verify (overwriting the stale record — it does not survive recovery) →
+    **clear the halt last** → return a fresh engine. The old engine's client
+    closes over the dead credential; apps must swap to the returned engine.
+  - **deviceId: the stale record's value wins.** Rotation only lands if
+    enrollment uses the identity the dead credential is bound to, and the
+    stored record is ground truth for that. An explicit `config.deviceId`
+    that differs is surfaced as `DEVICE_ID_CONFLICT`, never resolved
+    silently (either choice orphans a credential or a cursor). With no
+    readable record: explicit value, else the persisted package-owned id.
+    Never minted fresh.
+- **Halt-set identity rule** (the stale-engine hazard fix). The halt key is
+  shared by every engine instance on a device, so an old instance still
+  holding the superseded credential could previously re-halt the device
+  after recovery. Now, on `CREDENTIAL_INVALID`, the engine halts **unless**
+  the stored credential record exists, is readable, and holds a *different*
+  credential than the engine's bearer — definitive proof the instance was
+  superseded. **The rule fails toward halting**: a missing or unreadable
+  record still halts (the naive "no record → no halt" would retry a dead
+  credential forever, exactly what the halt exists to prevent). A superseded
+  instance goes **inert in memory** instead — surfaced once via
+  `onError('CREDENTIAL_INVALID', isHardStop true)`, then silent, zero
+  network, never touching the shared halt key — killing both the bricking
+  and the pointless server load. New state query: `isSuperseded()`.
+- The 1.4b minted-but-lost-persistence residual is now self-healing: a
+  credential minted and then lost to a persistence failure is superseded by
+  the next successful enrollment for the same `(accountId, deviceId)`.
+
+### Fixed
+
+- **Device cursors now report on installs that never configured a
+  `deviceId`, automatically on upgrade.** Such installs have never updated
+  their device cursor: the cursor step silently no-opped (`{updated:
+  false}`) whenever no `deviceId` was passed, so the server's tombstone GC
+  had no cursor for those devices. On upgrade, a device id is generated and
+  persisted (`{prefix}-device-id`) at engine construction, and cursor
+  reporting begins on the first completed sync cycle — **no app changes
+  required**; this applies to plain `createDbSyncEngine` construction, not
+  only the new connect flow, and identically in shared and per-account
+  mode. Installs that already pass an explicit `deviceId` are unaffected
+  and keep using it. Operator note: an install this fixes has no prior
+  cursor history, so the newly generated id strands nothing.
+  (This shipped in 1.7.0's device-identity work but 1.7.0 was never
+  published; it reaches consumers with this release.)
+
+### Upgrade notes
+
+Two behavior changes arrive with the fix above, without opt-in:
+
+- **Engine construction now touches localStorage.** `getOrCreateDeviceId`
+  reads and writes `{prefix}-device-id` at construction time; previously
+  construction deferred all localStorage access to cycle time. In an
+  environment without a `localStorage` global, construction now fails where
+  it previously succeeded and failed later at first sync. Theoretical for
+  browser and WebView hosts (all three GLANCE apps), but if construction
+  throws before any network activity, look here first.
+- **Injected `vaultClient` consumers see new `device()` calls.** An engine
+  constructed with `config.vaultClient` and no `deviceId` previously made
+  zero `device()` calls; it now makes one per cycle. A mock or bridge
+  client without a `device()` method will throw inside the cursor step —
+  swallowed and warned, never fatal — but the console warning is new and
+  recurring. Affects tests with mock clients and native shells that inject
+  a bridge client.
+
+## [1.7.0] - 2026-07-31
+
+### Added
+
+- **Per-account vault auth, client half (GLANCEvault Phase 1.4b).** A
+  GLANCEvault server can run `authMode: 'per-account'`, where each device
+  authenticates with its own enrolled credential instead of the instance-wide
+  shared device token. The package owns the whole client-side flow — apps hand
+  it a bootstrap secret (or the shared token) and get a working engine back,
+  without implementing discovery, branching, credential persistence, or
+  secret discard themselves. Shared mode remains the default path and is
+  byte-for-byte unchanged.
+  - **`connectVaultSyncEngine(config)`** — the packaged connect flow.
+    Discovers the server's auth mode from `/healthz`, then branches:
+    `shared` → builds the engine from `config.vaultToken` exactly as before;
+    `per-account` → uses the credential persisted under
+    `{prefix}-vault-credential`, enrolling with `config.enrollmentSecret`
+    only when no credential is stored for this exact (server, account). The
+    credential is persisted durably (write + read-back verify) **before**
+    enrollment is treated as complete, and a storage canary runs **before**
+    minting so broken storage cannot orphan one server row per launch. The
+    bootstrap secret is never written to storage, never logged, never placed
+    on the engine config, and not retained past the call. **No code path
+    re-enrolls automatically** — once a credential is stored it is always
+    reused, even after rejection (recovery is Phase 2.2), so revocation
+    (vault Phase 2.1) cannot be silently undone by a client loop. Discovery
+    failure (server unreachable, unexpected body) falls back to the last
+    known auth state — stored credential, else `vaultToken` — which is
+    exactly the pre-1.4b behavior for existing installs; an unrecognized
+    future mode string does the same rather than guessing. A `/healthz`
+    without `authMode` (a pre-1.4a server) is ordinary shared mode, not an
+    error. Typed failures: `VAULT_TOKEN_REQUIRED`,
+    `ENROLLMENT_SECRET_REQUIRED`, `CREDENTIAL_PERSIST_FAILED`,
+    `VAULT_UNREACHABLE`, plus the enrollment errors below.
+  - **`getOrCreateDeviceId(storageKeyPrefix)`** — the package now owns the
+    stable device identity: generated once (`crypto.randomUUID`, with a
+    `getRandomValues` fallback for old WebViews) and persisted under
+    `{prefix}-device-id`; used for both the device cursor and enrollment. An
+    explicit `config.deviceId` still wins. This also fixes a latent bug: the
+    engine's device-cursor update silently no-oped (`{updated:false}`) when
+    no `deviceId` was configured, so such installs never reported cursors —
+    on upgrade they get a generated id and start reporting for the first
+    time (server-side effect: a `devices` row appears and tombstone GC gains
+    an accurate cursor; pure improvement, no data risk).
+  - **Substrate exports** (used by the flow, available standalone):
+    `fetchVaultHealth({ vaultUrl, fetchImpl? })` — unauthenticated
+    `GET /healthz`, `authMode` normalized to `'shared'` when the server
+    predates the field; and `enrollVaultDevice({ vaultUrl, enrollmentSecret,
+    accountId, deviceId, fetchImpl? })` — `POST /enroll`, secret in the body
+    only, values sent byte-exact, typed `ENROLLMENT_REJECTED` (401) /
+    `ENROLLMENT_UNSUPPORTED` (404, shared-mode server) failures, and
+    `*_REQUIRED` codes thrown before touching the wire on missing fields.
+  - **`CREDENTIAL_INVALID` and the credential halt.** `createVaultClient`
+    now reads error bodies best-effort: a 401 whose body says
+    `invalid credential` (the server's chosen signal for "this device must
+    re-enroll") surfaces as code `CREDENTIAL_INVALID`; a missing, non-JSON,
+    or unrecognized body degrades to the generic `VAULT_ERROR`, so client
+    correctness never depends on server wording. On `CREDENTIAL_INVALID` the
+    DB engine persists a stop under `{prefix}-db-sync-credential-halt`,
+    surfaces `onError(message, 'CREDENTIAL_INVALID', /* isHardStop */ true)`,
+    and makes **no further network calls** — across restarts. Nothing in
+    this version clears the halt (that is Phase 2.2's recovery flow);
+    `isCredentialHalted()` / `getCredentialHalt()` expose the state.
+    Shared-mode 401s (`invalid device token`) keep today's retryable
+    behavior. The key verifier passes `CREDENTIAL_INVALID` through instead
+    of relabeling it `VERIFIER_UNSUPPORTED`, so `allowUnverified` can never
+    wave a rejected device through.
+  - `createVaultClient` is otherwise unchanged on the wire; `vaultToken`
+    accepts either the shared device token or an enrolled credential — the
+    credential IS the Bearer token.
+
+### Security notes
+
+- The persisted credential lives in localStorage, exactly as exposed as the
+  shared device token and the file tier's provider passwords are today:
+  parity, not a regression. Secure-storage migration is explicitly out of
+  scope for this phase.
+
 ## [1.6.1] - 2026-07-19
 
 ### Fixed

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@glance-apps/sync",
-  "version": "1.6.1",
+  "version": "1.10.0",
   "license": "MIT",
   "type": "module",
   "main": "src/index.js",