@glance-apps/sync 1.10.0 → 1.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +119 -0
- package/README.md +65 -0
- package/package.json +1 -1
- package/src/index.js +8 -0
- package/src/vaultClient.js +557 -20
- package/types/index.d.ts +109 -1
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,125 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [1.11.0] - 2026-08-30
|
|
11
|
+
|
|
12
|
+
**Should you take this release?** Yes if you use the GLANCEvault transport, and
|
|
13
|
+
immediately if anything in your bundle writes to it on a timer. This release is
|
|
14
|
+
the client-side answer to a 2026-08-30 incident in which an intent-cleanup loop
|
|
15
|
+
in the dayGLANCE Obsidian plugin deleted rows that were already tombstones. The
|
|
16
|
+
server re-tombstoned on every call — fresh seq, fresh SSE nudge — which put the
|
|
17
|
+
row back above the caller's cursor and fed the loop its own output. It ran for
|
|
18
|
+
hours and saturated the server's per-IP rate budget, and it was **invisible**:
|
|
19
|
+
every delete was fire-and-forget behind a `.catch(() => {})`, so nothing failed
|
|
20
|
+
loudly enough to notice. Two patches landed elsewhere (the plugin no longer
|
|
21
|
+
deletes tombstones; the vault's `softDeleteRow` is now idempotent). This release
|
|
22
|
+
adds the protection and the visibility **by construction**, so no caller has to
|
|
23
|
+
remember them and no loop can be silent. No server changes: the wire contract is
|
|
24
|
+
untouched.
|
|
25
|
+
|
|
26
|
+
### Added
|
|
27
|
+
|
|
28
|
+
- **The intents transport.** `intentsBatch(accountId, events)` and
|
|
29
|
+
`intentsList(accountId, { since, limit })` wrap the server's existing
|
|
30
|
+
`/intents/*` endpoints, so apps can drop their hand-rolled raw-fetch
|
|
31
|
+
transports and inherit this client's auth, error classification, brake and
|
|
32
|
+
meter. `intentsBatch` is insert-only server-side — a re-sent `eventId` is a
|
|
33
|
+
no-op — so retrying an ambiguous failure is safe by construction.
|
|
34
|
+
- **Pure transport**: the `envelope` is opaque here and is never decoded or
|
|
35
|
+
inspected. The codec, and the per-envelope key derivation it needs, stays
|
|
36
|
+
in `@glance-apps/intents`; this package carries the transport without
|
|
37
|
+
carrying the crypto.
|
|
38
|
+
- **A device-wide request brake.** Any real 429 now pauses *every* vault
|
|
39
|
+
request in the bundle realm: 30s, doubling per burst to a 10-minute ceiling,
|
|
40
|
+
one arming per burst (a concurrent fan-out that all meets the limiter costs
|
|
41
|
+
one escalation step, not ten). While paused, calls fail fast — **before
|
|
42
|
+
touching the network** — with a `VaultError` carrying `status: 429`,
|
|
43
|
+
`code: 'RATE_LIMITED'` and `retryInMs`.
|
|
44
|
+
- It is at **module scope, not per client instance**, deliberately: the thing
|
|
45
|
+
it protects is shared by every client in the process (the server's per-IP
|
|
46
|
+
budget), so the state that models it belongs to the realm. A separately
|
|
47
|
+
bundled copy — the Obsidian plugin's, say — is its own realm, which is
|
|
48
|
+
correct: separate process, separate traffic.
|
|
49
|
+
- **Decay, never amnesty.** A 2xx releases the pause but only *halves* the
|
|
50
|
+
escalation memory (below the 30s base it drains to zero). This is the fix
|
|
51
|
+
for the earlier app-side brake, which reset escalation to zero on *any*
|
|
52
|
+
success: on a saturated shared budget an occasional cheap request slips
|
|
53
|
+
into a fresh limiter window and returns 200, and each lucky 200 wiped the
|
|
54
|
+
whole 30s→480s escalation and re-licensed full cadence. Halving keeps the
|
|
55
|
+
storm's level in memory — the next 429 re-arms at 2× what is left — while a
|
|
56
|
+
genuine recovery still drains to nothing in a few quiet successes.
|
|
57
|
+
- Reads: `isVaultRateLimited()` and `vaultBrakeStatus()`, for callers that
|
|
58
|
+
prefer to sit a whole cycle out pre-flight. Escape hatch:
|
|
59
|
+
`createVaultClient({ brake: false })` opts a client out of both the gate
|
|
60
|
+
and the arming.
|
|
61
|
+
- **A budget meter.** Requests are counted per rolling minute and attributed by
|
|
62
|
+
method name. Crossing a soft threshold (default 300/min — half the server's
|
|
63
|
+
default per-IP budget, configurable) logs **one** line per window naming the
|
|
64
|
+
top contributors: `[vault] budget: 412 requests in the last minute (soft
|
|
65
|
+
limit 300) — top: intentsList 210, batch 130, deleteRow 72`. Visibility only;
|
|
66
|
+
it never throttles. Its whole argument is that it needs no theory about *why*
|
|
67
|
+
a loop is running, so the next storm is an attributed event on day one rather
|
|
68
|
+
than an afternoon of archaeology.
|
|
69
|
+
- **A write-loop detector — the success-side signal.** Failure visibility
|
|
70
|
+
already existed; what was missing on the day was the signal for everything
|
|
71
|
+
"succeeding" pathologically. The client now keeps a bounded per-`entityId`
|
|
72
|
+
history across `batch` upserts and `deleteRow`, and warns loudly (once per id
|
|
73
|
+
per window) when four **redundant** writes to one id land inside ten minutes.
|
|
74
|
+
Diagnostic only — never a brake.
|
|
75
|
+
- A write is redundant when it cannot have changed the row's state from what
|
|
76
|
+
the previous write to that id left it. Three shapes qualify: a **polarity
|
|
77
|
+
flip** (the delete-and-resupply loop), a **repeat delete** (re-deleting a
|
|
78
|
+
row that is already a tombstone — the incident's own shape: same polarity
|
|
79
|
+
every time, no content at all), and an **identical-content rewrite**.
|
|
80
|
+
- **Fails open by construction**: the rule needs repeated events on the
|
|
81
|
+
*same* id, so a first sync, a large import, or a bulk cleanup deleting many
|
|
82
|
+
different ids — many ids, one event each — can never trigger it, however
|
|
83
|
+
large. A bounded retry of one failed delete does not reach the threshold
|
|
84
|
+
either.
|
|
85
|
+
- Both thresholds are tunable via `configureVaultDiagnostics`. Four redundant
|
|
86
|
+
writes means five events (upsert/delete/upsert/delete/upsert, or five
|
|
87
|
+
deletes of one tombstone), which no human edit pattern produces and every
|
|
88
|
+
delete/resupply or re-tombstone loop produces within seconds; ten minutes
|
|
89
|
+
is long enough to catch a loop running on a 60s sync cycle and short enough
|
|
90
|
+
that a week of ordinary edits never accumulates into it.
|
|
91
|
+
- `getVaultStats()` returns all three in one read: brake status, the last
|
|
92
|
+
rolling minute of requests by method, and the current write-loop suspects.
|
|
93
|
+
`configureVaultDiagnostics(options)` tunes the visibility-only thresholds and
|
|
94
|
+
the log sink; `resetVaultDiagnostics()` clears everything (for tests).
|
|
95
|
+
|
|
96
|
+
### Changed
|
|
97
|
+
|
|
98
|
+
- Every `createVaultClient` method now passes through the brake and the meter.
|
|
99
|
+
The only behaviour change a healthy caller can observe is the new
|
|
100
|
+
`RATE_LIMITED` rejection while a window is open — and only after the server
|
|
101
|
+
has already answered a real 429.
|
|
102
|
+
- A **429 arms the brake only when it is a rate-limiter hit**, never when it is
|
|
103
|
+
an over-quota rejection. The server uses 429 for both and they call for
|
|
104
|
+
opposite handling: a limiter hit means "stop asking for a while" (exactly
|
|
105
|
+
what the brake does), while a quota rejection means the account is full,
|
|
106
|
+
clears only when an operator acts, and already has its own self-resuming
|
|
107
|
+
window in the DB engine. Gating on it would hide `QUOTA_EXCEEDED` and its
|
|
108
|
+
descriptor behind `RATE_LIMITED` and leave apps unable to render "X of Y
|
|
109
|
+
used". The quota shape is recognised as narrowly as before, so anything that
|
|
110
|
+
is not unmistakably a quota body is treated as a limiter hit.
|
|
111
|
+
|
|
112
|
+
### Notes
|
|
113
|
+
|
|
114
|
+
- **Not gated**: `fetchVaultHealth` and `enrollVaultDevice`. Both are
|
|
115
|
+
pre-credential, user-initiated, one-shot calls — `/healthz` is the
|
|
116
|
+
unauthenticated probe you reach for *while* diagnosing a stuck client, and
|
|
117
|
+
refusing it during a braked window would take away the diagnostic just when
|
|
118
|
+
it is wanted.
|
|
119
|
+
- **For lastGLANCE / lifeGLANCE**: safe to take at leisure (semver-minor).
|
|
120
|
+
During a braked window, client calls fail fast with `VaultError { status:
|
|
121
|
+
429, code: 'RATE_LIMITED' }` instead of hitting the wire; any ladder that
|
|
122
|
+
already treats a real 429 as transient — they all do — handles it unchanged.
|
|
123
|
+
The meter and the write-loop warnings come along for free.
|
|
124
|
+
- An AES-GCM envelope carries a fresh random IV, so identical plaintext does
|
|
125
|
+
not produce an identical envelope: for encrypted payloads it is the polarity
|
|
126
|
+
flip that fires the detector, and the identical-content rule covers plaintext
|
|
127
|
+
and deterministic payloads.
|
|
128
|
+
|
|
10
129
|
## [1.10.0] - 2026-08-01
|
|
11
130
|
|
|
12
131
|
**Should you take this release?** Yes, for any deployment using the database
|
package/README.md
CHANGED
|
@@ -158,6 +158,71 @@ Pass `encryptionEnabled: true` in the sync config object stored via `engine.setC
|
|
|
158
158
|
| `createProviders(config)` | Creates cloud sync provider objects (`nextcloud`, `koofr`, `webdav`). |
|
|
159
159
|
| `normalizeEtag(raw)` | Normalizes a raw ETag header value for If-Match use: strips a weak-validator prefix (`W/"abc"` → `"abc"`) and the content-coding suffixes some servers append inside the quotes (`"abc-gzip"` / `"abc-br"` → `"abc"`). Quotes are preserved; `null`/`undefined` pass through unchanged. |
|
|
160
160
|
|
|
161
|
+
### GLANCEvault client
|
|
162
|
+
|
|
163
|
+
| Export | Description |
|
|
164
|
+
|--------|-------------|
|
|
165
|
+
| `createVaultClient(config)` | HTTP client for a GLANCEvault server. `config` is `{ vaultUrl, vaultToken, fetchImpl?, brake? }`. Every method carries the Bearer token, classifies errors uniformly (`CREDENTIAL_INVALID`, `QUOTA_EXCEEDED`, `RATE_LIMITED`, else `VaultError` with `.status`), and passes through the module-scope brake and budget meter. |
|
|
166
|
+
| `fetchVaultHealth(config)` | Unauthenticated `GET /healthz`. Returns `{ status, version, schemaVersion, authMode }`, with `authMode` normalized to `'shared'` on servers that predate the field. |
|
|
167
|
+
| `enrollVaultDevice(config)` | Exchanges the admin-configured bootstrap secret for this device's own credential (`POST /enroll`). The credential is returned once and never again. |
|
|
168
|
+
|
|
169
|
+
**`VaultClient` methods:**
|
|
170
|
+
|
|
171
|
+
| Method | Description |
|
|
172
|
+
|--------|-------------|
|
|
173
|
+
| `batch(app, { accountId, rows })` | Upserts rows (`{ entityId, envelope, createdAt }`). Returns `{ written, maxSeq }`. |
|
|
174
|
+
| `list(app, { accountId, since })` | One page of rows with `seq > since`. Returns `{ rows, hasMore }`. |
|
|
175
|
+
| `getRow(app, entityId, accountId)` | Fetches one row; `null` on 404. |
|
|
176
|
+
| `deleteRow(app, entityId, accountId, opts?)` | Soft-deletes a row. `opts.deletedAt` (epoch ms) stamps the tombstone for delete-vs-edit LWW. |
|
|
177
|
+
| `device(app, { accountId, deviceId, lastSeenSeq })` | Updates this device's cursor. |
|
|
178
|
+
| `getSalt(accountId)` / `putSalt(accountId, salt)` | Reads/registers the account root-key salt as a `Uint8Array` (first-write-wins). |
|
|
179
|
+
| `intentsBatch(accountId, events)` | Appends intent events (`{ eventId, envelope, expiresAt }`) via `POST /intents/batch`. **Insert-only** — a re-sent `eventId` is a server no-op, so retrying an ambiguous failure is safe. Returns `{ written, maxSeq }`. |
|
|
180
|
+
| `intentsList(accountId, { since, limit })` | One ascending page of non-expired intent events with `seq > since` via `GET /intents/list`. Returns `{ rows, hasMore }`; the server pages at 500. |
|
|
181
|
+
|
|
182
|
+
The intents surface is **pure transport**: the `envelope` is opaque here and is
|
|
183
|
+
never decoded or inspected. The codec (and the per-envelope key derivation it
|
|
184
|
+
needs) stays in `@glance-apps/intents` — see `deriveKeyForSalt` above.
|
|
185
|
+
|
|
186
|
+
### Request diagnostics
|
|
187
|
+
|
|
188
|
+
One brake, one budget meter and one write-loop history per **bundle realm** —
|
|
189
|
+
not per client instance. They model a resource every client in the process
|
|
190
|
+
shares: the server's per-IP request budget. A separately bundled copy of this
|
|
191
|
+
package (an Obsidian plugin, say) is its own realm, which is correct — it is a
|
|
192
|
+
separate process with its own traffic.
|
|
193
|
+
|
|
194
|
+
| Export | Description |
|
|
195
|
+
|--------|-------------|
|
|
196
|
+
| `isVaultRateLimited()` | `true` while the brake is engaged. For callers that would rather sit a whole cycle out pre-flight than fire a call and catch the rejection. |
|
|
197
|
+
| `vaultBrakeStatus()` | `{ braked, until, memoryMs, retryInMs }`. |
|
|
198
|
+
| `getVaultStats()` | The brake status, the last rolling minute of requests attributed by method, and the entityIds whose write history looks loop-shaped. |
|
|
199
|
+
| `configureVaultDiagnostics(options?)` | Tunes the visibility-only thresholds (`softLimitPerMinute`, `loopTransitions`, `loopWindowMs`) and the log sink. The brake's curve is deliberately not configurable. |
|
|
200
|
+
| `resetVaultDiagnostics()` | Clears all three and restores the defaults. For tests. |
|
|
201
|
+
|
|
202
|
+
**The brake.** Any real 429 pauses *every* vault request in the realm: 30s,
|
|
203
|
+
doubling per burst to a 10-minute ceiling, with one arming per burst. While
|
|
204
|
+
paused, calls fail fast — before touching the network — with a `VaultError`
|
|
205
|
+
carrying `status: 429`, `code: 'RATE_LIMITED'` and `retryInMs`, so retry
|
|
206
|
+
ladders that already treat a real 429 as transient need no new code. A 2xx
|
|
207
|
+
releases the pause but only **halves** the escalation memory, so the next 429
|
|
208
|
+
re-arms at the storm's level; a genuine recovery drains to zero in a few quiet
|
|
209
|
+
successes. An over-quota 429 (`QUOTA_EXCEEDED`) is not a limiter hit and does
|
|
210
|
+
not arm it. `createVaultClient({ brake: false })` opts a client out of both the
|
|
211
|
+
gate and the arming — an escape hatch for tests.
|
|
212
|
+
|
|
213
|
+
**The budget meter and the write-loop detector** are visibility only; neither
|
|
214
|
+
ever throttles. The meter logs one attributable line per minute when a realm
|
|
215
|
+
crosses a soft threshold (default 300/min, half the server's default per-IP
|
|
216
|
+
budget). The detector keeps a bounded per-`entityId` write history across
|
|
217
|
+
`batch` upserts and `deleteRow` and warns once per id per window when four
|
|
218
|
+
**redundant** writes to one id land inside ten minutes. A write is redundant
|
|
219
|
+
when it cannot have changed the row's state from what the previous write to
|
|
220
|
+
that id left it: a polarity flip, a **repeat delete** (re-deleting a row that
|
|
221
|
+
is already a tombstone — the shape of the incident this was built for), or a
|
|
222
|
+
rewrite with identical content. It fails open by construction: the rule needs
|
|
223
|
+
repeated events on the *same* id, so a first sync, a large import, or a bulk
|
|
224
|
+
cleanup deleting many different ids can never trigger it.
|
|
225
|
+
|
|
161
226
|
### Auto-backup
|
|
162
227
|
|
|
163
228
|
| Export | Description |
|
package/package.json
CHANGED
package/src/index.js
CHANGED
|
@@ -47,6 +47,14 @@ export {
|
|
|
47
47
|
// discovery and the bootstrap-secret -> per-device-credential exchange.
|
|
48
48
|
fetchVaultHealth,
|
|
49
49
|
enrollVaultDevice,
|
|
50
|
+
// Module-scope request diagnostics (1.11.0). The brake gates every client
|
|
51
|
+
// call after a real 429; the budget meter and the write-loop detector are
|
|
52
|
+
// visibility only. All three are per bundle realm, not per client.
|
|
53
|
+
isVaultRateLimited,
|
|
54
|
+
vaultBrakeStatus,
|
|
55
|
+
getVaultStats,
|
|
56
|
+
configureVaultDiagnostics,
|
|
57
|
+
resetVaultDiagnostics,
|
|
50
58
|
} from './vaultClient.js';
|
|
51
59
|
// The packaged connect flow: discover -> branch -> enroll if needed ->
|
|
52
60
|
// persist credential -> build engine. Apps hand it a secret or token and get
|
package/src/vaultClient.js
CHANGED
|
@@ -14,6 +14,13 @@
|
|
|
14
14
|
// POST /sync/:app/device update device cursor, { updated }
|
|
15
15
|
// GET /salt/:accountId fetch the account root-key salt
|
|
16
16
|
// PUT /salt/:accountId register a salt (first-write-wins)
|
|
17
|
+
// POST /intents/batch append intent events, { written, maxSeq }
|
|
18
|
+
// GET /intents/list?accountId=&since=&limit= intent events after a cursor, { rows, hasMore }
|
|
19
|
+
//
|
|
20
|
+
// Every client method is gated by a module-scope brake and counted by a
|
|
21
|
+
// module-scope budget meter, and the two write methods keep a bounded
|
|
22
|
+
// per-entity history that warns on loop-shaped traffic. See the diagnostics
|
|
23
|
+
// block below for why those live at module scope and what they cost.
|
|
17
24
|
//
|
|
18
25
|
// Auth models (vault Phase 1.4b — the client half of per-account credentials):
|
|
19
26
|
// a vault runs in one of two modes, discoverable unauthenticated via
|
|
@@ -39,6 +46,425 @@ class VaultError extends Error {
|
|
|
39
46
|
}
|
|
40
47
|
}
|
|
41
48
|
|
|
49
|
+
// ═══════════════════════════════════════════════════════════════════════════
|
|
50
|
+
// Module-scope diagnostics (1.11.0): the brake, the budget meter, the
|
|
51
|
+
// write-loop detector.
|
|
52
|
+
//
|
|
53
|
+
// WHY MODULE SCOPE, NOT PER CLIENT INSTANCE. All three protect (or observe) a
|
|
54
|
+
// resource that is shared by every client in the process: the server's
|
|
55
|
+
// per-IP request budget. Two clients built by the same bundle are two views
|
|
56
|
+
// of one traffic stream, so the state that models that stream belongs to the
|
|
57
|
+
// module — one brake, one meter, one write history per *bundle realm*. The
|
|
58
|
+
// app is one realm; a plugin that bundles its own copy of this package is a
|
|
59
|
+
// separate realm, which is correct: it is a separate process with its own
|
|
60
|
+
// traffic and its own IP budget slice.
|
|
61
|
+
//
|
|
62
|
+
// WHY BY CONSTRUCTION. The incident these exist for (2026-08-30) was a
|
|
63
|
+
// fire-and-forget delete loop that re-tombstoned rows for hours and saturated
|
|
64
|
+
// the server's per-IP budget. Every request "succeeded" or failed into a
|
|
65
|
+
// swallowed `.catch(() => {})`. A protection a caller has to remember to
|
|
66
|
+
// apply is a protection that is missing on the day it matters, so the brake
|
|
67
|
+
// gates every network method of every client, and the meter and the write
|
|
68
|
+
// history observe them all whether or not anyone reads the stats.
|
|
69
|
+
// ═══════════════════════════════════════════════════════════════════════════
|
|
70
|
+
|
|
71
|
+
// --- The brake ---
|
|
72
|
+
// 30s doubling to a 10-minute ceiling. The ceiling is a deliberate stopping
|
|
73
|
+
// point rather than unbounded growth: a client that has waited ten minutes
|
|
74
|
+
// and is still being limited is in a condition an operator has to fix, and
|
|
75
|
+
// waiting twenty would only make the recovery slower once they do.
|
|
76
|
+
const BRAKE_BASE_MS = 30_000;
|
|
77
|
+
const BRAKE_CEILING_MS = 600_000;
|
|
78
|
+
|
|
79
|
+
// --- The budget meter ---
|
|
80
|
+
// 300/min is half of GLANCEvault's default 600/min per-IP budget: crossing
|
|
81
|
+
// half the budget in a minute is not yet a failure, which is exactly when a
|
|
82
|
+
// warning is still useful. The window is rolling; samples older than a minute
|
|
83
|
+
// are pruned on every record, and the buffer is hard-capped so a runaway loop
|
|
84
|
+
// can never grow it without bound (a dropped sample only under-counts a
|
|
85
|
+
// number that is already far past the threshold).
|
|
86
|
+
const BUDGET_WINDOW_MS = 60_000;
|
|
87
|
+
const DEFAULT_SOFT_LIMIT = 300;
|
|
88
|
+
const BUDGET_SAMPLE_CAP = 20_000;
|
|
89
|
+
const BUDGET_TOP_N = 3;
|
|
90
|
+
|
|
91
|
+
// --- The write-loop detector ---
|
|
92
|
+
// K=4 redundant writes inside M=10 minutes. A write is redundant when it
|
|
93
|
+
// cannot have changed the row's state from what the previous write to the
|
|
94
|
+
// same entityId left it: a polarity flip, a repeat delete, or an
|
|
95
|
+
// identical-content rewrite (see isRedundantWrite for why each counts). K=4
|
|
96
|
+
// means five events — upsert/delete/upsert/delete/upsert, or five deletes of
|
|
97
|
+
// one tombstone — which no human edit pattern produces and every
|
|
98
|
+
// delete/resupply or re-tombstone loop produces within seconds.
|
|
99
|
+
// M=10min is long enough to catch a slow loop running on a 60s sync cycle
|
|
100
|
+
// and short enough that a week of ordinary edits never accumulates into it.
|
|
101
|
+
// Both are configurable via configureVaultDiagnostics for callers whose
|
|
102
|
+
// cadence makes different numbers right.
|
|
103
|
+
//
|
|
104
|
+
// FAILS OPEN BY CONSTRUCTION: the rule is per-entityId and needs consecutive
|
|
105
|
+
// events on the SAME id, so a first sync or a large import — many DIFFERENT
|
|
106
|
+
// ids, one event each — can never trigger it however big it is.
|
|
107
|
+
const DEFAULT_LOOP_TRANSITIONS = 4;
|
|
108
|
+
const DEFAULT_LOOP_WINDOW_MS = 10 * 60_000;
|
|
109
|
+
const WRITE_HISTORY_PER_ID = 8; // >= K+1 events, so the ring always holds a full detection
|
|
110
|
+
const WRITE_HISTORY_IDS = 500; // LRU-capped: the map is bounded regardless of traffic
|
|
111
|
+
|
|
112
|
+
let diagnosticsConfig = {
|
|
113
|
+
softLimitPerMinute: DEFAULT_SOFT_LIMIT,
|
|
114
|
+
loopTransitions: DEFAULT_LOOP_TRANSITIONS,
|
|
115
|
+
loopWindowMs: DEFAULT_LOOP_WINDOW_MS,
|
|
116
|
+
logger: null,
|
|
117
|
+
};
|
|
118
|
+
|
|
119
|
+
const diagLog = (level, message) => {
|
|
120
|
+
const sink = diagnosticsConfig.logger || console;
|
|
121
|
+
const fn = typeof sink[level] === 'function' ? sink[level] : sink.log;
|
|
122
|
+
if (typeof fn === 'function') fn.call(sink, message);
|
|
123
|
+
};
|
|
124
|
+
|
|
125
|
+
const secs = (ms) => Math.round(ms / 1000);
|
|
126
|
+
|
|
127
|
+
// ---------------------------------------------------------------------------
|
|
128
|
+
// The brake
|
|
129
|
+
// ---------------------------------------------------------------------------
|
|
130
|
+
|
|
131
|
+
const brakeState = { until: 0, memoryMs: 0 };
|
|
132
|
+
|
|
133
|
+
/**
|
|
134
|
+
* The gate. Returns a typed error when the brake is engaged, or null.
|
|
135
|
+
*
|
|
136
|
+
* The error is deliberately shaped like a REAL 429 (`status: 429`) so every
|
|
137
|
+
* retry ladder already sitting on top of this client — the DB engine's
|
|
138
|
+
* backoff, an app's bounded retry — treats it as the transient it is with no
|
|
139
|
+
* new code. `code: 'RATE_LIMITED'` and `retryInMs` are additive: a caller
|
|
140
|
+
* that wants to sit the whole cycle out can branch on them.
|
|
141
|
+
*/
|
|
142
|
+
const brakeGate = () => {
|
|
143
|
+
const now = Date.now();
|
|
144
|
+
if (brakeState.until <= now) return null;
|
|
145
|
+
const retryInMs = brakeState.until - now;
|
|
146
|
+
const err = new VaultError(
|
|
147
|
+
`vault requests are paused: rate-limited by the server, retry in ~${Math.ceil(retryInMs / 1000)}s`,
|
|
148
|
+
429
|
|
149
|
+
);
|
|
150
|
+
err.code = 'RATE_LIMITED';
|
|
151
|
+
err.retryInMs = retryInMs;
|
|
152
|
+
return err;
|
|
153
|
+
};
|
|
154
|
+
|
|
155
|
+
/**
|
|
156
|
+
* Arms the brake on a real 429. ONE ARMING PER BURST: a 429 that arrives
|
|
157
|
+
* while the brake is already engaged never compounds the delay, so a
|
|
158
|
+
* concurrent fan-out of ten requests that all meet the limiter costs one
|
|
159
|
+
* escalation step, not ten.
|
|
160
|
+
*/
|
|
161
|
+
const brakeArm = (label) => {
|
|
162
|
+
const now = Date.now();
|
|
163
|
+
if (brakeState.until > now) return;
|
|
164
|
+
brakeState.memoryMs = brakeState.memoryMs === 0
|
|
165
|
+
? BRAKE_BASE_MS
|
|
166
|
+
: Math.min(brakeState.memoryMs * 2, BRAKE_CEILING_MS);
|
|
167
|
+
brakeState.until = now + brakeState.memoryMs;
|
|
168
|
+
// The once-per-incident report, and the only loud line the brake prints.
|
|
169
|
+
// It names the method that met the limiter because that is the single most
|
|
170
|
+
// useful fact when the storm is someone else's loop.
|
|
171
|
+
diagLog('warn', `[vault] BRAKE: rate-limited (429) on ${label} — vault requests paused for ~${secs(brakeState.memoryMs)}s`);
|
|
172
|
+
};
|
|
173
|
+
|
|
174
|
+
/**
|
|
175
|
+
* DECAY, NEVER AMNESTY. A success clears the gate but only HALVES the
|
|
176
|
+
* escalation memory (below the 30s base it drains to zero).
|
|
177
|
+
*
|
|
178
|
+
* This is the lesson from the first brake built for this ecosystem, which
|
|
179
|
+
* reset escalation to zero on ANY success. On a saturated SHARED budget the
|
|
180
|
+
* occasional cheap request slips into a fresh limiter window and returns 200
|
|
181
|
+
* — and each lucky 200 wiped the entire 30s->480s escalation and re-licensed
|
|
182
|
+
* full cadence. It backed off, escalated, forgot, and started over, forever.
|
|
183
|
+
*
|
|
184
|
+
* Halving keeps the storm's level in memory: the next 429 re-arms at 2x what
|
|
185
|
+
* is left (so a lucky success cannot buy back full cadence), while a genuine
|
|
186
|
+
* recovery drains to zero in a few quiet successes and costs nothing.
|
|
187
|
+
*/
|
|
188
|
+
const brakeNoteSuccess = () => {
|
|
189
|
+
const wasBraked = brakeState.until > Date.now();
|
|
190
|
+
brakeState.until = 0;
|
|
191
|
+
if (brakeState.memoryMs === 0) return;
|
|
192
|
+
const before = brakeState.memoryMs;
|
|
193
|
+
const halved = Math.floor(before / 2);
|
|
194
|
+
brakeState.memoryMs = halved < BRAKE_BASE_MS ? 0 : halved;
|
|
195
|
+
diagLog(
|
|
196
|
+
'info',
|
|
197
|
+
`[vault] brake ${wasBraked ? 'released' : 'decay'}: escalation memory ${secs(before)}s -> ${secs(brakeState.memoryMs)}s`
|
|
198
|
+
);
|
|
199
|
+
};
|
|
200
|
+
|
|
201
|
+
/**
|
|
202
|
+
* The brake's state, for callers that would rather sit a whole cycle out
|
|
203
|
+
* pre-flight than fire a call and catch RATE_LIMITED.
|
|
204
|
+
*
|
|
205
|
+
* @returns {{ braked: boolean, until: number|null, memoryMs: number, retryInMs: number }}
|
|
206
|
+
*/
|
|
207
|
+
export function vaultBrakeStatus() {
|
|
208
|
+
const now = Date.now();
|
|
209
|
+
const braked = brakeState.until > now;
|
|
210
|
+
return {
|
|
211
|
+
braked,
|
|
212
|
+
until: brakeState.until || null,
|
|
213
|
+
memoryMs: brakeState.memoryMs,
|
|
214
|
+
retryInMs: braked ? brakeState.until - now : 0,
|
|
215
|
+
};
|
|
216
|
+
}
|
|
217
|
+
|
|
218
|
+
/** Convenience read: is the shared brake engaged right now? */
|
|
219
|
+
export function isVaultRateLimited() {
|
|
220
|
+
return brakeState.until > Date.now();
|
|
221
|
+
}
|
|
222
|
+
|
|
223
|
+
// ---------------------------------------------------------------------------
|
|
224
|
+
// The budget meter
|
|
225
|
+
// ---------------------------------------------------------------------------
|
|
226
|
+
//
|
|
227
|
+
// VISIBILITY ONLY — the meter never throttles anything. Its argument is that
|
|
228
|
+
// it needs no theory about WHY a loop is running: it counts requests and
|
|
229
|
+
// names who made them, so the next storm is an attributed event on day one
|
|
230
|
+
// instead of an afternoon of archaeology.
|
|
231
|
+
|
|
232
|
+
const meterState = { samples: [], lastReportAt: 0, dropped: 0 };
|
|
233
|
+
|
|
234
|
+
const meterPrune = (now) => {
|
|
235
|
+
const cutoff = now - BUDGET_WINDOW_MS;
|
|
236
|
+
const s = meterState.samples;
|
|
237
|
+
let i = 0;
|
|
238
|
+
while (i < s.length && s[i].at <= cutoff) i += 1;
|
|
239
|
+
if (i > 0) s.splice(0, i);
|
|
240
|
+
};
|
|
241
|
+
|
|
242
|
+
const meterTally = () => {
|
|
243
|
+
const counts = new Map();
|
|
244
|
+
for (const sample of meterState.samples) {
|
|
245
|
+
counts.set(sample.label, (counts.get(sample.label) || 0) + 1);
|
|
246
|
+
}
|
|
247
|
+
return [...counts.entries()].sort((a, b) => b[1] - a[1]);
|
|
248
|
+
};
|
|
249
|
+
|
|
250
|
+
const meterRecord = (label) => {
|
|
251
|
+
const now = Date.now();
|
|
252
|
+
meterPrune(now);
|
|
253
|
+
if (meterState.samples.length >= BUDGET_SAMPLE_CAP) {
|
|
254
|
+
// Hard cap: a loop cannot grow the buffer without bound. Dropping the
|
|
255
|
+
// oldest samples only under-counts a number already far past the limit.
|
|
256
|
+
const excess = meterState.samples.length - BUDGET_SAMPLE_CAP + 1;
|
|
257
|
+
meterState.samples.splice(0, excess);
|
|
258
|
+
meterState.dropped += excess;
|
|
259
|
+
}
|
|
260
|
+
meterState.samples.push({ at: now, label });
|
|
261
|
+
|
|
262
|
+
const total = meterState.samples.length;
|
|
263
|
+
if (total <= diagnosticsConfig.softLimitPerMinute) return;
|
|
264
|
+
// ONE line per window: after reporting, stay quiet for a full minute. A
|
|
265
|
+
// storm that lasts an hour prints sixty attributable lines, not a million.
|
|
266
|
+
if (meterState.lastReportAt !== 0 && now - meterState.lastReportAt < BUDGET_WINDOW_MS) return;
|
|
267
|
+
meterState.lastReportAt = now;
|
|
268
|
+
const top = meterTally().slice(0, BUDGET_TOP_N).map(([name, n]) => `${name} ${n}`).join(', ');
|
|
269
|
+
diagLog('warn', `[vault] budget: ${total} requests in the last minute (soft limit ${diagnosticsConfig.softLimitPerMinute}) — top: ${top}`);
|
|
270
|
+
};
|
|
271
|
+
|
|
272
|
+
// ---------------------------------------------------------------------------
|
|
273
|
+
// The write-loop detector (success-side visibility)
|
|
274
|
+
// ---------------------------------------------------------------------------
|
|
275
|
+
//
|
|
276
|
+
// Failure visibility exists. What was missing on the day of the incident was
|
|
277
|
+
// the signal for everything "succeeding" pathologically: every request
|
|
278
|
+
// returned 200 and every caller swallowed its promise, so a delete/resupply
|
|
279
|
+
// loop was indistinguishable from healthy traffic until the server's budget
|
|
280
|
+
// ran out. This ring is the client-level answer — diagnostic only, never a
|
|
281
|
+
// brake, and it fails open.
|
|
282
|
+
|
|
283
|
+
const writeHistory = new Map(); // entityId -> { events: [...], warnedAt }
|
|
284
|
+
|
|
285
|
+
// FNV-1a over a bounded prefix. Cheap enough to run on every row of every
|
|
286
|
+
// batch, and only ever used to spot an IDENTICAL rewrite (see
|
|
287
|
+
// isRedundantWrite). Never decoded, only hashed.
|
|
288
|
+
const hashContent = (value) => {
|
|
289
|
+
if (typeof value !== 'string' || value === '') return null;
|
|
290
|
+
const span = value.length > 4096 ? value.slice(0, 4096) : value;
|
|
291
|
+
let h = 0x811c9dc5;
|
|
292
|
+
for (let i = 0; i < span.length; i += 1) {
|
|
293
|
+
h ^= span.charCodeAt(i);
|
|
294
|
+
h = Math.imul(h, 0x01000193);
|
|
295
|
+
}
|
|
296
|
+
return `${value.length}:${(h >>> 0).toString(36)}`;
|
|
297
|
+
};
|
|
298
|
+
|
|
299
|
+
/**
|
|
300
|
+
* Is this consecutive pair of writes to one entityId REDUNDANT — a write that
|
|
301
|
+
* cannot have changed the row's state from what the previous write left it?
|
|
302
|
+
*
|
|
303
|
+
* Three shapes qualify, and the third is the founding incident's:
|
|
304
|
+
*
|
|
305
|
+
* 1. A POLARITY FLIP (upsert -> delete or delete -> upsert). The
|
|
306
|
+
* write-flip / delete-and-resupply loop.
|
|
307
|
+
* 2. An IDENTICAL-CONTENT REWRITE (upsert -> upsert, same content). Note
|
|
308
|
+
* that an AES-GCM envelope carries a fresh random IV, so identical
|
|
309
|
+
* plaintext does NOT produce an identical envelope: this rule fires for
|
|
310
|
+
* plaintext and deterministic payloads, while shape 1 covers encrypted
|
|
311
|
+
* ones.
|
|
312
|
+
* 3. A REPEAT DELETE (delete -> delete). Re-deleting a row that is already
|
|
313
|
+
* a tombstone is redundant by definition — and it is exactly the loop
|
|
314
|
+
* this detector was built for: the Obsidian plugin's cleanup called
|
|
315
|
+
* deleteRow on rows that were already tombstones, the server re-tombstoned
|
|
316
|
+
* each time with a fresh seq, and that pushed the row back above the
|
|
317
|
+
* caller's cursor to be deleted again. Same polarity every time, no
|
|
318
|
+
* content at all, so neither of the first two rules sees it. A detector
|
|
319
|
+
* that misses its own origin story is not a detector.
|
|
320
|
+
*
|
|
321
|
+
* A bounded retry of one failed delete does not reach K on its own; a ladder
|
|
322
|
+
* that retries the same delete K times inside the window is itself worth
|
|
323
|
+
* knowing about, and is the same shape as the incident.
|
|
324
|
+
*/
|
|
325
|
+
const isRedundantWrite = (prev, cur) => {
|
|
326
|
+
if (prev.polarity !== cur.polarity) return true;
|
|
327
|
+
if (prev.polarity === 'delete') return true;
|
|
328
|
+
return prev.contentHash != null && prev.contentHash === cur.contentHash;
|
|
329
|
+
};
|
|
330
|
+
|
|
331
|
+
const countTransitions = (events, now) => {
|
|
332
|
+
const cutoff = now - diagnosticsConfig.loopWindowMs;
|
|
333
|
+
const recent = events.filter((e) => e.at >= cutoff);
|
|
334
|
+
let transitions = 0;
|
|
335
|
+
for (let i = 1; i < recent.length; i += 1) {
|
|
336
|
+
if (isRedundantWrite(recent[i - 1], recent[i])) transitions += 1;
|
|
337
|
+
}
|
|
338
|
+
return transitions;
|
|
339
|
+
};
|
|
340
|
+
|
|
341
|
+
const renderHistory = (events) => events
|
|
342
|
+
.map((e) => (e.polarity === 'upsert' ? 'upsert' : 'delete'))
|
|
343
|
+
.join(' -> ');
|
|
344
|
+
|
|
345
|
+
/**
|
|
346
|
+
* Records one write against an entityId and warns — loudly, at most once per
|
|
347
|
+
* id per window — when its recent history looks like a loop.
|
|
348
|
+
*
|
|
349
|
+
* Recorded at INTENT time (before the request), so a loop that is being
|
|
350
|
+
* gated by the brake or rejected by the server is still visible as a loop.
|
|
351
|
+
*/
|
|
352
|
+
const noteWrite = (entityId, polarity, envelope) => {
|
|
353
|
+
if (typeof entityId !== 'string' || entityId === '') return;
|
|
354
|
+
const now = Date.now();
|
|
355
|
+
let rec = writeHistory.get(entityId);
|
|
356
|
+
if (rec) writeHistory.delete(entityId); // re-insert keeps Map iteration order LRU
|
|
357
|
+
else rec = { events: [], warnedAt: 0 };
|
|
358
|
+
rec.events.push({
|
|
359
|
+
polarity,
|
|
360
|
+
at: now,
|
|
361
|
+
contentHash: polarity === 'upsert' ? hashContent(envelope) : null,
|
|
362
|
+
});
|
|
363
|
+
if (rec.events.length > WRITE_HISTORY_PER_ID) rec.events.shift();
|
|
364
|
+
writeHistory.set(entityId, rec);
|
|
365
|
+
while (writeHistory.size > WRITE_HISTORY_IDS) {
|
|
366
|
+
writeHistory.delete(writeHistory.keys().next().value);
|
|
367
|
+
}
|
|
368
|
+
|
|
369
|
+
const transitions = countTransitions(rec.events, now);
|
|
370
|
+
if (transitions < diagnosticsConfig.loopTransitions) return;
|
|
371
|
+
if (rec.warnedAt !== 0 && now - rec.warnedAt < diagnosticsConfig.loopWindowMs) return;
|
|
372
|
+
rec.warnedAt = now;
|
|
373
|
+
diagLog(
|
|
374
|
+
'warn',
|
|
375
|
+
`[vault] WRITE LOOP? entity ${entityId}: ${transitions} redundant writes (polarity flips, repeat deletes or identical rewrites) in the last ${Math.round(diagnosticsConfig.loopWindowMs / 60_000)} min — ${renderHistory(rec.events)}`
|
|
376
|
+
);
|
|
377
|
+
};
|
|
378
|
+
|
|
379
|
+
const writeLoopSuspects = () => {
|
|
380
|
+
const now = Date.now();
|
|
381
|
+
const suspects = [];
|
|
382
|
+
for (const [entityId, rec] of writeHistory) {
|
|
383
|
+
const transitions = countTransitions(rec.events, now);
|
|
384
|
+
if (transitions === 0) continue;
|
|
385
|
+
suspects.push({
|
|
386
|
+
entityId,
|
|
387
|
+
transitions,
|
|
388
|
+
warned: rec.warnedAt !== 0,
|
|
389
|
+
history: rec.events.map((e) => ({ polarity: e.polarity, at: e.at, contentHash: e.contentHash })),
|
|
390
|
+
});
|
|
391
|
+
}
|
|
392
|
+
return suspects.sort((a, b) => b.transitions - a.transitions).slice(0, 20);
|
|
393
|
+
};
|
|
394
|
+
|
|
395
|
+
// ---------------------------------------------------------------------------
|
|
396
|
+
// The shared stats surface
|
|
397
|
+
// ---------------------------------------------------------------------------
|
|
398
|
+
|
|
399
|
+
/**
|
|
400
|
+
* Everything the module-scope diagnostics know, in one read: the brake's
|
|
401
|
+
* state, the last rolling minute of request counts attributed by method, and
|
|
402
|
+
* the entityIds whose recent write history looks loop-shaped.
|
|
403
|
+
*
|
|
404
|
+
* @returns {{ brake: object, requests: object, writeLoopSuspects: Array }}
|
|
405
|
+
*/
|
|
406
|
+
export function getVaultStats() {
|
|
407
|
+
const now = Date.now();
|
|
408
|
+
meterPrune(now);
|
|
409
|
+
const byMethod = {};
|
|
410
|
+
for (const [label, n] of meterTally()) byMethod[label] = n;
|
|
411
|
+
return {
|
|
412
|
+
brake: vaultBrakeStatus(),
|
|
413
|
+
requests: {
|
|
414
|
+
lastMinute: meterState.samples.length,
|
|
415
|
+
softLimitPerMinute: diagnosticsConfig.softLimitPerMinute,
|
|
416
|
+
byMethod,
|
|
417
|
+
droppedSamples: meterState.dropped,
|
|
418
|
+
},
|
|
419
|
+
writeLoopSuspects: writeLoopSuspects(),
|
|
420
|
+
};
|
|
421
|
+
}
|
|
422
|
+
|
|
423
|
+
/**
|
|
424
|
+
* Tunes the visibility-only thresholds. The brake's curve is deliberately NOT
|
|
425
|
+
* configurable: it is a protection for someone else's server, and an app that
|
|
426
|
+
* could widen it would eventually widen it.
|
|
427
|
+
*
|
|
428
|
+
* @param {object} [options]
|
|
429
|
+
* @param {number} [options.softLimitPerMinute] - budget-meter warn threshold (default 300)
|
|
430
|
+
* @param {number} [options.loopTransitions] - write-loop K (default 4)
|
|
431
|
+
* @param {number} [options.loopWindowMs] - write-loop M (default 10min)
|
|
432
|
+
* @param {object|null} [options.logger] - console-shaped sink for the diagnostic lines
|
|
433
|
+
*/
|
|
434
|
+
export function configureVaultDiagnostics(options = {}) {
|
|
435
|
+
if (typeof options.softLimitPerMinute === 'number' && options.softLimitPerMinute > 0) {
|
|
436
|
+
diagnosticsConfig.softLimitPerMinute = options.softLimitPerMinute;
|
|
437
|
+
}
|
|
438
|
+
if (typeof options.loopTransitions === 'number' && options.loopTransitions > 0) {
|
|
439
|
+
diagnosticsConfig.loopTransitions = options.loopTransitions;
|
|
440
|
+
}
|
|
441
|
+
if (typeof options.loopWindowMs === 'number' && options.loopWindowMs > 0) {
|
|
442
|
+
diagnosticsConfig.loopWindowMs = options.loopWindowMs;
|
|
443
|
+
}
|
|
444
|
+
if ('logger' in options) diagnosticsConfig.logger = options.logger || null;
|
|
445
|
+
return { ...diagnosticsConfig };
|
|
446
|
+
}
|
|
447
|
+
|
|
448
|
+
/**
|
|
449
|
+
* Clears the brake, the meter and the write history, and restores the default
|
|
450
|
+
* thresholds. For tests — module-scope state is per realm and would otherwise
|
|
451
|
+
* carry between cases.
|
|
452
|
+
*/
|
|
453
|
+
export function resetVaultDiagnostics() {
|
|
454
|
+
brakeState.until = 0;
|
|
455
|
+
brakeState.memoryMs = 0;
|
|
456
|
+
meterState.samples.length = 0;
|
|
457
|
+
meterState.lastReportAt = 0;
|
|
458
|
+
meterState.dropped = 0;
|
|
459
|
+
writeHistory.clear();
|
|
460
|
+
diagnosticsConfig = {
|
|
461
|
+
softLimitPerMinute: DEFAULT_SOFT_LIMIT,
|
|
462
|
+
loopTransitions: DEFAULT_LOOP_TRANSITIONS,
|
|
463
|
+
loopWindowMs: DEFAULT_LOOP_WINDOW_MS,
|
|
464
|
+
logger: null,
|
|
465
|
+
};
|
|
466
|
+
}
|
|
467
|
+
|
|
42
468
|
// Quota rejection parsing (GLANCEvault Phase 3.2 / client Phase 3.3).
|
|
43
469
|
//
|
|
44
470
|
// The server rejects an over-quota write with 413 (storage-shaped: `storage`,
|
|
@@ -200,16 +626,25 @@ export async function enrollVaultDevice({ vaultUrl, enrollmentSecret, accountId,
|
|
|
200
626
|
/**
|
|
201
627
|
* Creates a vault client bound to a server URL and token.
|
|
202
628
|
*
|
|
629
|
+
* Every network method of the returned client passes through the module-scope
|
|
630
|
+
* brake and budget meter (see the diagnostics block at the top of this file).
|
|
631
|
+
* That is deliberate: protection a caller has to remember to apply is
|
|
632
|
+
* protection that is missing on the day it matters.
|
|
633
|
+
*
|
|
203
634
|
* @param {object} config
|
|
204
635
|
* @param {string} config.vaultUrl - base URL of the GLANCEvault server
|
|
205
636
|
* @param {string} config.vaultToken - Bearer token: the shared device token
|
|
206
637
|
* (shared mode) or this device's enrolled credential (per-account mode)
|
|
207
638
|
* @param {Function} [config.fetchImpl] - fetch implementation (defaults to global fetch)
|
|
639
|
+
* @param {boolean} [config.brake=true] - set false to opt this client out of the
|
|
640
|
+
* shared brake (it neither gates on nor arms it). An escape hatch for tests
|
|
641
|
+
* that need the wire on every call; production callers should leave it on.
|
|
208
642
|
*/
|
|
209
|
-
export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
643
|
+
export function createVaultClient({ vaultUrl, vaultToken, fetchImpl, brake = true } = {}) {
|
|
210
644
|
const base = requireBaseUrl(vaultUrl, 'createVaultClient');
|
|
211
645
|
if (!vaultToken) throw new Error('createVaultClient: vaultToken is required');
|
|
212
646
|
const doFetch = resolveFetch(fetchImpl, 'createVaultClient');
|
|
647
|
+
const brakeEnabled = brake !== false;
|
|
213
648
|
const authHeaders = (extra = {}) => ({
|
|
214
649
|
Authorization: `Bearer ${vaultToken}`,
|
|
215
650
|
...extra,
|
|
@@ -230,6 +665,29 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
230
665
|
return res;
|
|
231
666
|
};
|
|
232
667
|
|
|
668
|
+
/**
|
|
669
|
+
* The one door every network method goes through: gate, meter, send, and
|
|
670
|
+
* report a 2xx back to the brake.
|
|
671
|
+
*
|
|
672
|
+
* `label` is the METHOD NAME (batch, intentsList, deleteRow, ...) — the
|
|
673
|
+
* brake's arming line and the meter's attribution both name it, so the
|
|
674
|
+
* report says who met the limiter rather than just that someone did.
|
|
675
|
+
*
|
|
676
|
+
* A gated call throws BEFORE the fetch and is deliberately silent and
|
|
677
|
+
* unmetered: it never reaches the wire, so it is not budget the server
|
|
678
|
+
* spent, and the arming line already reported the incident once.
|
|
679
|
+
*/
|
|
680
|
+
const send = async (label, method, path, opts) => {
|
|
681
|
+
if (brakeEnabled) {
|
|
682
|
+
const gated = brakeGate();
|
|
683
|
+
if (gated) throw gated;
|
|
684
|
+
}
|
|
685
|
+
meterRecord(label);
|
|
686
|
+
const res = await request(method, path, opts);
|
|
687
|
+
if (brakeEnabled && res && res.ok) brakeNoteSuccess();
|
|
688
|
+
return res;
|
|
689
|
+
};
|
|
690
|
+
|
|
233
691
|
// Error-body inspection is BEST-EFFORT and deliberately narrow. The server
|
|
234
692
|
// made the 401 body wording the only signal separating shared mode's
|
|
235
693
|
// "invalid device token" (server misconfigured / wrong token) from
|
|
@@ -238,7 +696,7 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
238
696
|
// exact wording upgrades the code to CREDENTIAL_INVALID; a missing,
|
|
239
697
|
// non-JSON, or unrecognized body degrades to the generic VAULT_ERROR, so
|
|
240
698
|
// the client's correctness never depends on server wording.
|
|
241
|
-
const errorFromResponse = async (res, context) => {
|
|
699
|
+
const errorFromResponse = async (res, context, label) => {
|
|
242
700
|
let body = null;
|
|
243
701
|
let bodyError = null;
|
|
244
702
|
try {
|
|
@@ -247,13 +705,25 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
247
705
|
} catch {
|
|
248
706
|
// Empty or non-JSON error body: keep the generic classification.
|
|
249
707
|
}
|
|
708
|
+
// A 429 arms the shared brake — but only a RATE-LIMITER 429, not an
|
|
709
|
+
// over-quota one. The server uses 429 for both, and they call for
|
|
710
|
+
// opposite handling: a limiter hit means "you are asking too often, stop
|
|
711
|
+
// for a while" (exactly what the brake is), while a quota rejection means
|
|
712
|
+
// "this account is full", clears only when an operator acts, and already
|
|
713
|
+
// has its own self-resuming window in the DB engine. Gating on a quota
|
|
714
|
+
// 429 would mask QUOTA_EXCEEDED and its descriptor behind RATE_LIMITED
|
|
715
|
+
// and leave the engine unable to render "X of Y used". So the arming
|
|
716
|
+
// decision waits until the body has been parsed and the quota shape ruled
|
|
717
|
+
// out — see parseQuotaBody for how narrowly that shape is recognised.
|
|
718
|
+
const quotaShaped = parseQuotaBody(res.status, body, bodyError);
|
|
719
|
+
if (res.status === 429 && !quotaShaped && brakeEnabled) brakeArm(label || context);
|
|
250
720
|
if (res.status === 401 && bodyError === 'invalid credential') {
|
|
251
721
|
const err = new VaultError(`${context} failed: 401 — the server rejected this device's credential`, 401);
|
|
252
722
|
err.code = 'CREDENTIAL_INVALID';
|
|
253
723
|
return err;
|
|
254
724
|
}
|
|
255
|
-
|
|
256
|
-
|
|
725
|
+
if (quotaShaped) {
|
|
726
|
+
const quota = quotaShaped;
|
|
257
727
|
const err = new VaultError(
|
|
258
728
|
`${context} failed: ${res.status} — ${quota.quota} quota exceeded (${quota.used} of ${quota.limit}, requested ${quota.requested})`,
|
|
259
729
|
res.status
|
|
@@ -265,9 +735,9 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
265
735
|
return new VaultError(`${context} failed: ${res.status}`, res.status);
|
|
266
736
|
};
|
|
267
737
|
|
|
268
|
-
const jsonOrThrow = async (res, context) => {
|
|
738
|
+
const jsonOrThrow = async (res, context, label) => {
|
|
269
739
|
if (!res.ok) {
|
|
270
|
-
throw await errorFromResponse(res, context);
|
|
740
|
+
throw await errorFromResponse(res, context, label);
|
|
271
741
|
}
|
|
272
742
|
return res.json();
|
|
273
743
|
};
|
|
@@ -293,10 +763,17 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
293
763
|
*/
|
|
294
764
|
async batch(app, { accountId, rows }) {
|
|
295
765
|
requireAccountId(accountId, 'batch upsert');
|
|
296
|
-
|
|
766
|
+
// Recorded at intent time, before the wire: a loop that is being gated
|
|
767
|
+
// by the brake or rejected by the server is still a loop worth naming.
|
|
768
|
+
if (Array.isArray(rows)) {
|
|
769
|
+
for (const row of rows) {
|
|
770
|
+
if (row) noteWrite(row.entityId, 'upsert', row.envelope);
|
|
771
|
+
}
|
|
772
|
+
}
|
|
773
|
+
const res = await send('batch', 'POST', `/sync/${encodeURIComponent(app)}/batch`, {
|
|
297
774
|
body: { accountId, rows },
|
|
298
775
|
});
|
|
299
|
-
return jsonOrThrow(res, 'batch upsert');
|
|
776
|
+
return jsonOrThrow(res, 'batch upsert', 'batch');
|
|
300
777
|
},
|
|
301
778
|
|
|
302
779
|
/**
|
|
@@ -305,10 +782,10 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
305
782
|
*/
|
|
306
783
|
async list(app, { accountId, since }) {
|
|
307
784
|
requireAccountId(accountId, 'list');
|
|
308
|
-
const res = await
|
|
785
|
+
const res = await send('list', 'GET', `/sync/${encodeURIComponent(app)}/list`, {
|
|
309
786
|
query: { accountId, since: String(since) },
|
|
310
787
|
});
|
|
311
|
-
return jsonOrThrow(res, 'list');
|
|
788
|
+
return jsonOrThrow(res, 'list', 'list');
|
|
312
789
|
},
|
|
313
790
|
|
|
314
791
|
/**
|
|
@@ -316,11 +793,11 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
316
793
|
*/
|
|
317
794
|
async getRow(app, entityId, accountId) {
|
|
318
795
|
requireAccountId(accountId, 'get row');
|
|
319
|
-
const res = await
|
|
796
|
+
const res = await send('getRow', 'GET', `/sync/${encodeURIComponent(app)}/${encodeURIComponent(entityId)}`, {
|
|
320
797
|
query: { accountId },
|
|
321
798
|
});
|
|
322
799
|
if (res.status === 404) return null;
|
|
323
|
-
return jsonOrThrow(res, 'get row');
|
|
800
|
+
return jsonOrThrow(res, 'get row', 'getRow');
|
|
324
801
|
},
|
|
325
802
|
|
|
326
803
|
/**
|
|
@@ -334,13 +811,14 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
334
811
|
*/
|
|
335
812
|
async deleteRow(app, entityId, accountId, opts = {}) {
|
|
336
813
|
requireAccountId(accountId, 'delete row');
|
|
814
|
+
noteWrite(entityId, 'delete');
|
|
337
815
|
const query = { accountId };
|
|
338
816
|
if (opts && opts.deletedAt != null) query.deletedAt = String(opts.deletedAt);
|
|
339
|
-
const res = await
|
|
817
|
+
const res = await send('deleteRow', 'DELETE', `/sync/${encodeURIComponent(app)}/${encodeURIComponent(entityId)}`, {
|
|
340
818
|
query,
|
|
341
819
|
});
|
|
342
820
|
if (res.status === 404) return null;
|
|
343
|
-
return jsonOrThrow(res, 'delete row');
|
|
821
|
+
return jsonOrThrow(res, 'delete row', 'deleteRow');
|
|
344
822
|
},
|
|
345
823
|
|
|
346
824
|
/**
|
|
@@ -350,10 +828,10 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
350
828
|
*/
|
|
351
829
|
async device(app, { accountId, deviceId, lastSeenSeq }) {
|
|
352
830
|
requireAccountId(accountId, 'device cursor');
|
|
353
|
-
const res = await
|
|
831
|
+
const res = await send('device', 'POST', `/sync/${encodeURIComponent(app)}/device`, {
|
|
354
832
|
body: { accountId, deviceId, lastSeenSeq },
|
|
355
833
|
});
|
|
356
|
-
return jsonOrThrow(res, 'device cursor');
|
|
834
|
+
return jsonOrThrow(res, 'device cursor', 'device');
|
|
357
835
|
},
|
|
358
836
|
|
|
359
837
|
/**
|
|
@@ -361,9 +839,9 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
361
839
|
* salt is registered yet (404).
|
|
362
840
|
*/
|
|
363
841
|
async getSalt(accountId) {
|
|
364
|
-
const res = await
|
|
842
|
+
const res = await send('getSalt', 'GET', `/salt/${encodeURIComponent(accountId)}`);
|
|
365
843
|
if (res.status === 404) return null;
|
|
366
|
-
const body = await jsonOrThrow(res, 'get salt');
|
|
844
|
+
const body = await jsonOrThrow(res, 'get salt', 'getSalt');
|
|
367
845
|
return body && body.salt ? fromBase64(body.salt) : null;
|
|
368
846
|
},
|
|
369
847
|
|
|
@@ -377,11 +855,70 @@ export function createVaultClient({ vaultUrl, vaultToken, fetchImpl } = {}) {
|
|
|
377
855
|
* @returns {Promise<Uint8Array>}
|
|
378
856
|
*/
|
|
379
857
|
async putSalt(accountId, salt) {
|
|
380
|
-
const res = await
|
|
858
|
+
const res = await send('putSalt', 'PUT', `/salt/${encodeURIComponent(accountId)}`, {
|
|
381
859
|
body: { salt: toBase64(salt) },
|
|
382
860
|
});
|
|
383
|
-
const body = await jsonOrThrow(res, 'put salt');
|
|
861
|
+
const body = await jsonOrThrow(res, 'put salt', 'putSalt');
|
|
384
862
|
return body && body.salt ? fromBase64(body.salt) : salt;
|
|
385
863
|
},
|
|
864
|
+
|
|
865
|
+
// ───────────────────────── intents transport (1.11.0) ────────────────────
|
|
866
|
+
//
|
|
867
|
+
// The client half of GLANCEvault's /intents/* surface, so apps can drop
|
|
868
|
+
// their hand-rolled raw-fetch transports and inherit this client's auth,
|
|
869
|
+
// error classification, brake and meter. PURE TRANSPORT: the envelope is
|
|
870
|
+
// OPAQUE here and is never decoded, inspected or validated — the codec
|
|
871
|
+
// (and the per-envelope key derivation it needs) stays in
|
|
872
|
+
// @glance-apps/intents on the app side, which is why this package can
|
|
873
|
+
// carry the transport without carrying the crypto.
|
|
874
|
+
//
|
|
875
|
+
// The wire contract below is what the server already serves; nothing in
|
|
876
|
+
// this release changes it.
|
|
877
|
+
|
|
878
|
+
/**
|
|
879
|
+
* Appends intent events. INSERT-ONLY: the server ignores an eventId it
|
|
880
|
+
* has already stored, so a re-sent batch after an ambiguous failure is a
|
|
881
|
+
* no-op rather than a duplicate — retries are safe by construction.
|
|
882
|
+
*
|
|
883
|
+
* POST {vaultUrl}/intents/batch
|
|
884
|
+
* body { accountId, events: [{ eventId, envelope, expiresAt }] }
|
|
885
|
+
*
|
|
886
|
+
* @param {string} accountId
|
|
887
|
+
* @param {Array<{ eventId: string, envelope: string, expiresAt: string }>} events
|
|
888
|
+
* envelope is an opaque base64 string; expiresAt is an ISO timestamp.
|
|
889
|
+
* @returns {Promise<{ written: number, maxSeq: number }>}
|
|
890
|
+
*/
|
|
891
|
+
async intentsBatch(accountId, events) {
|
|
892
|
+
requireAccountId(accountId, 'intents batch');
|
|
893
|
+
if (!Array.isArray(events)) {
|
|
894
|
+
const err = new Error('intents batch: events must be an array of { eventId, envelope, expiresAt }.');
|
|
895
|
+
err.code = 'EVENTS_REQUIRED';
|
|
896
|
+
throw err;
|
|
897
|
+
}
|
|
898
|
+
const res = await send('intentsBatch', 'POST', '/intents/batch', {
|
|
899
|
+
body: { accountId, events },
|
|
900
|
+
});
|
|
901
|
+
return jsonOrThrow(res, 'intents batch', 'intentsBatch');
|
|
902
|
+
},
|
|
903
|
+
|
|
904
|
+
/**
|
|
905
|
+
* Fetches intent events with seq > since, ascending, one page per call.
|
|
906
|
+
* The server returns only non-expired rows and pages at 500.
|
|
907
|
+
*
|
|
908
|
+
* GET {vaultUrl}/intents/list?accountId=&since=&limit=
|
|
909
|
+
*
|
|
910
|
+
* @param {string} accountId
|
|
911
|
+
* @param {object} [opts]
|
|
912
|
+
* @param {number} [opts.since=0] - exclusive seq cursor
|
|
913
|
+
* @param {number} [opts.limit] - page size; omitted lets the server choose
|
|
914
|
+
* @returns {Promise<{ rows: Array<{ eventId: string, envelope: string, seq: number, expiresAt: string, serverMtime: string }>, hasMore: boolean }>}
|
|
915
|
+
*/
|
|
916
|
+
async intentsList(accountId, { since = 0, limit } = {}) {
|
|
917
|
+
requireAccountId(accountId, 'intents list');
|
|
918
|
+
const query = { accountId, since: String(since) };
|
|
919
|
+
if (limit != null) query.limit = String(limit);
|
|
920
|
+
const res = await send('intentsList', 'GET', '/intents/list', { query });
|
|
921
|
+
return jsonOrThrow(res, 'intents list', 'intentsList');
|
|
922
|
+
},
|
|
386
923
|
};
|
|
387
924
|
}
|
package/types/index.d.ts
CHANGED
|
@@ -40,7 +40,13 @@ export type SyncErrorCode =
|
|
|
40
40
|
// never halts on it. The parsed descriptor rides on the engine's
|
|
41
41
|
// getQuotaState(); the engine suppresses further writes for a bounded,
|
|
42
42
|
// self-resuming window while the pull keeps running.
|
|
43
|
-
| 'QUOTA_EXCEEDED'
|
|
43
|
+
| 'QUOTA_EXCEEDED'
|
|
44
|
+
// DB transport (1.11.0): the module-scope brake is engaged after a real 429,
|
|
45
|
+
// so this call failed FAST without touching the network. Carries
|
|
46
|
+
// `status: 429` and `retryInMs`, so any ladder that already treats a real
|
|
47
|
+
// 429 as transient handles it unchanged. Never a hard stop: the window is
|
|
48
|
+
// self-expiring and a success both releases it and halves the escalation.
|
|
49
|
+
| 'RATE_LIMITED';
|
|
44
50
|
|
|
45
51
|
export type SyncStatus = 'idle' | 'uploading' | 'downloading' | 'success' | 'error';
|
|
46
52
|
|
|
@@ -422,6 +428,23 @@ export interface VaultPulledRow extends Partial<VaultRow> {
|
|
|
422
428
|
envelope?: string;
|
|
423
429
|
}
|
|
424
430
|
|
|
431
|
+
/**
|
|
432
|
+
* One intent event on the wire. `envelope` is OPAQUE to this package — it is
|
|
433
|
+
* never decoded or inspected; the codec lives in `@glance-apps/intents`.
|
|
434
|
+
*/
|
|
435
|
+
export interface VaultIntentEvent {
|
|
436
|
+
eventId: string;
|
|
437
|
+
/** Opaque base64 payload. */
|
|
438
|
+
envelope: string;
|
|
439
|
+
/** ISO 8601. */
|
|
440
|
+
expiresAt: string;
|
|
441
|
+
}
|
|
442
|
+
|
|
443
|
+
export interface VaultIntentRow extends VaultIntentEvent {
|
|
444
|
+
seq: number;
|
|
445
|
+
serverMtime: string;
|
|
446
|
+
}
|
|
447
|
+
|
|
425
448
|
export interface VaultClient {
|
|
426
449
|
batch(app: string, args: { accountId: string; rows: VaultRow[] }): Promise<{ written: number; maxSeq: number }>;
|
|
427
450
|
list(app: string, args: { accountId: string; since: number }): Promise<{ rows: VaultPulledRow[]; hasMore: boolean }>;
|
|
@@ -430,6 +453,10 @@ export interface VaultClient {
|
|
|
430
453
|
device(app: string, args: { accountId: string; deviceId: string; lastSeenSeq: number }): Promise<{ updated: boolean }>;
|
|
431
454
|
getSalt(accountId: string): Promise<Uint8Array | null>;
|
|
432
455
|
putSalt(accountId: string, salt: Uint8Array): Promise<Uint8Array>;
|
|
456
|
+
/** Appends intent events. Insert-only: a re-sent eventId is a server no-op. */
|
|
457
|
+
intentsBatch(accountId: string, events: VaultIntentEvent[]): Promise<{ written: number; maxSeq: number }>;
|
|
458
|
+
/** Fetches non-expired intent events with seq > since, ascending. One page per call. */
|
|
459
|
+
intentsList(accountId: string, opts?: { since?: number; limit?: number }): Promise<{ rows: VaultIntentRow[]; hasMore: boolean }>;
|
|
433
460
|
}
|
|
434
461
|
|
|
435
462
|
export function createVaultClient(config: {
|
|
@@ -437,8 +464,89 @@ export function createVaultClient(config: {
|
|
|
437
464
|
/** Shared device token, or the device's enrolled credential in per-account mode. */
|
|
438
465
|
vaultToken: string;
|
|
439
466
|
fetchImpl?: typeof fetch;
|
|
467
|
+
/**
|
|
468
|
+
* Set false to opt this client out of the module-scope brake — it then
|
|
469
|
+
* neither gates on nor arms it. An escape hatch for tests that need the
|
|
470
|
+
* wire on every call; production callers should leave it on.
|
|
471
|
+
* @default true
|
|
472
|
+
*/
|
|
473
|
+
brake?: boolean;
|
|
440
474
|
}): VaultClient;
|
|
441
475
|
|
|
476
|
+
// ---------------------------------------------------------------------------
|
|
477
|
+
// Module-scope request diagnostics (1.11.0)
|
|
478
|
+
// ---------------------------------------------------------------------------
|
|
479
|
+
//
|
|
480
|
+
// One brake, one budget meter and one write-loop history per BUNDLE REALM,
|
|
481
|
+
// not per client instance: they model a resource every client in the process
|
|
482
|
+
// shares — the server's per-IP request budget. A separately bundled copy
|
|
483
|
+
// (e.g. an Obsidian plugin) is its own realm, which is correct: separate
|
|
484
|
+
// process, separate traffic.
|
|
485
|
+
|
|
486
|
+
export interface VaultBrakeStatus {
|
|
487
|
+
/** Is the brake engaged right now? While engaged, every client call fails fast with RATE_LIMITED. */
|
|
488
|
+
braked: boolean;
|
|
489
|
+
/** Epoch ms the current window expires, or null when not braked. */
|
|
490
|
+
until: number | null;
|
|
491
|
+
/** Escalation memory. Doubles per 429 burst to a 10-minute ceiling; a success halves it. */
|
|
492
|
+
memoryMs: number;
|
|
493
|
+
/** Milliseconds left in the window, 0 when not braked. */
|
|
494
|
+
retryInMs: number;
|
|
495
|
+
}
|
|
496
|
+
|
|
497
|
+
export interface VaultWriteLoopSuspect {
|
|
498
|
+
entityId: string;
|
|
499
|
+
/**
|
|
500
|
+
* Redundant writes counted inside the window: polarity flips, repeat
|
|
501
|
+
* deletes of an already-deleted row, or identical-content rewrites.
|
|
502
|
+
*/
|
|
503
|
+
transitions: number;
|
|
504
|
+
/** Whether the loud one-per-window warning has fired for this id. */
|
|
505
|
+
warned: boolean;
|
|
506
|
+
history: Array<{ polarity: 'upsert' | 'delete'; at: number; contentHash: string | null }>;
|
|
507
|
+
}
|
|
508
|
+
|
|
509
|
+
export interface VaultStats {
|
|
510
|
+
brake: VaultBrakeStatus;
|
|
511
|
+
requests: {
|
|
512
|
+
/** Requests that reached the wire in the last rolling minute. */
|
|
513
|
+
lastMinute: number;
|
|
514
|
+
softLimitPerMinute: number;
|
|
515
|
+
/** Attribution by client method name, e.g. { intentsList: 210, batch: 130 }. */
|
|
516
|
+
byMethod: Record<string, number>;
|
|
517
|
+
/** Samples dropped by the buffer's hard cap (only ever under-counts a storm). */
|
|
518
|
+
droppedSamples: number;
|
|
519
|
+
};
|
|
520
|
+
writeLoopSuspects: VaultWriteLoopSuspect[];
|
|
521
|
+
}
|
|
522
|
+
|
|
523
|
+
/** Is the shared brake engaged? For callers that prefer to sit a cycle out pre-flight. */
|
|
524
|
+
export function isVaultRateLimited(): boolean;
|
|
525
|
+
|
|
526
|
+
/** The brake's full state. */
|
|
527
|
+
export function vaultBrakeStatus(): VaultBrakeStatus;
|
|
528
|
+
|
|
529
|
+
/** Brake status, per-minute request counts by method, and write-loop suspects in one read. */
|
|
530
|
+
export function getVaultStats(): VaultStats;
|
|
531
|
+
|
|
532
|
+
/**
|
|
533
|
+
* Tunes the visibility-only thresholds. The brake's curve is deliberately not
|
|
534
|
+
* configurable — it protects someone else's server.
|
|
535
|
+
*/
|
|
536
|
+
export function configureVaultDiagnostics(options?: {
|
|
537
|
+
/** Budget-meter warn threshold. @default 300 */
|
|
538
|
+
softLimitPerMinute?: number;
|
|
539
|
+
/** Write-loop K: redundant writes to one entityId before warning. @default 4 */
|
|
540
|
+
loopTransitions?: number;
|
|
541
|
+
/** Write-loop M. @default 600000 (10 minutes) */
|
|
542
|
+
loopWindowMs?: number;
|
|
543
|
+
/** Console-shaped sink for the diagnostic lines. Defaults to `console`. */
|
|
544
|
+
logger?: { warn?: (m: string) => void; info?: (m: string) => void; log?: (m: string) => void } | null;
|
|
545
|
+
}): { softLimitPerMinute: number; loopTransitions: number; loopWindowMs: number; logger: unknown };
|
|
546
|
+
|
|
547
|
+
/** Clears the brake, the meter and the write history, and restores the defaults. For tests. */
|
|
548
|
+
export function resetVaultDiagnostics(): void;
|
|
549
|
+
|
|
442
550
|
// Per-account auth (vault Phase 1.4b).
|
|
443
551
|
|
|
444
552
|
/** Which trust model a GLANCEvault server runs. Discoverable via fetchVaultHealth. */
|