@syncular/server 0.30.16 → 0.32.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +196 -76
- package/dist/admin.d.ts +394 -50
- package/dist/admin.js +714 -82
- package/dist/context.d.ts +16 -6
- package/dist/d1-storage.d.ts +6 -5
- package/dist/d1-storage.js +148 -27
- package/dist/errors.d.ts +17 -4
- package/dist/errors.js +31 -6
- package/dist/events-ring.d.ts +12 -2
- package/dist/events-ring.js +27 -2
- package/dist/events.d.ts +21 -3
- package/dist/handler.js +19 -6
- package/dist/postgres-storage.d.ts +6 -5
- package/dist/postgres-storage.js +102 -18
- package/dist/pull.js +3 -26
- package/dist/realtime.d.ts +18 -1
- package/dist/realtime.js +31 -3
- package/dist/sqlite-image.d.ts +1 -1
- package/dist/sqlite-storage.d.ts +6 -5
- package/dist/sqlite-storage.js +115 -18
- package/dist/storage.d.ts +58 -7
- package/package.json +2 -2
- package/src/admin.ts +1215 -120
- package/src/context.ts +16 -6
- package/src/d1-storage.ts +210 -31
- package/src/errors.ts +45 -5
- package/src/events-ring.ts +31 -2
- package/src/events.ts +27 -4
- package/src/handler.ts +20 -6
- package/src/postgres-storage.ts +173 -28
- package/src/pull.ts +3 -29
- package/src/realtime.ts +58 -4
- package/src/sqlite-image.ts +1 -1
- package/src/sqlite-storage.ts +181 -26
- package/src/storage.ts +72 -9
package/README.md
CHANGED
|
@@ -28,8 +28,8 @@ telemetry. See the [domain event guide](https://syncular.dev/guide-domain-events
|
|
|
28
28
|
|
|
29
29
|
The server core is **runtime-neutral TypeScript** — `handleSyncRequest` and
|
|
30
30
|
the realtime session speak only Web `Request`/`Response`/`fetch`/Web-Crypto,
|
|
31
|
-
no Bun- or Node-only builtin (enforced by
|
|
32
|
-
`test/runtime-neutrality.test.ts`). Adapters wire that core to a runtime.
|
|
31
|
+
no Bun- or Node-only builtin (enforced by static and dynamic import scans
|
|
32
|
+
and an unminified browser bundle in `test/runtime-neutrality.test.ts`). Adapters wire that core to a runtime.
|
|
33
33
|
The supported set, and what deliberately does **not** get an adapter:
|
|
34
34
|
|
|
35
35
|
| Runtime | Adapter | Transport | Storage | Status |
|
|
@@ -83,11 +83,12 @@ import {
|
|
|
83
83
|
ensureSyncServerReady,
|
|
84
84
|
type SyncServerConfig,
|
|
85
85
|
} from '@syncular/server';
|
|
86
|
-
import { SqliteServerStorage } from '@syncular/server/sqlite';
|
|
86
|
+
import { buildSqliteImage, SqliteServerStorage } from '@syncular/server/sqlite';
|
|
87
87
|
|
|
88
88
|
const config: SyncServerConfig = {
|
|
89
89
|
schema,
|
|
90
90
|
storage: new SqliteServerStorage('./sync.db'),
|
|
91
|
+
sqliteImageBuilder: buildSqliteImage,
|
|
91
92
|
segments,
|
|
92
93
|
resolveScopes,
|
|
93
94
|
};
|
|
@@ -102,6 +103,14 @@ storage, leases, and SQLite-image generation without an external SQLite
|
|
|
102
103
|
package. The runtime-specific database wrappers are `BunSqliteDatabase` and
|
|
103
104
|
`NodeSqliteDatabase` when a host needs direct access to the native handle.
|
|
104
105
|
|
|
106
|
+
Hosts opt into building SQLite bootstrap images by supplying
|
|
107
|
+
`sqliteImageBuilder: buildSqliteImage` in the server config (and the realtime
|
|
108
|
+
hub config when it serves sync rounds). The neutral `@syncular/server` entry
|
|
109
|
+
never detects the runtime or loads a SQLite builder. Import the builder from
|
|
110
|
+
`@syncular/server/sqlite` on Bun or Node. Without a builder, the server serves
|
|
111
|
+
matching stored images and negotiates inline or external rows, including
|
|
112
|
+
signed URLs when configured, according to the client's accepted formats.
|
|
113
|
+
|
|
105
114
|
The helper accepts the generated `ServerSchema`, compiles it, and calls the
|
|
106
115
|
storage backend's low-level `ensureSchema(CompiledSchema)`. A failure is a
|
|
107
116
|
`SyncServerReadinessError` with stable code `sync.schema_not_ready`, a `phase`
|
|
@@ -119,7 +128,23 @@ After restoring an authoritative database, keep traffic stopped and call
|
|
|
119
128
|
`rotatePartitionLogEpoch({ storage, partition })` for every restored
|
|
120
129
|
partition. The rotation clears stale client cursors and requires version 2
|
|
121
130
|
clients to reset their server-derived rows while preserving the outbox. Follow
|
|
122
|
-
the complete [backup and restore runbook](https://syncular.dev/server-backup-restore
|
|
131
|
+
the complete [backup and restore runbook](https://syncular.dev/server-operations/#backup-and-restore).
|
|
132
|
+
|
|
133
|
+
## Adapter error hooks
|
|
134
|
+
|
|
135
|
+
A `SyncError` answers with its catalog code and status. By default, any other exception an
|
|
136
|
+
adapter catches answers `sync.internal_error` and is handed to
|
|
137
|
+
`SyncServerConfig.onError` exactly once; a throwing observer does not change
|
|
138
|
+
the response. The same config accepts an optional synchronous `mapError` hook:
|
|
139
|
+
after `onError` observes the original, `mapError` may return a catalog
|
|
140
|
+
`SyncError` (with structured `details`, for example `retryAfterMs`) that the
|
|
141
|
+
adapter answers instead. Return `undefined` to keep `sync.internal_error`. A
|
|
142
|
+
throw, a non-`SyncError` return, a code that is not an own catalog entry, or
|
|
143
|
+
`details` that is not a JSON document is contained as `sync.internal_error`.
|
|
144
|
+
The hook covers the sync, operations, segments, blobs, admin, and realtime
|
|
145
|
+
surfaces. It does not
|
|
146
|
+
change a remote operation's `operation.*` envelope, and a mapped `retryAfterMs`
|
|
147
|
+
is delivery metadata. Client retry scheduling remains unchanged.
|
|
123
148
|
|
|
124
149
|
## Write validators and recovery metadata
|
|
125
150
|
|
|
@@ -243,10 +268,55 @@ Do not set that assertion on a stateless Worker. Custom storages must implement
|
|
|
243
268
|
the pre-operation partition lock, locked idempotency re-check, atomic rejection
|
|
244
269
|
finalization, and—when `commitValidator` is used—candidate scans.
|
|
245
270
|
|
|
271
|
+
`D1ServerStorage` updates the client record in place instead of deleting and
|
|
272
|
+
reinserting it, and a row update whose scope map is unchanged leaves the
|
|
273
|
+
scope-index entries alone. Both liveness timestamps keep their cadence: the
|
|
274
|
+
partition's `last_authenticated_at_ms` and the client's `updated_at_ms`
|
|
275
|
+
retain their existing refresh cadence. A logical same-value commit still
|
|
276
|
+
applies in full: it advances `server_version`, records the change, and stores
|
|
277
|
+
the submitted payload. See the measured local baseline in the
|
|
278
|
+
[Cloudflare Workers guide](https://syncular.dev/server-workers/#routine-write-cost).
|
|
279
|
+
|
|
246
280
|
Whole-commit validation checks a client-proposed commit; it does not grant
|
|
247
281
|
authority. Privileged operations such as connecting facilities still belong in
|
|
248
282
|
explicit server-authoritative commands.
|
|
249
283
|
|
|
284
|
+
## Push idempotency and retry contract
|
|
285
|
+
|
|
286
|
+
The idempotency key for a push commit is the triple
|
|
287
|
+
**(`partition`, `clientId`, `clientCommitId`)**. The server persists the full
|
|
288
|
+
commit result in the same transaction as the writes, and a replay of the same
|
|
289
|
+
key returns that result byte-equivalent: an applied commit replays as `cached`,
|
|
290
|
+
a rejected commit replays as `rejected` (§2.3, §6.3).
|
|
291
|
+
|
|
292
|
+
The key contains the commit ID and nothing about the operations. Host
|
|
293
|
+
authentication, request-envelope validation, and the §1.5 clientId-actor
|
|
294
|
+
binding run before the lookup.
|
|
295
|
+
`clientId` is a client-supplied namespace inside the authenticated partition,
|
|
296
|
+
not proof of an authenticated device. When a retained result exists for the
|
|
297
|
+
key, the server skips `buildOperations`, the commit and write validators, and
|
|
298
|
+
the apply transaction: a retry that reuses the key with different operations
|
|
299
|
+
returns the persisted result, and those operations are never built, validated,
|
|
300
|
+
or applied. `packages/server/test/push.test.ts` pins this.
|
|
301
|
+
|
|
302
|
+
A client MUST NOT reuse a `clientCommitId` for changed intent, including after
|
|
303
|
+
the persisted result has been pruned. One ID identifies one logical commit
|
|
304
|
+
permanently, and a later edit is a later commit with a new ID.
|
|
305
|
+
|
|
306
|
+
Do not key idempotency on the request payload bytes. A §5.11 client re-encrypts
|
|
307
|
+
an encrypted column at every send with a fresh nonce, and a schema upgrade
|
|
308
|
+
re-encodes pending commits, so the wire payload of an unchanged commit differs
|
|
309
|
+
between a lost-ACK retry and the original send. A payload fingerprint rejects
|
|
310
|
+
those retries. The `encryption/lost-ack-retry-dedupes-changed-ciphertext`
|
|
311
|
+
conformance scenario exercises the changed-ciphertext retry against both cores.
|
|
312
|
+
|
|
313
|
+
Per-device namespacing and content binding are not implemented. Each needs an
|
|
314
|
+
input the ID does not carry: an authenticated device identity from the host, or
|
|
315
|
+
a client-side format that binds the commit content. Neither is specified here,
|
|
316
|
+
and the ID check does not authenticate the client. Within one partition the ID
|
|
317
|
+
contract is a correctness property: it stops a replayed ID from applying
|
|
318
|
+
different operations.
|
|
319
|
+
|
|
250
320
|
## Seed idempotency and safe revisioning
|
|
251
321
|
|
|
252
322
|
`seedMutations` uses the real push path and a stable `clientId`/`commitId`.
|
|
@@ -463,10 +533,10 @@ context). The demo server wires it behind `SYNCULAR_DEMO_EVENTS=1`.
|
|
|
463
533
|
|
|
464
534
|
| Event | When | Key fields |
|
|
465
535
|
| --- | --- | --- |
|
|
466
|
-
| `request.handled` | Once per `POST /sync`, after the response bytes are fully produced (or the request was rejected up front) | `kind` (`sync`), `partition`, `actorId`, `durationMs`, `bytesIn`, `bytesOut`, `outcome` (`ok` \| `schema_floor` \| `rejected` \| `error`), `errorCode?`, `pushCommits`, `pulled`, `subscriptions` |
|
|
536
|
+
| `request.handled` | Once per `POST /sync`, after the response bytes are fully produced (or the request was rejected up front) | `kind` (`sync`), `partition`, `actorId`, `durationMs`, `bytesIn`, `bytesOut`, `outcome` (`ok` \| `schema_floor` \| `epoch_acquired` \| `reset` \| `rejected` \| `error`; `epoch_acquired` answers a first round without `logEpoch`, `reset` a `logEpoch` the log no longer matches), `errorCode?`, `clientId?` (absent when the request did not decode), `pushCommits`, `pulled`, `subscriptions` |
|
|
467
537
|
| `push.applied` | A `PUSH_COMMIT` applied, or replayed from the idempotency cache (§2.3) | `clientId`, `clientCommitId`, `operations`, `commitSeq?`, `replay` |
|
|
468
|
-
| `push.rejected` | A commit rejected (§6.3) | `clientId`, `clientCommitId`, `operations`, `code` (§10.2), `opIndex` |
|
|
469
|
-
| `push.conflicted` | A commit terminated by a version conflict (§6.2) | `clientId`, `clientCommitId`, `operations`, `opIndex` |
|
|
538
|
+
| `push.rejected` | A commit rejected (§6.3) | `clientId`, `clientCommitId`, `operations`, `code` (§10.2), `opIndex`, `table?` and `rowId?` of the terminating operation |
|
|
539
|
+
| `push.conflicted` | A commit terminated by a version conflict (§6.2) | `clientId`, `clientCommitId`, `operations`, `opIndex`, `table?` and `rowId?` of the conflicting operation |
|
|
470
540
|
| `reaction.queued` | A planned reaction committed with its source push | `clientId`, `clientCommitId`, `commitSeq`, `idempotencyKey`, `reactionType`, `version` |
|
|
471
541
|
| `reaction.started` | A worker claimed and began one attempt | `workerId`, `idempotencyKey`, `reactionType`, `version`, `attempt` |
|
|
472
542
|
| `reaction.retried` | A retryable attempt failed and was rescheduled | started fields plus `nextAttemptAtMs`, `errorCode` |
|
|
@@ -484,26 +554,31 @@ context). The demo server wires it behind `SYNCULAR_DEMO_EVENTS=1`.
|
|
|
484
554
|
| `scopes.resolve_failed` | The host `resolveScopes` callback threw — the §3.2/§3.4 fail-loud path | `phase` (`request` \| `realtime` \| `segment-download`), `message` |
|
|
485
555
|
|
|
486
556
|
All events also carry `type`, `atMs`, and (where a request identity
|
|
487
|
-
exists) `partition` / `actorId`.
|
|
557
|
+
exists) `partition` / `actorId`. The events emitted while serving one sync
|
|
558
|
+
request (`request.handled`, `push.*`, `pull.served`, `lease.issued`,
|
|
559
|
+
`reaction.queued`, `scopes.resolve_failed`) share one `requestId`, so a sink
|
|
560
|
+
can group a request's events.
|
|
488
561
|
|
|
489
562
|
## Admin / console surface (`SyncularAdmin`)
|
|
490
563
|
|
|
491
|
-
The operator-facing
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
JSON-able queries.
|
|
564
|
+
The operator-facing surface over the server core lives in this package and
|
|
565
|
+
adds no wire protocol. Authorization is entirely the host's. It answers
|
|
566
|
+
who is connected, what is flowing and failing, why a client does or does not
|
|
567
|
+
see a row, and how far the horizon trails, through partition-scoped JSON-able
|
|
568
|
+
queries plus three operator actions.
|
|
497
569
|
|
|
498
570
|
### The event ring (the "event stream")
|
|
499
571
|
|
|
500
572
|
`RingBufferEvents` is a `SyncularServerEvents` sink that retains the last N
|
|
501
573
|
events in memory (bounded — oldest dropped when full) with a
|
|
502
|
-
`query({type?, sinceMs?, clientId?, actorId?, limit})` and a
|
|
574
|
+
`query({type?, partition?, sinceMs?, clientId?, actorId?, limit})` and a
|
|
503
575
|
`subscribe(listener)` hook for live tails (the SSE route below). It is the
|
|
504
|
-
event stream without any infrastructure dependency.
|
|
505
|
-
|
|
506
|
-
|
|
576
|
+
event stream without any infrastructure dependency. `evictedThroughMs` is
|
|
577
|
+
the newest `atMs` the ring dropped; the admin reports a window that starts
|
|
578
|
+
at or before it as incomplete. The `admin` option of `createSyncularHono`
|
|
579
|
+
and the Workers Durable Object create and compose the ring themselves. A
|
|
580
|
+
script that queries `SyncularAdmin` directly composes it with any other sink
|
|
581
|
+
so the ring and your logs/metrics see the same emissions:
|
|
507
582
|
|
|
508
583
|
```ts
|
|
509
584
|
import {
|
|
@@ -520,28 +595,51 @@ const admin = SyncularAdmin.fromConfig(config, { ring });
|
|
|
520
595
|
|
|
521
596
|
### Query surface
|
|
522
597
|
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
598
|
+
All methods are partition-scoped except the two fleet reads
|
|
599
|
+
(`listPartitions` / `partitionsOverview`), which enumerate partitions by
|
|
600
|
+
design. Row reads need the server `schema` (`fromConfig` passes it).
|
|
526
601
|
|
|
527
602
|
| Method | Returns |
|
|
528
603
|
| --- | --- |
|
|
529
|
-
| `
|
|
530
|
-
| `
|
|
531
|
-
| `
|
|
604
|
+
| `tables()` / `schemaVersion` | The schema tables with `columns[]` (`name`, `type`, `nullable`, `primaryKey`), and the server schema version. |
|
|
605
|
+
| `search(partition, query)` | Resolves one pasted id: clients whose id contains it or whose actor equals it, actors containing it, rows with that rowId in any table, the commit with that seq, and write outcomes with that clientCommitId. |
|
|
606
|
+
| `actors(partition)` | Each actor with its devices (clients), open sockets, worst lag, and failures in the last 5 minutes (the `errors().byClient` counts summed per actor). |
|
|
607
|
+
| `tableCounts(partition)` | Row count per schema table. |
|
|
608
|
+
| `listRows(partition, table, {afterRowId?, limit?, scope?})` | One page of decoded rows ordered by rowId, optionally only rows whose stored `scope` (`{variable, value}`) matches, read through the scope index; `nextAfterRowId` continues. |
|
|
609
|
+
| `rowDelivery(partition, table, rowId)` | The trace verdict for every known client on one row, sorted by verdict then last seen: who has the row and who does not. |
|
|
610
|
+
| `findWrite(partition, clientCommitId)` | Every stored idempotency outcome for that clientCommitId (applied with `commitSeq`, or rejected with per-operation results; conflict bytes as base64) and the ring events that carry it. |
|
|
611
|
+
| `listClients(partition, {limit?, sort?})` | `{clients, total}`. Each client: `clientId`, `actorId`, `wireVersion`, `schemaVersion?` (highest among its subscriptions), `cursor`, `lag` (`maxCommitSeq − max(cursor, 0)`), `updatedAtMs`, `subscriptions[]`, `active` (touched within the §4.6 active window), `connected?` and `connectedSinceMs?` (open realtime sockets on this process; present when a hub is wired, which `fromConfig` does when `config.realtime` is a `RealtimeHub`). `sort`: `seen` (default) or `lag`; `limit` default 200. |
|
|
612
|
+
| `clientDetail(partition, clientId, {eventLimit?})` | `{exists, client?, lease?, events}`: the record, its §7.3 lease when a lease store is wired, and its slice of the event tail. |
|
|
613
|
+
| `listCommits(partition, {afterSeq?, beforeSeq?, limit?, table?, changeSample?})` | Commit metadata, newest first: `commitSeq`, `clientId`, `clientCommitId`, `actorId`, `createdAtMs`, `changeCount`, `tables[]`, and with `changeSample` the first N changes as `sample[]` (`table`, `rowId`, `op`). Pass the oldest `commitSeq` of a page as `beforeSeq` for the next older page. |
|
|
614
|
+
| `commitDetail(partition, commitSeq)` | The commit's metadata plus `changes[]` (`table`, `rowId`, `op`, `rowVersion?`, `scopes`, `values?` rendered per SPEC §11, `undecodable?` for a payload written under an older column set). Throws `sync.not_found` for a missing or pruned commit. |
|
|
532
615
|
| `listReactions(partition, {statuses?, types?, limit?})` | Durable reaction lifecycle rows, newest first, including source commit, attempts, lease, completion, and bounded failure information. |
|
|
533
|
-
| `inspectRow(partition, table, rowId)` | `{exists, serverVersion?, scopes
|
|
616
|
+
| `inspectRow(partition, table, rowId, {historyLimit?})` | `{exists, serverVersion?, scopes?, values?, history}`: the row decoded per SPEC §11 and its retained changes, newest first (`commitSeq`, `op`, `rowVersion?`, `actorId`, `clientId`, `createdAtMs`). Throws `sync.unknown_table` for a table outside the schema. |
|
|
617
|
+
| `traceRow(partition, {table, rowId, clientId})` | Whether the client receives the row: a `verdict` (`client-unknown`, `row-missing`, `no-subscription`, `row-deleted`, `scope-mismatch`, `scope-denied`, `pending-pull`, `delivered`, evaluated in that order) with the row, its newest change, the horizon, the client, the actor's `allowed` scopes from `resolveScopes` (`outage` for the §7.3.3 sentinel), each subscription on the table with `requestedMatch`, `mismatches[]` and `effectiveMatch` (requested ∩ allowed, §3.2), and the client's retained push rejections. |
|
|
534
618
|
| `scopeActivity(partition, {variable, value}, {limit?})` | Recent commits touching one scope key, via the §3.1 change-scope index (never a log scan). |
|
|
535
|
-
| `horizonStatus(partition)` | `{maxCommitSeq, horizonSeq, retainedCommits, activeCursorFloor, recommendedHorizonSeq, recommendation}
|
|
536
|
-
| `
|
|
537
|
-
| `metrics(partition, {windowMs?, buckets?})` | Ring-derived request/push health over a trailing window (default 5 min): request count/rate, error share, p50/p95 duration, push applied/rejected/conflicted, and per-bucket counts
|
|
538
|
-
| `
|
|
539
|
-
| `
|
|
619
|
+
| `horizonStatus(partition)` | `{maxCommitSeq, horizonSeq, retainedCommits, activeCursorFloor, recommendedHorizonSeq, recommendation}`: the horizon a prune pass would reach now (§4.6) and `up-to-date` / `prune-recommended`. |
|
|
620
|
+
| `stats(partition)` | `{segments?, blobs?}` counts/bytes where the stores expose them (segments split rows/sqlite). |
|
|
621
|
+
| `metrics(partition, {windowMs?, buckets?})` | Ring-derived request/push health over a trailing window (default 5 min): request count/rate, error share, p50/p95 duration, push applied/rejected/conflicted, and per-bucket counts, errors, p95 and push outcomes. `coverage: {complete, fromMs}` is false when the ring dropped events inside the window. All zeros when no ring is wired. |
|
|
622
|
+
| `errors(partition, {windowMs?})` | Ring failures in the window: `byCode[]` (`source` `request`/`push`/`reaction`/`scopes`, `code`, `count`, distinct `clients` and `actors`, `lastAtMs`, `sampleClientIds`, `sampleRows`) and every client with a request or push failure in `byClient[]`, most failures first, with `coverage`. |
|
|
623
|
+
| `listPartitionRegistry()` / `partitionsOverview({limit?, offset?})` | Every authenticated partition; the overview pages them (default 100) as `{partitions, total}` with retained backlog, client counts, and the prune recommendation. |
|
|
624
|
+
| `events({type?, partition?, requestId?, sinceMs?, clientId?, actorId?, limit?})` | The ring tail, newest first. Empty when no ring is wired (`hasEventStream` reports which). |
|
|
540
625
|
| `subscribeEvents(listener)` | Live events as they land in the ring; returns the unsubscribe function, `undefined` when no ring is wired. |
|
|
541
626
|
|
|
627
|
+
Operator actions:
|
|
628
|
+
|
|
629
|
+
| Method | Effect |
|
|
630
|
+
| --- | --- |
|
|
631
|
+
| `retryReaction(partition, idempotencyKey)` | Requeues a dead-lettered reaction and returns it. Throws `sync.invalid_request` when the key names no dead-lettered reaction. |
|
|
632
|
+
| `prune(partition)` | One §4.6 prune pass now; returns `{previousHorizonSeq, horizonSeq}` and emits `prune.completed`. |
|
|
633
|
+
| `revokeLease(partition, clientId)` | Revokes the client's §7.3 lease (§7.3.4), emits `lease.revoked`, returns the revoked record. Throws `sync.invalid_request` when the client holds none. |
|
|
634
|
+
|
|
542
635
|
The query surface leans on **additive, optional** storage/store methods
|
|
543
|
-
(`ServerStorage.listClientRecords` / `listCommitMetadata` / `
|
|
544
|
-
`
|
|
636
|
+
(`ServerStorage.listClientRecords` / `listCommitMetadata` / `getCommit` /
|
|
637
|
+
`listRowChanges` / `listRows` / `countRows` / `listPushResults` /
|
|
638
|
+
`scopeActivity` / `listPartitions`; `SegmentStore.stats`;
|
|
639
|
+
`BlobStore.stats`). `listRowChanges` reads the partition's retained log
|
|
640
|
+
and `listPushResults` reads the stored idempotency outcomes, both without a
|
|
641
|
+
dedicated index, so the write path pays nothing for them and their cost is
|
|
642
|
+
bounded by retention
|
|
545
643
|
— the established
|
|
546
644
|
optional-method pattern. `SqliteServerStorage`, `PostgresServerStorage`,
|
|
547
645
|
`D1ServerStorage`, and the memory/sqlite stores implement them; the shared
|
|
@@ -555,72 +653,94 @@ stores (memory/sqlite) omit the marker.
|
|
|
555
653
|
|
|
556
654
|
### HTTP routes + the single console page
|
|
557
655
|
|
|
558
|
-
|
|
559
|
-
|
|
560
|
-
|
|
561
|
-
|
|
656
|
+
`createSyncularHono` mounts the console with its `admin` option, and
|
|
657
|
+
`createWorkersFetchHandler` takes the same guards (see the
|
|
658
|
+
`@syncular/server-workers` README). The option throws without the
|
|
659
|
+
`authorize` guard, so the admin never mounts open. Every endpoint, including
|
|
660
|
+
the page, runs the guard first; a falsy result is a 401. The `POST` actions
|
|
661
|
+
also run `authorizeActions`; without it, or when it returns falsy, they
|
|
662
|
+
answer 403 `sync.forbidden`.
|
|
562
663
|
|
|
563
664
|
```ts
|
|
564
|
-
import {
|
|
565
|
-
|
|
566
|
-
const
|
|
567
|
-
|
|
568
|
-
|
|
665
|
+
import { createSyncularHono } from '@syncular/server-hono';
|
|
666
|
+
|
|
667
|
+
const app = createSyncularHono({
|
|
668
|
+
config,
|
|
669
|
+
authenticate,
|
|
670
|
+
admin: {
|
|
671
|
+
authorize: ({ request }) => isOperator(request), // YOUR check, mandatory
|
|
672
|
+
authorizeActions: ({ request }) => isOnCall(request), // optional actions
|
|
673
|
+
// path: '/admin', ringCapacity: 1000 (the defaults)
|
|
674
|
+
},
|
|
569
675
|
});
|
|
570
|
-
app.route('/admin', routes);
|
|
571
676
|
```
|
|
572
677
|
|
|
678
|
+
The option creates a `RingBufferEvents` of `ringCapacity` events and
|
|
679
|
+
composes it into `config.events` and, when `config.realtime` is a
|
|
680
|
+
`RealtimeHub`, into the hub's events (`RealtimeHub.addEventSink`), so the
|
|
681
|
+
console sees HTTP and socket rounds. Admin errors go to `config.onError` and
|
|
682
|
+
`config.mapError`.
|
|
683
|
+
|
|
573
684
|
| Route | Mirrors |
|
|
574
685
|
| --- | --- |
|
|
575
686
|
| `GET /` | The console page (see below). |
|
|
576
|
-
| `GET /
|
|
687
|
+
| `GET /info` | `{partition, hasEventStream, actions, realtime, schemaVersion?, tables}` |
|
|
688
|
+
| `GET /search?q` | `search` |
|
|
689
|
+
| `GET /actors` | `actors` |
|
|
690
|
+
| `GET /tables` | `tables()` with each table's `rowCount` |
|
|
691
|
+
| `GET /rows/:table?afterRowId&limit&scope=variable:value` | `listRows` |
|
|
692
|
+
| `GET /rows/:table/:rowId/delivery` | `rowDelivery` |
|
|
693
|
+
| `GET /writes/:clientCommitId` | `findWrite` |
|
|
694
|
+
| `GET /clients?limit&sort` | `listClients` |
|
|
577
695
|
| `GET /clients/:clientId?eventLimit` | `clientDetail` |
|
|
578
|
-
| `GET /commits?afterSeq&limit&table` | `listCommits` |
|
|
696
|
+
| `GET /commits?afterSeq&beforeSeq&limit&table&sample` | `listCommits` (`sample` is `changeSample`) |
|
|
697
|
+
| `GET /commits/:commitSeq` | `commitDetail` |
|
|
579
698
|
| `GET /reactions?status&type&limit` | `listReactions` |
|
|
580
|
-
| `GET /rows/:table/:rowId` | `inspectRow` |
|
|
699
|
+
| `GET /rows/:table/:rowId?historyLimit` | `inspectRow` |
|
|
700
|
+
| `GET /trace?table&rowId&clientId` | `traceRow` |
|
|
581
701
|
| `GET /scope-activity?variable&value&limit` | `scopeActivity` |
|
|
582
702
|
| `GET /horizon` | `horizonStatus` |
|
|
583
703
|
| `GET /stats` | `stats` |
|
|
584
704
|
| `GET /metrics?windowMs` | `metrics` |
|
|
585
|
-
| `GET /
|
|
586
|
-
| `GET /
|
|
587
|
-
| `GET /events
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
`
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
|
|
594
|
-
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
705
|
+
| `GET /errors?windowMs` | `errors` |
|
|
706
|
+
| `GET /partitions?limit&offset` | `partitionsOverview` (fleet view) |
|
|
707
|
+
| `GET /events?type&requestId&sinceMs&clientId&actorId&limit` | `events` (ring tail) |
|
|
708
|
+
| `GET /events/stream?type&requestId&clientId&actorId&limit` | live SSE tail (backlog replay, then `subscribeEvents`) |
|
|
709
|
+
| `POST /reactions/:idempotencyKey/retry` | `retryReaction` |
|
|
710
|
+
| `POST /prune` | `prune` |
|
|
711
|
+
| `POST /clients/:clientId/revoke-lease` | `revokeLease` |
|
|
712
|
+
|
|
713
|
+
`?partition=` selects the partition. Without it, a deployment with exactly
|
|
714
|
+
one partition in its registry uses that one; any other count answers 400
|
|
715
|
+
`sync.invalid_request` with `details.parameter: 'partition'`, after the
|
|
716
|
+
guard ran with an empty partition. Every endpoint reads only the resolved
|
|
717
|
+
partition, event tail included. A malformed integer query parameter answers
|
|
718
|
+
400 `sync.invalid_request`. `GET /partitions` is the one cross-partition
|
|
719
|
+
endpoint: its guard receives an empty `partition`, so a guard that grants
|
|
720
|
+
single partitions does not grant the fleet view.
|
|
721
|
+
|
|
722
|
+
`GET /` serves a **single static HTML page**: no framework, no build step,
|
|
723
|
+
no external assets (about 110 KiB, 31 KiB gzipped). It fetches the sibling
|
|
724
|
+
JSON endpoints relative to its own mount path, so it works under any prefix
|
|
725
|
+
and the same guard covers its requests. Operator actions appear only when
|
|
726
|
+
`/info` reports `actions: true`. The views and the debugging workflows are
|
|
727
|
+
on the docs page [Admin console](https://syncular.dev/server-admin-console/).
|
|
605
728
|
|
|
606
729
|
**Live tail over SSE.** `GET /events/stream` streams the ring as
|
|
607
730
|
server-sent events: a recent backlog replays first, then events arrive as
|
|
608
731
|
they land (a `subscribe` hook on the ring), with a comment ping every 5 s
|
|
609
732
|
(under Bun.serve's 10 s default idle timeout, so a quiet stream survives).
|
|
610
733
|
It is plain web-streams, so it serves identically on Bun, Node, and
|
|
611
|
-
Workers. The page
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
734
|
+
Workers. The page opens the stream when `/info` reports `hasEventStream`;
|
|
735
|
+
when the stream fails (for example, `EventSource` cannot send the host's
|
|
736
|
+
auth headers, while cookie-authorized setups stream fine), events reload
|
|
737
|
+
with every refresh. The tail can carry sensitive identifiers (actorIds,
|
|
615
738
|
session ids), the same as `GET /events`; the admin guard gates both.
|
|
616
739
|
|
|
617
740
|
The demo server (`apps/demo`) mounts the admin behind a dev guard:
|
|
618
741
|
`SYNCULAR_DEMO_ADMIN=1` enables `/admin` (optionally token-gated with
|
|
619
|
-
`SYNCULAR_DEMO_ADMIN_TOKEN`
|
|
620
|
-
|
|
621
|
-
> Docs-site coverage of the console is a follow-up: the docs app is owned
|
|
622
|
-
> by a concurrent workstream this round (the schema-bump page), so this
|
|
623
|
-
> README is the console's documentation home for now.
|
|
742
|
+
`SYNCULAR_DEMO_ADMIN_TOKEN`; `SYNCULAR_DEMO_ADMIN_WRITES=1` adds the
|
|
743
|
+
operator actions), so the console is inspectable live.
|
|
624
744
|
|
|
625
745
|
## Segment storage on S3 / R2 (`S3SegmentStore`)
|
|
626
746
|
|
|
@@ -725,7 +845,7 @@ honor these conditional headers.
|
|
|
725
845
|
|
|
726
846
|
Even so, the counters are **approximate**, and `stats()` marks them
|
|
727
847
|
`approximate: true` (an additive field the admin surface carries through, so
|
|
728
|
-
`admin.
|
|
848
|
+
`admin.stats()` exposes it and the console labels the
|
|
729
849
|
numbers honestly). They can drift: a crash between the segment PUT and the
|
|
730
850
|
accumulator CAS, lifecycle GC deleting objects the accumulator still counts,
|
|
731
851
|
or enough concurrent writers to exhaust the CAS retry budget. They are a
|
|
@@ -887,7 +1007,7 @@ GET/HEAD.
|
|
|
887
1007
|
object (`{keyPrefix}stats/blobs.json`) folded read-modify-write on `put` under
|
|
888
1008
|
the same ETag compare-and-swap as segments, and marks the result
|
|
889
1009
|
`approximate: true` (the marker already present on `BlobStoreStats`, carried
|
|
890
|
-
through `admin.
|
|
1010
|
+
through `admin.stats()`). A `HEAD` before the PUT detects an idempotent
|
|
891
1011
|
re-upload so distinct blobs count once; the sweep decrements on delete. Same
|
|
892
1012
|
caveats as segment stats — a health gauge, not an invoice. The in-process
|
|
893
1013
|
stores (memory / sqlite) count on demand and omit the marker.
|