@quereus/sync 4.3.2 → 4.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -1
- package/dist/src/create-sync-module.d.ts +11 -1
- package/dist/src/create-sync-module.d.ts.map +1 -1
- package/dist/src/create-sync-module.js +2 -2
- package/dist/src/create-sync-module.js.map +1 -1
- package/dist/src/index.d.ts +4 -3
- package/dist/src/index.d.ts.map +1 -1
- package/dist/src/index.js +9 -3
- package/dist/src/index.js.map +1 -1
- package/dist/src/metadata/change-log.d.ts +39 -3
- package/dist/src/metadata/change-log.d.ts.map +1 -1
- package/dist/src/metadata/change-log.js +43 -8
- package/dist/src/metadata/change-log.js.map +1 -1
- package/dist/src/metadata/column-version.d.ts +62 -9
- package/dist/src/metadata/column-version.d.ts.map +1 -1
- package/dist/src/metadata/column-version.js +79 -25
- package/dist/src/metadata/column-version.js.map +1 -1
- package/dist/src/metadata/index.d.ts +1 -0
- package/dist/src/metadata/index.d.ts.map +1 -1
- package/dist/src/metadata/index.js +1 -0
- package/dist/src/metadata/index.js.map +1 -1
- package/dist/src/metadata/keys.d.ts +88 -22
- package/dist/src/metadata/keys.d.ts.map +1 -1
- package/dist/src/metadata/keys.js +134 -77
- package/dist/src/metadata/keys.js.map +1 -1
- package/dist/src/metadata/pk-identity.d.ts +60 -0
- package/dist/src/metadata/pk-identity.d.ts.map +1 -0
- package/dist/src/metadata/pk-identity.js +94 -0
- package/dist/src/metadata/pk-identity.js.map +1 -0
- package/dist/src/metadata/tombstones.d.ts +41 -11
- package/dist/src/metadata/tombstones.d.ts.map +1 -1
- package/dist/src/metadata/tombstones.js +58 -33
- package/dist/src/metadata/tombstones.js.map +1 -1
- package/dist/src/sync/change-applicator.d.ts +30 -1
- package/dist/src/sync/change-applicator.d.ts.map +1 -1
- package/dist/src/sync/change-applicator.js +161 -40
- package/dist/src/sync/change-applicator.js.map +1 -1
- package/dist/src/sync/maintenance.d.ts +70 -0
- package/dist/src/sync/maintenance.d.ts.map +1 -0
- package/dist/src/sync/maintenance.js +84 -0
- package/dist/src/sync/maintenance.js.map +1 -0
- package/dist/src/sync/protocol.d.ts +61 -10
- package/dist/src/sync/protocol.d.ts.map +1 -1
- package/dist/src/sync/protocol.js +15 -0
- package/dist/src/sync/protocol.js.map +1 -1
- package/dist/src/sync/snapshot-identity.d.ts +62 -0
- package/dist/src/sync/snapshot-identity.d.ts.map +1 -0
- package/dist/src/sync/snapshot-identity.js +88 -0
- package/dist/src/sync/snapshot-identity.js.map +1 -0
- package/dist/src/sync/snapshot-stream.d.ts +8 -1
- package/dist/src/sync/snapshot-stream.d.ts.map +1 -1
- package/dist/src/sync/snapshot-stream.js +166 -67
- package/dist/src/sync/snapshot-stream.js.map +1 -1
- package/dist/src/sync/snapshot.d.ts +1 -1
- package/dist/src/sync/snapshot.d.ts.map +1 -1
- package/dist/src/sync/snapshot.js +75 -44
- package/dist/src/sync/snapshot.js.map +1 -1
- package/dist/src/sync/staged-transaction-metadata.d.ts +75 -0
- package/dist/src/sync/staged-transaction-metadata.d.ts.map +1 -0
- package/dist/src/sync/staged-transaction-metadata.js +100 -0
- package/dist/src/sync/staged-transaction-metadata.js.map +1 -0
- package/dist/src/sync/store-adapter.d.ts +10 -4
- package/dist/src/sync/store-adapter.d.ts.map +1 -1
- package/dist/src/sync/store-adapter.js +244 -21
- package/dist/src/sync/store-adapter.js.map +1 -1
- package/dist/src/sync/sync-context.d.ts +54 -2
- package/dist/src/sync/sync-context.d.ts.map +1 -1
- package/dist/src/sync/sync-context.js +66 -0
- package/dist/src/sync/sync-context.js.map +1 -1
- package/dist/src/sync/sync-manager-impl.d.ts +70 -4
- package/dist/src/sync/sync-manager-impl.d.ts.map +1 -1
- package/dist/src/sync/sync-manager-impl.js +206 -36
- package/dist/src/sync/sync-manager-impl.js.map +1 -1
- package/dist/src/sync/wire.d.ts +47 -6
- package/dist/src/sync/wire.d.ts.map +1 -1
- package/dist/src/sync/wire.js +62 -11
- package/dist/src/sync/wire.js.map +1 -1
- package/package.json +6 -6
package/README.md
CHANGED
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
# @quereus/sync
|
|
2
2
|
|
|
3
|
+
> **Stability: Experimental** — a research track; the API, the wire protocol, and the
|
|
4
|
+
> stored bytes may change or disappear without notice, in any release including a patch.
|
|
5
|
+
> See [Stability Tiers](../../docs/stability.md#tiers).
|
|
6
|
+
|
|
3
7
|
CRDT-based multi-master sync framework for [Quereus](https://github.com/gotchoices/quereus). Enables offline-first applications with automatic conflict resolution.
|
|
4
8
|
|
|
5
9
|
## Features
|
|
@@ -126,6 +130,11 @@ if (checkpoint) {
|
|
|
126
130
|
}
|
|
127
131
|
```
|
|
128
132
|
|
|
133
|
+
A `SnapshotCheckpoint` holds a `Uint8Array` site id and a `bigint` HLC wall time,
|
|
134
|
+
so it is **not** JSON-safe as-is. To send one to a peer (e.g. in a
|
|
135
|
+
`resume_snapshot` message), run it through `serializeSnapshotCheckpoint` and
|
|
136
|
+
recover it on the far side with `deserializeSnapshotCheckpoint`.
|
|
137
|
+
|
|
129
138
|
## Events
|
|
130
139
|
|
|
131
140
|
Subscribe to sync events for UI updates:
|
|
@@ -169,6 +178,15 @@ This means concurrent updates to *different* columns of the same row both apply,
|
|
|
169
178
|
- `SyncManager` - Main sync coordination interface
|
|
170
179
|
- `SyncEventEmitter` / `SyncEventEmitterImpl` - Event subscription interface and implementation
|
|
171
180
|
|
|
181
|
+
### Maintenance Exports
|
|
182
|
+
|
|
183
|
+
The four housekeeping sweeps (`drainHeldChanges` / `pruneQuarantine` / `pruneTombstones` / `evictExpiredBasisTables`) are methods on `SyncManager`; this package deliberately schedules none of them — the host arms the timer. What it does ship is the *shape* of one pass, so every host runs the same semantics:
|
|
184
|
+
|
|
185
|
+
- `runSyncMaintenancePass(target, log)` - Runs all four sweeps in drain-before-prune order. Each sweep is error-isolated (a throw is reported to `log` and the remaining sweeps still run); the pass never rejects
|
|
186
|
+
- `createSyncMaintenanceTicker(getTarget, log)` - Wraps the pass with the single-flight and null-target guards a timer needs. The collapsed tick resolves immediately rather than joining the running pass — a host that must *await* the in-flight pass (to close stores at shutdown) should hold the pass promise itself
|
|
187
|
+
- `SYNC_MAINTENANCE_INTERVAL_MS` - Suggested cadence (5 minutes), sized for the one latency-sensitive sweep (`drainHeldChanges`). A host where that sweep is inert — a relay with no `getTableSchema` oracle — is free to run slower
|
|
188
|
+
- `SyncMaintenanceTarget` / `MaintenanceLogger` - The structural slice of `SyncManager` the pass calls, and the failure-reporting callback
|
|
189
|
+
|
|
172
190
|
### Clock Exports
|
|
173
191
|
|
|
174
192
|
- `HLCManager` - Hybrid Logical Clock manager
|
|
@@ -181,7 +199,7 @@ This means concurrent updates to *different* columns of the same row both apply,
|
|
|
181
199
|
- `ChangeSet` - Collection of changes from one transaction
|
|
182
200
|
- `Change` (`ColumnChange | RowDeletion`) - Single column or row change
|
|
183
201
|
- `SchemaMigration` - Schema change (CREATE/ALTER/DROP TABLE)
|
|
184
|
-
- `SnapshotChunk` - Streaming snapshot data
|
|
202
|
+
- `SnapshotChunk` - Streaming snapshot data, in emission order: header, schema-migration, table-start, column-versions, table-end, tombstone, footer. All DDL precedes all table data — see docs/sync.md § Streaming Snapshot API
|
|
185
203
|
- `ApplyResult` - Result of applying changes (applied, skipped, conflicts, transactions)
|
|
186
204
|
- `SyncConfig` / `DEFAULT_SYNC_CONFIG` - Configuration (retentionHorizonMs, allowResurrection, etc.)
|
|
187
205
|
|
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
* CRDT sync capabilities.
|
|
6
6
|
*/
|
|
7
7
|
import type { KVStore } from '@quereus/store';
|
|
8
|
-
import type { TableSchema, TransactionCommitBatch } from '@quereus/quereus';
|
|
8
|
+
import type { KeyNormalizerResolver, TableSchema, TransactionCommitBatch } from '@quereus/quereus';
|
|
9
9
|
import { SyncEventEmitterImpl } from './sync/events.js';
|
|
10
10
|
import { type SyncConfig, type ApplyToStoreCallback, type DropLocalTableCallback } from './sync/protocol.js';
|
|
11
11
|
import type { SyncManager } from './sync/manager.js';
|
|
@@ -70,6 +70,16 @@ export interface CreateSyncModuleOptions extends Partial<SyncConfig> {
|
|
|
70
70
|
* store) the sweep is a no-op.
|
|
71
71
|
*/
|
|
72
72
|
dropLocalTable?: DropLocalTableCallback;
|
|
73
|
+
/**
|
|
74
|
+
* Collation-name → key-normalizer resolver used to derive each row's pk
|
|
75
|
+
* IDENTITY (what per-row sync metadata is filed under). Pass the engine's
|
|
76
|
+
* `db.getKeyNormalizerResolver()` whenever a `getTableSchema` oracle is wired,
|
|
77
|
+
* so sync keys rows exactly as the database does — including collations
|
|
78
|
+
* registered with `db.registerCollation`. When omitted, a built-ins-only
|
|
79
|
+
* resolver (BINARY/NOCASE/RTRIM) is used, which throws on any custom
|
|
80
|
+
* collation name rather than mis-keying it.
|
|
81
|
+
*/
|
|
82
|
+
keyNormalizerResolver?: KeyNormalizerResolver;
|
|
73
83
|
/**
|
|
74
84
|
* Engine transaction-commit source for capturing local changes.
|
|
75
85
|
*
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"create-sync-module.d.ts","sourceRoot":"","sources":["../../src/create-sync-module.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,gBAAgB,CAAC;AAC9C,OAAO,KAAK,EAAE,WAAW,EAAE,sBAAsB,EAAE,MAAM,kBAAkB,CAAC;
|
|
1
|
+
{"version":3,"file":"create-sync-module.d.ts","sourceRoot":"","sources":["../../src/create-sync-module.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,gBAAgB,CAAC;AAC9C,OAAO,KAAK,EAAE,qBAAqB,EAAE,WAAW,EAAE,sBAAsB,EAAE,MAAM,kBAAkB,CAAC;AAEnG,OAAO,EAAE,oBAAoB,EAAE,MAAM,kBAAkB,CAAC;AACxD,OAAO,EAAuB,KAAK,UAAU,EAAE,KAAK,oBAAoB,EAAE,KAAK,sBAAsB,EAAE,MAAM,oBAAoB,CAAC;AAClI,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAErD;;;GAGG;AACH,MAAM,MAAM,sBAAsB,GAAG,CAAC,UAAU,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,KAAK,WAAW,GAAG,SAAS,CAAC;AAExG;;;;;;;;GAQG;AACH,MAAM,WAAW,uBAAuB;IACtC,mFAAmF;IACnF,mBAAmB,CAAC,QAAQ,EAAE,CAAC,KAAK,EAAE,sBAAsB,KAAK,IAAI,GAAG,MAAM,IAAI,CAAC;CACpF;AAED;;GAEG;AACH,MAAM,WAAW,sBAAsB;IACrC,2CAA2C;IAC3C,WAAW,EAAE,WAAW,CAAC;IACzB,gDAAgD;IAChD,UAAU,EAAE,oBAAoB,CAAC;CAClC;AAED;;GAEG;AACH,MAAM,WAAW,uBAAwB,SAAQ,OAAO,CAAC,UAAU,CAAC;IAClE;;;;;;;;;;OAUG;IACH,YAAY,CAAC,EAAE,oBAAoB,CAAC;IAEpC;;;;;;;;OAQG;IACH,cAAc,CAAC,EAAE,sBAAsB,CAAC;IAExC;;;;;;OAMG;IACH,cAAc,CAAC,EAAE,sBAAsB,CAAC;IAExC;;;;;;;;OAQG;IACH,qBAAqB,CAAC,EAAE,qBAAqB,CAAC;IAE9C;;;;;;;;;;;OAWG;IACH,iBAAiB,CAAC,EAAE,uBAAuB,CAAC;CAC7C;AAED;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAiCG;AACH,wBAAsB,gBAAgB,CACpC,EAAE,EAAE,OAAO,EACX,OAAO,GAAE,uBAA4B,GACpC,OAAO,CAAC,sBAAsB,CAAC,CAyBjC"}
|
|
@@ -42,13 +42,13 @@ import { DEFAULT_SYNC_CONFIG } from './sync/protocol.js';
|
|
|
42
42
|
* ```
|
|
43
43
|
*/
|
|
44
44
|
export async function createSyncModule(kv, options = {}) {
|
|
45
|
-
const { applyToStore, getTableSchema, dropLocalTable, transactionSource, ...configOverrides } = options;
|
|
45
|
+
const { applyToStore, getTableSchema, dropLocalTable, keyNormalizerResolver, transactionSource, ...configOverrides } = options;
|
|
46
46
|
const fullConfig = {
|
|
47
47
|
...DEFAULT_SYNC_CONFIG,
|
|
48
48
|
...configOverrides,
|
|
49
49
|
};
|
|
50
50
|
const syncEvents = new SyncEventEmitterImpl();
|
|
51
|
-
const syncManager = await SyncManagerImpl.create(kv, transactionSource, fullConfig, syncEvents, applyToStore, getTableSchema, dropLocalTable);
|
|
51
|
+
const syncManager = await SyncManagerImpl.create(kv, transactionSource, fullConfig, syncEvents, applyToStore, getTableSchema, dropLocalTable, keyNormalizerResolver);
|
|
52
52
|
return {
|
|
53
53
|
syncManager,
|
|
54
54
|
syncEvents,
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"create-sync-module.js","sourceRoot":"","sources":["../../src/create-sync-module.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAIH,OAAO,EAAE,eAAe,EAAE,MAAM,6BAA6B,CAAC;AAC9D,OAAO,EAAE,oBAAoB,EAAE,MAAM,kBAAkB,CAAC;AACxD,OAAO,EAAE,mBAAmB,EAA2E,MAAM,oBAAoB,CAAC;
|
|
1
|
+
{"version":3,"file":"create-sync-module.js","sourceRoot":"","sources":["../../src/create-sync-module.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAIH,OAAO,EAAE,eAAe,EAAE,MAAM,6BAA6B,CAAC;AAC9D,OAAO,EAAE,oBAAoB,EAAE,MAAM,kBAAkB,CAAC;AACxD,OAAO,EAAE,mBAAmB,EAA2E,MAAM,oBAAoB,CAAC;AAgGlI;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAiCG;AACH,MAAM,CAAC,KAAK,UAAU,gBAAgB,CACpC,EAAW,EACX,UAAmC,EAAE;IAErC,MAAM,EAAE,YAAY,EAAE,cAAc,EAAE,cAAc,EAAE,qBAAqB,EAAE,iBAAiB,EAAE,GAAG,eAAe,EAAE,GAAG,OAAO,CAAC;IAE/H,MAAM,UAAU,GAAe;QAC7B,GAAG,mBAAmB;QACtB,GAAG,eAAe;KACnB,CAAC;IAEF,MAAM,UAAU,GAAG,IAAI,oBAAoB,EAAE,CAAC;IAE9C,MAAM,WAAW,GAAG,MAAM,eAAe,CAAC,MAAM,CAC9C,EAAE,EACF,iBAAiB,EACjB,UAAU,EACV,UAAU,EACV,YAAY,EACZ,cAAc,EACd,cAAc,EACd,qBAAqB,CACtB,CAAC;IAEF,OAAO;QACL,WAAW;QACX,UAAU;KACX,CAAC;AACJ,CAAC"}
|
package/dist/src/index.d.ts
CHANGED
|
@@ -20,8 +20,9 @@
|
|
|
20
20
|
* const { syncManager, syncEvents } = await createSyncModule(kv, { transactionSource: db });
|
|
21
21
|
*/
|
|
22
22
|
export { type HLC, HLCManager, compareHLC, hlcEquals, maxHLC, createHLC, deterministicTxnId, MAX_OPSEQ, serializeHLC, deserializeHLC, type SerializedHLC, hlcToJson, hlcFromJson, type SiteId, generateSiteId, siteIdToBase64, siteIdFromBase64, toBase64Url, fromBase64Url, siteIdEquals, type SiteIdentity, serializeSiteIdentity, deserializeSiteIdentity, SITE_ID_KEY, } from './clock/index.js';
|
|
23
|
-
export { type ColumnChange, type RowDeletion, type Change, type SchemaMigrationType, type SchemaMigration, type ChangeSet, type ApplyRejection, type ApplyResult, type ColumnVersionEntry, type TableSnapshot, type Snapshot, type PeerSyncState, type SnapshotChunkType, type SnapshotHeaderChunk, type SnapshotTableStartChunk, type SnapshotColumnVersionsChunk, type SnapshotTableEndChunk, type SnapshotSchemaMigrationChunk, type SnapshotFooterChunk, type SnapshotChunk, type SnapshotProgress, type ApplyToStoreOptions, type DataChangeToApply, type SchemaChangeToApply, type ApplyToStoreResult, type ApplyToStoreCallback, type ConflictContext, type ConflictResolution, type ConflictResolver, type UnknownTableDisposition, type BasisEvictionConfig, type DropLocalTableCallback, type SyncConfig, DEFAULT_SYNC_CONFIG, } from './sync/protocol.js';
|
|
24
|
-
export { PROTOCOL_VERSION, bytesToBase64, base64ToBytes, serializeHLCForTransport, deserializeHLCFromTransport, type SerializedChangeSet, type SerializedChange, type SerializedSchemaMigration, type SerializedSnapshotChunk, type SerializedSnapshotHeaderChunk, type SerializedSnapshotTableStartChunk, type SerializedSnapshotColumnVersionsChunk, type SerializedSnapshotTombstoneChunk, type SerializedSnapshotTableEndChunk, type SerializedSnapshotSchemaMigrationChunk, type SerializedSnapshotFooterChunk, serializeChangeSet, deserializeChangeSet, serializeSnapshotChunk, deserializeSnapshotChunk, type ClientMessage, type HandshakeMessage, type GetChangesMessage, type ApplyChangesMessage, type GetSnapshotMessage, type ResumeSnapshotMessage, type PingMessage, type ServerMessage, type HandshakeAckMessage, type ChangesMessage, type PushChangesMessage, type ApplyResultMessage, type SnapshotChunkMessage, type SnapshotCompleteMessage, type RequestChangesMessage, type ErrorMessage, type PongMessage, } from './sync/wire.js';
|
|
23
|
+
export { type ColumnChange, type RowDeletion, type Change, type SchemaMigrationType, type SchemaMigration, type ChangeSet, type ApplyRejection, type ApplyResult, type ColumnVersionEntry, type TableSnapshot, type Snapshot, type SnapshotTombstone, type PeerSyncState, SNAPSHOT_WIRE_FORMAT_VERSION, type SnapshotChunkType, type SnapshotHeaderChunk, type SnapshotTableStartChunk, type SnapshotColumnVersionsChunk, type SnapshotTableEndChunk, type SnapshotSchemaMigrationChunk, type SnapshotFooterChunk, type SnapshotChunk, type SnapshotProgress, type ApplyToStoreOptions, type DataChangeToApply, type SchemaChangeToApply, type ApplyToStoreResult, type ApplyToStoreCallback, type ConflictContext, type ConflictResolution, type ConflictResolver, type UnknownTableDisposition, type BasisEvictionConfig, type DropLocalTableCallback, type SyncConfig, DEFAULT_SYNC_CONFIG, } from './sync/protocol.js';
|
|
24
|
+
export { PROTOCOL_VERSION, bytesToBase64, base64ToBytes, serializeHLCForTransport, deserializeHLCFromTransport, type SerializedChangeSet, type SerializedChange, type SerializedSchemaMigration, type SerializedSnapshotChunk, type SerializedSnapshotHeaderChunk, type SerializedSnapshotTableStartChunk, type SerializedSnapshotColumnVersionsChunk, type SerializedSnapshotTombstoneChunk, type SerializedSnapshotTableEndChunk, type SerializedSnapshotSchemaMigrationChunk, type SerializedSnapshotFooterChunk, type SerializedSnapshotCheckpoint, serializeChangeSet, deserializeChangeSet, serializeSnapshotChunk, deserializeSnapshotChunk, serializeSnapshotCheckpoint, deserializeSnapshotCheckpoint, type ClientMessage, type HandshakeMessage, type GetChangesMessage, type ApplyChangesMessage, type GetSnapshotMessage, type ResumeSnapshotMessage, type PingMessage, type ServerMessage, type HandshakeAckMessage, type ChangesMessage, type PushChangesMessage, type ApplyResultMessage, type SnapshotChunkMessage, type SnapshotCompleteMessage, type RequestChangesMessage, type ErrorMessage, type PongMessage, } from './sync/wire.js';
|
|
25
|
+
export { SYNC_MAINTENANCE_INTERVAL_MS, type SyncMaintenanceTarget, type MaintenanceLogger, runSyncMaintenancePass, createSyncMaintenanceTicker, } from './sync/maintenance.js';
|
|
25
26
|
export { lwwResolver, localWinsResolver, remoteWinsResolver, } from './sync/conflict-resolvers.js';
|
|
26
27
|
export { type SyncManager, type SnapshotCheckpoint } from './sync/manager.js';
|
|
27
28
|
export { SyncManagerImpl } from './sync/sync-manager-impl.js';
|
|
@@ -29,5 +30,5 @@ export { createStoreAdapter, type SyncStoreAdapterOptions } from './sync/store-a
|
|
|
29
30
|
export { registerBasisLifecycleTvf } from './sql/basis-lifecycle-tvf.js';
|
|
30
31
|
export { createSyncModule, type CreateSyncModuleResult, type CreateSyncModuleOptions, type GetTableSchemaCallback, type TransactionCommitSource, } from './create-sync-module.js';
|
|
31
32
|
export { type RemoteChangeEvent, type LocalChangeEvent, type ConflictEvent, type UnknownTableEvent, type AssertionViolationEvent, type BasisTableLifecycleEvent, type BasisTableEvictedEvent, type HeldChangesDrainedEvent, type SyncState, type Unsubscribe, type SyncEventEmitter, SyncEventEmitterImpl, } from './sync/events.js';
|
|
32
|
-
export { SYNC_KEY_PREFIX, buildColumnVersionKey, buildTombstoneKey, buildTransactionKey, buildPeerStateKey, buildPeerSentStateKey, buildSchemaMigrationKey, buildColumnVersionScanBounds, buildTableColumnVersionScanBounds, buildTombstoneScanBounds, buildSchemaMigrationScanBounds,
|
|
33
|
+
export { SYNC_KEY_PREFIX, buildColumnVersionKey, buildTombstoneKey, buildTransactionKey, buildPeerStateKey, buildPeerSentStateKey, buildSchemaMigrationKey, buildColumnVersionScanBounds, buildTableColumnVersionScanBounds, buildTombstoneScanBounds, buildSchemaMigrationScanBounds, type PkKeying, type PkKeyingResolver, encodePkIdentity, encodeRawPkIdentity, resolvePkKeying, createPkKeyingResolver, makePkIdentityEncoder, SYNC_METADATA_FORMAT_VERSION, type ColumnVersion, type ColumnVersionData, ColumnVersionStore, serializeColumnVersion, deserializeColumnVersion, encodeSqlValue, decodeSqlValue, type Tombstone, TombstoneStore, serializeTombstone, deserializeTombstone, type QuarantineEntry, QuarantineStore, serializeQuarantineEntry, deserializeQuarantineEntry, buildQuarantineKey, buildQuarantineScanBounds, type BasisLifecycleState, type BasisTableLifecycleRecord, type EvictPolicy, BasisLifecycleStore, classifyBasisLifecycle, parseEvictPolicyTag, effectiveEvictHorizonMs, quietSince, isEvictable, serializeBasisLifecycleRecord, deserializeBasisLifecycleRecord, buildBasisLifecycleKey, buildAllBasisLifecycleScanBounds, type PeerState, PeerStateStore, serializePeerState, deserializePeerState, type SchemaVersion, type SchemaVersionType, type SchemaChangeOperation, SchemaVersionStore, buildSchemaVersionKey, buildSchemaVersionScanBounds, buildAllSchemaVersionsScanBounds, serializeSchemaVersion, deserializeSchemaVersion, parseSchemaVersionKey, getDestructiveness, getOperationDestructiveness, shouldApplySchemaChangeByOperation, } from './metadata/index.js';
|
|
33
34
|
//# sourceMappingURL=index.d.ts.map
|
package/dist/src/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;GAoBG;AAGH,OAAO,EAEL,KAAK,GAAG,EACR,UAAU,EACV,UAAU,EACV,SAAS,EACT,MAAM,EACN,SAAS,EACT,kBAAkB,EAClB,SAAS,EACT,YAAY,EACZ,cAAc,EAEd,KAAK,aAAa,EAClB,SAAS,EACT,WAAW,EAEX,KAAK,MAAM,EACX,cAAc,EAEd,cAAc,EACd,gBAAgB,EAChB,WAAW,EACX,aAAa,EACb,YAAY,EACZ,KAAK,YAAY,EACjB,qBAAqB,EACrB,uBAAuB,EACvB,WAAW,GACZ,MAAM,kBAAkB,CAAC;AAG1B,OAAO,EAEL,KAAK,YAAY,EACjB,KAAK,WAAW,EAChB,KAAK,MAAM,EAEX,KAAK,mBAAmB,EACxB,KAAK,eAAe,EAEpB,KAAK,SAAS,EAEd,KAAK,cAAc,EACnB,KAAK,WAAW,EAChB,KAAK,kBAAkB,EACvB,KAAK,aAAa,EAClB,KAAK,QAAQ,EACb,KAAK,aAAa,EAElB,KAAK,iBAAiB,EACtB,KAAK,mBAAmB,EACxB,KAAK,uBAAuB,EAC5B,KAAK,2BAA2B,EAChC,KAAK,qBAAqB,EAC1B,KAAK,4BAA4B,EACjC,KAAK,mBAAmB,EACxB,KAAK,aAAa,EAClB,KAAK,gBAAgB,EAErB,KAAK,mBAAmB,EACxB,KAAK,iBAAiB,EACtB,KAAK,mBAAmB,EACxB,KAAK,kBAAkB,EACvB,KAAK,oBAAoB,EAEzB,KAAK,eAAe,EACpB,KAAK,kBAAkB,EACvB,KAAK,gBAAgB,EAErB,KAAK,uBAAuB,EAE5B,KAAK,mBAAmB,EACxB,KAAK,sBAAsB,EAE3B,KAAK,UAAU,EACf,mBAAmB,GACpB,MAAM,oBAAoB,CAAC;AAK5B,OAAO,EAEL,gBAAgB,EAEhB,aAAa,EACb,aAAa,EACb,wBAAwB,EACxB,2BAA2B,EAE3B,KAAK,mBAAmB,EACxB,KAAK,gBAAgB,EACrB,KAAK,yBAAyB,EAC9B,KAAK,uBAAuB,EAC5B,KAAK,6BAA6B,EAClC,KAAK,iCAAiC,EACtC,KAAK,qCAAqC,EAC1C,KAAK,gCAAgC,EACrC,KAAK,+BAA+B,EACpC,KAAK,sCAAsC,EAC3C,KAAK,6BAA6B,
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;GAoBG;AAGH,OAAO,EAEL,KAAK,GAAG,EACR,UAAU,EACV,UAAU,EACV,SAAS,EACT,MAAM,EACN,SAAS,EACT,kBAAkB,EAClB,SAAS,EACT,YAAY,EACZ,cAAc,EAEd,KAAK,aAAa,EAClB,SAAS,EACT,WAAW,EAEX,KAAK,MAAM,EACX,cAAc,EAEd,cAAc,EACd,gBAAgB,EAChB,WAAW,EACX,aAAa,EACb,YAAY,EACZ,KAAK,YAAY,EACjB,qBAAqB,EACrB,uBAAuB,EACvB,WAAW,GACZ,MAAM,kBAAkB,CAAC;AAG1B,OAAO,EAEL,KAAK,YAAY,EACjB,KAAK,WAAW,EAChB,KAAK,MAAM,EAEX,KAAK,mBAAmB,EACxB,KAAK,eAAe,EAEpB,KAAK,SAAS,EAEd,KAAK,cAAc,EACnB,KAAK,WAAW,EAChB,KAAK,kBAAkB,EACvB,KAAK,aAAa,EAClB,KAAK,QAAQ,EACb,KAAK,iBAAiB,EACtB,KAAK,aAAa,EAElB,4BAA4B,EAE5B,KAAK,iBAAiB,EACtB,KAAK,mBAAmB,EACxB,KAAK,uBAAuB,EAC5B,KAAK,2BAA2B,EAChC,KAAK,qBAAqB,EAC1B,KAAK,4BAA4B,EACjC,KAAK,mBAAmB,EACxB,KAAK,aAAa,EAClB,KAAK,gBAAgB,EAErB,KAAK,mBAAmB,EACxB,KAAK,iBAAiB,EACtB,KAAK,mBAAmB,EACxB,KAAK,kBAAkB,EACvB,KAAK,oBAAoB,EAEzB,KAAK,eAAe,EACpB,KAAK,kBAAkB,EACvB,KAAK,gBAAgB,EAErB,KAAK,uBAAuB,EAE5B,KAAK,mBAAmB,EACxB,KAAK,sBAAsB,EAE3B,KAAK,UAAU,EACf,mBAAmB,GACpB,MAAM,oBAAoB,CAAC;AAK5B,OAAO,EAEL,gBAAgB,EAEhB,aAAa,EACb,aAAa,EACb,wBAAwB,EACxB,2BAA2B,EAE3B,KAAK,mBAAmB,EACxB,KAAK,gBAAgB,EACrB,KAAK,yBAAyB,EAC9B,KAAK,uBAAuB,EAC5B,KAAK,6BAA6B,EAClC,KAAK,iCAAiC,EACtC,KAAK,qCAAqC,EAC1C,KAAK,gCAAgC,EACrC,KAAK,+BAA+B,EACpC,KAAK,sCAAsC,EAC3C,KAAK,6BAA6B,EAClC,KAAK,4BAA4B,EAEjC,kBAAkB,EAClB,oBAAoB,EACpB,sBAAsB,EACtB,wBAAwB,EACxB,2BAA2B,EAC3B,6BAA6B,EAE7B,KAAK,aAAa,EAClB,KAAK,gBAAgB,EACrB,KAAK,iBAAiB,EACtB,KAAK,mBAAmB,EACxB,KAAK,kBAAkB,EACvB,KAAK,qBAAqB,EAC1B,KAAK,WAAW,EAChB,KAAK,aAAa,EAClB,KAAK,mBAAmB,EACxB,KAAK,cAAc,EACnB,KAAK,kBAAkB,EACvB,KAAK,kBAAkB,EACvB,KAAK,oBAAoB,EACzB,KAAK,uBAAuB,EAC5B,KAAK,qBAAqB,EAC1B,KAAK,YAAY,EACjB,KAAK,WAAW,GACjB,MAAM,gBAAgB,CAAC;AAKxB,OAAO,EACL,4BAA4B,EAC5B,KAAK,qBAAqB,EAC1B,KAAK,iBAAiB,EACtB,sBAAsB,EACtB,2BAA2B,GAC5B,MAAM,uBAAuB,CAAC;AAG/B,OAAO,EACL,WAAW,EACX,iBAAiB,EACjB,kBAAkB,GACnB,MAAM,8BAA8B,CAAC;AAGtC,OAAO,EAAE,KAAK,WAAW,EAAE,KAAK,kBAAkB,EAAE,MAAM,mBAAmB,CAAC;AAC9E,OAAO,EAAE,eAAe,EAAE,MAAM,6BAA6B,CAAC;AAG9D,OAAO,EAAE,kBAAkB,EAAE,KAAK,uBAAuB,EAAE,MAAM,yBAAyB,CAAC;AAG3F,OAAO,EAAE,yBAAyB,EAAE,MAAM,8BAA8B,CAAC;AAGzE,OAAO,EACL,gBAAgB,EAChB,KAAK,sBAAsB,EAC3B,KAAK,uBAAuB,EAC5B,KAAK,sBAAsB,EAC3B,KAAK,uBAAuB,GAC7B,MAAM,yBAAyB,CAAC;AAGjC,OAAO,EACL,KAAK,iBAAiB,EACtB,KAAK,gBAAgB,EACrB,KAAK,aAAa,EAClB,KAAK,iBAAiB,EACtB,KAAK,uBAAuB,EAC5B,KAAK,wBAAwB,EAC7B,KAAK,sBAAsB,EAC3B,KAAK,uBAAuB,EAC5B,KAAK,SAAS,EACd,KAAK,WAAW,EAChB,KAAK,gBAAgB,EACrB,oBAAoB,GACrB,MAAM,kBAAkB,CAAC;AAG1B,OAAO,EAEL,eAAe,EACf,qBAAqB,EACrB,iBAAiB,EACjB,mBAAmB,EACnB,iBAAiB,EACjB,qBAAqB,EACrB,uBAAuB,EACvB,4BAA4B,EAC5B,iCAAiC,EACjC,wBAAwB,EACxB,8BAA8B,EAE9B,KAAK,QAAQ,EACb,KAAK,gBAAgB,EACrB,gBAAgB,EAChB,mBAAmB,EACnB,eAAe,EACf,sBAAsB,EACtB,qBAAqB,EACrB,4BAA4B,EAE5B,KAAK,aAAa,EAClB,KAAK,iBAAiB,EACtB,kBAAkB,EAClB,sBAAsB,EACtB,wBAAwB,EAExB,cAAc,EACd,cAAc,EAEd,KAAK,SAAS,EACd,cAAc,EACd,kBAAkB,EAClB,oBAAoB,EAEpB,KAAK,eAAe,EACpB,eAAe,EACf,wBAAwB,EACxB,0BAA0B,EAC1B,kBAAkB,EAClB,yBAAyB,EAEzB,KAAK,mBAAmB,EACxB,KAAK,yBAAyB,EAC9B,KAAK,WAAW,EAChB,mBAAmB,EACnB,sBAAsB,EACtB,mBAAmB,EACnB,uBAAuB,EACvB,UAAU,EACV,WAAW,EACX,6BAA6B,EAC7B,+BAA+B,EAC/B,sBAAsB,EACtB,gCAAgC,EAEhC,KAAK,SAAS,EACd,cAAc,EACd,kBAAkB,EAClB,oBAAoB,EAEpB,KAAK,aAAa,EAClB,KAAK,iBAAiB,EACtB,KAAK,qBAAqB,EAC1B,kBAAkB,EAClB,qBAAqB,EACrB,4BAA4B,EAC5B,gCAAgC,EAChC,sBAAsB,EACtB,wBAAwB,EACxB,qBAAqB,EAErB,kBAAkB,EAClB,2BAA2B,EAC3B,kCAAkC,GACnC,MAAM,qBAAqB,CAAC"}
|
package/dist/src/index.js
CHANGED
|
@@ -24,7 +24,9 @@ export { HLCManager, compareHLC, hlcEquals, maxHLC, createHLC, deterministicTxnI
|
|
|
24
24
|
// Base64url encoding
|
|
25
25
|
siteIdToBase64, siteIdFromBase64, toBase64Url, fromBase64Url, siteIdEquals, serializeSiteIdentity, deserializeSiteIdentity, SITE_ID_KEY, } from './clock/index.js';
|
|
26
26
|
// Sync protocol types
|
|
27
|
-
export {
|
|
27
|
+
export {
|
|
28
|
+
// Snapshot wire-format gate (both apply paths refuse a mismatched stamp)
|
|
29
|
+
SNAPSHOT_WIRE_FORMAT_VERSION, DEFAULT_SYNC_CONFIG, } from './sync/protocol.js';
|
|
28
30
|
// Wire protocol - shared transport/JSON layer (base64 helpers, Serialized* types,
|
|
29
31
|
// codec fns, message envelopes, PROTOCOL_VERSION). Single source of truth for the
|
|
30
32
|
// format the sync client and coordinator exchange.
|
|
@@ -34,7 +36,11 @@ PROTOCOL_VERSION,
|
|
|
34
36
|
// Base64 + HLC transport helpers
|
|
35
37
|
bytesToBase64, base64ToBytes, serializeHLCForTransport, deserializeHLCFromTransport,
|
|
36
38
|
// Codec functions
|
|
37
|
-
serializeChangeSet, deserializeChangeSet, serializeSnapshotChunk, deserializeSnapshotChunk, } from './sync/wire.js';
|
|
39
|
+
serializeChangeSet, deserializeChangeSet, serializeSnapshotChunk, deserializeSnapshotChunk, serializeSnapshotCheckpoint, deserializeSnapshotCheckpoint, } from './sync/wire.js';
|
|
40
|
+
// Host-driven maintenance pass. The library still schedules nothing — this is
|
|
41
|
+
// only the shape of one pass (ordering, error isolation, single-flight); the
|
|
42
|
+
// host arms the timer around it.
|
|
43
|
+
export { SYNC_MAINTENANCE_INTERVAL_MS, runSyncMaintenancePass, createSyncMaintenanceTicker, } from './sync/maintenance.js';
|
|
38
44
|
// Built-in conflict resolvers
|
|
39
45
|
export { lwwResolver, localWinsResolver, remoteWinsResolver, } from './sync/conflict-resolvers.js';
|
|
40
46
|
export { SyncManagerImpl } from './sync/sync-manager-impl.js';
|
|
@@ -49,7 +55,7 @@ export { SyncEventEmitterImpl, } from './sync/events.js';
|
|
|
49
55
|
// Metadata storage
|
|
50
56
|
export {
|
|
51
57
|
// Key builders
|
|
52
|
-
SYNC_KEY_PREFIX, buildColumnVersionKey, buildTombstoneKey, buildTransactionKey, buildPeerStateKey, buildPeerSentStateKey, buildSchemaMigrationKey, buildColumnVersionScanBounds, buildTableColumnVersionScanBounds, buildTombstoneScanBounds, buildSchemaMigrationScanBounds,
|
|
58
|
+
SYNC_KEY_PREFIX, buildColumnVersionKey, buildTombstoneKey, buildTransactionKey, buildPeerStateKey, buildPeerSentStateKey, buildSchemaMigrationKey, buildColumnVersionScanBounds, buildTableColumnVersionScanBounds, buildTombstoneScanBounds, buildSchemaMigrationScanBounds, encodePkIdentity, encodeRawPkIdentity, resolvePkKeying, createPkKeyingResolver, makePkIdentityEncoder, SYNC_METADATA_FORMAT_VERSION, ColumnVersionStore, serializeColumnVersion, deserializeColumnVersion,
|
|
53
59
|
// SqlValue JSON encoding (for Uint8Array/bigint in JSON transport)
|
|
54
60
|
encodeSqlValue, decodeSqlValue, TombstoneStore, serializeTombstone, deserializeTombstone, QuarantineStore, serializeQuarantineEntry, deserializeQuarantineEntry, buildQuarantineKey, buildQuarantineScanBounds, BasisLifecycleStore, classifyBasisLifecycle, parseEvictPolicyTag, effectiveEvictHorizonMs, quietSince, isEvictable, serializeBasisLifecycleRecord, deserializeBasisLifecycleRecord, buildBasisLifecycleKey, buildAllBasisLifecycleScanBounds, PeerStateStore, serializePeerState, deserializePeerState, SchemaVersionStore, buildSchemaVersionKey, buildSchemaVersionScanBounds, buildAllSchemaVersionsScanBounds, serializeSchemaVersion, deserializeSchemaVersion, parseSchemaVersionKey,
|
|
55
61
|
// Most destructive wins
|
package/dist/src/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;GAoBG;AAEH,eAAe;AACf,OAAO,EAGL,UAAU,EACV,UAAU,EACV,SAAS,EACT,MAAM,EACN,SAAS,EACT,kBAAkB,EAClB,SAAS,EACT,YAAY,EACZ,cAAc,EAGd,SAAS,EACT,WAAW,EAGX,cAAc;AACd,qBAAqB;AACrB,cAAc,EACd,gBAAgB,EAChB,WAAW,EACX,aAAa,EACb,YAAY,EAEZ,qBAAqB,EACrB,uBAAuB,EACvB,WAAW,GACZ,MAAM,kBAAkB,CAAC;AAE1B,sBAAsB;AACtB,OAAO,
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;GAoBG;AAEH,eAAe;AACf,OAAO,EAGL,UAAU,EACV,UAAU,EACV,SAAS,EACT,MAAM,EACN,SAAS,EACT,kBAAkB,EAClB,SAAS,EACT,YAAY,EACZ,cAAc,EAGd,SAAS,EACT,WAAW,EAGX,cAAc;AACd,qBAAqB;AACrB,cAAc,EACd,gBAAgB,EAChB,WAAW,EACX,aAAa,EACb,YAAY,EAEZ,qBAAqB,EACrB,uBAAuB,EACvB,WAAW,GACZ,MAAM,kBAAkB,CAAC;AAE1B,sBAAsB;AACtB,OAAO;AAkBL,yEAAyE;AACzE,4BAA4B,EA4B5B,mBAAmB,GACpB,MAAM,oBAAoB,CAAC;AAE5B,kFAAkF;AAClF,kFAAkF;AAClF,mDAAmD;AACnD,OAAO;AACL,UAAU;AACV,gBAAgB;AAChB,iCAAiC;AACjC,aAAa,EACb,aAAa,EACb,wBAAwB,EACxB,2BAA2B;AAc3B,kBAAkB;AAClB,kBAAkB,EAClB,oBAAoB,EACpB,sBAAsB,EACtB,wBAAwB,EACxB,2BAA2B,EAC3B,6BAA6B,GAmB9B,MAAM,gBAAgB,CAAC;AAExB,8EAA8E;AAC9E,6EAA6E;AAC7E,iCAAiC;AACjC,OAAO,EACL,4BAA4B,EAG5B,sBAAsB,EACtB,2BAA2B,GAC5B,MAAM,uBAAuB,CAAC;AAE/B,8BAA8B;AAC9B,OAAO,EACL,WAAW,EACX,iBAAiB,EACjB,kBAAkB,GACnB,MAAM,8BAA8B,CAAC;AAItC,OAAO,EAAE,eAAe,EAAE,MAAM,6BAA6B,CAAC;AAE9D,4CAA4C;AAC5C,OAAO,EAAE,kBAAkB,EAAgC,MAAM,yBAAyB,CAAC;AAE3F,uEAAuE;AACvE,OAAO,EAAE,yBAAyB,EAAE,MAAM,8BAA8B,CAAC;AAEzE,mBAAmB;AACnB,OAAO,EACL,gBAAgB,GAKjB,MAAM,yBAAyB,CAAC;AAEjC,kBAAkB;AAClB,OAAO,EAYL,oBAAoB,GACrB,MAAM,kBAAkB,CAAC;AAE1B,mBAAmB;AACnB,OAAO;AACL,eAAe;AACf,eAAe,EACf,qBAAqB,EACrB,iBAAiB,EACjB,mBAAmB,EACnB,iBAAiB,EACjB,qBAAqB,EACrB,uBAAuB,EACvB,4BAA4B,EAC5B,iCAAiC,EACjC,wBAAwB,EACxB,8BAA8B,EAI9B,gBAAgB,EAChB,mBAAmB,EACnB,eAAe,EACf,sBAAsB,EACtB,qBAAqB,EACrB,4BAA4B,EAI5B,kBAAkB,EAClB,sBAAsB,EACtB,wBAAwB;AACxB,mEAAmE;AACnE,cAAc,EACd,cAAc,EAGd,cAAc,EACd,kBAAkB,EAClB,oBAAoB,EAGpB,eAAe,EACf,wBAAwB,EACxB,0BAA0B,EAC1B,kBAAkB,EAClB,yBAAyB,EAKzB,mBAAmB,EACnB,sBAAsB,EACtB,mBAAmB,EACnB,uBAAuB,EACvB,UAAU,EACV,WAAW,EACX,6BAA6B,EAC7B,+BAA+B,EAC/B,sBAAsB,EACtB,gCAAgC,EAGhC,cAAc,EACd,kBAAkB,EAClB,oBAAoB,EAKpB,kBAAkB,EAClB,qBAAqB,EACrB,4BAA4B,EAC5B,gCAAgC,EAChC,sBAAsB,EACtB,wBAAwB,EACxB,qBAAqB;AACrB,wBAAwB;AACxB,kBAAkB,EAClB,2BAA2B,EAC3B,kCAAkC,GACnC,MAAM,qBAAqB,CAAC"}
|
|
@@ -8,15 +8,21 @@ import type { KVStore, WriteBatch } from '@quereus/store';
|
|
|
8
8
|
import type { SqlValue } from '@quereus/quereus';
|
|
9
9
|
import { type HLC } from '../clock/hlc.js';
|
|
10
10
|
import { type ChangeLogEntryType } from './keys.js';
|
|
11
|
+
import type { PkKeyingResolver } from './pk-identity.js';
|
|
11
12
|
/**
|
|
12
13
|
* Change log entry stored in KV.
|
|
14
|
+
*
|
|
15
|
+
* Carries the pk IDENTITY (the derived, lossy string the row's records are
|
|
16
|
+
* filed under), NOT the raw pk — the entry's value is empty and the identity
|
|
17
|
+
* cannot be decoded back. Consumers resolve the entry to its live `cv:`/`tb:`
|
|
18
|
+
* record (which holds the raw pk in its value) via the `*ByIdentity` getters.
|
|
13
19
|
*/
|
|
14
20
|
export interface ChangeLogEntry {
|
|
15
21
|
readonly hlc: HLC;
|
|
16
22
|
readonly entryType: ChangeLogEntryType;
|
|
17
23
|
readonly schema: string;
|
|
18
24
|
readonly table: string;
|
|
19
|
-
readonly
|
|
25
|
+
readonly identity: string;
|
|
20
26
|
readonly column?: string;
|
|
21
27
|
}
|
|
22
28
|
/**
|
|
@@ -25,16 +31,25 @@ export interface ChangeLogEntry {
|
|
|
25
31
|
* Each mutation (column update or row deletion) creates an entry in the
|
|
26
32
|
* change log, keyed by HLC. This allows efficient range scans to find
|
|
27
33
|
* all changes since a given timestamp.
|
|
34
|
+
*
|
|
35
|
+
* Every pk-taking method derives the key's pk IDENTITY through the per-table
|
|
36
|
+
* {@link PkKeyingResolver} passed at construction; `deleteEntryByIdentityBatch`
|
|
37
|
+
* serves callers that already hold a parsed identity.
|
|
28
38
|
*/
|
|
29
39
|
export declare class ChangeLogStore {
|
|
30
40
|
private readonly kv;
|
|
31
|
-
|
|
41
|
+
private readonly keying;
|
|
42
|
+
constructor(kv: KVStore, keying: PkKeyingResolver);
|
|
43
|
+
private identity;
|
|
32
44
|
/**
|
|
33
45
|
* Record a column change in the change log.
|
|
34
46
|
*/
|
|
35
47
|
recordColumnChange(hlc: HLC, schema: string, table: string, pk: SqlValue[], column: string): Promise<void>;
|
|
36
48
|
/**
|
|
37
|
-
* Record a column change in a batch.
|
|
49
|
+
* Record a column change in a batch. The key's pk IDENTITY is always derived
|
|
50
|
+
* locally — no caller-supplied-identity WRITE variant exists (see
|
|
51
|
+
* `ColumnVersionStore.setColumnVersionBatch`); `deleteEntryByIdentityBatch`
|
|
52
|
+
* below remains for DELETE-side callers holding a parsed key.
|
|
38
53
|
*/
|
|
39
54
|
recordColumnChangeBatch(batch: WriteBatch, hlc: HLC, schema: string, table: string, pk: SqlValue[], column: string): void;
|
|
40
55
|
/**
|
|
@@ -57,11 +72,32 @@ export declare class ChangeLogStore {
|
|
|
57
72
|
/**
|
|
58
73
|
* Delete change log entries up to a given HLC.
|
|
59
74
|
* Used for pruning old entries after they've been synced to all peers.
|
|
75
|
+
*
|
|
76
|
+
* NOTE: intentionally unwired — no production caller invokes this, and that is
|
|
77
|
+
* deliberate, not an oversight. The change log no longer leaks (entries are
|
|
78
|
+
* deleted with the `cv:`/`tb:` record they point at, so its size tracks live
|
|
79
|
+
* data), so horizon pruning is now a behaviour change rather than a leak fix:
|
|
80
|
+
* it drops index entries for cells that are still LIVE, which silently breaks
|
|
81
|
+
* any peer whose `sinceHLC` predates the horizon — their delta scan would start
|
|
82
|
+
* past the pruned range and never learn about those cells.
|
|
83
|
+
*
|
|
84
|
+
* Precondition before wiring it up: the server must first REFUSE delta sync for
|
|
85
|
+
* a too-old `sinceHLC` and fall back to a snapshot. `SyncManager.canDeltaSync`
|
|
86
|
+
* is that check, but no production caller invokes it either — the coordinator's
|
|
87
|
+
* `get_changes` handler passes `sinceHLC` straight through
|
|
88
|
+
* (packages/sync-coordinator/src/server/websocket.ts). Both halves must land
|
|
89
|
+
* together. Tracked as `feat-sync-changelog-horizon-pruning`.
|
|
60
90
|
*/
|
|
61
91
|
pruneEntriesBefore(beforeHLC: HLC): Promise<number>;
|
|
62
92
|
/**
|
|
63
93
|
* Delete a specific change log entry (used during snapshot application).
|
|
64
94
|
*/
|
|
65
95
|
deleteEntryBatch(batch: WriteBatch, hlc: HLC, entryType: ChangeLogEntryType, schema: string, table: string, pk: SqlValue[], column?: string): void;
|
|
96
|
+
/**
|
|
97
|
+
* Delete a specific change log entry by pk IDENTITY — for callers that hold a
|
|
98
|
+
* parsed key rather than a raw pk (e.g. the tombstone prune sweep, whose
|
|
99
|
+
* table may no longer have a resolvable schema).
|
|
100
|
+
*/
|
|
101
|
+
deleteEntryByIdentityBatch(batch: WriteBatch, hlc: HLC, entryType: ChangeLogEntryType, schema: string, table: string, identity: string, column?: string): void;
|
|
66
102
|
}
|
|
67
103
|
//# sourceMappingURL=change-log.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"change-log.d.ts","sourceRoot":"","sources":["../../../src/metadata/change-log.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAC1D,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AACjD,OAAO,EAAE,KAAK,GAAG,EAAc,MAAM,iBAAiB,CAAC;AACvD,OAAO,
|
|
1
|
+
{"version":3,"file":"change-log.d.ts","sourceRoot":"","sources":["../../../src/metadata/change-log.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAE,OAAO,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAC1D,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AACjD,OAAO,EAAE,KAAK,GAAG,EAAc,MAAM,iBAAiB,CAAC;AACvD,OAAO,EAML,KAAK,kBAAkB,EACxB,MAAM,WAAW,CAAC;AACnB,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AAEzD;;;;;;;GAOG;AACH,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,GAAG,EAAE,GAAG,CAAC;IAClB,QAAQ,CAAC,SAAS,EAAE,kBAAkB,CAAC;IACvC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED;;;;;;;;;;GAUG;AACH,qBAAa,cAAc;IAEvB,OAAO,CAAC,QAAQ,CAAC,EAAE;IACnB,OAAO,CAAC,QAAQ,CAAC,MAAM;gBADN,EAAE,EAAE,OAAO,EACX,MAAM,EAAE,gBAAgB;IAG3C,OAAO,CAAC,QAAQ;IAIhB;;OAEG;IACG,kBAAkB,CACtB,GAAG,EAAE,GAAG,EACR,MAAM,EAAE,MAAM,EACd,KAAK,EAAE,MAAM,EACb,EAAE,EAAE,QAAQ,EAAE,EACd,MAAM,EAAE,MAAM,GACb,OAAO,CAAC,IAAI,CAAC;IAMhB;;;;;OAKG;IACH,uBAAuB,CACrB,KAAK,EAAE,UAAU,EACjB,GAAG,EAAE,GAAG,EACR,MAAM,EAAE,MAAM,EACd,KAAK,EAAE,MAAM,EACb,EAAE,EAAE,QAAQ,EAAE,EACd,MAAM,EAAE,MAAM,GACb,IAAI;IAKP;;OAEG;IACG,cAAc,CAClB,GAAG,EAAE,GAAG,EACR,MAAM,EAAE,MAAM,EACd,KAAK,EAAE,MAAM,EACb,EAAE,EAAE,QAAQ,EAAE,GACb,OAAO,CAAC,IAAI,CAAC;IAKhB;;OAEG;IACH,mBAAmB,CACjB,KAAK,EAAE,UAAU,EACjB,GAAG,EAAE,GAAG,EACR,MAAM,EAAE,MAAM,EACd,KAAK,EAAE,MAAM,EACb,EAAE,EAAE,QAAQ,EAAE,GACb,IAAI;IAKP;;;OAGG;IACI,eAAe,CAAC,QAAQ,EAAE,GAAG,GAAG,aAAa,CAAC,cAAc,CAAC;IAUpE;;OAEG;IACI,aAAa,IAAI,aAAa,CAAC,cAAc,CAAC;IAUrD;;;;;;;;;;;;;;;;;;OAkBG;IACG,kBAAkB,CAAC,SAAS,EAAE,GAAG,GAAG,OAAO,CAAC,MAAM,CAAC;IAuBzD;;OAEG;IACH,gBAAgB,CACd,KAAK,EAAE,UAAU,EACjB,GAAG,EAAE,GAAG,EACR,SAAS,EAAE,kBAAkB,EAC7B,MAAM,EAAE,MAAM,EACd,KAAK,EAAE,MAAM,EACb,EAAE,EAAE,QAAQ,EAAE,EACd,MAAM,CAAC,EAAE,MAAM,GACd,IAAI;IAIP;;;;OAIG;IACH,0BAA0B,CACxB,KAAK,EAAE,UAAU,EACjB,GAAG,EAAE,GAAG,EACR,SAAS,EAAE,kBAAkB,EAC7B,MAAM,EAAE,MAAM,EACd,KAAK,EAAE,MAAM,EACb,QAAQ,EAAE,MAAM,EAChB,MAAM,CAAC,EAAE,MAAM,GACd,IAAI;CAIR"}
|
|
@@ -5,46 +5,58 @@
|
|
|
5
5
|
* enabling efficient getChangesSince() queries without scanning all data.
|
|
6
6
|
*/
|
|
7
7
|
import { compareHLC } from '../clock/hlc.js';
|
|
8
|
-
import { buildChangeLogKey, buildChangeLogScanBoundsAfter, buildAllChangeLogScanBounds, parseChangeLogKey, } from './keys.js';
|
|
8
|
+
import { buildChangeLogKey, buildChangeLogScanBoundsAfter, buildAllChangeLogScanBounds, parseChangeLogKey, encodePkIdentity, } from './keys.js';
|
|
9
9
|
/**
|
|
10
10
|
* Change log store for HLC-indexed change tracking.
|
|
11
11
|
*
|
|
12
12
|
* Each mutation (column update or row deletion) creates an entry in the
|
|
13
13
|
* change log, keyed by HLC. This allows efficient range scans to find
|
|
14
14
|
* all changes since a given timestamp.
|
|
15
|
+
*
|
|
16
|
+
* Every pk-taking method derives the key's pk IDENTITY through the per-table
|
|
17
|
+
* {@link PkKeyingResolver} passed at construction; `deleteEntryByIdentityBatch`
|
|
18
|
+
* serves callers that already hold a parsed identity.
|
|
15
19
|
*/
|
|
16
20
|
export class ChangeLogStore {
|
|
17
21
|
kv;
|
|
18
|
-
|
|
22
|
+
keying;
|
|
23
|
+
constructor(kv, keying) {
|
|
19
24
|
this.kv = kv;
|
|
25
|
+
this.keying = keying;
|
|
26
|
+
}
|
|
27
|
+
identity(schema, table, pk) {
|
|
28
|
+
return encodePkIdentity(pk, this.keying(schema, table));
|
|
20
29
|
}
|
|
21
30
|
/**
|
|
22
31
|
* Record a column change in the change log.
|
|
23
32
|
*/
|
|
24
33
|
async recordColumnChange(hlc, schema, table, pk, column) {
|
|
25
|
-
const key = buildChangeLogKey(hlc, 'column', schema, table, pk, column);
|
|
34
|
+
const key = buildChangeLogKey(hlc, 'column', schema, table, this.identity(schema, table, pk), column);
|
|
26
35
|
// Value is empty - all info is in the key
|
|
27
36
|
await this.kv.put(key, new Uint8Array(0));
|
|
28
37
|
}
|
|
29
38
|
/**
|
|
30
|
-
* Record a column change in a batch.
|
|
39
|
+
* Record a column change in a batch. The key's pk IDENTITY is always derived
|
|
40
|
+
* locally — no caller-supplied-identity WRITE variant exists (see
|
|
41
|
+
* `ColumnVersionStore.setColumnVersionBatch`); `deleteEntryByIdentityBatch`
|
|
42
|
+
* below remains for DELETE-side callers holding a parsed key.
|
|
31
43
|
*/
|
|
32
44
|
recordColumnChangeBatch(batch, hlc, schema, table, pk, column) {
|
|
33
|
-
const key = buildChangeLogKey(hlc, 'column', schema, table, pk, column);
|
|
45
|
+
const key = buildChangeLogKey(hlc, 'column', schema, table, this.identity(schema, table, pk), column);
|
|
34
46
|
batch.put(key, new Uint8Array(0));
|
|
35
47
|
}
|
|
36
48
|
/**
|
|
37
49
|
* Record a row deletion in the change log.
|
|
38
50
|
*/
|
|
39
51
|
async recordDeletion(hlc, schema, table, pk) {
|
|
40
|
-
const key = buildChangeLogKey(hlc, 'delete', schema, table, pk);
|
|
52
|
+
const key = buildChangeLogKey(hlc, 'delete', schema, table, this.identity(schema, table, pk));
|
|
41
53
|
await this.kv.put(key, new Uint8Array(0));
|
|
42
54
|
}
|
|
43
55
|
/**
|
|
44
56
|
* Record a row deletion in a batch.
|
|
45
57
|
*/
|
|
46
58
|
recordDeletionBatch(batch, hlc, schema, table, pk) {
|
|
47
|
-
const key = buildChangeLogKey(hlc, 'delete', schema, table, pk);
|
|
59
|
+
const key = buildChangeLogKey(hlc, 'delete', schema, table, this.identity(schema, table, pk));
|
|
48
60
|
batch.put(key, new Uint8Array(0));
|
|
49
61
|
}
|
|
50
62
|
/**
|
|
@@ -75,6 +87,21 @@ export class ChangeLogStore {
|
|
|
75
87
|
/**
|
|
76
88
|
* Delete change log entries up to a given HLC.
|
|
77
89
|
* Used for pruning old entries after they've been synced to all peers.
|
|
90
|
+
*
|
|
91
|
+
* NOTE: intentionally unwired — no production caller invokes this, and that is
|
|
92
|
+
* deliberate, not an oversight. The change log no longer leaks (entries are
|
|
93
|
+
* deleted with the `cv:`/`tb:` record they point at, so its size tracks live
|
|
94
|
+
* data), so horizon pruning is now a behaviour change rather than a leak fix:
|
|
95
|
+
* it drops index entries for cells that are still LIVE, which silently breaks
|
|
96
|
+
* any peer whose `sinceHLC` predates the horizon — their delta scan would start
|
|
97
|
+
* past the pruned range and never learn about those cells.
|
|
98
|
+
*
|
|
99
|
+
* Precondition before wiring it up: the server must first REFUSE delta sync for
|
|
100
|
+
* a too-old `sinceHLC` and fall back to a snapshot. `SyncManager.canDeltaSync`
|
|
101
|
+
* is that check, but no production caller invokes it either — the coordinator's
|
|
102
|
+
* `get_changes` handler passes `sinceHLC` straight through
|
|
103
|
+
* (packages/sync-coordinator/src/server/websocket.ts). Both halves must land
|
|
104
|
+
* together. Tracked as `feat-sync-changelog-horizon-pruning`.
|
|
78
105
|
*/
|
|
79
106
|
async pruneEntriesBefore(beforeHLC) {
|
|
80
107
|
const bounds = buildAllChangeLogScanBounds();
|
|
@@ -100,7 +127,15 @@ export class ChangeLogStore {
|
|
|
100
127
|
* Delete a specific change log entry (used during snapshot application).
|
|
101
128
|
*/
|
|
102
129
|
deleteEntryBatch(batch, hlc, entryType, schema, table, pk, column) {
|
|
103
|
-
|
|
130
|
+
this.deleteEntryByIdentityBatch(batch, hlc, entryType, schema, table, this.identity(schema, table, pk), column);
|
|
131
|
+
}
|
|
132
|
+
/**
|
|
133
|
+
* Delete a specific change log entry by pk IDENTITY — for callers that hold a
|
|
134
|
+
* parsed key rather than a raw pk (e.g. the tombstone prune sweep, whose
|
|
135
|
+
* table may no longer have a resolvable schema).
|
|
136
|
+
*/
|
|
137
|
+
deleteEntryByIdentityBatch(batch, hlc, entryType, schema, table, identity, column) {
|
|
138
|
+
const key = buildChangeLogKey(hlc, entryType, schema, table, identity, column);
|
|
104
139
|
batch.delete(key);
|
|
105
140
|
}
|
|
106
141
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"change-log.js","sourceRoot":"","sources":["../../../src/metadata/change-log.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAIH,OAAO,EAAY,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACvD,OAAO,EACL,iBAAiB,EACjB,6BAA6B,EAC7B,2BAA2B,EAC3B,iBAAiB,
|
|
1
|
+
{"version":3,"file":"change-log.js","sourceRoot":"","sources":["../../../src/metadata/change-log.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAIH,OAAO,EAAY,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACvD,OAAO,EACL,iBAAiB,EACjB,6BAA6B,EAC7B,2BAA2B,EAC3B,iBAAiB,EACjB,gBAAgB,GAEjB,MAAM,WAAW,CAAC;AAoBnB;;;;;;;;;;GAUG;AACH,MAAM,OAAO,cAAc;IAEN;IACA;IAFnB,YACmB,EAAW,EACX,MAAwB;QADxB,OAAE,GAAF,EAAE,CAAS;QACX,WAAM,GAAN,MAAM,CAAkB;IACxC,CAAC;IAEI,QAAQ,CAAC,MAAc,EAAE,KAAa,EAAE,EAAc;QAC5D,OAAO,gBAAgB,CAAC,EAAE,EAAE,IAAI,CAAC,MAAM,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC,CAAC;IAC1D,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,kBAAkB,CACtB,GAAQ,EACR,MAAc,EACd,KAAa,EACb,EAAc,EACd,MAAc;QAEd,MAAM,GAAG,GAAG,iBAAiB,CAAC,GAAG,EAAE,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,EAAE,MAAM,CAAC,CAAC;QACtG,0CAA0C;QAC1C,MAAM,IAAI,CAAC,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,IAAI,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;IAC5C,CAAC;IAED;;;;;OAKG;IACH,uBAAuB,CACrB,KAAiB,EACjB,GAAQ,EACR,MAAc,EACd,KAAa,EACb,EAAc,EACd,MAAc;QAEd,MAAM,GAAG,GAAG,iBAAiB,CAAC,GAAG,EAAE,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,EAAE,MAAM,CAAC,CAAC;QACtG,KAAK,CAAC,GAAG,CAAC,GAAG,EAAE,IAAI,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;IACpC,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,cAAc,CAClB,GAAQ,EACR,MAAc,EACd,KAAa,EACb,EAAc;QAEd,MAAM,GAAG,GAAG,iBAAiB,CAAC,GAAG,EAAE,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,CAAC,CAAC;QAC9F,MAAM,IAAI,CAAC,EAAE,CAAC,GAAG,CAAC,GAAG,EAAE,IAAI,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;IAC5C,CAAC;IAED;;OAEG;IACH,mBAAmB,CACjB,KAAiB,EACjB,GAAQ,EACR,MAAc,EACd,KAAa,EACb,EAAc;QAEd,MAAM,GAAG,GAAG,iBAAiB,CAAC,GAAG,EAAE,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,CAAC,CAAC;QAC9F,KAAK,CAAC,GAAG,CAAC,GAAG,EAAE,IAAI,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;IACpC,CAAC;IAED;;;OAGG;IACH,KAAK,CAAC,CAAC,eAAe,CAAC,QAAa;QAClC,MAAM,MAAM,GAAG,6BAA6B,CAAC,QAAQ,CAAC,CAAC;QACvD,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;YAClD,MAAM,MAAM,GAAG,iBAAiB,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;YAC5C,IAAI,MAAM,EAAE,CAAC;gBACX,MAAM,MAAM,CAAC;YACf,CAAC;QACH,CAAC;IACH,CAAC;IAED;;OAEG;IACH,KAAK,CAAC,CAAC,aAAa;QAClB,MAAM,MAAM,GAAG,2BAA2B,EAAE,CAAC;QAC7C,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;YAClD,MAAM,MAAM,GAAG,iBAAiB,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;YAC5C,IAAI,MAAM,EAAE,CAAC;gBACX,MAAM,MAAM,CAAC;YACf,CAAC;QACH,CAAC;IACH,CAAC;IAED;;;;;;;;;;;;;;;;;;OAkBG;IACH,KAAK,CAAC,kBAAkB,CAAC,SAAc;QACrC,MAAM,MAAM,GAAG,2BAA2B,EAAE,CAAC;QAC7C,MAAM,KAAK,GAAG,IAAI,CAAC,EAAE,CAAC,KAAK,EAAE,CAAC;QAC9B,IAAI,KAAK,GAAG,CAAC,CAAC;QAEd,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,IAAI,CAAC,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;YAClD,MAAM,MAAM,GAAG,iBAAiB,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;YAC5C,IAAI,CAAC,MAAM;gBAAE,SAAS;YAEtB,0EAA0E;YAC1E,0EAA0E;YAC1E,oEAAoE;YACpE,kEAAkE;YAClE,IAAI,UAAU,CAAC,MAAM,CAAC,GAAG,EAAE,SAAS,CAAC,IAAI,CAAC;gBAAE,MAAM;YAElD,KAAK,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;YACxB,KAAK,EAAE,CAAC;QACV,CAAC;QAED,MAAM,KAAK,CAAC,KAAK,EAAE,CAAC;QACpB,OAAO,KAAK,CAAC;IACf,CAAC;IAED;;OAEG;IACH,gBAAgB,CACd,KAAiB,EACjB,GAAQ,EACR,SAA6B,EAC7B,MAAc,EACd,KAAa,EACb,EAAc,EACd,MAAe;QAEf,IAAI,CAAC,0BAA0B,CAAC,KAAK,EAAE,GAAG,EAAE,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,EAAE,MAAM,CAAC,CAAC;IAClH,CAAC;IAED;;;;OAIG;IACH,0BAA0B,CACxB,KAAiB,EACjB,GAAQ,EACR,SAA6B,EAC7B,MAAc,EACd,KAAa,EACb,QAAgB,EAChB,MAAe;QAEf,MAAM,GAAG,GAAG,iBAAiB,CAAC,GAAG,EAAE,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,QAAQ,EAAE,MAAM,CAAC,CAAC;QAC/E,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;IACpB,CAAC;CACF"}
|
|
@@ -7,9 +7,20 @@
|
|
|
7
7
|
import type { SqlValue } from '@quereus/quereus';
|
|
8
8
|
import type { KVStore, WriteBatch } from '@quereus/store';
|
|
9
9
|
import { type HLC } from '../clock/hlc.js';
|
|
10
|
+
import type { PkKeyingResolver } from './pk-identity.js';
|
|
10
11
|
/**
|
|
11
12
|
* Column version record stored in the KV store.
|
|
12
13
|
*
|
|
14
|
+
* `pk` is the row's ADDRESS — the raw primary-key values, one spelling from the
|
|
15
|
+
* row's equivalence class. It lives in the VALUE because the record's KEY files
|
|
16
|
+
* under the derived, lossy pk IDENTITY (see `keys.ts`) which cannot be decoded
|
|
17
|
+
* back to values.
|
|
18
|
+
*
|
|
19
|
+
* NOTE: this repeats the pk in EVERY cell record of a row, so metadata size grows
|
|
20
|
+
* with (pk width × column count). Fine at current scale; if sync metadata volume
|
|
21
|
+
* ever becomes a concern for wide tables with long pks, add one per-row
|
|
22
|
+
* identity→pk record and drop `pk` from the cell records.
|
|
23
|
+
*
|
|
13
24
|
* `priorHlc`/`priorValue` are an optional per-cell before-image: the cell version
|
|
14
25
|
* this one replaced (its replica-local lineage). They are written together (both
|
|
15
26
|
* present or both absent) and are absent on the first write of a cell and on
|
|
@@ -18,17 +29,24 @@ import { type HLC } from '../clock/hlc.js';
|
|
|
18
29
|
export interface ColumnVersion {
|
|
19
30
|
hlc: HLC;
|
|
20
31
|
value: SqlValue;
|
|
32
|
+
pk: SqlValue[];
|
|
21
33
|
priorHlc?: HLC;
|
|
22
34
|
priorValue?: SqlValue;
|
|
23
35
|
}
|
|
36
|
+
/**
|
|
37
|
+
* Write-side shape: everything but the pk, which the store methods take as
|
|
38
|
+
* their own parameter (they need it for the key identity anyway) and embed.
|
|
39
|
+
*/
|
|
40
|
+
export type ColumnVersionData = Omit<ColumnVersion, 'pk'>;
|
|
24
41
|
/**
|
|
25
42
|
* Serialize a column version for storage.
|
|
26
|
-
* Format: 30 bytes HLC + JSON payload `{ v, pv?, ph? }`.
|
|
43
|
+
* Format: 30 bytes HLC + JSON payload `{ v, k, pv?, ph? }`.
|
|
27
44
|
*
|
|
28
45
|
* Uint8Array values are encoded as `{"__bin":"<base64>"}` (bigint as
|
|
29
46
|
* `{"__bigint":"..."}`) so they survive the JSON round-trip; the same encoding
|
|
30
|
-
* covers the before-image (`pv`). The before-image fields are
|
|
31
|
-
* when the version has no prior, keeping first-writes and
|
|
47
|
+
* covers the pk cells and the before-image (`pv`). The before-image fields are
|
|
48
|
+
* omitted entirely when the version has no prior, keeping first-writes and
|
|
49
|
+
* snapshot cells compact.
|
|
32
50
|
*/
|
|
33
51
|
export declare function serializeColumnVersion(cv: ColumnVersion): Uint8Array;
|
|
34
52
|
/**
|
|
@@ -51,30 +69,65 @@ export declare function encodeSqlValue(v: SqlValue): unknown;
|
|
|
51
69
|
export declare function decodeSqlValue(v: unknown): SqlValue;
|
|
52
70
|
/**
|
|
53
71
|
* Column version store operations.
|
|
72
|
+
*
|
|
73
|
+
* Every pk-taking method derives the record key's pk IDENTITY through the
|
|
74
|
+
* per-table {@link PkKeyingResolver} passed at construction, so callers keep
|
|
75
|
+
* addressing rows by raw pk while the storage keys collapse spellings.
|
|
54
76
|
*/
|
|
55
77
|
export declare class ColumnVersionStore {
|
|
56
78
|
private readonly kv;
|
|
57
|
-
|
|
79
|
+
private readonly keying;
|
|
80
|
+
constructor(kv: KVStore, keying: PkKeyingResolver);
|
|
81
|
+
private identity;
|
|
58
82
|
/**
|
|
59
83
|
* Get the version of a specific column.
|
|
60
84
|
*/
|
|
61
85
|
getColumnVersion(schemaName: string, tableName: string, pk: SqlValue[], column: string): Promise<ColumnVersion | undefined>;
|
|
86
|
+
/**
|
|
87
|
+
* Get the version of a specific column by pk IDENTITY (as recovered from a
|
|
88
|
+
* parsed `cv:`/`cl:` key — no raw pk needed).
|
|
89
|
+
*/
|
|
90
|
+
getColumnVersionByIdentity(schemaName: string, tableName: string, identity: string, column: string): Promise<ColumnVersion | undefined>;
|
|
62
91
|
/**
|
|
63
92
|
* Set the version of a specific column.
|
|
64
93
|
*/
|
|
65
|
-
setColumnVersion(schemaName: string, tableName: string, pk: SqlValue[], column: string, version:
|
|
94
|
+
setColumnVersion(schemaName: string, tableName: string, pk: SqlValue[], column: string, version: ColumnVersionData): Promise<void>;
|
|
66
95
|
/**
|
|
67
|
-
* Set column version in a batch.
|
|
96
|
+
* Set column version in a batch. The key's pk IDENTITY is always derived
|
|
97
|
+
* locally through this store's keying resolver — there is deliberately no
|
|
98
|
+
* caller-supplied-identity variant (a snapshot receiver trusting a sender's
|
|
99
|
+
* identity was the bug fixed by deriving on ingress; see docs/sync.md
|
|
100
|
+
* § Row identity vs. address).
|
|
68
101
|
*/
|
|
69
|
-
setColumnVersionBatch(batch: WriteBatch, schemaName: string, tableName: string, pk: SqlValue[], column: string, version:
|
|
102
|
+
setColumnVersionBatch(batch: WriteBatch, schemaName: string, tableName: string, pk: SqlValue[], column: string, version: ColumnVersionData): void;
|
|
70
103
|
/**
|
|
71
104
|
* Get all column versions for a row.
|
|
72
105
|
*/
|
|
73
106
|
getRowVersions(schemaName: string, tableName: string, pk: SqlValue[]): Promise<Map<string, ColumnVersion>>;
|
|
74
107
|
/**
|
|
75
|
-
*
|
|
108
|
+
* Queue deletion of ONE column's cell record into `batch`, whether or not a
|
|
109
|
+
* committed record exists — the caller may be removing a record that is only
|
|
110
|
+
* STAGED in this same batch (the local-capture delete cleanup; see
|
|
111
|
+
* `deleteRowVersionsAndLogEntries`).
|
|
112
|
+
*/
|
|
113
|
+
deleteColumnVersionBatch(batch: WriteBatch, schemaName: string, tableName: string, pk: SqlValue[], column: string): void;
|
|
114
|
+
/**
|
|
115
|
+
* Queue deletion of every column version of a row into `batch`, returning the
|
|
116
|
+
* versions that were removed (column → version). A column in `keepColumns` is
|
|
117
|
+
* skipped entirely — neither staged for deletion nor included in the returned
|
|
118
|
+
* map (its paired `cl:` entry must survive too; see
|
|
119
|
+
* `deleteRowVersionsAndLogEntries`).
|
|
120
|
+
*
|
|
121
|
+
* The returned map is what lets a caller drop the paired `cl:` change-log
|
|
122
|
+
* entries in the SAME batch — each entry's key embeds the version's HLC, which
|
|
123
|
+
* is only recoverable from the record being deleted.
|
|
124
|
+
*
|
|
125
|
+
* NOTE: this fully deserializes each cell (a JSON parse of the value) when the
|
|
126
|
+
* change-log caller only needs the 30-byte HLC prefix. If deleting wide rows or
|
|
127
|
+
* rows with large blob cells ever shows up as slow, read just the HLC prefix
|
|
128
|
+
* here instead of reusing `getRowVersions`.
|
|
76
129
|
*/
|
|
77
|
-
|
|
130
|
+
deleteRowVersionsBatch(batch: WriteBatch, schemaName: string, tableName: string, pk: SqlValue[], keepColumns?: ReadonlySet<string>): Promise<Map<string, ColumnVersion>>;
|
|
78
131
|
/**
|
|
79
132
|
* Check if a column write should be applied (LWW comparison).
|
|
80
133
|
* Returns true if the incoming HLC is newer than the current version.
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"column-version.d.ts","sourceRoot":"","sources":["../../../src/metadata/column-version.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AACjD,OAAO,KAAK,EAAE,OAAO,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAC1D,OAAO,EAAE,KAAK,GAAG,EAAwF,MAAM,iBAAiB,CAAC;
|
|
1
|
+
{"version":3,"file":"column-version.d.ts","sourceRoot":"","sources":["../../../src/metadata/column-version.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AACjD,OAAO,KAAK,EAAE,OAAO,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAC1D,OAAO,EAAE,KAAK,GAAG,EAAwF,MAAM,iBAAiB,CAAC;AAEjI,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AAEzD;;;;;;;;;;;;;;;;;GAiBG;AACH,MAAM,WAAW,aAAa;IAC5B,GAAG,EAAE,GAAG,CAAC;IACT,KAAK,EAAE,QAAQ,CAAC;IAChB,EAAE,EAAE,QAAQ,EAAE,CAAC;IACf,QAAQ,CAAC,EAAE,GAAG,CAAC;IACf,UAAU,CAAC,EAAE,QAAQ,CAAC;CACvB;AAED;;;GAGG;AACH,MAAM,MAAM,iBAAiB,GAAG,IAAI,CAAC,aAAa,EAAE,IAAI,CAAC,CAAC;AAe1D;;;;;;;;;GASG;AACH,wBAAgB,sBAAsB,CAAC,EAAE,EAAE,aAAa,GAAG,UAAU,CAgBpE;AAED;;;GAGG;AACH,wBAAgB,wBAAwB,CAAC,MAAM,EAAE,UAAU,GAAG,aAAa,CAS1E;AAMD;;;GAGG;AACH,wBAAgB,cAAc,CAAC,CAAC,EAAE,QAAQ,GAAG,OAAO,CAUnD;AAED;;;;;;GAMG;AACH,wBAAgB,cAAc,CAAC,CAAC,EAAE,OAAO,GAAG,QAAQ,CA2BnD;AAED;;;;;;GAMG;AACH,qBAAa,kBAAkB;IAE3B,OAAO,CAAC,QAAQ,CAAC,EAAE;IACnB,OAAO,CAAC,QAAQ,CAAC,MAAM;gBADN,EAAE,EAAE,OAAO,EACX,MAAM,EAAE,gBAAgB;IAG3C,OAAO,CAAC,QAAQ;IAIhB;;OAEG;IACG,gBAAgB,CACpB,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,EAAE,EAAE,QAAQ,EAAE,EACd,MAAM,EAAE,MAAM,GACb,OAAO,CAAC,aAAa,GAAG,SAAS,CAAC;IAIrC;;;OAGG;IACG,0BAA0B,CAC9B,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,QAAQ,EAAE,MAAM,EAChB,MAAM,EAAE,MAAM,GACb,OAAO,CAAC,aAAa,GAAG,SAAS,CAAC;IAOrC;;OAEG;IACG,gBAAgB,CACpB,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,EAAE,EAAE,QAAQ,EAAE,EACd,MAAM,EAAE,MAAM,EACd,OAAO,EAAE,iBAAiB,GACzB,OAAO,CAAC,IAAI,CAAC;IAKhB;;;;;;OAMG;IACH,qBAAqB,CACnB,KAAK,EAAE,UAAU,EACjB,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,EAAE,EAAE,QAAQ,EAAE,EACd,MAAM,EAAE,MAAM,EACd,OAAO,EAAE,iBAAiB,GACzB,IAAI;IAKP;;OAEG;IACG,cAAc,CAClB,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,EAAE,EAAE,QAAQ,EAAE,GACb,OAAO,CAAC,GAAG,CAAC,MAAM,EAAE,aAAa,CAAC,CAAC;IAoBtC;;;;;OAKG;IACH,wBAAwB,CACtB,KAAK,EAAE,UAAU,EACjB,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,EAAE,EAAE,QAAQ,EAAE,EACd,MAAM,EAAE,MAAM,GACb,IAAI;IAIP;;;;;;;;;;;;;;;OAeG;IACG,sBAAsB,CAC1B,KAAK,EAAE,UAAU,EACjB,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,EAAE,EAAE,QAAQ,EAAE,EACd,WAAW,CAAC,EAAE,WAAW,CAAC,MAAM,CAAC,GAChC,OAAO,CAAC,GAAG,CAAC,MAAM,EAAE,aAAa,CAAC,CAAC;IAatC;;;OAGG;IACG,gBAAgB,CACpB,UAAU,EAAE,MAAM,EAClB,SAAS,EAAE,MAAM,EACjB,EAAE,EAAE,QAAQ,EAAE,EACd,MAAM,EAAE,MAAM,EACd,WAAW,EAAE,GAAG,GACf,OAAO,CAAC,OAAO,CAAC;CAKpB"}
|