@zukhruf/sqlite-changefeed 0.8.2 → 0.8.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. package/README.md +50 -20
  2. package/dist/baseline.d.ts +11 -0
  3. package/dist/baseline.d.ts.map +1 -0
  4. package/dist/baseline.js +11 -0
  5. package/dist/baseline.js.map +1 -0
  6. package/dist/changefeed.d.ts +10 -3
  7. package/dist/changefeed.d.ts.map +1 -1
  8. package/dist/changefeed.js +12 -21
  9. package/dist/changefeed.js.map +1 -1
  10. package/dist/index.d.ts +9 -1
  11. package/dist/index.d.ts.map +1 -1
  12. package/dist/index.js +5 -1
  13. package/dist/index.js.map +1 -1
  14. package/dist/pass.d.ts +2 -1
  15. package/dist/pass.d.ts.map +1 -1
  16. package/dist/pass.js +11 -4
  17. package/dist/pass.js.map +1 -1
  18. package/dist/position-file.d.ts +8 -1
  19. package/dist/position-file.d.ts.map +1 -1
  20. package/dist/position-file.js +11 -3
  21. package/dist/position-file.js.map +1 -1
  22. package/dist/position-in-memory.d.ts +3 -0
  23. package/dist/position-in-memory.d.ts.map +1 -1
  24. package/dist/position-in-memory.js +19 -6
  25. package/dist/position-in-memory.js.map +1 -1
  26. package/dist/position.d.ts +19 -1
  27. package/dist/position.d.ts.map +1 -1
  28. package/dist/read-every-row.d.ts +8 -0
  29. package/dist/read-every-row.d.ts.map +1 -0
  30. package/dist/read-every-row.js +8 -0
  31. package/dist/read-every-row.js.map +1 -0
  32. package/dist/start.d.ts +12 -0
  33. package/dist/start.d.ts.map +1 -0
  34. package/dist/start.js +2 -0
  35. package/dist/start.js.map +1 -0
  36. package/dist/table-changes.d.ts +7 -4
  37. package/dist/table-changes.d.ts.map +1 -1
  38. package/dist/table-changes.js +19 -9
  39. package/dist/table-changes.js.map +1 -1
  40. package/dist/tables.js +9 -7
  41. package/dist/tables.js.map +1 -1
  42. package/package.json +8 -8
package/README.md CHANGED
@@ -22,11 +22,21 @@ npm install @zukhruf/sqlite-changefeed
22
22
  The owner is a notes app. It keeps its database in WAL mode. The reader copies each table into its own store:
23
23
 
24
24
  ```ts
25
- import { Changefeed } from '@zukhruf/sqlite-changefeed';
25
+ import {
26
+ Baseline,
27
+ Changefeed,
28
+ PositionFile,
29
+ ReadEveryRow,
30
+ } from '@zukhruf/sqlite-changefeed';
26
31
 
27
32
  using feed = new Changefeed(
28
33
  `${process.env.HOME}/Library/Application Support/Notes/notes.sqlite`,
29
- `${process.env.HOME}/.notes-copy/notes.position.sqlite`,
34
+ {
35
+ position: new PositionFile(
36
+ `${process.env.HOME}/.notes-copy/notes.position.sqlite`,
37
+ ),
38
+ start: new ReadEveryRow(),
39
+ },
30
40
  );
31
41
 
32
42
  const stop = new AbortController();
@@ -46,11 +56,19 @@ for await (const batch of feed.watch({
46
56
  }
47
57
  ```
48
58
 
49
- This is a durable feed: it has a position file, and a new run goes on from the last saved batch. Without a position path, the feed observes from now. It reads each table one time with no events, and then gives only the changes after that moment:
59
+ The feed gets two objects. The position tells where the feed keeps what it saw. The start tells what the first pass gives. This is a durable feed: its position is a file, and a new run goes on from the last saved batch. Its start is `ReadEveryRow`, so the first run gives each row as a read event.
60
+
61
+ A host that sends a message for each change does not want the rows that were there before it started. Give it the start `Baseline`. The first pass is then a baseline: it keeps the fingerprints, and gives no events. A position file keeps the baseline, so a new run gives the changes that the owner made while the feed was stopped:
50
62
 
51
63
  ```ts
52
64
  using feed = new Changefeed(
53
65
  `${process.env.HOME}/Library/Application Support/Notes/notes.sqlite`,
66
+ {
67
+ position: new PositionFile(
68
+ `${process.env.HOME}/.notes-alerts/notes.position.sqlite`,
69
+ ),
70
+ start: new Baseline(),
71
+ },
54
72
  );
55
73
 
56
74
  for await (const batch of feed.watch({
@@ -62,14 +80,14 @@ for await (const batch of feed.watch({
62
80
  }
63
81
  ```
64
82
 
65
- The first run of a durable feed gives a `read` event for each row of each table. Each later batch gives only the rows that changed:
83
+ With `ReadEveryRow`, the first run gives a `read` event for each row of each table. Each later batch gives only the rows that changed. Each integer is a `bigint`:
66
84
 
67
85
  ```
68
- { table: 'notes', op: 'read', key: { id: 1 }, after: { id: 1, folder_id: 7, title: 'Plan', body: '…' } }
69
- { table: 'tags', op: 'read', key: { rowid: 3 }, after: { note_id: 1, tag: 'q4' } }
86
+ { table: 'notes', op: 'read', key: { id: 1n }, after: { id: 1n, folder_id: 7n, title: 'Plan', body: '…' } }
87
+ { table: 'tags', op: 'read', key: { rowid: 3n }, after: { note_id: 1n, tag: 'q4' } }
70
88
  …
71
- { table: 'notes', op: 'update', key: { id: 1 }, after: { id: 1, folder_id: 7, title: 'Plan v2', body: '…' } }
72
- { table: 'tags', op: 'delete', key: { rowid: 3 } }
89
+ { table: 'notes', op: 'update', key: { id: 1n }, after: { id: 1n, folder_id: 7n, title: 'Plan v2', body: '…' } }
90
+ { table: 'tags', op: 'delete', key: { rowid: 3n } }
73
91
  { table: 'drafts', op: 'truncate' }
74
92
  ```
75
93
 
@@ -77,7 +95,7 @@ The table `tags` has no primary key, so its key is the `rowid`. The owner droppe
77
95
 
78
96
  ## The rules
79
97
 
80
- - The database must be in WAL mode. `new Changefeed(databasePath, positionPath?)` reads `PRAGMA journal_mode`, and throws `JournalModeError` for each other mode. Each pass reads it again, because the owner can replace the file. [ADR 0002](./docs/adr/0002-wal-is-required.md) tells why.
98
+ - The database must be in WAL mode. `new Changefeed(databasePath, { position, start })` reads `PRAGMA journal_mode`, and throws `JournalModeError` for each other mode. Each pass reads it again, because the owner can replace the file. [ADR 0002](./docs/adr/0002-wal-is-required.md) tells why.
81
99
  - The feed reads each table that `PRAGMA table_list` gives with type `table`, except a name that starts with `sqlite_`. It does not read an internal table of SQLite (`sqlite_schema`, `sqlite_sequence`, `sqlite_stat*`), a `view`, a `virtual` table, or a `shadow` table. A shadow table holds the data of a virtual table, for example the `notes_fts_data` of an FTS5 table. It is internal to the module of the virtual table. [ADR 0006](./docs/adr/0006-every-table-keyed-by-its-primary-key.md) tells why.
82
100
  - The key of a row is the identity that SQLite makes unique and not `NULL`. The feed finds it from the schema of each table:
83
101
  - A `WITHOUT ROWID` table: its declared primary key. SQLite makes each column of it `NOT NULL`, and `PRAGMA table_info` gives `notnull` 1 for each column of it.
@@ -85,14 +103,19 @@ The table `tags` has no primary key, so its key is the `rowid`. The owner droppe
85
103
  - Each other rowid table: the key `{ rowid }`. A declared primary key of a rowid table can hold `NULL` in many rows, so it does not identify a row.
86
104
  - The columns of a primary key are the columns of `PRAGMA table_info` with a `pk` above 0, in the order of `pk`.
87
105
  - `VACUUM` can change the `rowid` of each row of a table with no primary key. The feed compares the rows by `rowid`, so it then gives an event for each `rowid` whose row changed: an `insert` for a new `rowid`, an `update` for a `rowid` that now holds a different row, and a `delete` for a `rowid` that is gone. The SQLite documentation advises an `INTEGER PRIMARY KEY` for a stable `rowid`.
88
- - A pass copies each row into a plain object, because `node:sqlite` gives rows with no prototype. `after` keeps the values that SQLite gives: a number, a string, `null`, or a `Uint8Array` for a `BLOB`.
106
+ - A pass copies each row into a plain object, because `node:sqlite` gives rows with no prototype. `after` keeps the values that SQLite gives: a `bigint` for an `INTEGER`, a number for a `REAL`, a string, `null`, or a `Uint8Array` for a `BLOB`. A key value is a `bigint` for an `INTEGER` too, such as each `rowid`. [ADR 0008](./docs/adr/0008-every-integer-is-a-bigint.md) tells why.
89
107
  - A change event is `{ table, op, key, after? }`, or `{ table, op: 'truncate' }`. `op` is `read`, `insert`, `update`, `delete` or `truncate`. `after` is the full row for `read`, `insert` and `update`. There is no `before`. [ADR 0003](./docs/adr/0003-the-change-event.md) tells why.
90
108
  - `read` is each row of a table that the position does not have: each row at the first run, and each row of a new table.
91
109
  - `truncate` tells that the owner dropped a table, or changed the columns of its primary key. Remove all rows of that table from your copy. When the primary key changed, a `read` for each row comes after the `truncate`.
92
110
  - A change to the other columns of a table, for example a new column, changes the `after` of its rows. Each row then gives an `update`.
93
- - The feed owns its position. `positionPath` is optional, and it sets the kind of the feed. [ADR 0004](./docs/adr/0004-the-feed-owns-its-position.md) tells why.
94
- - With `positionPath`, the feed is durable. It keeps its position in its own SQLite file at that path: one fingerprint for each row of each table. It creates the file when the file is missing. A missing or empty file is a first run, and the first pass gives each row as `read`. A later run goes on from the saved position. Give each durable feed its own position file.
95
- - Without `positionPath`, the feed observes from now. The first pass of each `watch` is the baseline: it reads each table, keeps the fingerprints in memory, and gives no batch. Then it gives only the changes after the baseline. It writes nothing to disk. A new run takes a new baseline, so it does not see the changes that the owner made while the feed was stopped.
111
+ - `new Changefeed(databasePath, { position, start })` takes two strategies. Both are required. [ADR 0007](./docs/adr/0007-the-host-gives-the-position-and-the-start.md) tells why.
112
+ - `position` is where the feed keeps what it saw. The feed owns it: the feed disposes it, also when `new Changefeed` throws. A new position is a class that implements `Position`.
113
+ - `new PositionFile(path)` keeps the position in its own SQLite file at that path: one fingerprint for each row of each table. It creates the file when the file is missing. A missing or empty file is a position that no pass saved. A later run goes on from the saved position. Give each feed its own position file.
114
+ - `new PositionInMemory()` keeps the position in memory. It writes nothing to disk. A new feed starts with a new position, so it does not see the changes that the owner made while no feed ran.
115
+ - `start` is what the first pass gives for each table, until a pass saves the position. A new start is a class that implements `Start`.
116
+ - `new ReadEveryRow()` gives each row as `read`.
117
+ - `new Baseline()` keeps the fingerprint of each row, and gives no events. The feed then gives only the changes after the baseline.
118
+ - After the first save, a table that the position does not have gives a `read` for each row, with each start. Thus a table that the owner creates after a baseline gives its rows.
96
119
  - `watch({ interval, signal })` gives batches. A batch is an array of change events. The feed makes a first pass at once. Then it polls the data version each `interval` milliseconds, and makes a pass only when the data version changed. A pass that finds no change gives no batch, and the feed saves its position at once: for example, the key columns of a new table with no rows.
97
120
  - A durable feed saves the position of a batch when the loop asks for the next batch. It writes the position in one transaction. Apply each event of a batch before the loop goes on.
98
121
  - Delivery of a durable feed is at least once. If the process stops before the loop asks for the next batch, the next run gives the same batch again. A loop that stops with `break`, an error or an abort also gives its last batch again at the next run. Apply each event so that a second apply has no effect: a `put` by key, a `remove` by key, a `clear` by table.
@@ -104,8 +127,9 @@ The table `tags` has no primary key, so its key is the `rowid`. The owner droppe
104
127
 
105
128
  - You see only the net change between two passes. A row that the owner inserts and deletes between two passes gives no event. A row that the owner changes two times gives one `update`.
106
129
  - Each commit of the owner makes a pass that reads each table in full. The cost of a pass grows with the size of the database, also when the commit changed one row. [ADR 0001](./docs/adr/0001-capture-by-snapshot-differential.md) names a later option that reads only the changed tables.
107
- - The position keeps one fingerprint for each row of each table. It grows with the number of rows. A durable feed keeps it on disk. A feed that observes from now keeps it in memory.
108
- - A feed that observes from now does not see the changes that the owner made while the feed was stopped.
130
+ - The position keeps one fingerprint for each row of each table. It grows with the number of rows. A `PositionFile` keeps it on disk. A `PositionInMemory` keeps it in memory.
131
+ - A feed with a `PositionInMemory` does not see the changes that the owner made while no feed ran.
132
+ - A batch holds all the change events of one pass, with the full rows. A first run with `ReadEveryRow`, a new table, or a new primary key of a large table gives a large batch. A `Baseline` keeps no rows, only the fingerprints.
109
133
  - Precedent for scale: Fivetran's Teleport Sync, which also compares data sets, gives a warning when more than 50,000 records change between two syncs.
110
134
 
111
135
  ## Errors
@@ -123,7 +147,8 @@ The table `tags` has no primary key, so its key is the `rowid`. The owner droppe
123
147
  | CockroachDB `CREATE CHANGEFEED` | A changefeed for each table that it names. Each message is keyed by the primary key of the row. |
124
148
  | Azure Cosmos DB change feed | A feed of the changes to the items of a container. A reader goes on from the last position that it read. |
125
149
  | Debezium | Change data capture from the log of the database. It gives one topic for each table, keyed by the primary key. Its event has the envelope `before`, `after`, `op` (`c`, `u`, `d`, `r`, `t`), `source` and `ts_ms`. This package keeps `op` and `after`. Its `read` is the `r`, and its `truncate` is the `t`. The embedded engine keeps its offsets in a file that the host names (`offset.storage.file.filename`). |
126
- | Kafka `auto.offset.reset` | It applies when a consumer has no committed offset. `earliest` reads each record again, and `latest` reads only the records that come after the consumer starts. Spring Cloud Stream gives `earliest` to a consumer with a group, which keeps committed offsets, and `latest` to a consumer with no group. A feed with a position path is the first case, and a feed with no path is the second. |
150
+ | Kafka `auto.offset.reset` | It applies when a consumer has no committed offset. `earliest` reads each record again, and `latest` reads only the records that come after the consumer starts. The group, which keeps the committed offsets, is a separate choice. The `position` of the feed is the group, and its `start` is the reset: `ReadEveryRow` is `earliest`, and `Baseline` is `latest`. |
151
+ | Debezium `snapshot.mode` | `initial` reads each row of each table at the first start, and `no_data` reads none. A custom mode is a class of its `Snapshotter` interface. The offsets go to a store that the host selects (`offset.storage`, a file or memory). `ReadEveryRow` is `initial`, `Baseline` is `no_data`, and `Position` is the store. |
127
152
  | Kafka consumer auto-commit | A poll commits the offsets of the records that the last poll gave. Delivery is at least once. The feed saves its position in the same way: at the next request for a batch. |
128
153
  | Kafka Connect JDBC source | Change data capture by query. Its modes are `bulk`, `incrementing`, `timestamp` and `timestamp+incrementing`. It does not see a hard delete: a deleted row is not in the next query. This package compares the full table, so it sees each delete. |
129
154
  | Fivetran Teleport Sync | It compares the data set of the source with the data set of the last sync, with no log. It needs only a read-only user and no timestamp columns. This package does the same for a SQLite file. |
@@ -149,6 +174,8 @@ This package needed four changes in other packages. All four are done:
149
174
  - [0004 The feed owns its position](./docs/adr/0004-the-feed-owns-its-position.md)
150
175
  - [0005 One snapshot for each pass](./docs/adr/0005-one-snapshot-for-each-pass.md)
151
176
  - [0006 Every table, keyed by its primary key](./docs/adr/0006-every-table-keyed-by-its-primary-key.md)
177
+ - [0007 The host gives the position and the start](./docs/adr/0007-the-host-gives-the-position-and-the-start.md)
178
+ - [0008 Every integer is a bigint](./docs/adr/0008-every-integer-is-a-bigint.md)
152
179
 
153
180
  ## Development
154
181
 
@@ -164,10 +191,13 @@ src/
164
191
  changefeed.ts Changefeed: the connections, the runs of watch, and the save of a batch
165
192
  pass.ts one pass in one snapshot, and the WAL check
166
193
  tables.ts the tables of PRAGMA table_list, the key of each table, and its rows
167
- table-changes.ts the change events of one table, from the differential
168
- position.ts the position, and what a pass changes in it
169
- position-file.ts the position of a durable feed, in its SQLite file
170
- position-in-memory.ts the position of an observing feed
194
+ table-changes.ts the change events of one table, and its snapshot with no events
195
+ position.ts Position, and what a pass changes in it
196
+ position-file.ts PositionFile: the position in its own SQLite file
197
+ position-in-memory.ts PositionInMemory: the position in memory
198
+ start.ts Start: what the first pass gives for a table
199
+ read-every-row.ts ReadEveryRow: a read for each row
200
+ baseline.ts Baseline: fingerprints, and no events
171
201
  change-event.ts the change event
172
202
  journal-mode-error.ts JournalModeError
173
203
  index.ts the public exports
@@ -0,0 +1,11 @@
1
+ import type { Start } from './start.ts';
2
+ import { type TableChanges } from './table-changes.ts';
3
+ import type { Table } from './tables.ts';
4
+ /**
5
+ * The first pass is a baseline: it keeps the fingerprint of each row, and
6
+ * gives no events. Later passes give only the changes after it.
7
+ */
8
+ export declare class Baseline implements Start {
9
+ table(table: Table): Promise<TableChanges>;
10
+ }
11
+ //# sourceMappingURL=baseline.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"baseline.d.ts","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACxC,OAAO,EAAE,KAAK,YAAY,EAAc,MAAM,oBAAoB,CAAC;AACnE,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,aAAa,CAAC;AAEzC;;;GAGG;AACH,qBAAa,QAAS,YAAW,KAAK;IAC9B,KAAK,CAAC,KAAK,EAAE,KAAK,GAAG,OAAO,CAAC,YAAY,CAAC;CAGjD"}
@@ -0,0 +1,11 @@
1
+ import { snapshotOf } from "./table-changes.js";
2
+ /**
3
+ * The first pass is a baseline: it keeps the fingerprint of each row, and
4
+ * gives no events. Later passes give only the changes after it.
5
+ */
6
+ export class Baseline {
7
+ async table(table) {
8
+ return { events: [], state: await snapshotOf(table) };
9
+ }
10
+ }
11
+ //# sourceMappingURL=baseline.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"baseline.js","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AACA,OAAO,EAAqB,UAAU,EAAE,MAAM,oBAAoB,CAAC;AAGnE;;;GAGG;AACH,MAAM,OAAO,QAAQ;IACnB,KAAK,CAAC,KAAK,CAAC,KAAY;QACtB,OAAO,EAAE,MAAM,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,UAAU,CAAC,KAAK,CAAC,EAAE,CAAC;IACxD,CAAC;CACF"}
@@ -1,5 +1,7 @@
1
1
  import { type WallClock } from '@zukhruf/async';
2
2
  import type { ChangeEvent } from './change-event.ts';
3
+ import type { Position } from './position.ts';
4
+ import type { Start } from './start.ts';
3
5
  export type WatchOptions = {
4
6
  /** Milliseconds between two polls of the data version. */
5
7
  readonly interval: number;
@@ -8,14 +10,19 @@ export type WatchOptions = {
8
10
  /** The clock of the poll: `globalThis` in production, a fake clock in a test. */
9
11
  readonly clock: WallClock;
10
12
  };
13
+ export type ChangefeedOptions = {
14
+ /** Where the feed keeps what it saw. The feed owns it, and disposes it. */
15
+ readonly position: Position;
16
+ /** What the first pass gives, until the position is saved once. */
17
+ readonly start: Start;
18
+ };
11
19
  /**
12
20
  * A change event for each changed row of each table of a SQLite database
13
- * that another process owns. With a position path the feed is durable and
14
- * goes on from its last saved batch; without one it observes from now.
21
+ * that another process owns.
15
22
  */
16
23
  export declare class Changefeed implements Disposable {
17
24
  #private;
18
- constructor(databasePath: string, positionPath?: string);
25
+ constructor(databasePath: string, { position, start }: ChangefeedOptions);
19
26
  /**
20
27
  * Batches of change events. The feed saves the position of a batch when
21
28
  * the loop asks for the next one.
@@ -1 +1 @@
1
- {"version":3,"file":"changefeed.d.ts","sourceRoot":"","sources":["../src/changefeed.ts"],"names":[],"mappings":"AAAA,OAAO,EAAiB,KAAK,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAI/D,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAMrD,MAAM,MAAM,YAAY,GAAG;IACzB,0DAA0D;IAC1D,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,kEAAkE;IAClE,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAC;IAC7B,iFAAiF;IACjF,QAAQ,CAAC,KAAK,EAAE,SAAS,CAAC;CAC3B,CAAC;AAMF;;;;GAIG;AACH,qBAAa,UAAW,YAAW,UAAU;;gBAM/B,YAAY,EAAE,MAAM,EAAE,YAAY,CAAC,EAAE,MAAM;IAiBvD;;;OAGG;IACI,KAAK,CAAC,EACX,QAAQ,EACR,MAAM,EACN,KAAK,GACN,EAAE,YAAY,GAAG,cAAc,CAAC,WAAW,EAAE,EAAE,IAAI,EAAE,SAAS,CAAC;IA+BhE,CAAC,MAAM,CAAC,OAAO,CAAC,IAAI,IAAI;CAGzB"}
1
+ {"version":3,"file":"changefeed.d.ts","sourceRoot":"","sources":["../src/changefeed.ts"],"names":[],"mappings":"AAAA,OAAO,EAAiB,KAAK,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAI/D,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAErD,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,eAAe,CAAC;AAC9C,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AAExC,MAAM,MAAM,YAAY,GAAG;IACzB,0DAA0D;IAC1D,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,kEAAkE;IAClE,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAC;IAC7B,iFAAiF;IACjF,QAAQ,CAAC,KAAK,EAAE,SAAS,CAAC;CAC3B,CAAC;AAEF,MAAM,MAAM,iBAAiB,GAAG;IAC9B,2EAA2E;IAC3E,QAAQ,CAAC,QAAQ,EAAE,QAAQ,CAAC;IAC5B,mEAAmE;IACnE,QAAQ,CAAC,KAAK,EAAE,KAAK,CAAC;CACvB,CAAC;AAEF;;;GAGG;AACH,qBAAa,UAAW,YAAW,UAAU;;gBAO/B,YAAY,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,KAAK,EAAE,EAAE,iBAAiB;IAcxE;;;OAGG;IACI,KAAK,CAAC,EACX,QAAQ,EACR,MAAM,EACN,KAAK,GACN,EAAE,YAAY,GAAG,cAAc,CAAC,WAAW,EAAE,EAAE,IAAI,EAAE,SAAS,CAAC;IAgChE,CAAC,MAAM,CAAC,OAAO,CAAC,IAAI,IAAI;CAGzB"}
@@ -2,33 +2,27 @@ import { ChangeWatcher } from '@zukhruf/async';
2
2
  import { DataVersion } from '@zukhruf/sqlite-data-version';
3
3
  import { Snapshot } from '@zukhruf/sqlite-snapshot';
4
4
  import { pass, requireWal } from "./pass.js";
5
- import { PositionFile } from "./position-file.js";
6
- import { PositionInMemory } from "./position-in-memory.js";
7
5
  /**
8
6
  * A change event for each changed row of each table of a SQLite database
9
- * that another process owns. With a position path the feed is durable and
10
- * goes on from its last saved batch; without one it observes from now.
7
+ * that another process owns.
11
8
  */
12
9
  export class Changefeed {
13
10
  #path;
14
11
  #version;
15
- #run;
12
+ #position;
13
+ #start;
16
14
  #resources;
17
- constructor(databasePath, positionPath) {
15
+ constructor(databasePath, { position, start }) {
18
16
  using resources = new DisposableStack();
17
+ // The feed owns its position also when the database fails to open.
18
+ this.#position = resources.use(position);
19
+ this.#start = start;
19
20
  this.#path = databasePath;
20
21
  this.#version = resources.use(new DataVersion(databasePath));
21
22
  {
22
23
  using snapshot = new Snapshot(databasePath);
23
24
  requireWal(snapshot);
24
25
  }
25
- if (positionPath === undefined) {
26
- this.#run = () => ({ position: new PositionInMemory(), baseline: true });
27
- }
28
- else {
29
- const file = resources.use(new PositionFile(positionPath));
30
- this.#run = () => ({ position: file, baseline: false });
31
- }
32
26
  this.#resources = resources.move();
33
27
  }
34
28
  /**
@@ -36,8 +30,6 @@ export class Changefeed {
36
30
  * the loop asks for the next one.
37
31
  */
38
32
  async *watch({ interval, signal, clock, }) {
39
- const { position, baseline } = this.#run();
40
- let silent = baseline;
41
33
  // The data version is a local read: a check that takes the whole
42
34
  // interval is stuck.
43
35
  const watcher = new ChangeWatcher([this.#version], {
@@ -52,17 +44,16 @@ export class Changefeed {
52
44
  throw outcome.error;
53
45
  if (outcome?.state !== 'changed')
54
46
  continue;
55
- const { events, update } = await pass(this.#path, position);
56
- if (silent || events.length === 0) {
57
- // A baseline, or a pass with no events, has no batch to deliver.
58
- position.save(update);
59
- silent = false;
47
+ const { events, update } = await pass(this.#path, this.#position, this.#start);
48
+ if (events.length === 0) {
49
+ // A pass with no events, a baseline too, has no batch to deliver.
50
+ this.#position.save(update);
60
51
  continue;
61
52
  }
62
53
  yield events;
63
54
  // The loop may not have applied a batch that an abort stopped.
64
55
  signal.throwIfAborted();
65
- position.save(update);
56
+ this.#position.save(update);
66
57
  }
67
58
  // The poll ends only when the signal aborts.
68
59
  signal.throwIfAborted();
@@ -1 +1 @@
1
- {"version":3,"file":"changefeed.js","sourceRoot":"","sources":["../src/changefeed.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAkB,MAAM,gBAAgB,CAAC;AAC/D,OAAO,EAAE,WAAW,EAAE,MAAM,8BAA8B,CAAC;AAC3D,OAAO,EAAE,QAAQ,EAAE,MAAM,0BAA0B,CAAC;AAGpD,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,MAAM,WAAW,CAAC;AAC7C,OAAO,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClD,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAgB3D;;;;GAIG;AACH,MAAM,OAAO,UAAU;IACZ,KAAK,CAAS;IACd,QAAQ,CAAc;IACtB,IAAI,CAAY;IAChB,UAAU,CAAkB;IAErC,YAAY,YAAoB,EAAE,YAAqB;QACrD,MAAM,SAAS,GAAG,IAAI,eAAe,EAAE,CAAC;QACxC,IAAI,CAAC,KAAK,GAAG,YAAY,CAAC;QAC1B,IAAI,CAAC,QAAQ,GAAG,SAAS,CAAC,GAAG,CAAC,IAAI,WAAW,CAAC,YAAY,CAAC,CAAC,CAAC;QAC7D,CAAC;YACC,MAAM,QAAQ,GAAG,IAAI,QAAQ,CAAC,YAAY,CAAC,CAAC;YAC5C,UAAU,CAAC,QAAQ,CAAC,CAAC;QACvB,CAAC;QACD,IAAI,YAAY,KAAK,SAAS,EAAE,CAAC;YAC/B,IAAI,CAAC,IAAI,GAAG,GAAG,EAAE,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,gBAAgB,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC;QAC3E,CAAC;aAAM,CAAC;YACN,MAAM,IAAI,GAAG,SAAS,CAAC,GAAG,CAAC,IAAI,YAAY,CAAC,YAAY,CAAC,CAAC,CAAC;YAC3D,IAAI,CAAC,IAAI,GAAG,GAAG,EAAE,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,QAAQ,EAAE,KAAK,EAAE,CAAC,CAAC;QAC1D,CAAC;QACD,IAAI,CAAC,UAAU,GAAG,SAAS,CAAC,IAAI,EAAE,CAAC;IACrC,CAAC;IAED;;;OAGG;IACH,KAAK,CAAC,CAAC,KAAK,CAAC,EACX,QAAQ,EACR,MAAM,EACN,KAAK,GACQ;QACb,MAAM,EAAE,QAAQ,EAAE,QAAQ,EAAE,GAAG,IAAI,CAAC,IAAI,EAAE,CAAC;QAC3C,IAAI,MAAM,GAAG,QAAQ,CAAC;QACtB,iEAAiE;QACjE,qBAAqB;QACrB,MAAM,OAAO,GAAG,IAAI,aAAa,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE;YACjD,QAAQ;YACR,OAAO,EAAE,QAAQ;YACjB,KAAK;SACN,CAAC,CAAC;QACH,IAAI,KAAK,EAAE,MAAM,QAAQ,IAAI,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC;YACnD,MAAM,OAAO,GAAG,QAAQ,CAAC,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;YAC5C,kDAAkD;YAClD,IAAI,OAAO,EAAE,KAAK,KAAK,QAAQ;gBAAE,MAAM,OAAO,CAAC,KAAK,CAAC;YACrD,IAAI,OAAO,EAAE,KAAK,KAAK,SAAS;gBAAE,SAAS;YAC3C,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAC;YAC5D,IAAI,MAAM,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBAClC,iEAAiE;gBACjE,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;gBACtB,MAAM,GAAG,KAAK,CAAC;gBACf,SAAS;YACX,CAAC;YACD,MAAM,MAAM,CAAC;YACb,+DAA+D;YAC/D,MAAM,CAAC,cAAc,EAAE,CAAC;YACxB,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxB,CAAC;QACD,6CAA6C;QAC7C,MAAM,CAAC,cAAc,EAAE,CAAC;IAC1B,CAAC;IAED,CAAC,MAAM,CAAC,OAAO,CAAC;QACd,IAAI,CAAC,UAAU,CAAC,OAAO,EAAE,CAAC;IAC5B,CAAC;CACF"}
1
+ {"version":3,"file":"changefeed.js","sourceRoot":"","sources":["../src/changefeed.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAkB,MAAM,gBAAgB,CAAC;AAC/D,OAAO,EAAE,WAAW,EAAE,MAAM,8BAA8B,CAAC;AAC3D,OAAO,EAAE,QAAQ,EAAE,MAAM,0BAA0B,CAAC;AAGpD,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,MAAM,WAAW,CAAC;AAoB7C;;;GAGG;AACH,MAAM,OAAO,UAAU;IACZ,KAAK,CAAS;IACd,QAAQ,CAAc;IACtB,SAAS,CAAW;IACpB,MAAM,CAAQ;IACd,UAAU,CAAkB;IAErC,YAAY,YAAoB,EAAE,EAAE,QAAQ,EAAE,KAAK,EAAqB;QACtE,MAAM,SAAS,GAAG,IAAI,eAAe,EAAE,CAAC;QACxC,mEAAmE;QACnE,IAAI,CAAC,SAAS,GAAG,SAAS,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;QACzC,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC;QACpB,IAAI,CAAC,KAAK,GAAG,YAAY,CAAC;QAC1B,IAAI,CAAC,QAAQ,GAAG,SAAS,CAAC,GAAG,CAAC,IAAI,WAAW,CAAC,YAAY,CAAC,CAAC,CAAC;QAC7D,CAAC;YACC,MAAM,QAAQ,GAAG,IAAI,QAAQ,CAAC,YAAY,CAAC,CAAC;YAC5C,UAAU,CAAC,QAAQ,CAAC,CAAC;QACvB,CAAC;QACD,IAAI,CAAC,UAAU,GAAG,SAAS,CAAC,IAAI,EAAE,CAAC;IACrC,CAAC;IAED;;;OAGG;IACH,KAAK,CAAC,CAAC,KAAK,CAAC,EACX,QAAQ,EACR,MAAM,EACN,KAAK,GACQ;QACb,iEAAiE;QACjE,qBAAqB;QACrB,MAAM,OAAO,GAAG,IAAI,aAAa,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE;YACjD,QAAQ;YACR,OAAO,EAAE,QAAQ;YACjB,KAAK;SACN,CAAC,CAAC;QACH,IAAI,KAAK,EAAE,MAAM,QAAQ,IAAI,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC;YACnD,MAAM,OAAO,GAAG,QAAQ,CAAC,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;YAC5C,kDAAkD;YAClD,IAAI,OAAO,EAAE,KAAK,KAAK,QAAQ;gBAAE,MAAM,OAAO,CAAC,KAAK,CAAC;YACrD,IAAI,OAAO,EAAE,KAAK,KAAK,SAAS;gBAAE,SAAS;YAC3C,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,GAAG,MAAM,IAAI,CACnC,IAAI,CAAC,KAAK,EACV,IAAI,CAAC,SAAS,EACd,IAAI,CAAC,MAAM,CACZ,CAAC;YACF,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBACxB,kEAAkE;gBAClE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;gBAC5B,SAAS;YACX,CAAC;YACD,MAAM,MAAM,CAAC;YACb,+DAA+D;YAC/D,MAAM,CAAC,cAAc,EAAE,CAAC;YACxB,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QAC9B,CAAC;QACD,6CAA6C;QAC7C,MAAM,CAAC,cAAc,EAAE,CAAC;IAC1B,CAAC;IAED,CAAC,MAAM,CAAC,OAAO,CAAC;QACd,IAAI,CAAC,UAAU,CAAC,OAAO,EAAE,CAAC;IAC5B,CAAC;CACF"}
package/dist/index.d.ts CHANGED
@@ -1,5 +1,13 @@
1
+ export { Baseline } from './baseline.ts';
1
2
  export type { ChangeEvent, Key, Row } from './change-event.ts';
2
- export { Changefeed, type WatchOptions } from './changefeed.ts';
3
+ export { Changefeed, type ChangefeedOptions, type WatchOptions, } from './changefeed.ts';
3
4
  export { JournalModeError } from './journal-mode-error.ts';
5
+ export { PositionFile } from './position-file.ts';
6
+ export { PositionInMemory } from './position-in-memory.ts';
7
+ export type { Position, PositionUpdate, TablePosition } from './position.ts';
8
+ export { ReadEveryRow } from './read-every-row.ts';
9
+ export type { Start } from './start.ts';
10
+ export type { TableChanges } from './table-changes.ts';
11
+ export type { KeyedRow, Table } from './tables.ts';
4
12
  export { UnavailableError } from '@zukhruf/sqlite-open';
5
13
  //# sourceMappingURL=index.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,YAAY,EAAE,WAAW,EAAE,GAAG,EAAE,GAAG,EAAE,MAAM,mBAAmB,CAAC;AAC/D,OAAO,EAAE,UAAU,EAAE,KAAK,YAAY,EAAE,MAAM,iBAAiB,CAAC;AAChE,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAC3D,OAAO,EAAE,gBAAgB,EAAE,MAAM,sBAAsB,CAAC"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,eAAe,CAAC;AACzC,YAAY,EAAE,WAAW,EAAE,GAAG,EAAE,GAAG,EAAE,MAAM,mBAAmB,CAAC;AAC/D,OAAO,EACL,UAAU,EACV,KAAK,iBAAiB,EACtB,KAAK,YAAY,GAClB,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAC3D,OAAO,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClD,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAC3D,YAAY,EAAE,QAAQ,EAAE,cAAc,EAAE,aAAa,EAAE,MAAM,eAAe,CAAC;AAC7E,OAAO,EAAE,YAAY,EAAE,MAAM,qBAAqB,CAAC;AACnD,YAAY,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACxC,YAAY,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AACvD,YAAY,EAAE,QAAQ,EAAE,KAAK,EAAE,MAAM,aAAa,CAAC;AACnD,OAAO,EAAE,gBAAgB,EAAE,MAAM,sBAAsB,CAAC"}
package/dist/index.js CHANGED
@@ -1,4 +1,8 @@
1
- export { Changefeed } from "./changefeed.js";
1
+ export { Baseline } from "./baseline.js";
2
+ export { Changefeed, } from "./changefeed.js";
2
3
  export { JournalModeError } from "./journal-mode-error.js";
4
+ export { PositionFile } from "./position-file.js";
5
+ export { PositionInMemory } from "./position-in-memory.js";
6
+ export { ReadEveryRow } from "./read-every-row.js";
3
7
  export { UnavailableError } from '@zukhruf/sqlite-open';
4
8
  //# sourceMappingURL=index.js.map
package/dist/index.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,UAAU,EAAqB,MAAM,iBAAiB,CAAC;AAChE,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAC3D,OAAO,EAAE,gBAAgB,EAAE,MAAM,sBAAsB,CAAC"}
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,eAAe,CAAC;AAEzC,OAAO,EACL,UAAU,GAGX,MAAM,iBAAiB,CAAC;AACzB,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAC3D,OAAO,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClD,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAE3D,OAAO,EAAE,YAAY,EAAE,MAAM,qBAAqB,CAAC;AAInD,OAAO,EAAE,gBAAgB,EAAE,MAAM,sBAAsB,CAAC"}
package/dist/pass.d.ts CHANGED
@@ -1,7 +1,8 @@
1
1
  import { Snapshot } from '@zukhruf/sqlite-snapshot';
2
2
  import type { ChangeEvent } from './change-event.ts';
3
3
  import type { Position, PositionUpdate } from './position.ts';
4
- export declare function pass(path: string, position: Position): Promise<{
4
+ import type { Start } from './start.ts';
5
+ export declare function pass(path: string, position: Position, start: Start): Promise<{
5
6
  events: ChangeEvent[];
6
7
  update: PositionUpdate;
7
8
  }>;
@@ -1 +1 @@
1
- {"version":3,"file":"pass.d.ts","sourceRoot":"","sources":["../src/pass.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,0BAA0B,CAAC;AAEpD,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAErD,OAAO,KAAK,EAAE,QAAQ,EAAE,cAAc,EAAiB,MAAM,eAAe,CAAC;AAM7E,wBAAsB,IAAI,CACxB,IAAI,EAAE,MAAM,EACZ,QAAQ,EAAE,QAAQ,GACjB,OAAO,CAAC;IAAE,MAAM,EAAE,WAAW,EAAE,CAAC;IAAC,MAAM,EAAE,cAAc,CAAA;CAAE,CAAC,CA6B5D;AAGD,wBAAgB,UAAU,CAAC,QAAQ,EAAE,QAAQ,GAAG,IAAI,CAKnD"}
1
+ {"version":3,"file":"pass.d.ts","sourceRoot":"","sources":["../src/pass.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,0BAA0B,CAAC;AAEpD,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAErD,OAAO,KAAK,EAAE,QAAQ,EAAE,cAAc,EAAiB,MAAM,eAAe,CAAC;AAC7E,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AAOxC,wBAAsB,IAAI,CACxB,IAAI,EAAE,MAAM,EACZ,QAAQ,EAAE,QAAQ,EAClB,KAAK,EAAE,KAAK,GACX,OAAO,CAAC;IAAE,MAAM,EAAE,WAAW,EAAE,CAAC;IAAC,MAAM,EAAE,cAAc,CAAA;CAAE,CAAC,CA+B5D;AAGD,wBAAgB,UAAU,CAAC,QAAQ,EAAE,QAAQ,GAAG,IAAI,CAKnD"}
package/dist/pass.js CHANGED
@@ -3,10 +3,12 @@ import { JournalModeError } from "./journal-mode-error.js";
3
3
  import { tableChanges } from "./table-changes.js";
4
4
  import { tablesOf } from "./tables.js";
5
5
  // A pass reads each table in one snapshot, and compares it with the
6
- // position. It holds the snapshot only while it reads.
7
- export async function pass(path, position) {
6
+ // position. It holds the snapshot only while it reads. Until the position is
7
+ // saved once, the start gives what each table gives.
8
+ export async function pass(path, position, start) {
8
9
  using snapshot = new Snapshot(path);
9
10
  requireWal(snapshot);
11
+ const first = !position.started();
10
12
  const known = position.keyColumns();
11
13
  const events = [];
12
14
  const tables = new Map();
@@ -17,8 +19,13 @@ export async function pass(path, position) {
17
19
  JSON.stringify(keyColumns) === JSON.stringify(table.keyColumns);
18
20
  if (keyColumns !== undefined && !same)
19
21
  events.push({ table: table.name, op: 'truncate' });
20
- const changes = await tableChanges(table, same ? position.state(table.name) : null);
21
- events.push(...changes.events);
22
+ const changes = first
23
+ ? await start.table(table)
24
+ : await tableChanges(table, same ? position.state(table.name) : null);
25
+ // One push for each event: a spread of a large table's events into one
26
+ // call overflows the stack.
27
+ for (const event of changes.events)
28
+ events.push(event);
22
29
  if (!same || changes.events.length > 0)
23
30
  tables.set(table.name, {
24
31
  keyColumns: table.keyColumns,
package/dist/pass.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"pass.js","sourceRoot":"","sources":["../src/pass.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,0BAA0B,CAAC;AAGpD,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAE3D,OAAO,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClD,OAAO,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAEvC,oEAAoE;AACpE,uDAAuD;AACvD,MAAM,CAAC,KAAK,UAAU,IAAI,CACxB,IAAY,EACZ,QAAkB;IAElB,MAAM,QAAQ,GAAG,IAAI,QAAQ,CAAC,IAAI,CAAC,CAAC;IACpC,UAAU,CAAC,QAAQ,CAAC,CAAC;IACrB,MAAM,KAAK,GAAG,QAAQ,CAAC,UAAU,EAAE,CAAC;IACpC,MAAM,MAAM,GAAkB,EAAE,CAAC;IACjC,MAAM,MAAM,GAAG,IAAI,GAAG,EAAyB,CAAC;IAChD,MAAM,OAAO,GAAG,QAAQ,CAAC,QAAQ,CAAC,CAAC;IACnC,KAAK,MAAM,KAAK,IAAI,OAAO,EAAE,CAAC;QAC5B,MAAM,UAAU,GAAG,KAAK,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACzC,MAAM,IAAI,GACR,UAAU,KAAK,SAAS;YACxB,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,KAAK,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;QAClE,IAAI,UAAU,KAAK,SAAS,IAAI,CAAC,IAAI;YACnC,MAAM,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,IAAI,EAAE,EAAE,EAAE,UAAU,EAAE,CAAC,CAAC;QACrD,MAAM,OAAO,GAAG,MAAM,YAAY,CAChC,KAAK,EACL,IAAI,CAAC,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CACzC,CAAC;QACF,MAAM,CAAC,IAAI,CAAC,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC;QAC/B,IAAI,CAAC,IAAI,IAAI,OAAO,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC;YACpC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,EAAE;gBACrB,UAAU,EAAE,KAAK,CAAC,UAAU;gBAC5B,KAAK,EAAE,OAAO,CAAC,KAAK;aACrB,CAAC,CAAC;IACP,CAAC;IACD,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC;IAC1D,MAAM,OAAO,GAAG,CAAC,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC;IACvE,KAAK,MAAM,KAAK,IAAI,OAAO;QAAE,MAAM,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,EAAE,EAAE,UAAU,EAAE,CAAC,CAAC;IACpE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,EAAE,MAAM,EAAE,OAAO,EAAE,EAAE,CAAC;AACjD,CAAC;AAED,0EAA0E;AAC1E,MAAM,UAAU,UAAU,CAAC,QAAkB;IAC3C,MAAM,CAAC,GAAG,CAAC,GAAG,QAAQ,CAAC,GAAG,CAAC,qBAAqB,CAAC,CAAC;IAClD,IAAI,GAAG,KAAK,SAAS;QAAE,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;IAC9E,MAAM,IAAI,GAAG,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC;IACtC,IAAI,IAAI,KAAK,KAAK;QAAE,MAAM,IAAI,gBAAgB,CAAC,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;AACtE,CAAC"}
1
+ {"version":3,"file":"pass.js","sourceRoot":"","sources":["../src/pass.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,0BAA0B,CAAC;AAGpD,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAG3D,OAAO,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAClD,OAAO,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAEvC,oEAAoE;AACpE,6EAA6E;AAC7E,qDAAqD;AACrD,MAAM,CAAC,KAAK,UAAU,IAAI,CACxB,IAAY,EACZ,QAAkB,EAClB,KAAY;IAEZ,MAAM,QAAQ,GAAG,IAAI,QAAQ,CAAC,IAAI,CAAC,CAAC;IACpC,UAAU,CAAC,QAAQ,CAAC,CAAC;IACrB,MAAM,KAAK,GAAG,CAAC,QAAQ,CAAC,OAAO,EAAE,CAAC;IAClC,MAAM,KAAK,GAAG,QAAQ,CAAC,UAAU,EAAE,CAAC;IACpC,MAAM,MAAM,GAAkB,EAAE,CAAC;IACjC,MAAM,MAAM,GAAG,IAAI,GAAG,EAAyB,CAAC;IAChD,MAAM,OAAO,GAAG,QAAQ,CAAC,QAAQ,CAAC,CAAC;IACnC,KAAK,MAAM,KAAK,IAAI,OAAO,EAAE,CAAC;QAC5B,MAAM,UAAU,GAAG,KAAK,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACzC,MAAM,IAAI,GACR,UAAU,KAAK,SAAS;YACxB,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,KAAK,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;QAClE,IAAI,UAAU,KAAK,SAAS,IAAI,CAAC,IAAI;YACnC,MAAM,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,IAAI,EAAE,EAAE,EAAE,UAAU,EAAE,CAAC,CAAC;QACrD,MAAM,OAAO,GAAG,KAAK;YACnB,CAAC,CAAC,MAAM,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC;YAC1B,CAAC,CAAC,MAAM,YAAY,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QACxE,uEAAuE;QACvE,4BAA4B;QAC5B,KAAK,MAAM,KAAK,IAAI,OAAO,CAAC,MAAM;YAAE,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;QACvD,IAAI,CAAC,IAAI,IAAI,OAAO,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC;YACpC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,EAAE;gBACrB,UAAU,EAAE,KAAK,CAAC,UAAU;gBAC5B,KAAK,EAAE,OAAO,CAAC,KAAK;aACrB,CAAC,CAAC;IACP,CAAC;IACD,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC;IAC1D,MAAM,OAAO,GAAG,CAAC,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC;IACvE,KAAK,MAAM,KAAK,IAAI,OAAO;QAAE,MAAM,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,EAAE,EAAE,UAAU,EAAE,CAAC,CAAC;IACpE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,EAAE,MAAM,EAAE,OAAO,EAAE,EAAE,CAAC;AACjD,CAAC;AAED,0EAA0E;AAC1E,MAAM,UAAU,UAAU,CAAC,QAAkB;IAC3C,MAAM,CAAC,GAAG,CAAC,GAAG,QAAQ,CAAC,GAAG,CAAC,qBAAqB,CAAC,CAAC;IAClD,IAAI,GAAG,KAAK,SAAS;QAAE,MAAM,IAAI,SAAS,CAAC,iCAAiC,CAAC,CAAC;IAC9E,MAAM,IAAI,GAAG,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC;IACtC,IAAI,IAAI,KAAK,KAAK;QAAE,MAAM,IAAI,gBAAgB,CAAC,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;AACtE,CAAC"}
@@ -1,8 +1,15 @@
1
1
  import type { SnapshotState } from '@zukhruf/snapshot-differential';
2
2
  import type { Position, PositionUpdate } from './position.ts';
3
- export declare class PositionFile implements Position, Disposable {
3
+ /**
4
+ * A position in a SQLite file of the feed, at the path that the host gives:
5
+ * the key columns of each table and one fingerprint for each (table, key). A
6
+ * new run of a feed goes on from it. A missing or empty file is a position
7
+ * that no pass saved.
8
+ */
9
+ export declare class PositionFile implements Position {
4
10
  #private;
5
11
  constructor(path: string);
12
+ started(): boolean;
6
13
  keyColumns(): ReadonlyMap<string, readonly string[]>;
7
14
  state(table: string): SnapshotState;
8
15
  save({ tables, dropped }: PositionUpdate): void;
@@ -1 +1 @@
1
- {"version":3,"file":"position-file.d.ts","sourceRoot":"","sources":["../src/position-file.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,gCAAgC,CAAC;AAIpE,OAAO,KAAK,EAAE,QAAQ,EAAE,cAAc,EAAE,MAAM,eAAe,CAAC;AAK9D,qBAAa,YAAa,YAAW,QAAQ,EAAE,UAAU;;gBAG3C,IAAI,EAAE,MAAM;IAqBxB,UAAU,IAAI,WAAW,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC;IAYpD,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,aAAa;IAcnC,IAAI,CAAC,EAAE,MAAM,EAAE,OAAO,EAAE,EAAE,cAAc,GAAG,IAAI;IA8B/C,CAAC,MAAM,CAAC,OAAO,CAAC,IAAI,IAAI;CAGzB"}
1
+ {"version":3,"file":"position-file.d.ts","sourceRoot":"","sources":["../src/position-file.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,gCAAgC,CAAC;AAIpE,OAAO,KAAK,EAAE,QAAQ,EAAE,cAAc,EAAE,MAAM,eAAe,CAAC;AAE9D;;;;;GAKG;AACH,qBAAa,YAAa,YAAW,QAAQ;;gBAG/B,IAAI,EAAE,MAAM;IAsBxB,OAAO,IAAI,OAAO;IAMlB,UAAU,IAAI,WAAW,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC;IAYpD,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,aAAa;IAcnC,IAAI,CAAC,EAAE,MAAM,EAAE,OAAO,EAAE,EAAE,cAAc,GAAG,IAAI;IA+B/C,CAAC,MAAM,CAAC,OAAO,CAAC,IAAI,IAAI;CAGzB"}
@@ -1,8 +1,11 @@
1
1
  import { open } from '@zukhruf/sqlite-open';
2
2
  import { transaction } from '@zukhruf/sqlite-transaction';
3
- // The position of a durable feed: a SQLite file of the feed, with the key
4
- // columns of each table and one fingerprint for each (table, key). A missing
5
- // or empty file is a position with no tables.
3
+ /**
4
+ * A position in a SQLite file of the feed, at the path that the host gives:
5
+ * the key columns of each table and one fingerprint for each (table, key). A
6
+ * new run of a feed goes on from it. A missing or empty file is a position
7
+ * that no pass saved.
8
+ */
6
9
  export class PositionFile {
7
10
  #database;
8
11
  constructor(path) {
@@ -26,6 +29,10 @@ export class PositionFile {
26
29
  throw cause;
27
30
  }
28
31
  }
32
+ // The user version of the file is 1 after the first save.
33
+ started() {
34
+ return (this.#database.prepare('PRAGMA user_version').get()?.user_version !== 0);
35
+ }
29
36
  keyColumns() {
30
37
  return new Map(this.#database
31
38
  .prepare('SELECT name, key_columns FROM tables')
@@ -59,6 +66,7 @@ export class PositionFile {
59
66
  for (const [key, entry] of Object.entries(state.snapshot))
60
67
  keepRow.run(table, key, fingerprintOf(entry));
61
68
  }
69
+ this.#database.exec('PRAGMA user_version = 1');
62
70
  }, { mode: 'immediate' });
63
71
  }
64
72
  [Symbol.dispose]() {
@@ -1 +1 @@
1
- {"version":3,"file":"position-file.js","sourceRoot":"","sources":["../src/position-file.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,IAAI,EAAE,MAAM,sBAAsB,CAAC;AAC5C,OAAO,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAI1D,0EAA0E;AAC1E,6EAA6E;AAC7E,8CAA8C;AAC9C,MAAM,OAAO,YAAY;IACd,SAAS,CAAe;IAEjC,YAAY,IAAY;QACtB,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC;QAC5B,IAAI,CAAC;YACH,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;;;;;;;;;;;OAWnB,CAAC,CAAC;QACL,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,CAAC;YACvB,MAAM,KAAK,CAAC;QACd,CAAC;IACH,CAAC;IAED,UAAU;QACR,OAAO,IAAI,GAAG,CACZ,IAAI,CAAC,SAAS;aACX,OAAO,CAAC,sCAAsC,CAAC;aAC/C,GAAG,EAAE;aACL,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC;YAC9B,MAAM,CAAC,IAAI,CAAC;YACZ,SAAS,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;SAC/B,CAAC,CACL,CAAC;IACJ,CAAC;IAED,KAAK,CAAC,KAAa;QACjB,OAAO;YACL,QAAQ,EAAE,MAAM,CAAC,WAAW,CAC1B,IAAI,CAAC,SAAS;iBACX,OAAO,CACN,gEAAgE,CACjE;iBACA,GAAG,CAAC,KAAK,CAAC;iBACV,GAAG,CAAC,CAAC,EAAE,GAAG,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC,WAAW,CAAC,CAAC,CAAC,CACrE;SACF,CAAC;IACJ,CAAC;IAED,iDAAiD;IACjD,IAAI,CAAC,EAAE,MAAM,EAAE,OAAO,EAAkB;QACtC,MAAM,WAAW,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACxC,mCAAmC,CACpC,CAAC;QACF,MAAM,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACvC,+CAA+C,CAChD,CAAC;QACF,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACtC,sDAAsD,CACvD,CAAC;QACF,MAAM,OAAO,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACpC,0EAA0E,CAC3E,CAAC;QACF,WAAW,CACT,IAAI,CAAC,SAAS,EACd,GAAG,EAAE;YACH,KAAK,MAAM,KAAK,IAAI,CAAC,GAAG,OAAO,EAAE,GAAG,MAAM,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC;gBACnD,WAAW,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;gBACvB,UAAU,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;YACxB,CAAC;YACD,KAAK,MAAM,CAAC,KAAK,EAAE,EAAE,UAAU,EAAE,KAAK,EAAE,CAAC,IAAI,MAAM,EAAE,CAAC;gBACpD,SAAS,CAAC,GAAG,CAAC,KAAK,EAAE,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,CAAC,CAAC;gBACjD,KAAK,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,QAAQ,CAAC;oBACvD,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,GAAG,EAAE,aAAa,CAAC,KAAK,CAAC,CAAC,CAAC;YAClD,CAAC;QACH,CAAC,EACD,EAAE,IAAI,EAAE,WAAW,EAAE,CACtB,CAAC;IACJ,CAAC;IAED,CAAC,MAAM,CAAC,OAAO,CAAC;QACd,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,CAAC;IACzB,CAAC;CACF;AAED,SAAS,SAAS,CAAC,IAAY;IAC7B,MAAM,OAAO,GAAY,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC1C,IACE,CAAC,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC;QACvB,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,OAAO,MAAM,KAAK,QAAQ,CAAC;QAEtD,MAAM,IAAI,SAAS,CAAC,2BAA2B,IAAI,iBAAiB,CAAC,CAAC;IACxE,OAAO,OAAO,CAAC;AACjB,CAAC;AAED,mEAAmE;AACnE,qBAAqB;AACrB,SAAS,aAAa,CAAC,KAAwC;IAC7D,IAAI,OAAO,KAAK,KAAK,QAAQ;QAC3B,MAAM,IAAI,SAAS,CAAC,8BAA8B,CAAC,CAAC;IACtD,OAAO,KAAK,CAAC;AACf,CAAC"}
1
+ {"version":3,"file":"position-file.js","sourceRoot":"","sources":["../src/position-file.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,IAAI,EAAE,MAAM,sBAAsB,CAAC;AAC5C,OAAO,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAI1D;;;;;GAKG;AACH,MAAM,OAAO,YAAY;IACd,SAAS,CAAe;IAEjC,YAAY,IAAY;QACtB,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC;QAC5B,IAAI,CAAC;YACH,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;;;;;;;;;;;OAWnB,CAAC,CAAC;QACL,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,CAAC;YACvB,MAAM,KAAK,CAAC;QACd,CAAC;IACH,CAAC;IAED,0DAA0D;IAC1D,OAAO;QACL,OAAO,CACL,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,qBAAqB,CAAC,CAAC,GAAG,EAAE,EAAE,YAAY,KAAK,CAAC,CACxE,CAAC;IACJ,CAAC;IAED,UAAU;QACR,OAAO,IAAI,GAAG,CACZ,IAAI,CAAC,SAAS;aACX,OAAO,CAAC,sCAAsC,CAAC;aAC/C,GAAG,EAAE;aACL,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC;YAC9B,MAAM,CAAC,IAAI,CAAC;YACZ,SAAS,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;SAC/B,CAAC,CACL,CAAC;IACJ,CAAC;IAED,KAAK,CAAC,KAAa;QACjB,OAAO;YACL,QAAQ,EAAE,MAAM,CAAC,WAAW,CAC1B,IAAI,CAAC,SAAS;iBACX,OAAO,CACN,gEAAgE,CACjE;iBACA,GAAG,CAAC,KAAK,CAAC;iBACV,GAAG,CAAC,CAAC,EAAE,GAAG,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC,WAAW,CAAC,CAAC,CAAC,CACrE;SACF,CAAC;IACJ,CAAC;IAED,iDAAiD;IACjD,IAAI,CAAC,EAAE,MAAM,EAAE,OAAO,EAAkB;QACtC,MAAM,WAAW,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACxC,mCAAmC,CACpC,CAAC;QACF,MAAM,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACvC,+CAA+C,CAChD,CAAC;QACF,MAAM,SAAS,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACtC,sDAAsD,CACvD,CAAC;QACF,MAAM,OAAO,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CACpC,0EAA0E,CAC3E,CAAC;QACF,WAAW,CACT,IAAI,CAAC,SAAS,EACd,GAAG,EAAE;YACH,KAAK,MAAM,KAAK,IAAI,CAAC,GAAG,OAAO,EAAE,GAAG,MAAM,CAAC,IAAI,EAAE,CAAC,EAAE,CAAC;gBACnD,WAAW,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;gBACvB,UAAU,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;YACxB,CAAC;YACD,KAAK,MAAM,CAAC,KAAK,EAAE,EAAE,UAAU,EAAE,KAAK,EAAE,CAAC,IAAI,MAAM,EAAE,CAAC;gBACpD,SAAS,CAAC,GAAG,CAAC,KAAK,EAAE,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,CAAC,CAAC;gBACjD,KAAK,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,KAAK,CAAC,QAAQ,CAAC;oBACvD,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,GAAG,EAAE,aAAa,CAAC,KAAK,CAAC,CAAC,CAAC;YAClD,CAAC;YACD,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,yBAAyB,CAAC,CAAC;QACjD,CAAC,EACD,EAAE,IAAI,EAAE,WAAW,EAAE,CACtB,CAAC;IACJ,CAAC;IAED,CAAC,MAAM,CAAC,OAAO,CAAC;QACd,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,CAAC;IACzB,CAAC;CACF;AAED,SAAS,SAAS,CAAC,IAAY;IAC7B,MAAM,OAAO,GAAY,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC1C,IACE,CAAC,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC;QACvB,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,OAAO,MAAM,KAAK,QAAQ,CAAC;QAEtD,MAAM,IAAI,SAAS,CAAC,2BAA2B,IAAI,iBAAiB,CAAC,CAAC;IACxE,OAAO,OAAO,CAAC;AACjB,CAAC;AAED,mEAAmE;AACnE,qBAAqB;AACrB,SAAS,aAAa,CAAC,KAAwC;IAC7D,IAAI,OAAO,KAAK,KAAK,QAAQ;QAC3B,MAAM,IAAI,SAAS,CAAC,8BAA8B,CAAC,CAAC;IACtD,OAAO,KAAK,CAAC;AACf,CAAC"}
@@ -1,9 +1,12 @@
1
1
  import type { SnapshotState } from '@zukhruf/snapshot-differential';
2
2
  import type { Position, PositionUpdate } from './position.ts';
3
+ /** A position in memory: it lasts as long as its feed. */
3
4
  export declare class PositionInMemory implements Position {
4
5
  #private;
6
+ started(): boolean;
5
7
  keyColumns(): ReadonlyMap<string, readonly string[]>;
6
8
  state(table: string): SnapshotState;
7
9
  save({ tables, dropped }: PositionUpdate): void;
10
+ [Symbol.dispose](): void;
8
11
  }
9
12
  //# sourceMappingURL=position-in-memory.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"position-in-memory.d.ts","sourceRoot":"","sources":["../src/position-in-memory.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,gCAAgC,CAAC;AAEpE,OAAO,KAAK,EAAE,QAAQ,EAAE,cAAc,EAAiB,MAAM,eAAe,CAAC;AAG7E,qBAAa,gBAAiB,YAAW,QAAQ;;IAG/C,UAAU,IAAI,WAAW,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC;IAMpD,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,aAAa;IAOnC,IAAI,CAAC,EAAE,MAAM,EAAE,OAAO,EAAE,EAAE,cAAc,GAAG,IAAI;CAIhD"}
1
+ {"version":3,"file":"position-in-memory.d.ts","sourceRoot":"","sources":["../src/position-in-memory.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,gCAAgC,CAAC;AAEpE,OAAO,KAAK,EAAE,QAAQ,EAAE,cAAc,EAAiB,MAAM,eAAe,CAAC;AAQ7E,0DAA0D;AAC1D,qBAAa,gBAAiB,YAAW,QAAQ;;IAG/C,OAAO,IAAI,OAAO;IAIlB,UAAU,IAAI,WAAW,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC;IAUpD,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,aAAa;IASnC,IAAI,CAAC,EAAE,MAAM,EAAE,OAAO,EAAE,EAAE,cAAc,GAAG,IAAI;IAO/C,CAAC,MAAM,CAAC,OAAO,CAAC,IAAI,IAAI;CACzB"}
@@ -1,20 +1,33 @@
1
- // The position of an observing feed: in memory, for one run.
1
+ /** A position in memory: it lasts as long as its feed. */
2
2
  export class PositionInMemory {
3
- #tables = new Map();
3
+ #phase = { saved: false };
4
+ started() {
5
+ return this.#phase.saved;
6
+ }
4
7
  keyColumns() {
5
- return new Map([...this.#tables].map(([table, { keyColumns }]) => [table, keyColumns]));
8
+ if (!this.#phase.saved)
9
+ return new Map();
10
+ return new Map([...this.#phase.tables].map(([table, { keyColumns }]) => [
11
+ table,
12
+ keyColumns,
13
+ ]));
6
14
  }
7
15
  state(table) {
8
- const position = this.#tables.get(table);
16
+ const position = this.#phase.saved
17
+ ? this.#phase.tables.get(table)
18
+ : undefined;
9
19
  if (position === undefined)
10
20
  throw new RangeError(`The position has no table ${table}`);
11
21
  return position.state;
12
22
  }
13
23
  save({ tables, dropped }) {
24
+ const kept = this.#phase.saved ? this.#phase.tables : new Map();
14
25
  for (const table of dropped)
15
- this.#tables.delete(table);
26
+ kept.delete(table);
16
27
  for (const [table, position] of tables)
17
- this.#tables.set(table, position);
28
+ kept.set(table, position);
29
+ this.#phase = { saved: true, tables: kept };
18
30
  }
31
+ [Symbol.dispose]() { }
19
32
  }
20
33
  //# sourceMappingURL=position-in-memory.js.map
@@ -1 +1 @@
1
- {"version":3,"file":"position-in-memory.js","sourceRoot":"","sources":["../src/position-in-memory.ts"],"names":[],"mappings":"AAIA,6DAA6D;AAC7D,MAAM,OAAO,gBAAgB;IAClB,OAAO,GAAG,IAAI,GAAG,EAAyB,CAAC;IAEpD,UAAU;QACR,OAAO,IAAI,GAAG,CACZ,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,EAAE,EAAE,UAAU,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,EAAE,UAAU,CAAC,CAAC,CACxE,CAAC;IACJ,CAAC;IAED,KAAK,CAAC,KAAa;QACjB,MAAM,QAAQ,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;QACzC,IAAI,QAAQ,KAAK,SAAS;YACxB,MAAM,IAAI,UAAU,CAAC,6BAA6B,KAAK,EAAE,CAAC,CAAC;QAC7D,OAAO,QAAQ,CAAC,KAAK,CAAC;IACxB,CAAC;IAED,IAAI,CAAC,EAAE,MAAM,EAAE,OAAO,EAAkB;QACtC,KAAK,MAAM,KAAK,IAAI,OAAO;YAAE,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACxD,KAAK,MAAM,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,MAAM;YAAE,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAC;IAC5E,CAAC;CACF"}
1
+ {"version":3,"file":"position-in-memory.js","sourceRoot":"","sources":["../src/position-in-memory.ts"],"names":[],"mappings":"AAUA,0DAA0D;AAC1D,MAAM,OAAO,gBAAgB;IAC3B,MAAM,GAAU,EAAE,KAAK,EAAE,KAAK,EAAE,CAAC;IAEjC,OAAO;QACL,OAAO,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC;IAC3B,CAAC;IAED,UAAU;QACR,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,KAAK;YAAE,OAAO,IAAI,GAAG,EAAE,CAAC;QACzC,OAAO,IAAI,GAAG,CACZ,CAAC,GAAG,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,EAAE,EAAE,UAAU,EAAE,CAAC,EAAE,EAAE,CAAC;YACvD,KAAK;YACL,UAAU;SACX,CAAC,CACH,CAAC;IACJ,CAAC;IAED,KAAK,CAAC,KAAa;QACjB,MAAM,QAAQ,GAAG,IAAI,CAAC,MAAM,CAAC,KAAK;YAChC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC;YAC/B,CAAC,CAAC,SAAS,CAAC;QACd,IAAI,QAAQ,KAAK,SAAS;YACxB,MAAM,IAAI,UAAU,CAAC,6BAA6B,KAAK,EAAE,CAAC,CAAC;QAC7D,OAAO,QAAQ,CAAC,KAAK,CAAC;IACxB,CAAC;IAED,IAAI,CAAC,EAAE,MAAM,EAAE,OAAO,EAAkB;QACtC,MAAM,IAAI,GAAG,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,GAAG,EAAE,CAAC;QAChE,KAAK,MAAM,KAAK,IAAI,OAAO;YAAE,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QAChD,KAAK,MAAM,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,MAAM;YAAE,IAAI,CAAC,GAAG,CAAC,KAAK,EAAE,QAAQ,CAAC,CAAC;QAClE,IAAI,CAAC,MAAM,GAAG,EAAE,KAAK,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC;IAC9C,CAAC;IAED,CAAC,MAAM,CAAC,OAAO,CAAC,KAAU,CAAC;CAC5B"}
@@ -1,15 +1,33 @@
1
1
  import type { SnapshotState } from '@zukhruf/snapshot-differential';
2
+ /**
3
+ * What the feed saw of one table: its key columns, and the snapshot of the
4
+ * differential, one fingerprint for each key.
5
+ */
2
6
  export type TablePosition = {
3
7
  readonly keyColumns: readonly string[];
4
8
  readonly state: SnapshotState;
5
9
  };
10
+ /**
11
+ * What a pass changes in the position: the tables it saw differently, and the
12
+ * tables the owner dropped.
13
+ */
6
14
  export type PositionUpdate = {
7
15
  readonly tables: ReadonlyMap<string, TablePosition>;
8
16
  readonly dropped: readonly string[];
9
17
  };
10
- export interface Position {
18
+ /**
19
+ * Where the feed keeps what it saw at its last saved batch. The feed owns the
20
+ * position that it gets, and disposes it. A new position is a class that
21
+ * implements this interface.
22
+ */
23
+ export interface Position extends Disposable {
24
+ /** Whether a pass saved this position before. The start applies until one did. */
25
+ started(): boolean;
26
+ /** The key columns of each table that the position has. */
11
27
  keyColumns(): ReadonlyMap<string, readonly string[]>;
28
+ /** The snapshot of one table that the position has, for the differential. */
12
29
  state(table: string): SnapshotState;
30
+ /** Keeps the update of one batch, all of it or none of it. */
13
31
  save(update: PositionUpdate): void;
14
32
  }
15
33
  //# sourceMappingURL=position.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"position.d.ts","sourceRoot":"","sources":["../src/position.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,gCAAgC,CAAC;AAIpE,MAAM,MAAM,aAAa,GAAG;IAC1B,QAAQ,CAAC,UAAU,EAAE,SAAS,MAAM,EAAE,CAAC;IACvC,QAAQ,CAAC,KAAK,EAAE,aAAa,CAAC;CAC/B,CAAC;AAIF,MAAM,MAAM,cAAc,GAAG;IAC3B,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAC,MAAM,EAAE,aAAa,CAAC,CAAC;IACpD,QAAQ,CAAC,OAAO,EAAE,SAAS,MAAM,EAAE,CAAC;CACrC,CAAC;AAGF,MAAM,WAAW,QAAQ;IAEvB,UAAU,IAAI,WAAW,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC,CAAC;IAErD,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,aAAa,CAAC;IACpC,IAAI,CAAC,MAAM,EAAE,cAAc,GAAG,IAAI,CAAC;CACpC"}
1
+ {"version":3,"file":"position.d.ts","sourceRoot":"","sources":["../src/position.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,gCAAgC,CAAC;AAEpE;;;GAGG;AACH,MAAM,MAAM,aAAa,GAAG;IAC1B,QAAQ,CAAC,UAAU,EAAE,SAAS,MAAM,EAAE,CAAC;IACvC,QAAQ,CAAC,KAAK,EAAE,aAAa,CAAC;CAC/B,CAAC;AAEF;;;GAGG;AACH,MAAM,MAAM,cAAc,GAAG;IAC3B,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAC,MAAM,EAAE,aAAa,CAAC,CAAC;IACpD,QAAQ,CAAC,OAAO,EAAE,SAAS,MAAM,EAAE,CAAC;CACrC,CAAC;AAEF;;;;GAIG;AACH,MAAM,WAAW,QAAS,SAAQ,UAAU;IAC1C,kFAAkF;IAClF,OAAO,IAAI,OAAO,CAAC;IACnB,2DAA2D;IAC3D,UAAU,IAAI,WAAW,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC,CAAC;IACrD,6EAA6E;IAC7E,KAAK,CAAC,KAAK,EAAE,MAAM,GAAG,aAAa,CAAC;IACpC,8DAA8D;IAC9D,IAAI,CAAC,MAAM,EAAE,cAAc,GAAG,IAAI,CAAC;CACpC"}
@@ -0,0 +1,8 @@
1
+ import type { Start } from './start.ts';
2
+ import { type TableChanges } from './table-changes.ts';
3
+ import type { Table } from './tables.ts';
4
+ /** The first pass gives a read for each row of each table. */
5
+ export declare class ReadEveryRow implements Start {
6
+ table(table: Table): Promise<TableChanges>;
7
+ }
8
+ //# sourceMappingURL=read-every-row.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"read-every-row.d.ts","sourceRoot":"","sources":["../src/read-every-row.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACxC,OAAO,EAAE,KAAK,YAAY,EAAgB,MAAM,oBAAoB,CAAC;AACrE,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,aAAa,CAAC;AAEzC,8DAA8D;AAC9D,qBAAa,YAAa,YAAW,KAAK;IACxC,KAAK,CAAC,KAAK,EAAE,KAAK,GAAG,OAAO,CAAC,YAAY,CAAC;CAG3C"}
@@ -0,0 +1,8 @@
1
+ import { tableChanges } from "./table-changes.js";
2
+ /** The first pass gives a read for each row of each table. */
3
+ export class ReadEveryRow {
4
+ table(table) {
5
+ return tableChanges(table, null);
6
+ }
7
+ }
8
+ //# sourceMappingURL=read-every-row.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"read-every-row.js","sourceRoot":"","sources":["../src/read-every-row.ts"],"names":[],"mappings":"AACA,OAAO,EAAqB,YAAY,EAAE,MAAM,oBAAoB,CAAC;AAGrE,8DAA8D;AAC9D,MAAM,OAAO,YAAY;IACvB,KAAK,CAAC,KAAY;QAChB,OAAO,YAAY,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC;IACnC,CAAC;CACF"}
@@ -0,0 +1,12 @@
1
+ import type { TableChanges } from './table-changes.ts';
2
+ import type { Table } from './tables.ts';
3
+ /**
4
+ * What the first pass of a position that no pass saved gives for each table.
5
+ * After that save, a pass gives a read for each row of a new table. A new
6
+ * start is a class that implements this interface.
7
+ */
8
+ export interface Start {
9
+ /** The change events and the snapshot of one table at the first pass. */
10
+ table(table: Table): Promise<TableChanges>;
11
+ }
12
+ //# sourceMappingURL=start.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"start.d.ts","sourceRoot":"","sources":["../src/start.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,oBAAoB,CAAC;AACvD,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,aAAa,CAAC;AAEzC;;;;GAIG;AACH,MAAM,WAAW,KAAK;IACpB,yEAAyE;IACzE,KAAK,CAAC,KAAK,EAAE,KAAK,GAAG,OAAO,CAAC,YAAY,CAAC,CAAC;CAC5C"}
package/dist/start.js ADDED
@@ -0,0 +1,2 @@
1
+ export {};
2
+ //# sourceMappingURL=start.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"start.js","sourceRoot":"","sources":["../src/start.ts"],"names":[],"mappings":""}
@@ -1,8 +1,11 @@
1
1
  import { type SnapshotState } from '@zukhruf/snapshot-differential';
2
2
  import type { ChangeEvent } from './change-event.ts';
3
3
  import type { Table } from './tables.ts';
4
- export declare function tableChanges(table: Table, saved: SnapshotState | null): Promise<{
5
- events: ChangeEvent[];
6
- state: SnapshotState;
7
- }>;
4
+ /** The change events of one table in a pass, and its new snapshot. */
5
+ export type TableChanges = {
6
+ readonly events: ChangeEvent[];
7
+ readonly state: SnapshotState;
8
+ };
9
+ export declare function tableChanges(table: Table, saved: SnapshotState | null): Promise<TableChanges>;
10
+ export declare function snapshotOf(table: Table): Promise<SnapshotState>;
8
11
  //# sourceMappingURL=table-changes.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"table-changes.d.ts","sourceRoot":"","sources":["../src/table-changes.ts"],"names":[],"mappings":"AAAA,OAAO,EAGL,KAAK,aAAa,EACnB,MAAM,gCAAgC,CAAC;AAExC,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AACrD,OAAO,KAAK,EAAY,KAAK,EAAE,MAAM,aAAa,CAAC;AAKnD,wBAAsB,YAAY,CAChC,KAAK,EAAE,KAAK,EACZ,KAAK,EAAE,aAAa,GAAG,IAAI,GAC1B,OAAO,CAAC;IAAE,MAAM,EAAE,WAAW,EAAE,CAAC;IAAC,KAAK,EAAE,aAAa,CAAA;CAAE,CAAC,CAoC1D"}
1
+ {"version":3,"file":"table-changes.d.ts","sourceRoot":"","sources":["../src/table-changes.ts"],"names":[],"mappings":"AAAA,OAAO,EAGL,KAAK,aAAa,EACnB,MAAM,gCAAgC,CAAC;AAExC,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AACrD,OAAO,KAAK,EAAY,KAAK,EAAE,MAAM,aAAa,CAAC;AAEnD,sEAAsE;AACtE,MAAM,MAAM,YAAY,GAAG;IACzB,QAAQ,CAAC,MAAM,EAAE,WAAW,EAAE,CAAC;IAC/B,QAAQ,CAAC,KAAK,EAAE,aAAa,CAAC;CAC/B,CAAC;AAKF,wBAAsB,YAAY,CAChC,KAAK,EAAE,KAAK,EACZ,KAAK,EAAE,aAAa,GAAG,IAAI,GAC1B,OAAO,CAAC,YAAY,CAAC,CA4BvB;AAID,wBAAsB,UAAU,CAAC,KAAK,EAAE,KAAK,GAAG,OAAO,CAAC,aAAa,CAAC,CAIrE"}
@@ -3,16 +3,8 @@ import { ClosedWorld, SnapshotDifferential, } from '@zukhruf/snapshot-differenti
3
3
  // its new snapshot. null as the snapshot: the position does not have the
4
4
  // table, so each row is a read.
5
5
  export async function tableChanges(table, saved) {
6
- const differential = new SnapshotDifferential({
7
- name: table.name,
8
- primaryKey: table.keyColumns,
9
- key: (row) => table.keyColumns.map((column) => keyValueOf(row, column)),
10
- // A pass reads each row of the table, so a key it does not find was
11
- // deleted.
12
- policy: new ClosedWorld(),
13
- });
14
6
  const events = [];
15
- for await (const change of differential.diff(table.rows(), saved)) {
7
+ for await (const change of differentialOf(table).diff(table.rows(), saved)) {
16
8
  switch (change.type) {
17
9
  case 'insert':
18
10
  events.push({
@@ -39,6 +31,24 @@ export async function tableChanges(table, saved) {
39
31
  }
40
32
  throw new TypeError(`The differential of ${table.name} gave no snapshot`);
41
33
  }
34
+ // The snapshot of one table with no events: it keeps no row, only the
35
+ // fingerprint of each.
36
+ export async function snapshotOf(table) {
37
+ for await (const change of differentialOf(table).diff(table.rows(), null))
38
+ if (change.type === 'state')
39
+ return change.state;
40
+ throw new TypeError(`The differential of ${table.name} gave no snapshot`);
41
+ }
42
+ function differentialOf(table) {
43
+ return new SnapshotDifferential({
44
+ name: table.name,
45
+ primaryKey: table.keyColumns,
46
+ key: (row) => table.keyColumns.map((column) => keyValueOf(row, column)),
47
+ // A pass reads each row of the table, so a key it does not find was
48
+ // deleted.
49
+ policy: new ClosedWorld(),
50
+ });
51
+ }
42
52
  function keyValueOf(row, column) {
43
53
  const value = row.key[column];
44
54
  if (value === undefined)
@@ -1 +1 @@
1
- {"version":3,"file":"table-changes.js","sourceRoot":"","sources":["../src/table-changes.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,WAAW,EACX,oBAAoB,GAErB,MAAM,gCAAgC,CAAC;AAKxC,2EAA2E;AAC3E,yEAAyE;AACzE,gCAAgC;AAChC,MAAM,CAAC,KAAK,UAAU,YAAY,CAChC,KAAY,EACZ,KAA2B;IAE3B,MAAM,YAAY,GAAG,IAAI,oBAAoB,CAAW;QACtD,IAAI,EAAE,KAAK,CAAC,IAAI;QAChB,UAAU,EAAE,KAAK,CAAC,UAAU;QAC5B,GAAG,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,UAAU,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;QACvE,oEAAoE;QACpE,WAAW;QACX,MAAM,EAAE,IAAI,WAAW,EAAE;KAC1B,CAAC,CAAC;IACH,MAAM,MAAM,GAAkB,EAAE,CAAC;IACjC,IAAI,KAAK,EAAE,MAAM,MAAM,IAAI,YAAY,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,EAAE,KAAK,CAAC,EAAE,CAAC;QAClE,QAAQ,MAAM,CAAC,IAAI,EAAE,CAAC;YACpB,KAAK,QAAQ;gBACX,MAAM,CAAC,IAAI,CAAC;oBACV,KAAK,EAAE,KAAK,CAAC,IAAI;oBACjB,EAAE,EAAE,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ;oBACtC,GAAG,EAAE,MAAM,CAAC,MAAM,CAAC,GAAG;oBACtB,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK;iBAC3B,CAAC,CAAC;gBACH,MAAM;YACR,KAAK,QAAQ;gBACX,MAAM,CAAC,IAAI,CAAC;oBACV,KAAK,EAAE,KAAK,CAAC,IAAI;oBACjB,EAAE,EAAE,QAAQ;oBACZ,GAAG,EAAE,MAAM,CAAC,MAAM,CAAC,GAAG;oBACtB,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK;iBAC3B,CAAC,CAAC;gBACH,MAAM;YACR,KAAK,QAAQ;gBACX,MAAM,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,IAAI,EAAE,EAAE,EAAE,QAAQ,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,CAAC,CAAC;gBAClE,MAAM;YACR,KAAK,OAAO;gBACV,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,CAAC;QAC3C,CAAC;IACH,CAAC;IACD,MAAM,IAAI,SAAS,CAAC,uBAAuB,KAAK,CAAC,IAAI,mBAAmB,CAAC,CAAC;AAC5E,CAAC;AAED,SAAS,UAAU,CAAC,GAAa,EAAE,MAAc;IAC/C,MAAM,KAAK,GAAG,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;IAC9B,IAAI,KAAK,KAAK,SAAS;QACrB,MAAM,IAAI,SAAS,CAAC,yCAAyC,MAAM,EAAE,CAAC,CAAC;IACzE,OAAO,KAAK,CAAC;AACf,CAAC"}
1
+ {"version":3,"file":"table-changes.js","sourceRoot":"","sources":["../src/table-changes.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,WAAW,EACX,oBAAoB,GAErB,MAAM,gCAAgC,CAAC;AAWxC,2EAA2E;AAC3E,yEAAyE;AACzE,gCAAgC;AAChC,MAAM,CAAC,KAAK,UAAU,YAAY,CAChC,KAAY,EACZ,KAA2B;IAE3B,MAAM,MAAM,GAAkB,EAAE,CAAC;IACjC,IAAI,KAAK,EAAE,MAAM,MAAM,IAAI,cAAc,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,EAAE,KAAK,CAAC,EAAE,CAAC;QAC3E,QAAQ,MAAM,CAAC,IAAI,EAAE,CAAC;YACpB,KAAK,QAAQ;gBACX,MAAM,CAAC,IAAI,CAAC;oBACV,KAAK,EAAE,KAAK,CAAC,IAAI;oBACjB,EAAE,EAAE,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ;oBACtC,GAAG,EAAE,MAAM,CAAC,MAAM,CAAC,GAAG;oBACtB,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK;iBAC3B,CAAC,CAAC;gBACH,MAAM;YACR,KAAK,QAAQ;gBACX,MAAM,CAAC,IAAI,CAAC;oBACV,KAAK,EAAE,KAAK,CAAC,IAAI;oBACjB,EAAE,EAAE,QAAQ;oBACZ,GAAG,EAAE,MAAM,CAAC,MAAM,CAAC,GAAG;oBACtB,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK;iBAC3B,CAAC,CAAC;gBACH,MAAM;YACR,KAAK,QAAQ;gBACX,MAAM,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,IAAI,EAAE,EAAE,EAAE,QAAQ,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,CAAC,CAAC;gBAClE,MAAM;YACR,KAAK,OAAO;gBACV,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,CAAC;QAC3C,CAAC;IACH,CAAC;IACD,MAAM,IAAI,SAAS,CAAC,uBAAuB,KAAK,CAAC,IAAI,mBAAmB,CAAC,CAAC;AAC5E,CAAC;AAED,sEAAsE;AACtE,uBAAuB;AACvB,MAAM,CAAC,KAAK,UAAU,UAAU,CAAC,KAAY;IAC3C,IAAI,KAAK,EAAE,MAAM,MAAM,IAAI,cAAc,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,EAAE,IAAI,CAAC;QACvE,IAAI,MAAM,CAAC,IAAI,KAAK,OAAO;YAAE,OAAO,MAAM,CAAC,KAAK,CAAC;IACnD,MAAM,IAAI,SAAS,CAAC,uBAAuB,KAAK,CAAC,IAAI,mBAAmB,CAAC,CAAC;AAC5E,CAAC;AAED,SAAS,cAAc,CAAC,KAAY;IAClC,OAAO,IAAI,oBAAoB,CAAW;QACxC,IAAI,EAAE,KAAK,CAAC,IAAI;QAChB,UAAU,EAAE,KAAK,CAAC,UAAU;QAC5B,GAAG,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,UAAU,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;QACvE,oEAAoE;QACpE,WAAW;QACX,MAAM,EAAE,IAAI,WAAW,EAAE;KAC1B,CAAC,CAAC;AACL,CAAC;AAED,SAAS,UAAU,CAAC,GAAa,EAAE,MAAc;IAC/C,MAAM,KAAK,GAAG,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;IAC9B,IAAI,KAAK,KAAK,SAAS;QACrB,MAAM,IAAI,SAAS,CAAC,yCAAyC,MAAM,EAAE,CAAC,CAAC;IACzE,OAAO,KAAK,CAAC;AACf,CAAC"}
package/dist/tables.js CHANGED
@@ -15,9 +15,7 @@ export function tablesOf(snapshot) {
15
15
  return {
16
16
  name: table,
17
17
  keyColumns: ['rowid'],
18
- rows: () => snapshot
19
- .iterate(`SELECT rowid AS rowid, * ${from}`)
20
- .map(({ rowid, ...after }) => ({
18
+ rows: () => rowsOf(snapshot, `SELECT rowid AS rowid, * ${from}`).map(({ rowid, ...after }) => ({
21
19
  key: { rowid: keyValue(rowid) },
22
20
  after,
23
21
  })),
@@ -25,7 +23,7 @@ export function tablesOf(snapshot) {
25
23
  return {
26
24
  name: table,
27
25
  keyColumns: primaryKey,
28
- rows: () => snapshot.iterate(`SELECT * ${from}`).map((row) => {
26
+ rows: () => rowsOf(snapshot, `SELECT * ${from}`).map((row) => {
29
27
  const after = { ...row };
30
28
  return {
31
29
  key: Object.fromEntries(primaryKey.map((column) => [column, keyValue(after[column])])),
@@ -58,10 +56,14 @@ function text(value) {
58
56
  throw new TypeError(`SQLite gave ${String(value)} for a name`);
59
57
  return value;
60
58
  }
61
- // The schema rules above keep NULL out of a key, and node:sqlite gives a
62
- // bigint only to a statement that asks for it.
59
+ // The rows of a table, each integer as a bigint: an INTEGER of SQLite holds
60
+ // 64 bits, and a number holds an integer exactly only up to 2^53.
61
+ function rowsOf(snapshot, sql) {
62
+ return snapshot.prepare(sql, { readBigInts: true }).iterate();
63
+ }
64
+ // The schema rules above keep NULL out of a key.
63
65
  function keyValue(value) {
64
- if (value === null || value === undefined || typeof value === 'bigint')
66
+ if (value === null || value === undefined)
65
67
  throw new TypeError(`SQLite gave ${String(value)} for a key value`);
66
68
  return value;
67
69
  }
@@ -1 +1 @@
1
- {"version":3,"file":"tables.js","sourceRoot":"","sources":["../src/tables.ts"],"names":[],"mappings":"AAkBA,uEAAuE;AACvE,4EAA4E;AAC5E,yBAAyB;AACzB,MAAM,UAAU,QAAQ,CAAC,QAAkB;IACzC,OAAO,QAAQ;SACZ,GAAG,CACF;;;qBAGe,CAChB;SACA,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,EAAE,EAAE;QAChB,MAAM,KAAK,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC;QACzB,MAAM,UAAU,GAAG,qBAAqB,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC;QAC1D,MAAM,IAAI,GAAG,SAAS,KAAK,CAAC,UAAU,CAAC,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC;QACrD,IAAI,UAAU,KAAK,IAAI;YACrB,OAAO;gBACL,IAAI,EAAE,KAAK;gBACX,UAAU,EAAE,CAAC,OAAO,CAAC;gBACrB,IAAI,EAAE,GAAG,EAAE,CACT,QAAQ;qBACL,OAAO,CAAC,4BAA4B,IAAI,EAAE,CAAC;qBAC3C,GAAG,CAAC,CAAC,EAAE,KAAK,EAAE,GAAG,KAAK,EAAE,EAAE,EAAE,CAAC,CAAC;oBAC7B,GAAG,EAAE,EAAE,KAAK,EAAE,QAAQ,CAAC,KAAK,CAAC,EAAE;oBAC/B,KAAK;iBACN,CAAC,CAAC;aACR,CAAC;QACJ,OAAO;YACL,IAAI,EAAE,KAAK;YACX,UAAU,EAAE,UAAU;YACtB,IAAI,EAAE,GAAG,EAAE,CACT,QAAQ,CAAC,OAAO,CAAC,YAAY,IAAI,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE;gBAC/C,MAAM,KAAK,GAAG,EAAE,GAAG,GAAG,EAAE,CAAC;gBACzB,OAAO;oBACL,GAAG,EAAE,MAAM,CAAC,WAAW,CACrB,UAAU,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,MAAM,EAAE,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAC9D;oBACD,KAAK;iBACN,CAAC;YACJ,CAAC,CAAC;SACL,CAAC;IACJ,CAAC,CAAC,CAAC;AACP,CAAC;AAED,0EAA0E;AAC1E,yEAAyE;AACzE,0EAA0E;AAC1E,yGAAyG;AACzG,0EAA0E;AAC1E,aAAa;AACb,SAAS,qBAAqB,CAC5B,QAAkB,EAClB,KAAa;IAEb,MAAM,OAAO,GAAG,QAAQ,CAAC,GAAG,CAC1B,2EAA2E,EAC3E,KAAK,CACN,CAAC;IACF,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IACtC,wEAAwE;IACxE,wEAAwE;IACxE,6CAA6C;IAC7C,MAAM,UAAU,GACd,QAAQ,CAAC,GAAG,CACV,wDAAwD,EACxD,KAAK,CACN,CAAC,MAAM,KAAK,CAAC,CAAC;IACjB,IAAI,UAAU,IAAI,OAAO,CAAC,KAAK,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,OAAO,KAAK,CAAC,CAAC;QAC/D,OAAO,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC;IACpD,OAAO,IAAI,CAAC;AACd,CAAC;AAED,SAAS,IAAI,CAAC,KAAiC;IAC7C,IAAI,OAAO,KAAK,KAAK,QAAQ;QAC3B,MAAM,IAAI,SAAS,CAAC,eAAe,MAAM,CAAC,KAAK,CAAC,aAAa,CAAC,CAAC;IACjE,OAAO,KAAK,CAAC;AACf,CAAC;AAED,yEAAyE;AACzE,+CAA+C;AAC/C,SAAS,QAAQ,CAAC,KAAiC;IACjD,IAAI,KAAK,KAAK,IAAI,IAAI,KAAK,KAAK,SAAS,IAAI,OAAO,KAAK,KAAK,QAAQ;QACpE,MAAM,IAAI,SAAS,CAAC,eAAe,MAAM,CAAC,KAAK,CAAC,kBAAkB,CAAC,CAAC;IACtE,OAAO,KAAK,CAAC;AACf,CAAC"}
1
+ {"version":3,"file":"tables.js","sourceRoot":"","sources":["../src/tables.ts"],"names":[],"mappings":"AAkBA,uEAAuE;AACvE,4EAA4E;AAC5E,yBAAyB;AACzB,MAAM,UAAU,QAAQ,CAAC,QAAkB;IACzC,OAAO,QAAQ;SACZ,GAAG,CACF;;;qBAGe,CAChB;SACA,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,EAAE,EAAE;QAChB,MAAM,KAAK,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC;QACzB,MAAM,UAAU,GAAG,qBAAqB,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC;QAC1D,MAAM,IAAI,GAAG,SAAS,KAAK,CAAC,UAAU,CAAC,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC;QACrD,IAAI,UAAU,KAAK,IAAI;YACrB,OAAO;gBACL,IAAI,EAAE,KAAK;gBACX,UAAU,EAAE,CAAC,OAAO,CAAC;gBACrB,IAAI,EAAE,GAAG,EAAE,CACT,MAAM,CAAC,QAAQ,EAAE,4BAA4B,IAAI,EAAE,CAAC,CAAC,GAAG,CACtD,CAAC,EAAE,KAAK,EAAE,GAAG,KAAK,EAAE,EAAE,EAAE,CAAC,CAAC;oBACxB,GAAG,EAAE,EAAE,KAAK,EAAE,QAAQ,CAAC,KAAK,CAAC,EAAE;oBAC/B,KAAK;iBACN,CAAC,CACH;aACJ,CAAC;QACJ,OAAO;YACL,IAAI,EAAE,KAAK;YACX,UAAU,EAAE,UAAU;YACtB,IAAI,EAAE,GAAG,EAAE,CACT,MAAM,CAAC,QAAQ,EAAE,YAAY,IAAI,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,EAAE;gBAC/C,MAAM,KAAK,GAAG,EAAE,GAAG,GAAG,EAAE,CAAC;gBACzB,OAAO;oBACL,GAAG,EAAE,MAAM,CAAC,WAAW,CACrB,UAAU,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,MAAM,EAAE,QAAQ,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAC9D;oBACD,KAAK;iBACN,CAAC;YACJ,CAAC,CAAC;SACL,CAAC;IACJ,CAAC,CAAC,CAAC;AACP,CAAC;AAED,0EAA0E;AAC1E,yEAAyE;AACzE,0EAA0E;AAC1E,yGAAyG;AACzG,0EAA0E;AAC1E,aAAa;AACb,SAAS,qBAAqB,CAC5B,QAAkB,EAClB,KAAa;IAEb,MAAM,OAAO,GAAG,QAAQ,CAAC,GAAG,CAC1B,2EAA2E,EAC3E,KAAK,CACN,CAAC;IACF,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IACtC,wEAAwE;IACxE,wEAAwE;IACxE,6CAA6C;IAC7C,MAAM,UAAU,GACd,QAAQ,CAAC,GAAG,CACV,wDAAwD,EACxD,KAAK,CACN,CAAC,MAAM,KAAK,CAAC,CAAC;IACjB,IAAI,UAAU,IAAI,OAAO,CAAC,KAAK,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,OAAO,KAAK,CAAC,CAAC;QAC/D,OAAO,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC;IACpD,OAAO,IAAI,CAAC;AACd,CAAC;AAED,SAAS,IAAI,CAAC,KAAiC;IAC7C,IAAI,OAAO,KAAK,KAAK,QAAQ;QAC3B,MAAM,IAAI,SAAS,CAAC,eAAe,MAAM,CAAC,KAAK,CAAC,aAAa,CAAC,CAAC;IACjE,OAAO,KAAK,CAAC;AACf,CAAC;AAED,4EAA4E;AAC5E,kEAAkE;AAClE,SAAS,MAAM,CACb,QAAkB,EAClB,GAAW;IAEX,OAAO,QAAQ,CAAC,OAAO,CAAC,GAAG,EAAE,EAAE,WAAW,EAAE,IAAI,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC;AAChE,CAAC;AAED,iDAAiD;AACjD,SAAS,QAAQ,CAAC,KAAiC;IACjD,IAAI,KAAK,KAAK,IAAI,IAAI,KAAK,KAAK,SAAS;QACvC,MAAM,IAAI,SAAS,CAAC,eAAe,MAAM,CAAC,KAAK,CAAC,kBAAkB,CAAC,CAAC;IACtE,OAAO,KAAK,CAAC;AACf,CAAC"}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@zukhruf/sqlite-changefeed",
3
- "version": "0.8.2",
3
+ "version": "0.8.3",
4
4
  "description": "A changefeed of a SQLite database that another process owns and writes: a change event for each changed row of each table, keyed by its primary key, from a snapshot differential over a read-only connection. The feed keeps its own position. Needs WAL mode.",
5
5
  "license": "MIT",
6
6
  "repository": {
@@ -41,15 +41,15 @@
41
41
  "access": "public"
42
42
  },
43
43
  "dependencies": {
44
- "@zukhruf/async": "^0.8.2",
45
- "@zukhruf/snapshot-differential": "^0.8.2",
46
- "@zukhruf/sqlite-data-version": "^0.8.2",
47
- "@zukhruf/sqlite-open": "^0.8.2",
48
- "@zukhruf/sqlite-snapshot": "^0.8.2",
49
- "@zukhruf/sqlite-transaction": "^0.8.2"
44
+ "@zukhruf/async": "^0.8.3",
45
+ "@zukhruf/snapshot-differential": "^0.8.3",
46
+ "@zukhruf/sqlite-data-version": "^0.8.3",
47
+ "@zukhruf/sqlite-open": "^0.8.3",
48
+ "@zukhruf/sqlite-snapshot": "^0.8.3",
49
+ "@zukhruf/sqlite-transaction": "^0.8.3"
50
50
  },
51
51
  "devDependencies": {
52
52
  "@sinonjs/fake-timers": "15.4.0",
53
- "@zukhruf/testing": "^0.8.2"
53
+ "@zukhruf/testing": "^0.8.3"
54
54
  }
55
55
  }