@hachej/boring-files 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +21 -1
- package/dist/journal.d.ts +7 -0
- package/dist/journal.js +8 -3
- package/dist/revision-contracts.d.ts +41 -0
- package/dist/revision-handler.d.ts +3 -0
- package/dist/revision-handler.js +109 -1
- package/dist/revision-protocol.d.ts +8 -1
- package/dist/revision-protocol.js +65 -1
- package/dist/revision.js +89 -1
- package/dist/sqlite-filesystem.js +53 -11
- package/dist/workspace.d.ts +59 -6
- package/dist/workspace.js +250 -12
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -10,6 +10,12 @@ Opaque resource view IDs are provider-scoped and bound to the actual backing/inc
|
|
|
10
10
|
|
|
11
11
|
The provider accepts all-or-nothing create and replace. It refuses working views, per-change requests and deletes (files are removed with the workspace's own tools) before any effect. Logical paths are relative to the workspace root; a path whose real location leaves the root is denied. A single file is replaced atomically by every backend; a multi-file batch only by the SQLite backend, together with its receipt when the journal shares the database. Access control is the host's, at the workspace boundary: there are no per-file permissions inside a workspace, and external revocation fencing is not implemented.
|
|
12
12
|
|
|
13
|
+
### Agent writes and the change feed
|
|
14
|
+
|
|
15
|
+
Agent writes are observed, never receipted ([owner ruling 2](../../docs/architecture/FILES-GIT-EXEC.md#one-place-for-files)). `provider.record({ path, before?, after, source: 'agent', conversationId? }, access)` is what the file guard (`@hachej/boring-agent/file-guard`) calls after a Pi `write` or `edit` succeeded: the file's history gains the written revision with `source: 'agent'` (and the replaced revision, `source: 'observed'`, when `before` carries its bytes), and the change feed an `agent` event. It never writes the publication journal and never mints a receipt. It does not enter the provider's queue (the guard already runs inside it), so calling it from queued work cannot deadlock. History entries (`saves`, `catalog.history`) carry `source`: `publish`, `agent` or `observed` (`keep` and replaced revisions).
|
|
16
|
+
|
|
17
|
+
`provider.changes({ since?, signal }, access)` is one change feed for every file view: an `AsyncIterable` of `{ kind: 'change', path, revision | null, source: 'publish' | 'agent' | 'poll' | 'watch', seq, at }`, ordered by `seq` and replayable from `since` (the last `seq` seen; `changeHead()` is "now"). The provider retains the last `changeBuffer` events (default 1024); a cursor older than that, from another provider lifetime or ahead of this one, and a listener that falls behind get `{ kind: 'resnapshot', seq, at }`: reload everything shown, then continue. Events are invalidation hints; bytes and the journal stay authoritative. Sources: conditional writes (`publish`), `record` (`agent`), `poll` and, while someone listens (and `watchLingerMs` after, default 30 s), the file system's own `watch` (`watch`; a reported directory is `{ revision: null, directory: true }`). The SQLite file system reports writes through its own connection at once and polls every second for other connections. When the file system cannot watch (the virtual and remote environments report `not_supported`), the host calls `provider.poll()` at the end of each agent turn: without paths it hashes every file (not `.git` or `node_modules`); the first call takes the baseline, later ones report changed, new and removed files. A `bash` write gets change events only, never history. After the last listener leaves the watcher lingers and then closes; changes made while no watcher was open are unknown, so the first listener afterwards makes the feed append a `resnapshot` (a client that stays connected is never affected). `changeIncarnation` names the provider lifetime: a cursor sent with another `incarnation` is a `resnapshot` whatever its number. `provider.close()` ends every change stream, closes the watcher and drops the listeners.
|
|
18
|
+
|
|
13
19
|
Receipts are scoped by workspace identity, principal, scope and initiator. Identical committed retries return the original receipt; changed arguments conflict. Refused operations do not reserve an ID. A receipt from an earlier incarnation of the workspace reads `unknown`. A not-found lookup is not permission to replay an uncertain operation.
|
|
14
20
|
|
|
15
21
|
`@hachej/boring-files/publication` is browser-safe. It captures request bytes, computes a versioned canonical SHA-256 digest with WebCrypto, and parses result shapes. Digests carry `boring-publication-v1:sha256:` and bind that format identifier in the hashed payload. Consumers must also check operation, target, scope and index association against their own request. This module confers no authorization. The SQLite connection (`openNodeConnection` from `@hachej/boring-files/sqlite`) is a separate server entry point.
|
|
@@ -138,7 +144,21 @@ const revisionProvider = await connectRevisionProvider({
|
|
|
138
144
|
|
|
139
145
|
The binding discovers identity and workspace incarnation from the authenticated
|
|
140
146
|
server. Reconnect when the selected workspace changes. It exposes `list`,
|
|
141
|
-
filename `search`, retained `history`, `locate`, `read`, `publish` and
|
|
147
|
+
filename `search`, retained `history`, `locate`, `read`, `publish`, `lookup` and
|
|
148
|
+
`changes({ since?, signal })`, the workspace's change feed.
|
|
149
|
+
|
|
150
|
+
The feed is `GET <endpoint>?op=changes[&since=<seq>&incarnation=<id>]` with the binding header:
|
|
151
|
+
server-sent events (`event: ready` with the current seq and the provider incarnation, then `event: change` or
|
|
152
|
+
`event: resnapshot` with `id:` the seq and the JSON event as `data:`), and a
|
|
153
|
+
comment line every `heartbeatMs` (`createRevisionHandler({ resolve, heartbeatMs })`,
|
|
154
|
+
default 15 s). It is a plain streamed response with no queue (pull-based), so a slow client
|
|
155
|
+
costs the provider's bounded replay buffer and a `resnapshot`, never memory in
|
|
156
|
+
the handler; revocation discards anything unread. It runs on Node and Workers
|
|
157
|
+
alike. The handler holds the workspace lease for the life of the stream and
|
|
158
|
+
releases it when the client disconnects or its access signal aborts; a request
|
|
159
|
+
the host refuses, or whose binding names another workspace or identity, gets no
|
|
160
|
+
event (403 or 409). The client reconnects after the last seq it delivered, with
|
|
161
|
+
backoff, until its `signal` aborts; a refusal ends it with a `RevisionError`.
|
|
142
162
|
All writes use conditional publication; uploading a new file requires absence,
|
|
143
163
|
and replacing one requires its exact resource revision. An unknown result must
|
|
144
164
|
be reconciled by operation ID before another upload to that path.
|
package/dist/journal.d.ts
CHANGED
|
@@ -38,7 +38,13 @@ export interface StoredVersion {
|
|
|
38
38
|
readonly mediaType: string;
|
|
39
39
|
/** When this revision was saved, in milliseconds since the epoch. Kept the first time the revision is retained. */
|
|
40
40
|
readonly savedAt?: number;
|
|
41
|
+
/**
|
|
42
|
+
* Who produced the revision: `publish` (a conditional write), `agent` (a Pi `write` or `edit`, recorded by `provider.record`: it
|
|
43
|
+
* has no receipt) or `observed` (retained as found: `keep`, or the replaced revision of a write). Default `publish`.
|
|
44
|
+
*/
|
|
45
|
+
readonly source?: HistorySource;
|
|
41
46
|
}
|
|
47
|
+
export type HistorySource = 'publish' | 'agent' | 'observed';
|
|
42
48
|
/**
|
|
43
49
|
* The operation journal plus what a workspace needs beyond receipts: an intent row written before an operation's first effect
|
|
44
50
|
* (an intent without a completion is an operation of unknown outcome), and the last versions of each file when there is no Git.
|
|
@@ -68,6 +74,7 @@ export interface WorkspaceJournal extends OperationJournal {
|
|
|
68
74
|
readonly saves: (scope: string, path: string) => {
|
|
69
75
|
readonly revision: string;
|
|
70
76
|
readonly savedAt: number;
|
|
77
|
+
readonly source: HistorySource;
|
|
71
78
|
}[];
|
|
72
79
|
}
|
|
73
80
|
export declare function createWorkspaceJournal(db: SqliteConnection, options?: {
|
package/dist/journal.js
CHANGED
|
@@ -59,6 +59,7 @@ export function committedChanges(changes, ref) {
|
|
|
59
59
|
throw new Error('Stored operation has no changes');
|
|
60
60
|
return [first, ...rest];
|
|
61
61
|
}
|
|
62
|
+
const historySource = (value) => value === 'agent' || value === 'observed' ? value : 'publish';
|
|
62
63
|
export function createWorkspaceJournal(db, options = {}) {
|
|
63
64
|
const limit = options.historyLimit ?? 20;
|
|
64
65
|
if (!Number.isSafeInteger(limit) || limit < 1)
|
|
@@ -74,18 +75,22 @@ export function createWorkspaceJournal(db, options = {}) {
|
|
|
74
75
|
) STRICT;
|
|
75
76
|
CREATE TABLE IF NOT EXISTS boring_history (
|
|
76
77
|
scope TEXT NOT NULL, path TEXT NOT NULL, revision TEXT NOT NULL, sequence INTEGER NOT NULL,
|
|
77
|
-
bytes BLOB NOT NULL, media_type TEXT NOT NULL, saved_at INTEGER NOT NULL DEFAULT 0,
|
|
78
|
+
bytes BLOB NOT NULL, media_type TEXT NOT NULL, saved_at INTEGER NOT NULL DEFAULT 0, source TEXT NOT NULL DEFAULT 'publish',
|
|
78
79
|
PRIMARY KEY (scope, path, revision)
|
|
79
80
|
) STRICT;
|
|
80
81
|
`);
|
|
81
82
|
// A journal created before save times were recorded gets the column; its old rows read as unknown (0).
|
|
82
83
|
if (!db.all("SELECT name FROM pragma_table_info('boring_history')").some(column => column.name === 'saved_at'))
|
|
83
84
|
db.exec('ALTER TABLE boring_history ADD COLUMN saved_at INTEGER NOT NULL DEFAULT 0');
|
|
85
|
+
// A journal created before history recorded who wrote each revision: its old rows were all conditional writes.
|
|
86
|
+
if (!db.all("SELECT name FROM pragma_table_info('boring_history')").some(column => column.name === 'source'))
|
|
87
|
+
db.exec("ALTER TABLE boring_history ADD COLUMN source TEXT NOT NULL DEFAULT 'publish'");
|
|
84
88
|
});
|
|
85
89
|
const remember = (scope, path, version) => {
|
|
86
90
|
const last = db.get('SELECT COALESCE(MAX(sequence), 0) AS last FROM boring_history WHERE scope = ? AND path = ?', scope, path);
|
|
87
91
|
const sequence = Number(last?.last ?? 0) + 1;
|
|
88
|
-
|
|
92
|
+
// A revision found again (`observed`) keeps the writer it was first recorded with; a new write of the same bytes names its writer.
|
|
93
|
+
db.run("INSERT INTO boring_history (scope, path, revision, sequence, bytes, media_type, saved_at, source) VALUES (?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT(scope, path, revision) DO UPDATE SET sequence = excluded.sequence, bytes = excluded.bytes, media_type = excluded.media_type, saved_at = CASE WHEN boring_history.saved_at = 0 THEN excluded.saved_at ELSE boring_history.saved_at END, source = CASE WHEN excluded.source = 'observed' THEN boring_history.source ELSE excluded.source END", scope, path, version.revision, sequence, version.bytes, version.mediaType, version.savedAt ?? 0, historySource(version.source));
|
|
89
94
|
db.run('DELETE FROM boring_history WHERE scope = ? AND path = ? AND revision NOT IN (SELECT revision FROM boring_history WHERE scope = ? AND path = ? ORDER BY sequence DESC LIMIT ?)', scope, path, scope, path, limit);
|
|
90
95
|
};
|
|
91
96
|
const journal = {
|
|
@@ -112,7 +117,7 @@ export function createWorkspaceJournal(db, options = {}) {
|
|
|
112
117
|
return { revision, bytes: Uint8Array.from(row.bytes), mediaType: text(row.media_type) };
|
|
113
118
|
},
|
|
114
119
|
revisions: (scope, path) => db.all('SELECT revision FROM boring_history WHERE scope = ? AND path = ? ORDER BY sequence DESC', scope, path).map(row => text(row.revision)),
|
|
115
|
-
saves: (scope, path) => db.all('SELECT revision, saved_at FROM boring_history WHERE scope = ? AND path = ? ORDER BY sequence DESC', scope, path).map(row => ({ revision: text(row.revision), savedAt: Number(row.saved_at) })),
|
|
120
|
+
saves: (scope, path) => db.all('SELECT revision, saved_at, source FROM boring_history WHERE scope = ? AND path = ? ORDER BY sequence DESC', scope, path).map(row => ({ revision: text(row.revision), savedAt: Number(row.saved_at), source: historySource(row.source) })),
|
|
116
121
|
};
|
|
117
122
|
registerJournalConnection(journal, db);
|
|
118
123
|
return journal;
|
|
@@ -24,6 +24,41 @@ export interface FileSearchRequest {
|
|
|
24
24
|
export interface SavedRevision {
|
|
25
25
|
readonly revision: string;
|
|
26
26
|
readonly savedAt: number;
|
|
27
|
+
/** Who wrote it: `publish` (a conditional write), `agent` (a Pi `write` or `edit`: observed, never receipted) or `observed` (retained as found). */
|
|
28
|
+
readonly source?: 'publish' | 'agent' | 'observed';
|
|
29
|
+
}
|
|
30
|
+
/**
|
|
31
|
+
* One entry of a workspace's change feed (ruling 2 of 2026-10-10): an invalidation hint, never content or a receipt. Bytes and
|
|
32
|
+
* the publication journal stay authoritative; a viewer that receives one reads again.
|
|
33
|
+
*
|
|
34
|
+
* - `change`: something at `path` changed. `revision` is the Git blob id now there, or `null` when the path is absent or is a
|
|
35
|
+
* directory (`directory: true`: something below it may have changed). `source`: `publish` (a conditional write), `agent` (a Pi
|
|
36
|
+
* `write` or `edit`), `poll` (found by hashing, for example after a `bash` command) or `watch` (reported by the file system).
|
|
37
|
+
* - `resnapshot`: events were lost (the cursor is older than the retained window, from another provider lifetime, or the watcher
|
|
38
|
+
* overflowed or restarted). Reload everything shown, then continue after this `seq`.
|
|
39
|
+
*
|
|
40
|
+
* `seq` increases strictly within one provider lifetime; `at` is milliseconds since the epoch.
|
|
41
|
+
*/
|
|
42
|
+
export type WorkspaceChangeEvent = {
|
|
43
|
+
readonly kind: 'change';
|
|
44
|
+
readonly path: string;
|
|
45
|
+
readonly revision: string | null;
|
|
46
|
+
readonly source: 'publish' | 'agent' | 'poll' | 'watch';
|
|
47
|
+
readonly seq: number;
|
|
48
|
+
readonly at: number;
|
|
49
|
+
readonly directory?: true;
|
|
50
|
+
readonly conversationId?: string;
|
|
51
|
+
} | {
|
|
52
|
+
readonly kind: 'resnapshot';
|
|
53
|
+
readonly seq: number;
|
|
54
|
+
readonly at: number;
|
|
55
|
+
};
|
|
56
|
+
export interface ChangeFeedRequest {
|
|
57
|
+
/** The last `seq` already seen: only later events are delivered (a gap is a `resnapshot`). Absent: live events only. */
|
|
58
|
+
readonly since?: number | undefined;
|
|
59
|
+
/** The provider lifetime `since` came from (`ready` event). A cursor from another lifetime is a `resnapshot`, whatever its number. */
|
|
60
|
+
readonly incarnation?: string | undefined;
|
|
61
|
+
readonly signal?: AbortSignal | undefined;
|
|
27
62
|
}
|
|
28
63
|
export interface WorkspaceCatalog {
|
|
29
64
|
readonly list: (request: FileListRequest, access: ResourceAccess) => Promise<FilePage>;
|
|
@@ -44,5 +79,11 @@ export interface RevisionProvider extends ResourceClient {
|
|
|
44
79
|
readonly list: (request: FileListRequest, signal?: AbortSignal) => Promise<FilePage>;
|
|
45
80
|
readonly search: (request: FileSearchRequest, signal?: AbortSignal) => Promise<FilePage>;
|
|
46
81
|
readonly history: (path: string, signal?: AbortSignal) => Promise<readonly SavedRevision[]>;
|
|
82
|
+
/**
|
|
83
|
+
* The workspace's change feed (`GET ?op=changes`, server-sent events), reconnecting after the last `seq` seen until `signal`
|
|
84
|
+
* aborts; a refused binding (another workspace, revoked access) ends it with a `RevisionError`. Optional: a hand-built provider
|
|
85
|
+
* may have none, and a viewer then refreshes on its own schedule.
|
|
86
|
+
*/
|
|
87
|
+
readonly changes?: (request?: ChangeFeedRequest) => AsyncIterable<WorkspaceChangeEvent>;
|
|
47
88
|
}
|
|
48
89
|
//# sourceMappingURL=revision-contracts.d.ts.map
|
|
@@ -1,11 +1,14 @@
|
|
|
1
1
|
import type { ResourceAccess } from './contracts.js';
|
|
2
2
|
import type { WorkspaceResourceProvider } from './workspace.js';
|
|
3
3
|
export interface RevisionHandlerOptions {
|
|
4
|
+
/** The host's authentication and workspace selection, once per request. For the change feed the lease is held until the stream ends. */
|
|
4
5
|
readonly resolve: (request: Request) => Promise<null | {
|
|
5
6
|
readonly provider: WorkspaceResourceProvider;
|
|
6
7
|
readonly access: ResourceAccess;
|
|
7
8
|
readonly release?: () => void | Promise<void>;
|
|
8
9
|
}>;
|
|
10
|
+
/** How often the change feed sends a comment line so proxies keep it open, in milliseconds (default 15 000). */
|
|
11
|
+
readonly heartbeatMs?: number;
|
|
9
12
|
}
|
|
10
13
|
export declare function createRevisionHandler(options: RevisionHandlerOptions): (request: Request) => Promise<Response>;
|
|
11
14
|
//# sourceMappingURL=revision-handler.d.ts.map
|
package/dist/revision-handler.js
CHANGED
|
@@ -3,6 +3,90 @@ import { binding, bindingHeader, bindingValue, catalogHeader, filePage, record,
|
|
|
3
3
|
import { guardStatus, readJsonBody, RequestGuardError } from './request-guard.js';
|
|
4
4
|
import { accessSnapshot, identifier } from './publication-input.js';
|
|
5
5
|
import { RevisionError } from './revision-contracts.js';
|
|
6
|
+
const encoder = new TextEncoder();
|
|
7
|
+
/**
|
|
8
|
+
* `GET ?op=changes[&since=<seq>&incarnation=<id>]`: the workspace's change feed as server-sent events (`event: change | resnapshot`,
|
|
9
|
+
* `id:` the seq, `data:` the JSON event), after an `event: ready` naming the current seq and the provider's incarnation, with a
|
|
10
|
+
* comment every `heartbeatMs` while idle. Plain streamed responses, so it runs on Node and on Workers alike.
|
|
11
|
+
*
|
|
12
|
+
* The stream is pull-based with no queue (`highWaterMark: 0`): the provider is read only while the client is reading, so a slow
|
|
13
|
+
* client costs the provider's bounded replay buffer (and a `resnapshot` when it falls behind), never unbounded memory here. The
|
|
14
|
+
* stream ends when the person disconnects or their access is revoked, discarding anything not yet read, and the workspace lease is
|
|
15
|
+
* released then.
|
|
16
|
+
*/
|
|
17
|
+
function changeStream(provider, access, request, heartbeatMs, release) {
|
|
18
|
+
const stop = new AbortController();
|
|
19
|
+
const signal = AbortSignal.any([access.signal, stop.signal, provider.lifetime]);
|
|
20
|
+
let iterator;
|
|
21
|
+
let pending;
|
|
22
|
+
let ready = false, ended = false;
|
|
23
|
+
const finish = () => {
|
|
24
|
+
if (ended)
|
|
25
|
+
return;
|
|
26
|
+
ended = true;
|
|
27
|
+
stop.abort();
|
|
28
|
+
void iterator?.return?.().catch(() => { });
|
|
29
|
+
void release();
|
|
30
|
+
};
|
|
31
|
+
const body = new ReadableStream({
|
|
32
|
+
start: controller => {
|
|
33
|
+
const close = () => { finish(); try {
|
|
34
|
+
controller.close();
|
|
35
|
+
}
|
|
36
|
+
catch { /* already closed or cancelled */ } };
|
|
37
|
+
signal.addEventListener('abort', close, { once: true });
|
|
38
|
+
if (signal.aborted)
|
|
39
|
+
close();
|
|
40
|
+
},
|
|
41
|
+
pull: async (controller) => {
|
|
42
|
+
if (ended)
|
|
43
|
+
return;
|
|
44
|
+
const send = (text) => { try {
|
|
45
|
+
controller.enqueue(encoder.encode(text));
|
|
46
|
+
}
|
|
47
|
+
catch {
|
|
48
|
+
finish();
|
|
49
|
+
} };
|
|
50
|
+
const end = () => { finish(); try {
|
|
51
|
+
controller.close();
|
|
52
|
+
}
|
|
53
|
+
catch { /* already closed */ } };
|
|
54
|
+
try {
|
|
55
|
+
if (!ready) {
|
|
56
|
+
ready = true;
|
|
57
|
+
// Without a cursor the feed starts at the current seq, which the client learns first and resumes from after a disconnect.
|
|
58
|
+
const start = request.since ?? provider.changeHead();
|
|
59
|
+
iterator = provider.changes({ since: start, ...(request.incarnation === undefined ? {} : { incarnation: request.incarnation }), signal }, { ...access, signal })[Symbol.asyncIterator]();
|
|
60
|
+
// An async iterator runs its body only on the first pull, so prime it here: the watcher then starts before `ready`, and a
|
|
61
|
+
// write the client makes after reading `ready` is not missed.
|
|
62
|
+
pending = iterator.next();
|
|
63
|
+
pending.catch(() => { });
|
|
64
|
+
send(`event: ready\ndata: ${JSON.stringify({ seq: start, incarnation: provider.changeIncarnation })}\n\n`);
|
|
65
|
+
return;
|
|
66
|
+
}
|
|
67
|
+
pending ??= iterator.next();
|
|
68
|
+
pending.catch(() => { });
|
|
69
|
+
let timer;
|
|
70
|
+
const beat = new Promise(resolve => { timer = setTimeout(() => resolve('beat'), heartbeatMs); });
|
|
71
|
+
const outcome = await Promise.race([pending, beat]).finally(() => clearTimeout(timer));
|
|
72
|
+
if (signal.aborted)
|
|
73
|
+
return end();
|
|
74
|
+
if (outcome === 'beat')
|
|
75
|
+
return send(': heartbeat\n\n');
|
|
76
|
+
pending = undefined;
|
|
77
|
+
if (outcome.done)
|
|
78
|
+
return end();
|
|
79
|
+
const event = outcome.value;
|
|
80
|
+
send(`id: ${event.seq}\nevent: ${event.kind}\ndata: ${JSON.stringify(event)}\n\n`);
|
|
81
|
+
}
|
|
82
|
+
catch {
|
|
83
|
+
end(); /* the client reconnects from its last seq */
|
|
84
|
+
}
|
|
85
|
+
},
|
|
86
|
+
cancel: () => { finish(); },
|
|
87
|
+
}, { highWaterMark: 0 });
|
|
88
|
+
return new Response(body, { headers: { 'content-type': 'text/event-stream; charset=utf-8', 'cache-control': 'no-store', 'x-accel-buffering': 'no' } });
|
|
89
|
+
}
|
|
6
90
|
function listRequest(value) {
|
|
7
91
|
const data = record(value);
|
|
8
92
|
if ((data.directory !== undefined && typeof data.directory !== 'string') || (data.cursor !== undefined && typeof data.cursor !== 'string') || (data.limit !== undefined && typeof data.limit !== 'number'))
|
|
@@ -10,6 +94,9 @@ function listRequest(value) {
|
|
|
10
94
|
return { ...(typeof data.directory === 'string' ? { directory: data.directory } : {}), ...(typeof data.cursor === 'string' ? { cursor: data.cursor } : {}), ...(typeof data.limit === 'number' ? { limit: data.limit } : {}) };
|
|
11
95
|
}
|
|
12
96
|
export function createRevisionHandler(options) {
|
|
97
|
+
const heartbeatMs = options.heartbeatMs ?? 15_000;
|
|
98
|
+
if (!Number.isSafeInteger(heartbeatMs) || heartbeatMs < 1)
|
|
99
|
+
throw new RangeError('The heartbeat interval must be a positive integer');
|
|
13
100
|
return async (request) => {
|
|
14
101
|
let release;
|
|
15
102
|
const operations = new Set();
|
|
@@ -34,7 +121,8 @@ export function createRevisionHandler(options) {
|
|
|
34
121
|
return refuse(403);
|
|
35
122
|
const association = binding({ identity: captured, workspace: selected.provider.identity });
|
|
36
123
|
const respond = (value) => Response.json(value, { headers: { 'cache-control': 'no-store' } });
|
|
37
|
-
|
|
124
|
+
const op = new URL(request.url).searchParams.get('op');
|
|
125
|
+
if (request.method === 'GET' && op === null)
|
|
38
126
|
return respond(association);
|
|
39
127
|
const header = request.headers.get(bindingHeader);
|
|
40
128
|
if (!header || header.length > 16384)
|
|
@@ -49,6 +137,26 @@ export function createRevisionHandler(options) {
|
|
|
49
137
|
if (requested !== bindingValue(association))
|
|
50
138
|
return refuse(409);
|
|
51
139
|
const provider = selected.provider;
|
|
140
|
+
if (request.method === 'GET') {
|
|
141
|
+
if (op !== 'changes')
|
|
142
|
+
return refuse(400);
|
|
143
|
+
const cursor = new URL(request.url).searchParams.get('since');
|
|
144
|
+
if (cursor !== null && !/^\d{1,16}$/.test(cursor))
|
|
145
|
+
return refuse(400);
|
|
146
|
+
const since = cursor === null ? undefined : Number(cursor);
|
|
147
|
+
const incarnation = new URL(request.url).searchParams.get('incarnation') ?? undefined;
|
|
148
|
+
if (incarnation !== undefined && !/^[\w-]{1,64}$/.test(incarnation))
|
|
149
|
+
return refuse(400);
|
|
150
|
+
if (since !== undefined && !Number.isSafeInteger(since))
|
|
151
|
+
return refuse(400);
|
|
152
|
+
// The stream owns the lease from here: it is released when the stream ends, not when this function returns.
|
|
153
|
+
const lease = release;
|
|
154
|
+
release = undefined;
|
|
155
|
+
return changeStream(provider, access, { since, incarnation }, heartbeatMs, async () => { try {
|
|
156
|
+
await lease?.();
|
|
157
|
+
}
|
|
158
|
+
catch { /* the stream already ended */ } });
|
|
159
|
+
}
|
|
52
160
|
if (!request.headers.has(catalogHeader))
|
|
53
161
|
return await createResourceHandler({
|
|
54
162
|
authenticate: async () => access,
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import type { ResourceIdentity } from './remote-protocol.js';
|
|
2
2
|
import type { WorkspaceKey } from './workspace.js';
|
|
3
|
-
import type { FilePage, SavedRevision } from './revision-contracts.js';
|
|
3
|
+
import type { FilePage, SavedRevision, WorkspaceChangeEvent } from './revision-contracts.js';
|
|
4
4
|
export declare const bindingHeader = "x-boring-workspace";
|
|
5
5
|
export declare const catalogHeader = "x-boring-catalog";
|
|
6
6
|
export interface RevisionBinding {
|
|
@@ -12,4 +12,11 @@ export declare function binding(value: unknown): RevisionBinding;
|
|
|
12
12
|
export declare const bindingValue: (value: RevisionBinding) => string;
|
|
13
13
|
export declare function filePage(value: unknown): FilePage;
|
|
14
14
|
export declare function savedRevisions(value: unknown): readonly SavedRevision[];
|
|
15
|
+
/** One change-feed event as the server sent it, checked field by field (`data:` of `event: change | resnapshot`). */
|
|
16
|
+
export declare function changeEvent(value: unknown): WorkspaceChangeEvent;
|
|
17
|
+
/** Server-sent events from a response body: `{ event, data }` per message, comments skipped, a line longer than `limit` refused. */
|
|
18
|
+
export declare function serverEvents(body: ReadableStream<Uint8Array>, limit?: number): AsyncGenerator<{
|
|
19
|
+
readonly event: string;
|
|
20
|
+
readonly data: string;
|
|
21
|
+
}>;
|
|
15
22
|
//# sourceMappingURL=revision-protocol.d.ts.map
|
|
@@ -37,6 +37,70 @@ export function savedRevisions(value) {
|
|
|
37
37
|
const entry = record(item);
|
|
38
38
|
if (typeof entry.savedAt !== 'number' || !Number.isSafeInteger(entry.savedAt) || entry.savedAt < 0)
|
|
39
39
|
throw new RevisionError('invalid', 'Invalid save timestamp');
|
|
40
|
-
|
|
40
|
+
const source = entry.source === 'publish' || entry.source === 'agent' || entry.source === 'observed' ? entry.source : undefined;
|
|
41
|
+
return { revision: identifier(entry.revision), savedAt: entry.savedAt, ...(source ? { source } : {}) };
|
|
41
42
|
});
|
|
42
43
|
}
|
|
44
|
+
const sources = new Set(['publish', 'agent', 'poll', 'watch']);
|
|
45
|
+
const sequence = (value, what) => {
|
|
46
|
+
if (typeof value !== 'number' || !Number.isSafeInteger(value) || value < 0)
|
|
47
|
+
throw new RevisionError('invalid', `Invalid ${what}`);
|
|
48
|
+
return value;
|
|
49
|
+
};
|
|
50
|
+
/** One change-feed event as the server sent it, checked field by field (`data:` of `event: change | resnapshot`). */
|
|
51
|
+
export function changeEvent(value) {
|
|
52
|
+
const data = record(value);
|
|
53
|
+
const seq = sequence(data.seq, 'change sequence'), at = sequence(data.at, 'change time');
|
|
54
|
+
if (data.kind === 'resnapshot')
|
|
55
|
+
return { kind: 'resnapshot', seq, at };
|
|
56
|
+
if (data.kind !== 'change' || typeof data.path !== 'string' || typeof data.source !== 'string' || !sources.has(data.source))
|
|
57
|
+
throw new RevisionError('invalid', 'Invalid change event');
|
|
58
|
+
if (data.directory !== undefined && data.directory !== true)
|
|
59
|
+
throw new RevisionError('invalid', 'Invalid change event');
|
|
60
|
+
const path = data.path === '' && data.directory === true ? '' : catalogPath(data.path);
|
|
61
|
+
const revision = data.revision === null ? null : identifier(data.revision);
|
|
62
|
+
return {
|
|
63
|
+
kind: 'change', path, revision, source: data.source, seq, at,
|
|
64
|
+
...(data.directory === true ? { directory: true } : {}),
|
|
65
|
+
...(data.conversationId !== undefined ? { conversationId: identifier(data.conversationId) } : {}),
|
|
66
|
+
};
|
|
67
|
+
}
|
|
68
|
+
/** Server-sent events from a response body: `{ event, data }` per message, comments skipped, a line longer than `limit` refused. */
|
|
69
|
+
export async function* serverEvents(body, limit = 1_048_576) {
|
|
70
|
+
const reader = body.getReader(), decoder = new TextDecoder();
|
|
71
|
+
let pending = '', event = 'message', data = [];
|
|
72
|
+
try {
|
|
73
|
+
for (;;) {
|
|
74
|
+
const { done, value } = await reader.read();
|
|
75
|
+
if (done)
|
|
76
|
+
return;
|
|
77
|
+
pending += decoder.decode(value, { stream: true });
|
|
78
|
+
let newline;
|
|
79
|
+
while ((newline = pending.search(/\r?\n/)) !== -1) {
|
|
80
|
+
const line = pending.slice(0, newline);
|
|
81
|
+
pending = pending.slice(pending[newline] === '\r' ? newline + 2 : newline + 1);
|
|
82
|
+
if (line === '') {
|
|
83
|
+
if (data.length)
|
|
84
|
+
yield { event, data: data.join('\n') };
|
|
85
|
+
event = 'message';
|
|
86
|
+
data = [];
|
|
87
|
+
continue;
|
|
88
|
+
}
|
|
89
|
+
if (line.startsWith(':'))
|
|
90
|
+
continue;
|
|
91
|
+
const colon = line.indexOf(':');
|
|
92
|
+
const field = colon === -1 ? line : line.slice(0, colon);
|
|
93
|
+
const content = colon === -1 ? '' : line.slice(colon + 1).replace(/^ /, '');
|
|
94
|
+
if (field === 'event')
|
|
95
|
+
event = content;
|
|
96
|
+
else if (field === 'data')
|
|
97
|
+
data.push(content);
|
|
98
|
+
}
|
|
99
|
+
if (pending.length > limit)
|
|
100
|
+
throw new RevisionError('invalid', 'Change feed line exceeds the limit');
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
finally {
|
|
104
|
+
void reader.cancel().catch(() => { });
|
|
105
|
+
}
|
|
106
|
+
}
|
package/dist/revision.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { createResourceClient } from './remote.js';
|
|
2
|
-
import { binding, bindingHeader, bindingValue, catalogHeader, filePage, record, savedRevisions } from './revision-protocol.js';
|
|
2
|
+
import { binding, bindingHeader, bindingValue, catalogHeader, changeEvent, filePage, record, savedRevisions, serverEvents } from './revision-protocol.js';
|
|
3
3
|
import { readJsonBody } from './request-guard.js';
|
|
4
4
|
import { RevisionError } from './revision-contracts.js';
|
|
5
5
|
import { observe } from './remote-protocol.js';
|
|
@@ -45,6 +45,93 @@ export async function connectRevisionProvider(options) {
|
|
|
45
45
|
throw new RevisionError('denied', 'The response belongs to another workspace');
|
|
46
46
|
return result.value;
|
|
47
47
|
}
|
|
48
|
+
/** One feed across reconnections: each resumes after the last seq delivered; a refused binding ends it. */
|
|
49
|
+
async function* changes(request = {}) {
|
|
50
|
+
const signal = request.signal;
|
|
51
|
+
let cursor = request.since, incarnation = request.incarnation, staged, delay = 500;
|
|
52
|
+
const pause = () => new Promise(resolve => {
|
|
53
|
+
const timer = setTimeout(done, delay);
|
|
54
|
+
function done() { clearTimeout(timer); signal?.removeEventListener('abort', done); resolve(); }
|
|
55
|
+
signal?.addEventListener('abort', done, { once: true });
|
|
56
|
+
delay = Math.min(delay * 2, 30_000);
|
|
57
|
+
});
|
|
58
|
+
while (!signal?.aborted) {
|
|
59
|
+
const url = new URL(endpoint);
|
|
60
|
+
url.searchParams.set('op', 'changes');
|
|
61
|
+
if (cursor !== undefined) {
|
|
62
|
+
url.searchParams.set('since', String(cursor));
|
|
63
|
+
if (incarnation !== undefined)
|
|
64
|
+
url.searchParams.set('incarnation', incarnation);
|
|
65
|
+
}
|
|
66
|
+
const connection = new AbortController();
|
|
67
|
+
const stop = () => connection.abort();
|
|
68
|
+
signal?.addEventListener('abort', stop, { once: true });
|
|
69
|
+
try {
|
|
70
|
+
let response;
|
|
71
|
+
try {
|
|
72
|
+
response = await fetch(new Request(url, { method: 'GET', redirect: 'error', headers: { accept: 'text/event-stream', [bindingHeader]: bound }, signal: connection.signal }));
|
|
73
|
+
}
|
|
74
|
+
catch {
|
|
75
|
+
if (signal?.aborted)
|
|
76
|
+
return;
|
|
77
|
+
await pause();
|
|
78
|
+
continue;
|
|
79
|
+
}
|
|
80
|
+
if (response.redirected || !response.ok || !response.body) {
|
|
81
|
+
void response.body?.cancel().catch(() => { });
|
|
82
|
+
if (response.status === 403 || response.status === 409)
|
|
83
|
+
throw new RevisionError('denied', 'The selected workspace is unavailable or no longer authorized');
|
|
84
|
+
if (response.status === 400)
|
|
85
|
+
throw new RevisionError('invalid', 'The change feed request was refused');
|
|
86
|
+
await pause();
|
|
87
|
+
continue;
|
|
88
|
+
}
|
|
89
|
+
try {
|
|
90
|
+
for await (const message of serverEvents(response.body)) {
|
|
91
|
+
if (message.event === 'ready') {
|
|
92
|
+
const ready = record(JSON.parse(message.data)), seq = ready.seq;
|
|
93
|
+
// A new provider lifetime is adopted with its resnapshot, not with `ready`: a disconnect in between must reconnect with the
|
|
94
|
+
// old cursor and incarnation, which the server answers with a resnapshot again.
|
|
95
|
+
if (typeof ready.incarnation === 'string') {
|
|
96
|
+
if (cursor === undefined || incarnation === undefined)
|
|
97
|
+
incarnation = ready.incarnation;
|
|
98
|
+
else
|
|
99
|
+
staged = ready.incarnation;
|
|
100
|
+
}
|
|
101
|
+
if (cursor === undefined && typeof seq === 'number' && Number.isSafeInteger(seq))
|
|
102
|
+
cursor = seq;
|
|
103
|
+
delay = 500;
|
|
104
|
+
continue;
|
|
105
|
+
}
|
|
106
|
+
if (message.event !== 'change' && message.event !== 'resnapshot')
|
|
107
|
+
continue;
|
|
108
|
+
const event = changeEvent(JSON.parse(message.data));
|
|
109
|
+
if (event.kind === 'change' && cursor !== undefined && event.seq <= cursor)
|
|
110
|
+
continue;
|
|
111
|
+
if (event.kind === 'resnapshot' && staged !== undefined) {
|
|
112
|
+
incarnation = staged;
|
|
113
|
+
staged = undefined;
|
|
114
|
+
}
|
|
115
|
+
cursor = event.seq;
|
|
116
|
+
yield event;
|
|
117
|
+
}
|
|
118
|
+
}
|
|
119
|
+
catch (error) {
|
|
120
|
+
if (signal?.aborted)
|
|
121
|
+
return;
|
|
122
|
+
if (error instanceof RevisionError && error.kind === 'invalid')
|
|
123
|
+
throw error;
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
finally {
|
|
127
|
+
signal?.removeEventListener('abort', stop);
|
|
128
|
+
connection.abort();
|
|
129
|
+
}
|
|
130
|
+
if (signal?.aborted)
|
|
131
|
+
return;
|
|
132
|
+
await pause();
|
|
133
|
+
}
|
|
134
|
+
}
|
|
48
135
|
if (!client.publish || !client.lookup)
|
|
49
136
|
throw new Error('Publishing client lacks reconciliation');
|
|
50
137
|
return {
|
|
@@ -53,5 +140,6 @@ export async function connectRevisionProvider(options) {
|
|
|
53
140
|
list: async (request, signal) => filePage(await invoke('list', request, signal)),
|
|
54
141
|
search: async (request, signal) => filePage(await invoke('search', request, signal)),
|
|
55
142
|
history: async (path, signal) => savedRevisions(await invoke('history', path, signal)),
|
|
143
|
+
changes: request => changes(request),
|
|
56
144
|
};
|
|
57
145
|
}
|
|
@@ -20,7 +20,10 @@ const within = (base, path) => path === base || path.startsWith(base === '/' ? '
|
|
|
20
20
|
// system answers exactly like Pi's environments.
|
|
21
21
|
const fileError = (code, message, path) => new FileError(code, message, path);
|
|
22
22
|
const isFileError = (error) => error instanceof FileError;
|
|
23
|
-
/**
|
|
23
|
+
/**
|
|
24
|
+
* How often a watcher compares snapshots. A write through this file system (this connection) is compared at once, right after
|
|
25
|
+
* its transaction; polling remains for writes through another connection to the same database (`mode: 'polling'`).
|
|
26
|
+
*/
|
|
24
27
|
const WATCH_INTERVAL_MS = 1000;
|
|
25
28
|
/** A FileError whose message starts with the POSIX code, as shells and Git read it. */
|
|
26
29
|
const failure = (code, errno, path, what) => fileError(code, `${errno}: ${what}, '${path}'`, path);
|
|
@@ -65,6 +68,8 @@ export function openSqliteFileSystem(options) {
|
|
|
65
68
|
if (found.kind !== 'directory')
|
|
66
69
|
throw failure('not_directory', 'ENOTDIR', path, 'not a directory');
|
|
67
70
|
}
|
|
71
|
+
/** Paths written, renamed or removed by this file system since the last comparison: reported as changed even when size and time look alike. */
|
|
72
|
+
const touched = new Set();
|
|
68
73
|
function bytes(path) {
|
|
69
74
|
if (existing(path).kind === 'directory')
|
|
70
75
|
throw failure('is_directory', 'EISDIR', path, 'illegal operation on a directory');
|
|
@@ -74,6 +79,7 @@ export function openSqliteFileSystem(options) {
|
|
|
74
79
|
return Uint8Array.from(found.bytes);
|
|
75
80
|
}
|
|
76
81
|
function put(path, kind, content) {
|
|
82
|
+
touched.add(path);
|
|
77
83
|
db.run('INSERT INTO boring_workspace_files (workspace, path, parent, kind, bytes, mtime_ms) VALUES (?, ?, ?, ?, ?, ?) ON CONFLICT(workspace, path) DO UPDATE SET kind = excluded.kind, bytes = excluded.bytes, mtime_ms = excluded.mtime_ms', workspace, path, dirname(path), kind, content, now());
|
|
78
84
|
}
|
|
79
85
|
function write(path, content) {
|
|
@@ -125,13 +131,31 @@ export function openSqliteFileSystem(options) {
|
|
|
125
131
|
const incarnation = db.get('SELECT incarnation FROM boring_workspaces WHERE workspace = ?', workspace)?.incarnation;
|
|
126
132
|
if (typeof incarnation !== 'string')
|
|
127
133
|
throw new Error('The workspace has no incarnation');
|
|
134
|
+
/** Each open watcher's comparison, run right after a write through this file system commits. */
|
|
135
|
+
const watchers = new Set();
|
|
136
|
+
let notifying = false;
|
|
137
|
+
function notify() {
|
|
138
|
+
if (watchers.size === 0) {
|
|
139
|
+
touched.clear();
|
|
140
|
+
return;
|
|
141
|
+
}
|
|
142
|
+
if (notifying)
|
|
143
|
+
return;
|
|
144
|
+
notifying = true;
|
|
145
|
+
// After the current synchronous work: a batch write inside a caller's transaction is compared once it has committed.
|
|
146
|
+
queueMicrotask(() => { notifying = false; const paths = [...touched]; touched.clear(); for (const check of [...watchers])
|
|
147
|
+
check(paths); });
|
|
148
|
+
}
|
|
128
149
|
async function run(path, context, mode, work) {
|
|
129
150
|
if (context.abortSignal?.aborted)
|
|
130
151
|
return err(fileError('aborted', 'Operation aborted', path));
|
|
131
152
|
if (typeof path !== 'string' || path.includes('\0'))
|
|
132
153
|
return err(fileError('invalid', 'Invalid path', String(path)));
|
|
133
154
|
try {
|
|
134
|
-
|
|
155
|
+
const value = db.transaction(mode, () => work(normalize(path.startsWith('/') ? path : `${fs.cwd}/${path}`)));
|
|
156
|
+
if (mode === 'write')
|
|
157
|
+
notify();
|
|
158
|
+
return ok(value);
|
|
135
159
|
}
|
|
136
160
|
catch (error) {
|
|
137
161
|
return err(isFileError(error) ? error : fileError('unknown', error instanceof Error ? error.message : String(error), path));
|
|
@@ -264,6 +288,8 @@ export function openSqliteFileSystem(options) {
|
|
|
264
288
|
const moved = existing(from);
|
|
265
289
|
if (from === to)
|
|
266
290
|
return;
|
|
291
|
+
touched.add(from);
|
|
292
|
+
touched.add(to);
|
|
267
293
|
if (from === '/' || within(from, to))
|
|
268
294
|
throw failure('invalid', 'EINVAL', to, 'cannot move a directory into itself');
|
|
269
295
|
const target = row(to);
|
|
@@ -306,8 +332,9 @@ export function openSqliteFileSystem(options) {
|
|
|
306
332
|
};
|
|
307
333
|
return reader;
|
|
308
334
|
}),
|
|
309
|
-
// SQLite reports no changes, so the watcher compares snapshots of the watched rows
|
|
310
|
-
// watcher is returned, so a change made
|
|
335
|
+
// SQLite reports no changes, so the watcher compares snapshots of the watched rows: at once after each write through this file
|
|
336
|
+
// system, and every second for other connections. The first snapshot is taken before the watcher is returned, so a change made
|
|
337
|
+
// after `watch` resolves is always seen.
|
|
311
338
|
watch: async (targets, onChange, context) => {
|
|
312
339
|
if (context.abortSignal?.aborted)
|
|
313
340
|
return err(fileError('aborted', 'Operation aborted'));
|
|
@@ -326,7 +353,18 @@ export function openSqliteFileSystem(options) {
|
|
|
326
353
|
onChange(change);
|
|
327
354
|
}
|
|
328
355
|
catch { /* the host's callback cannot stop the watcher */ } };
|
|
329
|
-
const
|
|
356
|
+
const hidden = (name, target) => (target.exclude?.hidden && name.startsWith('.')) || target.exclude?.names?.includes(name);
|
|
357
|
+
/** Whether a path this file system just wrote lies under a watched target, as the snapshot would have listed it. */
|
|
358
|
+
const covered = (path) => targets.some(target => {
|
|
359
|
+
const root = normalize(target.path.startsWith('/') ? target.path : `${fs.cwd}/${target.path}`);
|
|
360
|
+
if (path === root)
|
|
361
|
+
return true;
|
|
362
|
+
if (!within(root, path))
|
|
363
|
+
return false;
|
|
364
|
+
const names = path.slice(root.length + (root === '/' ? 0 : 1)).split('/');
|
|
365
|
+
return (target.recursive || names.length === 1) && !names.some(name => hidden(name, target));
|
|
366
|
+
});
|
|
367
|
+
const check = (direct = []) => {
|
|
330
368
|
if (stopped)
|
|
331
369
|
return;
|
|
332
370
|
let next;
|
|
@@ -335,16 +373,19 @@ export function openSqliteFileSystem(options) {
|
|
|
335
373
|
}
|
|
336
374
|
catch (error) {
|
|
337
375
|
report({ error: isFileError(error) ? error : fileError('unknown', String(error)) });
|
|
338
|
-
|
|
339
|
-
clearInterval(timer);
|
|
376
|
+
stop();
|
|
340
377
|
return;
|
|
341
378
|
}
|
|
342
|
-
const
|
|
379
|
+
const written = new Set(direct.filter(covered));
|
|
380
|
+
const changed = [...new Set([...previous.keys(), ...next.keys(), ...written])].filter(path => written.has(path) || previous.get(path) !== next.get(path));
|
|
343
381
|
previous = next;
|
|
344
382
|
if (changed.length)
|
|
345
383
|
report({ paths: changed });
|
|
346
|
-
}
|
|
347
|
-
const
|
|
384
|
+
};
|
|
385
|
+
const timer = setInterval(() => check(), WATCH_INTERVAL_MS);
|
|
386
|
+
const stop = () => { stopped = true; clearInterval(timer); watchers.delete(check); };
|
|
387
|
+
watchers.add(check);
|
|
388
|
+
const watcher = { mode: 'polling', close: async () => { stop(); } };
|
|
348
389
|
return ok(watcher);
|
|
349
390
|
},
|
|
350
391
|
canonicalPath: (path, context) => run(path, context, 'read', absolute => { existing(absolute); return absolute; }),
|
|
@@ -366,6 +407,7 @@ export function openSqliteFileSystem(options) {
|
|
|
366
407
|
throw failure('permission_denied', 'EPERM', absolute, 'the workspace root cannot be removed');
|
|
367
408
|
if (found.kind === 'directory' && !options?.recursive && children(absolute).length)
|
|
368
409
|
throw failure('invalid', 'ENOTEMPTY', absolute, 'directory not empty');
|
|
410
|
+
touched.add(absolute);
|
|
369
411
|
db.run(`DELETE FROM boring_workspace_files WHERE ${subtree(absolute)}`, ...subtreeArgs(absolute));
|
|
370
412
|
}),
|
|
371
413
|
createTempDir: (prefix, context) => temporary(prefix ?? 'tmp-', '', 'directory', context),
|
|
@@ -375,7 +417,7 @@ export function openSqliteFileSystem(options) {
|
|
|
375
417
|
registerSqliteBatch(fs, {
|
|
376
418
|
connection: db,
|
|
377
419
|
bytes: path => { const found = row(normalize(path)); return !found ? null : found.kind === 'directory' ? undefined : bytes(normalize(path)); },
|
|
378
|
-
write: (path, content) => { const absolute = normalize(path); makeDirectory(dirname(absolute), true); write(absolute, content); },
|
|
420
|
+
write: (path, content) => { const absolute = normalize(path); makeDirectory(dirname(absolute), true); write(absolute, content); notify(); },
|
|
379
421
|
});
|
|
380
422
|
return fs;
|
|
381
423
|
}
|
package/dist/workspace.d.ts
CHANGED
|
@@ -1,16 +1,32 @@
|
|
|
1
1
|
import type { FileSystem } from '@earendil-works/pi-durable/env';
|
|
2
2
|
import type { PublicationLookupResult, PublicationResult, ReadResult, ResourceAccess, ResourceProvider } from './contracts.js';
|
|
3
3
|
import type { WorkspaceCatalogOptions } from './workspace-catalog.js';
|
|
4
|
-
import type { WorkspaceCatalog } from './revision-contracts.js';
|
|
4
|
+
import type { ChangeFeedRequest, WorkspaceCatalog, WorkspaceChangeEvent } from './revision-contracts.js';
|
|
5
|
+
export type { ChangeFeedRequest, WorkspaceChangeEvent } from './revision-contracts.js';
|
|
5
6
|
export type { WorkspaceCatalogOptions } from './workspace-catalog.js';
|
|
6
|
-
import type { WorkspaceJournal } from './journal.js';
|
|
7
|
+
import type { HistorySource, WorkspaceJournal } from './journal.js';
|
|
7
8
|
import type { WorkspaceKey } from './workspace-identity.js';
|
|
8
9
|
export type { WorkspaceKey } from './workspace-identity.js';
|
|
9
10
|
export interface WorkspaceChange {
|
|
10
11
|
readonly path: string;
|
|
11
|
-
/** `null` when the file is absent. */
|
|
12
|
+
/** `null` when the file is absent (or, from a watcher, when the path is a directory). */
|
|
12
13
|
readonly revision: string | null;
|
|
13
|
-
|
|
14
|
+
/** `publish`: a conditional write. `agent`: `record` (a Pi `write` or `edit`). `poll`: found by hashing. `watch`: the file system reported it. */
|
|
15
|
+
readonly source: 'publish' | 'agent' | 'poll' | 'watch';
|
|
16
|
+
}
|
|
17
|
+
/** What `record` is told about an agent write (ruling 2: observed, never receipted). */
|
|
18
|
+
export interface AgentWrite {
|
|
19
|
+
readonly path: string;
|
|
20
|
+
/** The file before the write, when it existed: its revision and the bytes of that revision (kept in history as the replaced version). */
|
|
21
|
+
readonly before?: {
|
|
22
|
+
readonly revision: string;
|
|
23
|
+
readonly bytes: Uint8Array;
|
|
24
|
+
} | null | undefined;
|
|
25
|
+
/** The revision the write produced, as the caller observed it right after. */
|
|
26
|
+
readonly after: string;
|
|
27
|
+
readonly source: 'agent';
|
|
28
|
+
/** The Pi conversation that wrote it, carried on the change event. */
|
|
29
|
+
readonly conversationId?: string | undefined;
|
|
14
30
|
}
|
|
15
31
|
/** Runs one piece of work at a time, in order. A failed piece does not stop the ones behind it. */
|
|
16
32
|
export interface WorkspaceQueue {
|
|
@@ -27,25 +43,62 @@ export interface WorkspaceResourceProvider extends ResourceProvider {
|
|
|
27
43
|
};
|
|
28
44
|
/** The workspace's mutation queue. Native tool calls that must not interleave with a conditional write run through it. */
|
|
29
45
|
readonly queue: WorkspaceQueue;
|
|
46
|
+
/**
|
|
47
|
+
* Record a write the agent already made through the native tools (Pi `write`/`edit`, called by the file guard right after the
|
|
48
|
+
* tool succeeded): the file's history gains the written revision (source `agent`, and the replaced one when `before` carries its
|
|
49
|
+
* bytes) and the change feed an `agent` event. It writes history rows only: no publish, no receipt, no operation or intent rows,
|
|
50
|
+
* so agent writes are observed, not published. It does not enter `queue` (the guard already runs inside it), so it is safe to call from queued work.
|
|
51
|
+
* When the bytes there no longer have the `after` revision (a shell wrote in between), only the change event is recorded.
|
|
52
|
+
*/
|
|
53
|
+
readonly record: (write: AgentWrite, access: ResourceAccess) => Promise<void>;
|
|
54
|
+
/**
|
|
55
|
+
* The change feed: every write this provider makes or records, every change `poll` finds and, while someone is listening, every
|
|
56
|
+
* change the file system's `watch` reports (the SQLite file system reports its own connection's writes at once). Ordered by
|
|
57
|
+
* `seq` and replayable from `since` within a bounded window; a gap yields a `resnapshot` event. The iterator ends when
|
|
58
|
+
* `request.signal` or `access.signal` aborts, or the provider closes. Events are invalidation hints: read the bytes again.
|
|
59
|
+
*
|
|
60
|
+
* After the last listener leaves, the watcher stays open for `watchLingerMs` and then closes. Changes made while no watcher was
|
|
61
|
+
* open are unknown, so the first listener afterwards makes the feed append a `resnapshot`: a client that reconnects after a
|
|
62
|
+
* longer absence reloads everything once, and one that stays connected is never affected.
|
|
63
|
+
*/
|
|
64
|
+
readonly changes: (request: ChangeFeedRequest, access: ResourceAccess) => AsyncIterable<WorkspaceChangeEvent>;
|
|
65
|
+
/** The seq of the newest change event (the cursor of "now" for `changes({ since })`). */
|
|
66
|
+
readonly changeHead: () => number;
|
|
67
|
+
/** Names this provider's lifetime; a cursor from another one (`ChangeFeedRequest.incarnation`) is answered with a `resnapshot`. */
|
|
68
|
+
readonly changeIncarnation: string;
|
|
69
|
+
/** Ends every change stream, closes the file watcher and drops the change listeners. Idempotent; the provider still reads and publishes. */
|
|
70
|
+
readonly close: () => void;
|
|
71
|
+
/** Aborts when the provider closes, so a change stream nobody is reading still ends and releases its lease. */
|
|
72
|
+
readonly lifetime: AbortSignal;
|
|
30
73
|
/** Revisions still retained for a file, newest first. */
|
|
31
74
|
readonly history: (path: string) => string[];
|
|
32
75
|
/** The same revisions with the time each was saved (milliseconds since the epoch; 0 when unknown), newest first. */
|
|
33
76
|
readonly saves: (path: string) => {
|
|
34
77
|
readonly revision: string;
|
|
35
78
|
readonly savedAt: number;
|
|
79
|
+
readonly source: HistorySource;
|
|
36
80
|
}[];
|
|
37
81
|
/** Read the file as it is now and retain that revision in its history (what `present` shows). Serialised with writes. */
|
|
38
82
|
readonly keep: (path: string, access: ResourceAccess) => Promise<ReadResult>;
|
|
39
83
|
/** Provider writes and polled detections. The listener must not throw. */
|
|
40
84
|
readonly onChange: (listener: (change: WorkspaceChange) => void) => () => void;
|
|
41
|
-
/**
|
|
42
|
-
|
|
85
|
+
/**
|
|
86
|
+
* Hash each path now and report those that differ from the last revision this provider wrote or polled. Finds shell writes.
|
|
87
|
+
* Without `paths`, every file of the workspace (except `.git` and `node_modules`): the first such call takes the baseline and
|
|
88
|
+
* reports nothing; later calls also report new and removed files. A host whose file system cannot `watch` calls it at the end of
|
|
89
|
+
* each agent turn (after `bash`), so the change feed still sees shell writes.
|
|
90
|
+
*/
|
|
91
|
+
readonly poll: (paths?: readonly string[], signal?: AbortSignal) => Promise<WorkspaceChange[]>;
|
|
43
92
|
}
|
|
44
93
|
export interface WorkspaceProviderOptions {
|
|
45
94
|
readonly identity: WorkspaceKey;
|
|
46
95
|
readonly fs: FileSystem;
|
|
47
96
|
readonly journal: WorkspaceJournal;
|
|
48
97
|
readonly catalog?: WorkspaceCatalogOptions;
|
|
98
|
+
/** How many change events the feed retains for replay (default 1024). An older cursor gets a `resnapshot`. */
|
|
99
|
+
readonly changeBuffer?: number;
|
|
100
|
+
/** How long the watcher stays open after the last change-feed listener leaves, in milliseconds (default 30 000). */
|
|
101
|
+
readonly watchLingerMs?: number;
|
|
49
102
|
}
|
|
50
103
|
/** The Git blob id of `bytes` (`sha1("blob <length>\0" + bytes)`), computed without Git. */
|
|
51
104
|
export declare function blobRevision(bytes: Uint8Array): Promise<string>;
|
package/dist/workspace.js
CHANGED
|
@@ -71,14 +71,38 @@ export function createWorkspaceProvider(options) {
|
|
|
71
71
|
principal: access.principalId, initiator: access.initiatorId, operation: operationId,
|
|
72
72
|
});
|
|
73
73
|
const ownEvidence = (evidence) => evidence.startsWith(`${key.incarnation}:`);
|
|
74
|
-
|
|
75
|
-
|
|
74
|
+
// ---- The change feed: a bounded, seq-ordered buffer of invalidation hints ----------------------------------------------------
|
|
75
|
+
const bufferLimit = options.changeBuffer ?? 1024;
|
|
76
|
+
if (!Number.isSafeInteger(bufferLimit) || bufferLimit < 1)
|
|
77
|
+
throw new RangeError('The change buffer must be a positive integer');
|
|
78
|
+
const lingerMs = options.watchLingerMs ?? 30_000;
|
|
79
|
+
// Sequence numbers start from the clock (microseconds), so a cursor from an earlier provider lifetime is older than this one's
|
|
80
|
+
// first event and reads as a gap, never as a position in this lifetime.
|
|
81
|
+
const firstSeq = Date.now() * 1000;
|
|
82
|
+
let head = firstSeq;
|
|
83
|
+
// Names this provider lifetime. A cursor from another lifetime is refused by this, not only by the clock: the clock may go back.
|
|
84
|
+
const incarnationOfFeed = randomUUID();
|
|
85
|
+
const lifetime = new AbortController();
|
|
86
|
+
const buffer = [];
|
|
87
|
+
const waiting = new Set();
|
|
88
|
+
function append(event) {
|
|
89
|
+
head += 1;
|
|
90
|
+
buffer.push(Object.freeze({ ...event, seq: head, at: Date.now() }));
|
|
91
|
+
if (buffer.length > bufferLimit)
|
|
92
|
+
buffer.shift();
|
|
93
|
+
for (const wake of [...waiting])
|
|
94
|
+
wake();
|
|
95
|
+
}
|
|
96
|
+
function emit(change, extra = {}) {
|
|
97
|
+
if (!extra.directory)
|
|
98
|
+
known.set(change.path, change.revision);
|
|
76
99
|
for (const listener of [...listeners]) {
|
|
77
100
|
try {
|
|
78
101
|
listener(change);
|
|
79
102
|
}
|
|
80
103
|
catch { /* a listener failure cannot undo a write */ }
|
|
81
104
|
}
|
|
105
|
+
append({ kind: 'change', ...change, ...extra });
|
|
82
106
|
}
|
|
83
107
|
const dirname = (path) => path.slice(0, path.lastIndexOf('/')) || '/';
|
|
84
108
|
const inside = (base, path) => path === base || path.startsWith(base === '/' ? '/' : `${base}/`);
|
|
@@ -182,7 +206,7 @@ export function createWorkspaceProvider(options) {
|
|
|
182
206
|
const result = await read({ target: { resource: { providerId, path }, view: { kind: 'published' } }, revision: { kind: 'latest' } }, access);
|
|
183
207
|
if (result.kind === 'available') {
|
|
184
208
|
const path = result.snapshot.ref.resource.path;
|
|
185
|
-
journal.remember(historyScope, path, { revision: result.snapshot.ref.revision, bytes: result.snapshot.bytes, mediaType: result.snapshot.mediaType, savedAt: await savedAtOf(path) });
|
|
209
|
+
journal.remember(historyScope, path, { revision: result.snapshot.ref.revision, bytes: result.snapshot.bytes, mediaType: result.snapshot.mediaType, savedAt: await savedAtOf(path), source: 'observed' });
|
|
186
210
|
}
|
|
187
211
|
return result;
|
|
188
212
|
});
|
|
@@ -276,11 +300,11 @@ export function createWorkspaceProvider(options) {
|
|
|
276
300
|
}
|
|
277
301
|
const evidence = `${key.incarnation}:${randomUUID()}`;
|
|
278
302
|
const versions = plans.flatMap(({ change, path, before, after, unchanged, beforeSavedAt }) => {
|
|
279
|
-
const kept = (revision, bytes, mediaType, savedAt) => ({ scope: historyScope, path, version: { revision, bytes, mediaType, savedAt } });
|
|
303
|
+
const kept = (revision, bytes, mediaType, savedAt, source) => ({ scope: historyScope, path, version: { revision, bytes, mediaType, savedAt, source } });
|
|
280
304
|
return unchanged ? [] : [
|
|
281
305
|
...(change.kind === 'replace' && before?.kind === 'file'
|
|
282
|
-
? [kept(before.revision, before.bytes, journal.version(historyScope, path, before.revision)?.mediaType ?? mediaTypeOf(path), beforeSavedAt)] : []),
|
|
283
|
-
kept(after, change.bytes, change.mediaType, Date.now()),
|
|
306
|
+
? [kept(before.revision, before.bytes, journal.version(historyScope, path, before.revision)?.mediaType ?? mediaTypeOf(path), beforeSavedAt, 'observed')] : []),
|
|
307
|
+
kept(after, change.bytes, change.mediaType, Date.now(), 'publish'),
|
|
284
308
|
];
|
|
285
309
|
});
|
|
286
310
|
const record = {
|
|
@@ -343,7 +367,7 @@ export function createWorkspaceProvider(options) {
|
|
|
343
367
|
}
|
|
344
368
|
for (const { path, after, unchanged } of plans)
|
|
345
369
|
if (!unchanged)
|
|
346
|
-
emit({ path, revision: after, source: '
|
|
370
|
+
emit({ path, revision: after, source: 'publish' });
|
|
347
371
|
return { kind: 'committed', receipt: receipt(request.operationId, access, record) };
|
|
348
372
|
}
|
|
349
373
|
catch (error) {
|
|
@@ -369,20 +393,63 @@ export function createWorkspaceProvider(options) {
|
|
|
369
393
|
? { kind: 'unknown', operationId: id, reason: 'The operation began and has no recorded outcome' }
|
|
370
394
|
: { kind: 'not-found' };
|
|
371
395
|
};
|
|
396
|
+
/** Every file of the workspace, relative to the root (not `.git`, `node_modules` or the provider's temporary files). */
|
|
397
|
+
async function everyFile(signal) {
|
|
398
|
+
const found = [];
|
|
399
|
+
let scanned = 0;
|
|
400
|
+
const walk = async (directory) => {
|
|
401
|
+
const listing = await fs.listDir(directory ? absolute(directory) : root, fsContext(signal));
|
|
402
|
+
if (!listing.ok)
|
|
403
|
+
return true;
|
|
404
|
+
for (const entry of listing.value) {
|
|
405
|
+
if (++scanned > 100_000)
|
|
406
|
+
return false;
|
|
407
|
+
const path = directory ? `${directory}/${entry.name}` : entry.name;
|
|
408
|
+
if (entry.kind === 'directory') {
|
|
409
|
+
if (entry.name !== '.git' && entry.name !== 'node_modules' && !await walk(path))
|
|
410
|
+
return false;
|
|
411
|
+
}
|
|
412
|
+
else if (entry.kind === 'file' && !isTemporary(path))
|
|
413
|
+
found.push(path);
|
|
414
|
+
}
|
|
415
|
+
return true;
|
|
416
|
+
};
|
|
417
|
+
return await walk('') ? found : undefined;
|
|
418
|
+
}
|
|
419
|
+
let baselined = false;
|
|
372
420
|
const poll = async (paths, signal) => {
|
|
373
421
|
const changed = [];
|
|
374
422
|
await sweep();
|
|
375
|
-
|
|
423
|
+
let inputs;
|
|
424
|
+
let report = (path) => known.has(path);
|
|
425
|
+
if (paths === undefined) {
|
|
426
|
+
const files = await everyFile(signal);
|
|
427
|
+
// Too many files to hash on each turn: say so to every listener instead of reporting a partial picture.
|
|
428
|
+
if (files === undefined) {
|
|
429
|
+
append({ kind: 'resnapshot' });
|
|
430
|
+
return changed;
|
|
431
|
+
}
|
|
432
|
+
const present = new Set(files);
|
|
433
|
+
const removed = [...known].filter(([path, revision]) => revision !== null && !present.has(path)).map(([path]) => path);
|
|
434
|
+
inputs = [...files, ...removed];
|
|
435
|
+
if (baselined)
|
|
436
|
+
report = () => true;
|
|
437
|
+
baselined = true;
|
|
438
|
+
}
|
|
439
|
+
else
|
|
440
|
+
inputs = paths;
|
|
441
|
+
for (const input of inputs) {
|
|
376
442
|
const path = locator({ resource: { providerId, path: input }, view: { kind: 'published' } }).resource.path;
|
|
377
443
|
const now = await observe(path, signal);
|
|
378
444
|
if (now.kind === 'error' || now.kind === 'outside' || isTemporary(path))
|
|
379
445
|
continue;
|
|
380
446
|
const revision = now.kind === 'file' ? now.revision : null;
|
|
381
|
-
|
|
382
|
-
|
|
447
|
+
if (known.has(path) && known.get(path) === revision)
|
|
448
|
+
continue;
|
|
449
|
+
if (!known.has(path) && revision === null && paths === undefined)
|
|
383
450
|
continue;
|
|
384
451
|
const change = { path, revision, source: 'poll' };
|
|
385
|
-
if (
|
|
452
|
+
if (!report(path))
|
|
386
453
|
known.set(path, revision);
|
|
387
454
|
else {
|
|
388
455
|
emit(change);
|
|
@@ -391,10 +458,181 @@ export function createWorkspaceProvider(options) {
|
|
|
391
458
|
}
|
|
392
459
|
return changed;
|
|
393
460
|
};
|
|
461
|
+
const record = async (write, context) => {
|
|
462
|
+
const access = accessSnapshot(context);
|
|
463
|
+
if (write.source !== 'agent')
|
|
464
|
+
throw new TypeError('Only agent writes are recorded');
|
|
465
|
+
const path = locator({ resource: { providerId, path: write.path }, view: { kind: 'published' } }).resource.path;
|
|
466
|
+
const after = identifier(write.after);
|
|
467
|
+
const conversationId = write.conversationId === undefined ? undefined : identifier(write.conversationId);
|
|
468
|
+
// Known before the await: a watcher event for this same write, handled meanwhile, then finds nothing new and emits nothing twice.
|
|
469
|
+
if (!isTemporary(path))
|
|
470
|
+
known.set(path, after);
|
|
471
|
+
// Deliberately not `queue.run`: the file guard calls this from inside the queue, and the queue runs one piece at a time.
|
|
472
|
+
const now = await observe(path, access.signal);
|
|
473
|
+
if (now.kind === 'outside' || now.kind === 'error')
|
|
474
|
+
return;
|
|
475
|
+
if (now.kind === 'file' && now.revision === after) {
|
|
476
|
+
const mediaType = journal.version(historyScope, path, after)?.mediaType ?? mediaTypeOf(path);
|
|
477
|
+
const before = write.before ?? undefined;
|
|
478
|
+
// The replaced bytes are kept only when they really are that revision.
|
|
479
|
+
if (before && before.revision !== after && before.bytes instanceof Uint8Array && await blobRevision(before.bytes) === identifier(before.revision)) {
|
|
480
|
+
journal.remember(historyScope, path, { revision: before.revision, bytes: Uint8Array.from(before.bytes), mediaType: journal.version(historyScope, path, before.revision)?.mediaType ?? mediaType, savedAt: 0, source: 'observed' });
|
|
481
|
+
}
|
|
482
|
+
journal.remember(historyScope, path, { revision: after, bytes: now.bytes, mediaType, savedAt: Date.now(), source: 'agent' });
|
|
483
|
+
}
|
|
484
|
+
emit({ path, revision: now.kind === 'file' ? now.revision : null, source: 'agent' }, conversationId === undefined ? {} : { conversationId });
|
|
485
|
+
};
|
|
486
|
+
// ---- The watcher: open while someone listens to the feed (and `lingerMs` after), when the file system can watch ----------------
|
|
487
|
+
let listening = 0;
|
|
488
|
+
let watcher;
|
|
489
|
+
let watchUnsupported = false, watchedBefore = false;
|
|
490
|
+
let linger;
|
|
491
|
+
let watchWork = Promise.resolve();
|
|
492
|
+
const relativeTo = (absolutePath, base) => {
|
|
493
|
+
base = base.replace(/\/+$/, '') || '/';
|
|
494
|
+
if (absolutePath === base)
|
|
495
|
+
return '';
|
|
496
|
+
const prefix = base === '/' ? '/' : `${base}/`;
|
|
497
|
+
return absolutePath.startsWith(prefix) ? absolutePath.slice(prefix.length) : undefined;
|
|
498
|
+
};
|
|
499
|
+
async function watched(input) {
|
|
500
|
+
// The watcher may name the root as given or as resolved (a symbolic link on the way to the workspace).
|
|
501
|
+
const real = await realRoot();
|
|
502
|
+
const path = relativeTo(input, root) ?? (real === undefined ? undefined : relativeTo(input, real));
|
|
503
|
+
if (path === undefined || isTemporary(path) || path.split('/').some(name => name === '.git' || name === 'node_modules'))
|
|
504
|
+
return;
|
|
505
|
+
if (path === '') {
|
|
506
|
+
emit({ path, revision: null, source: 'watch' }, { directory: true });
|
|
507
|
+
return;
|
|
508
|
+
}
|
|
509
|
+
const info = await fs.fileInfo(absolute(path), fsContext());
|
|
510
|
+
if (info.ok && info.value.kind === 'directory') {
|
|
511
|
+
emit({ path, revision: null, source: 'watch' }, { directory: true });
|
|
512
|
+
return;
|
|
513
|
+
}
|
|
514
|
+
const now = await observe(path);
|
|
515
|
+
if (now.kind === 'error' || now.kind === 'outside')
|
|
516
|
+
return;
|
|
517
|
+
const revision = now.kind === 'file' ? now.revision : null;
|
|
518
|
+
if (known.has(path) && known.get(path) === revision)
|
|
519
|
+
return;
|
|
520
|
+
emit({ path, revision, source: 'watch' });
|
|
521
|
+
}
|
|
522
|
+
function onWatch(change) {
|
|
523
|
+
if ('paths' in change) {
|
|
524
|
+
const paths = [...change.paths];
|
|
525
|
+
watchWork = watchWork.then(async () => { for (const path of paths)
|
|
526
|
+
await watched(path).catch(() => undefined); });
|
|
527
|
+
return;
|
|
528
|
+
}
|
|
529
|
+
// Overflow or a failed watcher: what changed is unknown, so every listener reloads. A failed watcher is not reopened (it would
|
|
530
|
+
// fail again, for example on a tree past the environment's limit); the host's `poll()` at turn end remains.
|
|
531
|
+
if ('error' in change) {
|
|
532
|
+
watcher = undefined;
|
|
533
|
+
watchUnsupported = true;
|
|
534
|
+
}
|
|
535
|
+
append({ kind: 'resnapshot' });
|
|
536
|
+
}
|
|
537
|
+
function startWatching() {
|
|
538
|
+
if (lifetime.signal.aborted)
|
|
539
|
+
return;
|
|
540
|
+
if (linger !== undefined) {
|
|
541
|
+
clearTimeout(linger);
|
|
542
|
+
linger = undefined;
|
|
543
|
+
}
|
|
544
|
+
if (watcher || watchUnsupported)
|
|
545
|
+
return;
|
|
546
|
+
// Changes made while nobody watched were not seen: a listener resuming from before that reloads.
|
|
547
|
+
if (watchedBefore)
|
|
548
|
+
append({ kind: 'resnapshot' });
|
|
549
|
+
watchedBefore = true;
|
|
550
|
+
watcher = fs.watch([{ path: root, recursive: true, exclude: { names: ['.git', 'node_modules'] } }], onWatch, fsContext()).then(result => {
|
|
551
|
+
if (result.ok)
|
|
552
|
+
return result.value;
|
|
553
|
+
watchUnsupported = true;
|
|
554
|
+
return undefined;
|
|
555
|
+
}, () => { watchUnsupported = true; return undefined; });
|
|
556
|
+
}
|
|
557
|
+
function stopWatching() {
|
|
558
|
+
const current = watcher;
|
|
559
|
+
watcher = undefined;
|
|
560
|
+
void current?.then(value => value?.close(fsContext()));
|
|
561
|
+
}
|
|
562
|
+
async function* changes(request, context) {
|
|
563
|
+
const access = accessSnapshot(context);
|
|
564
|
+
identifier(access.scopeId);
|
|
565
|
+
const since = request.since;
|
|
566
|
+
if (since !== undefined && (!Number.isSafeInteger(since) || since < 0))
|
|
567
|
+
throw new TypeError('Invalid change cursor');
|
|
568
|
+
if (lifetime.signal.aborted)
|
|
569
|
+
return;
|
|
570
|
+
const signals = [request.signal, access.signal, lifetime.signal].filter((value) => value !== undefined);
|
|
571
|
+
const signal = signals.length ? AbortSignal.any(signals) : new AbortController().signal;
|
|
572
|
+
listening += 1;
|
|
573
|
+
startWatching();
|
|
574
|
+
try {
|
|
575
|
+
let cursor = head;
|
|
576
|
+
if (since !== undefined) {
|
|
577
|
+
const oldest = buffer[0]?.seq ?? head + 1;
|
|
578
|
+
const foreign = request.incarnation !== undefined && request.incarnation !== incarnationOfFeed;
|
|
579
|
+
if (foreign || since > head || since < firstSeq || since < oldest - 1)
|
|
580
|
+
yield { kind: 'resnapshot', seq: head, at: Date.now() };
|
|
581
|
+
else
|
|
582
|
+
cursor = since;
|
|
583
|
+
}
|
|
584
|
+
while (!signal.aborted) {
|
|
585
|
+
const oldest = buffer[0];
|
|
586
|
+
// This listener fell behind the retained window: it reloads, then continues with what is retained.
|
|
587
|
+
if (oldest && oldest.seq > cursor + 1) {
|
|
588
|
+
cursor = oldest.seq - 1;
|
|
589
|
+
yield { kind: 'resnapshot', seq: cursor, at: Date.now() };
|
|
590
|
+
continue;
|
|
591
|
+
}
|
|
592
|
+
const next = buffer.find(event => event.seq > cursor);
|
|
593
|
+
if (next) {
|
|
594
|
+
cursor = next.seq;
|
|
595
|
+
// The access projection: nothing after revocation, never the provider's temporary files.
|
|
596
|
+
if (signal.aborted)
|
|
597
|
+
return;
|
|
598
|
+
if (next.kind === 'change' && isTemporary(next.path))
|
|
599
|
+
continue;
|
|
600
|
+
yield next;
|
|
601
|
+
continue;
|
|
602
|
+
}
|
|
603
|
+
await new Promise(resolve => {
|
|
604
|
+
const wake = () => { waiting.delete(wake); signal.removeEventListener('abort', wake); resolve(); };
|
|
605
|
+
waiting.add(wake);
|
|
606
|
+
signal.addEventListener('abort', wake, { once: true });
|
|
607
|
+
});
|
|
608
|
+
}
|
|
609
|
+
}
|
|
610
|
+
finally {
|
|
611
|
+
listening -= 1;
|
|
612
|
+
if (listening === 0 && watcher && !lifetime.signal.aborted) {
|
|
613
|
+
linger = setTimeout(() => { linger = undefined; if (listening === 0)
|
|
614
|
+
stopWatching(); }, lingerMs);
|
|
615
|
+
linger.unref?.();
|
|
616
|
+
}
|
|
617
|
+
}
|
|
618
|
+
}
|
|
394
619
|
return {
|
|
395
620
|
providerId, read, queue, keep, identity: Object.freeze(key),
|
|
396
621
|
catalog: createWorkspaceCatalog({ fs, ...(options.catalog ? { config: options.catalog } : {}), saves: path => journal.saves(historyScope, path) }),
|
|
397
|
-
publication: { publish }, reconciliation: { lookup }, poll,
|
|
622
|
+
publication: { publish }, reconciliation: { lookup }, poll, record, changes, changeHead: () => head, changeIncarnation: incarnationOfFeed,
|
|
623
|
+
lifetime: lifetime.signal,
|
|
624
|
+
close: () => {
|
|
625
|
+
// Every change stream ends, the watcher closes and the listeners are dropped: nothing of this provider keeps running.
|
|
626
|
+
lifetime.abort();
|
|
627
|
+
if (linger !== undefined) {
|
|
628
|
+
clearTimeout(linger);
|
|
629
|
+
linger = undefined;
|
|
630
|
+
}
|
|
631
|
+
stopWatching();
|
|
632
|
+
listeners.clear();
|
|
633
|
+
for (const wake of [...waiting])
|
|
634
|
+
wake();
|
|
635
|
+
},
|
|
398
636
|
history: path => journal.revisions(historyScope, locator({ resource: { providerId, path }, view: { kind: 'published' } }).resource.path),
|
|
399
637
|
saves: path => journal.saves(historyScope, locator({ resource: { providerId, path }, view: { kind: 'published' } }).resource.path),
|
|
400
638
|
onChange: listener => { listeners.add(listener); return () => { listeners.delete(listener); }; },
|