sensemaking 0.25.0 → 0.26.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -10
- package/dist/cjs/cli/index.d.cts +1 -1
- package/dist/cjs/cli/index.d.ts +1 -1
- package/dist/cjs/cli/index.js +1 -1
- package/dist/cjs/cli/index.js.map +1 -1
- package/dist/cjs/cli/path.js +5 -29
- package/dist/cjs/cli/path.js.map +1 -1
- package/dist/cjs/cli/related.js +4 -3
- package/dist/cjs/cli/related.js.map +1 -1
- package/dist/cjs/commands/index.d.cts +6 -3
- package/dist/cjs/commands/index.d.ts +6 -3
- package/dist/cjs/commands/index.js +4 -0
- package/dist/cjs/commands/index.js.map +1 -1
- package/dist/cjs/commands/map.d.cts +14 -5
- package/dist/cjs/commands/map.d.ts +14 -5
- package/dist/cjs/commands/map.js +42 -35
- package/dist/cjs/commands/map.js.map +1 -1
- package/dist/cjs/commands/path.d.cts +14 -0
- package/dist/cjs/commands/path.d.ts +14 -0
- package/dist/cjs/commands/path.js +209 -0
- package/dist/cjs/commands/path.js.map +1 -0
- package/dist/cjs/commands/peek.d.cts +16 -1
- package/dist/cjs/commands/peek.d.ts +16 -1
- package/dist/cjs/commands/peek.js +71 -33
- package/dist/cjs/commands/peek.js.map +1 -1
- package/dist/cjs/commands/related.d.cts +16 -3
- package/dist/cjs/commands/related.d.ts +16 -3
- package/dist/cjs/commands/related.js +131 -98
- package/dist/cjs/commands/related.js.map +1 -1
- package/dist/cjs/commands/scope.d.cts +1 -0
- package/dist/cjs/commands/scope.d.ts +1 -0
- package/dist/cjs/commands/scope.js +149 -0
- package/dist/cjs/commands/scope.js.map +1 -1
- package/dist/cjs/commands/search.d.cts +26 -3
- package/dist/cjs/commands/search.d.ts +26 -3
- package/dist/cjs/commands/search.js +211 -71
- package/dist/cjs/commands/search.js.map +1 -1
- package/dist/cjs/commands/signals.d.cts +4 -3
- package/dist/cjs/commands/signals.d.ts +4 -3
- package/dist/cjs/commands/signals.js +13 -4
- package/dist/cjs/commands/signals.js.map +1 -1
- package/dist/cjs/commands/snapshot.d.cts +3 -0
- package/dist/cjs/commands/snapshot.d.ts +3 -0
- package/dist/cjs/commands/snapshot.js +180 -0
- package/dist/cjs/commands/snapshot.js.map +1 -0
- package/dist/cjs/embed/cohere.js +41 -16
- package/dist/cjs/embed/cohere.js.map +1 -1
- package/dist/cjs/embed/endpoint.d.cts +2 -0
- package/dist/cjs/embed/endpoint.d.ts +2 -0
- package/dist/cjs/embed/endpoint.js +30 -0
- package/dist/cjs/embed/endpoint.js.map +1 -0
- package/dist/cjs/embed/handoff.d.cts +7 -2
- package/dist/cjs/embed/handoff.d.ts +7 -2
- package/dist/cjs/embed/handoff.js +10 -0
- package/dist/cjs/embed/handoff.js.map +1 -1
- package/dist/cjs/embed/http.d.cts +18 -1
- package/dist/cjs/embed/http.d.ts +18 -1
- package/dist/cjs/embed/http.js +239 -40
- package/dist/cjs/embed/http.js.map +1 -1
- package/dist/cjs/embed/openai.js +39 -17
- package/dist/cjs/embed/openai.js.map +1 -1
- package/dist/cjs/embed/query.d.cts +9 -3
- package/dist/cjs/embed/query.d.ts +9 -3
- package/dist/cjs/embed/query.js +257 -90
- package/dist/cjs/embed/query.js.map +1 -1
- package/dist/cjs/embed/registry.d.cts +2 -2
- package/dist/cjs/embed/registry.d.ts +2 -2
- package/dist/cjs/embed/registry.js +36 -4
- package/dist/cjs/embed/registry.js.map +1 -1
- package/dist/cjs/embed/static.js +6 -2
- package/dist/cjs/embed/static.js.map +1 -1
- package/dist/cjs/embed/store.js +42 -77
- package/dist/cjs/embed/store.js.map +1 -1
- package/dist/cjs/embed/types.d.cts +5 -2
- package/dist/cjs/embed/types.d.ts +5 -2
- package/dist/cjs/embed/types.js.map +1 -1
- package/dist/cjs/features/embed.js +290 -43
- package/dist/cjs/features/embed.js.map +1 -1
- package/dist/cjs/graph/traverse.d.cts +2 -2
- package/dist/cjs/graph/traverse.d.ts +2 -2
- package/dist/cjs/graph/traverse.js +15 -5
- package/dist/cjs/graph/traverse.js.map +1 -1
- package/dist/cjs/index.d.cts +2 -2
- package/dist/cjs/index.d.ts +2 -2
- package/dist/cjs/index.js +6 -0
- package/dist/cjs/index.js.map +1 -1
- package/dist/cjs/output/output.d.cts +4 -36
- package/dist/cjs/output/output.d.ts +4 -36
- package/dist/cjs/output/output.js +1 -2
- package/dist/cjs/output/output.js.map +1 -1
- package/dist/cjs/scan/pool.d.cts +2 -1
- package/dist/cjs/scan/pool.d.ts +2 -1
- package/dist/cjs/scan/pool.js +116 -27
- package/dist/cjs/scan/pool.js.map +1 -1
- package/dist/cjs/scan/reparse.d.cts +1 -0
- package/dist/cjs/scan/reparse.d.ts +1 -0
- package/dist/cjs/scan/reparse.js +75 -21
- package/dist/cjs/scan/reparse.js.map +1 -1
- package/dist/cjs/store/builder.d.cts +11 -21
- package/dist/cjs/store/builder.d.ts +11 -21
- package/dist/cjs/store/builder.js +1077 -438
- package/dist/cjs/store/builder.js.map +1 -1
- package/dist/cjs/store/duckdb/connection.js +16 -2
- package/dist/cjs/store/duckdb/connection.js.map +1 -1
- package/dist/cjs/store/duckdb/open.d.cts +1 -1
- package/dist/cjs/store/duckdb/open.d.ts +1 -1
- package/dist/cjs/store/duckdb/open.js +73 -65
- package/dist/cjs/store/duckdb/open.js.map +1 -1
- package/dist/cjs/store/duckdb/reconcile.js +533 -15
- package/dist/cjs/store/duckdb/reconcile.js.map +1 -1
- package/dist/cjs/store/embed-scope.js +19 -9
- package/dist/cjs/store/embed-scope.js.map +1 -1
- package/dist/cjs/store/index.d.cts +6 -0
- package/dist/cjs/store/index.d.ts +6 -0
- package/dist/cjs/store/index.js +8 -4
- package/dist/cjs/store/index.js.map +1 -1
- package/dist/cjs/store/open.d.cts +6 -1
- package/dist/cjs/store/open.d.ts +6 -1
- package/dist/cjs/store/open.js +230 -307
- package/dist/cjs/store/open.js.map +1 -1
- package/dist/cjs/store/readiness.d.cts +5 -0
- package/dist/cjs/store/readiness.d.ts +5 -0
- package/dist/cjs/store/readiness.js +39 -0
- package/dist/cjs/store/readiness.js.map +1 -0
- package/dist/cjs/store/reconcile.d.cts +26 -1
- package/dist/cjs/store/reconcile.d.ts +26 -1
- package/dist/cjs/store/reconcile.js +670 -576
- package/dist/cjs/store/reconcile.js.map +1 -1
- package/dist/cjs/store/signature.d.cts +1 -0
- package/dist/cjs/store/signature.d.ts +1 -0
- package/dist/cjs/store/signature.js +9 -0
- package/dist/cjs/store/signature.js.map +1 -1
- package/dist/cjs/store/sqlite/open.d.cts +1 -1
- package/dist/cjs/store/sqlite/open.d.ts +1 -1
- package/dist/cjs/store/sqlite/open.js +84 -90
- package/dist/cjs/store/sqlite/open.js.map +1 -1
- package/dist/cjs/store/sqlite/reconcile.js.map +1 -1
- package/dist/cjs/store/sqlite/store.js +63 -60
- package/dist/cjs/store/sqlite/store.js.map +1 -1
- package/dist/cjs/store/sqlite/vectors.d.cts +2 -2
- package/dist/cjs/store/sqlite/vectors.d.ts +2 -2
- package/dist/cjs/store/sqlite/vectors.js +8 -4
- package/dist/cjs/store/sqlite/vectors.js.map +1 -1
- package/dist/cjs/store/stages.js.map +1 -1
- package/dist/cjs/store/turso/connection.d.cts +1 -0
- package/dist/cjs/store/turso/connection.d.ts +1 -0
- package/dist/cjs/store/turso/connection.js +319 -94
- package/dist/cjs/store/turso/connection.js.map +1 -1
- package/dist/cjs/store/turso/open.d.cts +1 -1
- package/dist/cjs/store/turso/open.d.ts +1 -1
- package/dist/cjs/store/turso/open.js +70 -74
- package/dist/cjs/store/turso/open.js.map +1 -1
- package/dist/cjs/store/turso/store.js +25 -23
- package/dist/cjs/store/turso/store.js.map +1 -1
- package/dist/cjs/store/turso/vectors.d.cts +2 -2
- package/dist/cjs/store/turso/vectors.d.ts +2 -2
- package/dist/cjs/store/turso/vectors.js +26 -16
- package/dist/cjs/store/turso/vectors.js.map +1 -1
- package/dist/cjs/store/types.d.cts +4 -1
- package/dist/cjs/store/types.d.ts +4 -1
- package/dist/cjs/store/types.js.map +1 -1
- package/dist/cjs/store/vectors.d.cts +14 -1
- package/dist/cjs/store/vectors.d.ts +14 -1
- package/dist/cjs/store/vectors.js +61 -0
- package/dist/cjs/store/vectors.js.map +1 -1
- package/dist/cjs/watch.d.cts +6 -0
- package/dist/cjs/watch.d.ts +6 -0
- package/dist/cjs/watch.js +42 -10
- package/dist/cjs/watch.js.map +1 -1
- package/dist/cjs/workers/parse.d.cts +4 -3
- package/dist/cjs/workers/parse.d.ts +4 -3
- package/dist/cjs/workers/parse.js +19 -17
- package/dist/cjs/workers/parse.js.map +1 -1
- package/dist/esm/cli/index.d.ts +1 -1
- package/dist/esm/cli/index.js +1 -1
- package/dist/esm/cli/index.js.map +1 -1
- package/dist/esm/cli/path.js +4 -10
- package/dist/esm/cli/path.js.map +1 -1
- package/dist/esm/cli/related.js +5 -3
- package/dist/esm/cli/related.js.map +1 -1
- package/dist/esm/commands/index.d.ts +6 -3
- package/dist/esm/commands/index.js +1 -0
- package/dist/esm/commands/index.js.map +1 -1
- package/dist/esm/commands/map.d.ts +14 -5
- package/dist/esm/commands/map.js +2 -0
- package/dist/esm/commands/map.js.map +1 -1
- package/dist/esm/commands/path.d.ts +14 -0
- package/dist/esm/commands/path.js +33 -0
- package/dist/esm/commands/path.js.map +1 -0
- package/dist/esm/commands/peek.d.ts +16 -1
- package/dist/esm/commands/peek.js +35 -9
- package/dist/esm/commands/peek.js.map +1 -1
- package/dist/esm/commands/related.d.ts +16 -3
- package/dist/esm/commands/related.js +45 -28
- package/dist/esm/commands/related.js.map +1 -1
- package/dist/esm/commands/scope.d.ts +1 -0
- package/dist/esm/commands/scope.js +58 -0
- package/dist/esm/commands/scope.js.map +1 -1
- package/dist/esm/commands/search.d.ts +26 -3
- package/dist/esm/commands/search.js +85 -18
- package/dist/esm/commands/search.js.map +1 -1
- package/dist/esm/commands/signals.d.ts +4 -3
- package/dist/esm/commands/signals.js +11 -4
- package/dist/esm/commands/signals.js.map +1 -1
- package/dist/esm/commands/snapshot.d.ts +3 -0
- package/dist/esm/commands/snapshot.js +25 -0
- package/dist/esm/commands/snapshot.js.map +1 -0
- package/dist/esm/embed/cohere.js +35 -10
- package/dist/esm/embed/cohere.js.map +1 -1
- package/dist/esm/embed/endpoint.d.ts +2 -0
- package/dist/esm/embed/endpoint.js +11 -0
- package/dist/esm/embed/endpoint.js.map +1 -0
- package/dist/esm/embed/handoff.d.ts +7 -2
- package/dist/esm/embed/handoff.js +7 -0
- package/dist/esm/embed/handoff.js.map +1 -1
- package/dist/esm/embed/http.d.ts +18 -1
- package/dist/esm/embed/http.js +137 -17
- package/dist/esm/embed/http.js.map +1 -1
- package/dist/esm/embed/openai.js +34 -12
- package/dist/esm/embed/openai.js.map +1 -1
- package/dist/esm/embed/query.d.ts +9 -3
- package/dist/esm/embed/query.js +98 -26
- package/dist/esm/embed/query.js.map +1 -1
- package/dist/esm/embed/registry.d.ts +2 -2
- package/dist/esm/embed/registry.js +34 -5
- package/dist/esm/embed/registry.js.map +1 -1
- package/dist/esm/embed/static.js +10 -2
- package/dist/esm/embed/static.js.map +1 -1
- package/dist/esm/embed/store.js +30 -21
- package/dist/esm/embed/store.js.map +1 -1
- package/dist/esm/embed/types.d.ts +5 -2
- package/dist/esm/embed/types.js.map +1 -1
- package/dist/esm/features/embed.js +82 -21
- package/dist/esm/features/embed.js.map +1 -1
- package/dist/esm/graph/traverse.d.ts +2 -2
- package/dist/esm/graph/traverse.js +4 -2
- package/dist/esm/graph/traverse.js.map +1 -1
- package/dist/esm/index.d.ts +2 -2
- package/dist/esm/index.js +1 -1
- package/dist/esm/index.js.map +1 -1
- package/dist/esm/output/output.d.ts +4 -36
- package/dist/esm/output/output.js +1 -2
- package/dist/esm/output/output.js.map +1 -1
- package/dist/esm/scan/pool.d.ts +2 -1
- package/dist/esm/scan/pool.js +80 -23
- package/dist/esm/scan/pool.js.map +1 -1
- package/dist/esm/scan/reparse.d.ts +1 -0
- package/dist/esm/scan/reparse.js +40 -12
- package/dist/esm/scan/reparse.js.map +1 -1
- package/dist/esm/store/builder.d.ts +11 -21
- package/dist/esm/store/builder.js +337 -133
- package/dist/esm/store/builder.js.map +1 -1
- package/dist/esm/store/duckdb/connection.js +9 -1
- package/dist/esm/store/duckdb/connection.js.map +1 -1
- package/dist/esm/store/duckdb/open.d.ts +1 -1
- package/dist/esm/store/duckdb/open.js +13 -9
- package/dist/esm/store/duckdb/open.js.map +1 -1
- package/dist/esm/store/duckdb/reconcile.js +145 -10
- package/dist/esm/store/duckdb/reconcile.js.map +1 -1
- package/dist/esm/store/embed-scope.js +16 -6
- package/dist/esm/store/embed-scope.js.map +1 -1
- package/dist/esm/store/index.d.ts +6 -0
- package/dist/esm/store/index.js +10 -4
- package/dist/esm/store/index.js.map +1 -1
- package/dist/esm/store/open.d.ts +6 -1
- package/dist/esm/store/open.js +77 -87
- package/dist/esm/store/open.js.map +1 -1
- package/dist/esm/store/readiness.d.ts +5 -0
- package/dist/esm/store/readiness.js +11 -0
- package/dist/esm/store/readiness.js.map +1 -0
- package/dist/esm/store/reconcile.d.ts +26 -1
- package/dist/esm/store/reconcile.js +142 -120
- package/dist/esm/store/reconcile.js.map +1 -1
- package/dist/esm/store/signature.d.ts +1 -0
- package/dist/esm/store/signature.js +8 -0
- package/dist/esm/store/signature.js.map +1 -1
- package/dist/esm/store/sqlite/open.d.ts +1 -1
- package/dist/esm/store/sqlite/open.js +12 -12
- package/dist/esm/store/sqlite/open.js.map +1 -1
- package/dist/esm/store/sqlite/reconcile.js +2 -2
- package/dist/esm/store/sqlite/reconcile.js.map +1 -1
- package/dist/esm/store/sqlite/store.js +21 -18
- package/dist/esm/store/sqlite/store.js.map +1 -1
- package/dist/esm/store/sqlite/vectors.d.ts +2 -2
- package/dist/esm/store/sqlite/vectors.js +7 -5
- package/dist/esm/store/sqlite/vectors.js.map +1 -1
- package/dist/esm/store/stages.js.map +1 -1
- package/dist/esm/store/turso/connection.d.ts +1 -0
- package/dist/esm/store/turso/connection.js +97 -28
- package/dist/esm/store/turso/connection.js.map +1 -1
- package/dist/esm/store/turso/open.d.ts +1 -1
- package/dist/esm/store/turso/open.js +9 -8
- package/dist/esm/store/turso/open.js.map +1 -1
- package/dist/esm/store/turso/store.js +16 -14
- package/dist/esm/store/turso/store.js.map +1 -1
- package/dist/esm/store/turso/vectors.d.ts +2 -2
- package/dist/esm/store/turso/vectors.js +27 -17
- package/dist/esm/store/turso/vectors.js.map +1 -1
- package/dist/esm/store/types.d.ts +4 -1
- package/dist/esm/store/types.js.map +1 -1
- package/dist/esm/store/vectors.d.ts +14 -1
- package/dist/esm/store/vectors.js +55 -0
- package/dist/esm/store/vectors.js.map +1 -1
- package/dist/esm/watch.d.ts +6 -0
- package/dist/esm/watch.js +40 -14
- package/dist/esm/watch.js.map +1 -1
- package/dist/esm/workers/parse.d.ts +4 -3
- package/dist/esm/workers/parse.js +19 -17
- package/dist/esm/workers/parse.js.map +1 -1
- package/package.json +1 -1
- package/skills/sense/SKILL.md +4 -2
- package/skills/sense/references/stores/duckdb.md +1 -1
- package/skills/sense-setup/SKILL.md +2 -2
- package/skills/sense-setup/references/embeddings.md +1 -1
- package/skills/sense-setup/references/store-benchmarks.md +5 -5
- package/skills/sense-setup/references/stores/duckdb.md +1 -1
package/dist/esm/watch.js
CHANGED
|
@@ -7,10 +7,19 @@ import { startWatchClaim, WATCH_HEARTBEAT_INTERVAL_MS } from './watch-claim.js';
|
|
|
7
7
|
// Watch prepares the index for snapshot queries; filesystem events request debounced builds
|
|
8
8
|
// and periodic reconciliation recovers missed notifications.
|
|
9
9
|
const DEBOUNCE_MS = 200;
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
10
|
+
function isCallerAbort(signal, reason) {
|
|
11
|
+
return (signal === null || signal === void 0 ? void 0 : signal.aborted) === true && Object.is(reason, signal.reason);
|
|
12
|
+
}
|
|
13
|
+
function errorMessage(reason) {
|
|
14
|
+
return reason instanceof Error ? reason.message : String(reason);
|
|
15
|
+
}
|
|
16
|
+
/**
|
|
17
|
+
* Runs in the foreground. Caller abort waits for active native work and cleanup; SIGINT/SIGTERM
|
|
18
|
+
* also run one final freshness pass. Cleanup failures reject either shutdown path.
|
|
19
|
+
*/ export async function runWatch(cfg, opts = {}) {
|
|
13
20
|
var _opts_onEvent, _opts_debounceMs, _opts_heartbeatIntervalMs, _cfg_rootDir, _cfg_configDir, _opts_force;
|
|
21
|
+
var _opts_signal;
|
|
22
|
+
if ((_opts_signal = opts.signal) === null || _opts_signal === void 0 ? void 0 : _opts_signal.aborted) return;
|
|
14
23
|
const onEvent = (_opts_onEvent = opts.onEvent) !== null && _opts_onEvent !== void 0 ? _opts_onEvent : ()=>{};
|
|
15
24
|
const debounceMs = (_opts_debounceMs = opts.debounceMs) !== null && _opts_debounceMs !== void 0 ? _opts_debounceMs : DEBOUNCE_MS;
|
|
16
25
|
const reconcileIntervalMs = (_opts_heartbeatIntervalMs = opts.heartbeatIntervalMs) !== null && _opts_heartbeatIntervalMs !== void 0 ? _opts_heartbeatIntervalMs : WATCH_HEARTBEAT_INTERVAL_MS;
|
|
@@ -35,8 +44,10 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
35
44
|
let startupNotification = false;
|
|
36
45
|
let scheduleReconcile;
|
|
37
46
|
let stopping = false;
|
|
47
|
+
let callerCanceled = false;
|
|
38
48
|
try {
|
|
39
|
-
var
|
|
49
|
+
var _opts_signal1, _opts_signal2, _opts_signal3;
|
|
50
|
+
(_opts_signal1 = opts.signal) === null || _opts_signal1 === void 0 ? void 0 : _opts_signal1.throwIfAborted();
|
|
40
51
|
// Subscribe before the first build. Native watchers have no ready barrier, so startup also
|
|
41
52
|
// performs an unconditional catch-up pass after the initial build.
|
|
42
53
|
watcher = fsWatch(rootDir, {
|
|
@@ -51,7 +62,9 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
51
62
|
requestShutdown === null || requestShutdown === void 0 ? void 0 : requestShutdown(err);
|
|
52
63
|
});
|
|
53
64
|
const startupEvents = [];
|
|
54
|
-
const { store: initialStore, dbPath, warnings: initialWarnings, parsed: initialParsed } = await openStore(cfg
|
|
65
|
+
const { store: initialStore, dbPath, warnings: initialWarnings, parsed: initialParsed } = await openStore(cfg, {
|
|
66
|
+
signal: opts.signal
|
|
67
|
+
});
|
|
55
68
|
let initialTotal = 0;
|
|
56
69
|
try {
|
|
57
70
|
// total is read before close: an event fires only once nothing is held open again.
|
|
@@ -59,6 +72,7 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
59
72
|
} finally{
|
|
60
73
|
await initialStore.close();
|
|
61
74
|
}
|
|
75
|
+
(_opts_signal2 = opts.signal) === null || _opts_signal2 === void 0 ? void 0 : _opts_signal2.throwIfAborted();
|
|
62
76
|
if (claimFailure) throw claimFailure;
|
|
63
77
|
startupEvents.push({
|
|
64
78
|
parsed: initialParsed,
|
|
@@ -68,8 +82,12 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
68
82
|
// Always reopen once after subscribing, then drain notifications observed while startup was
|
|
69
83
|
// in progress. Each pass closes before the next one starts, so no stores overlap.
|
|
70
84
|
do {
|
|
85
|
+
var _opts_signal4, _opts_signal5;
|
|
71
86
|
startupNotification = false;
|
|
72
|
-
|
|
87
|
+
(_opts_signal4 = opts.signal) === null || _opts_signal4 === void 0 ? void 0 : _opts_signal4.throwIfAborted();
|
|
88
|
+
const { store, parsed, warnings } = await openStore(cfg, {
|
|
89
|
+
signal: opts.signal
|
|
90
|
+
});
|
|
73
91
|
try {
|
|
74
92
|
if (parsed > 0 || warnings.length > 0) startupEvents.push({
|
|
75
93
|
parsed,
|
|
@@ -79,8 +97,9 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
79
97
|
} finally{
|
|
80
98
|
await store.close();
|
|
81
99
|
}
|
|
100
|
+
(_opts_signal5 = opts.signal) === null || _opts_signal5 === void 0 ? void 0 : _opts_signal5.throwIfAborted();
|
|
82
101
|
if (claimFailure) throw claimFailure;
|
|
83
|
-
}while (startupNotification && !((
|
|
102
|
+
}while (startupNotification && !((_opts_signal3 = opts.signal) === null || _opts_signal3 === void 0 ? void 0 : _opts_signal3.aborted))
|
|
84
103
|
startup = false;
|
|
85
104
|
// At most one open+reconcile+close cycle runs at a time; a trigger that arrives mid-cycle is
|
|
86
105
|
// coalesced into a single rerun instead of opening a second, overlapping connection.
|
|
@@ -88,7 +107,9 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
88
107
|
let queuedAlways = null;
|
|
89
108
|
const tick = async (alwaysEmit)=>{
|
|
90
109
|
try {
|
|
91
|
-
const { store, parsed, warnings } = await openStore(cfg
|
|
110
|
+
const { store, parsed, warnings } = await openStore(cfg, {
|
|
111
|
+
signal: opts.signal
|
|
112
|
+
});
|
|
92
113
|
try {
|
|
93
114
|
if (alwaysEmit || parsed > 0 || warnings.length > 0) {
|
|
94
115
|
onEvent({
|
|
@@ -102,9 +123,10 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
102
123
|
await store.close();
|
|
103
124
|
}
|
|
104
125
|
} catch (err) {
|
|
126
|
+
if (isCallerAbort(opts.signal, err)) return;
|
|
105
127
|
onEvent({
|
|
106
128
|
type: 'reconcile-error',
|
|
107
|
-
message: err
|
|
129
|
+
message: errorMessage(err)
|
|
108
130
|
});
|
|
109
131
|
}
|
|
110
132
|
};
|
|
@@ -140,15 +162,16 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
140
162
|
return await new Promise((resolveShutdown, rejectShutdown)=>{
|
|
141
163
|
var _opts_signal;
|
|
142
164
|
let shutdownReason = null;
|
|
143
|
-
const shutdown = (reason)=>{
|
|
165
|
+
const shutdown = (reason, canceledByCaller = false)=>{
|
|
144
166
|
shutdownReason !== null && shutdownReason !== void 0 ? shutdownReason : shutdownReason = reason !== null && reason !== void 0 ? reason : null;
|
|
167
|
+
callerCanceled || (callerCanceled = canceledByCaller);
|
|
145
168
|
if (stopping) return;
|
|
146
169
|
stopping = true;
|
|
147
170
|
const shutdownTask = (async ()=>{
|
|
148
171
|
var _opts_signal;
|
|
149
172
|
process.off('SIGINT', stopNormally);
|
|
150
173
|
process.off('SIGTERM', stopNormally);
|
|
151
|
-
(_opts_signal = opts.signal) === null || _opts_signal === void 0 ? void 0 : _opts_signal.removeEventListener('abort',
|
|
174
|
+
(_opts_signal = opts.signal) === null || _opts_signal === void 0 ? void 0 : _opts_signal.removeEventListener('abort', stopForCaller);
|
|
152
175
|
clearInterval(reconcileTimer);
|
|
153
176
|
if (debounceTimer) clearTimeout(debounceTimer);
|
|
154
177
|
watcher === null || watcher === void 0 ? void 0 : watcher.close();
|
|
@@ -157,7 +180,7 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
157
180
|
try {
|
|
158
181
|
// A normal stop performs one final freshness pass. Ownership loss drains existing work
|
|
159
182
|
// but starts nothing new because the replacement watcher now owns that responsibility.
|
|
160
|
-
if (!shutdownReason) {
|
|
183
|
+
if (!shutdownReason && !callerCanceled) {
|
|
161
184
|
const { store: finalStore } = await openStore(cfg);
|
|
162
185
|
await finalStore.close();
|
|
163
186
|
}
|
|
@@ -179,14 +202,15 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
179
202
|
shutdownTask.then(resolveShutdown, rejectShutdown);
|
|
180
203
|
};
|
|
181
204
|
const stopNormally = ()=>shutdown();
|
|
205
|
+
const stopForCaller = ()=>shutdown(undefined, true);
|
|
182
206
|
requestShutdown = shutdown;
|
|
183
207
|
process.once('SIGINT', stopNormally);
|
|
184
208
|
process.once('SIGTERM', stopNormally);
|
|
185
209
|
if (claimFailure) shutdown(claimFailure);
|
|
186
|
-
else if ((_opts_signal = opts.signal) === null || _opts_signal === void 0 ? void 0 : _opts_signal.aborted)
|
|
210
|
+
else if ((_opts_signal = opts.signal) === null || _opts_signal === void 0 ? void 0 : _opts_signal.aborted) stopForCaller();
|
|
187
211
|
else {
|
|
188
212
|
var _opts_signal1;
|
|
189
|
-
(_opts_signal1 = opts.signal) === null || _opts_signal1 === void 0 ? void 0 : _opts_signal1.addEventListener('abort',
|
|
213
|
+
(_opts_signal1 = opts.signal) === null || _opts_signal1 === void 0 ? void 0 : _opts_signal1.addEventListener('abort', stopForCaller, {
|
|
190
214
|
once: true
|
|
191
215
|
});
|
|
192
216
|
try {
|
|
@@ -210,6 +234,7 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
210
234
|
}
|
|
211
235
|
});
|
|
212
236
|
} catch (err) {
|
|
237
|
+
const canceledByCaller = isCallerAbort(opts.signal, err);
|
|
213
238
|
watcher === null || watcher === void 0 ? void 0 : watcher.close();
|
|
214
239
|
if (claimCloseStarted) throw err;
|
|
215
240
|
try {
|
|
@@ -220,6 +245,7 @@ export async function runWatch(cfg, opts = {}) {
|
|
|
220
245
|
closeError
|
|
221
246
|
], 'watch setup and claim cleanup both failed');
|
|
222
247
|
}
|
|
248
|
+
if (canceledByCaller) return;
|
|
223
249
|
throw err;
|
|
224
250
|
} finally{
|
|
225
251
|
requestShutdown = undefined;
|
package/dist/esm/watch.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/watch.ts"],"sourcesContent":["import { watch as fsWatch } from 'node:fs';\nimport type { ResolvedConfig } from './config/index.ts';\nimport { SenseError } from './errors.ts';\nimport { guardedTick } from './lib/guarded-tick.ts';\nimport { shouldReconcileWatchNotification } from './lib/watch-notification.ts';\nimport { docCount, openStore } from './store/index.ts';\nimport { startWatchClaim, WATCH_HEARTBEAT_INTERVAL_MS } from './watch-claim.ts';\n\n// Watch prepares the index for snapshot queries; filesystem events request debounced builds\n// and periodic reconciliation recovers missed notifications.\nconst DEBOUNCE_MS = 200;\n\nexport type WatchEvent = { type: 'started'; rootDir: string; dbPath: string } | { type: 'reconciled'; parsed: number; total: number; warnings: string[] } | { type: 'reconcile-error'; message: string };\n\nexport interface WatchOptions {\n force?: boolean;\n onEvent?: (event: WatchEvent) => void;\n // Aborting runs the same shutdown path as SIGINT/SIGTERM.\n signal?: AbortSignal;\n debounceMs?: number;\n heartbeatIntervalMs?: number;\n}\n\n// Runs in the foreground until SIGINT/SIGTERM/signal abort. DuckDB and Turso hand the cache file to\n// one process at a time, so every reconcile cycle opens and closes instead of holding an idle lock.\nexport async function runWatch(cfg: ResolvedConfig, opts: WatchOptions = {}): Promise<void> {\n const onEvent = opts.onEvent ?? (() => {});\n const debounceMs = opts.debounceMs ?? DEBOUNCE_MS;\n const reconcileIntervalMs = opts.heartbeatIntervalMs ?? WATCH_HEARTBEAT_INTERVAL_MS;\n if (!Number.isFinite(reconcileIntervalMs) || reconcileIntervalMs <= 0) throw new SenseError('CONFIG_INVALID', 'watch heartbeatIntervalMs must be a finite positive number');\n // Reconciliation may be parked for a caller, but live ownership must renew before fixed expiry.\n const claimIntervalMs = Math.min(reconcileIntervalMs, WATCH_HEARTBEAT_INTERVAL_MS);\n const rootDir = cfg.rootDir ?? cfg.baseDir;\n const configDir = cfg.configDir ?? rootDir;\n let claimFailure: Error | null = null;\n let requestShutdown: ((reason: Error) => void) | undefined;\n const claim = await startWatchClaim(configDir, opts.force ?? false, claimIntervalMs, (err) => {\n claimFailure ??= err;\n requestShutdown?.(err);\n });\n let claimCloseStarted = false;\n const closeClaim = (): Promise<void> => {\n claimCloseStarted = true;\n return claim.close();\n };\n let watcher: ReturnType<typeof fsWatch> | undefined;\n let startup = true;\n let startupNotification = false;\n let scheduleReconcile: (() => void) | undefined;\n let stopping = false;\n try {\n // Subscribe before the first build. Native watchers have no ready barrier, so startup also\n // performs an unconditional catch-up pass after the initial build.\n watcher = fsWatch(rootDir, { recursive: true }, (_event, filename) => {\n if (!shouldReconcileWatchNotification(filename)) return;\n if (startup) startupNotification = true;\n else scheduleReconcile?.();\n });\n watcher.on('error', (err) => {\n claimFailure ??= err;\n requestShutdown?.(err);\n });\n\n const startupEvents: Array<{ parsed: number; total: number; warnings: string[] }> = [];\n const { store: initialStore, dbPath, warnings: initialWarnings, parsed: initialParsed } = await openStore(cfg);\n let initialTotal = 0;\n try {\n // total is read before close: an event fires only once nothing is held open again.\n initialTotal = initialWarnings.length > 0 || initialParsed > 0 ? await docCount(initialStore) : 0;\n } finally {\n await initialStore.close();\n }\n if (claimFailure) throw claimFailure;\n\n startupEvents.push({ parsed: initialParsed, total: initialTotal, warnings: initialWarnings });\n // Always reopen once after subscribing, then drain notifications observed while startup was\n // in progress. Each pass closes before the next one starts, so no stores overlap.\n do {\n startupNotification = false;\n const { store, parsed, warnings } = await openStore(cfg);\n try {\n if (parsed > 0 || warnings.length > 0) startupEvents.push({ parsed, total: await docCount(store), warnings });\n } finally {\n await store.close();\n }\n if (claimFailure) throw claimFailure;\n } while (startupNotification && !opts.signal?.aborted);\n startup = false;\n\n // At most one open+reconcile+close cycle runs at a time; a trigger that arrives mid-cycle is\n // coalesced into a single rerun instead of opening a second, overlapping connection.\n let running: Promise<void> | null = null;\n let queuedAlways: boolean | null = null;\n\n const tick = async (alwaysEmit: boolean): Promise<void> => {\n try {\n const { store, parsed, warnings } = await openStore(cfg);\n try {\n if (alwaysEmit || parsed > 0 || warnings.length > 0) {\n onEvent({ type: 'reconciled', parsed, total: await docCount(store), warnings });\n }\n } finally {\n await store.close();\n }\n } catch (err) {\n onEvent({ type: 'reconcile-error', message: (err as Error).message });\n }\n };\n\n const startCycle = (alwaysEmit: boolean) => {\n running = tick(alwaysEmit).then(() => {\n running = null;\n if (queuedAlways !== null && !stopping) {\n const again = queuedAlways;\n queuedAlways = null;\n startCycle(again);\n }\n });\n };\n\n const requestCycle = (alwaysEmit: boolean) => {\n if (stopping) return;\n if (running) {\n queuedAlways = queuedAlways || alwaysEmit;\n return;\n }\n startCycle(alwaysEmit);\n };\n\n let debounceTimer: NodeJS.Timeout | null = null;\n // Ignore our own state files, or cache and claim writes would retrigger reconciliation forever.\n // An unresolvable filename reconciles because parsing nothing costs less than missing a real edit.\n scheduleReconcile = () => {\n if (debounceTimer) clearTimeout(debounceTimer);\n debounceTimer = setTimeout(() => {\n debounceTimer = null;\n requestCycle(true);\n }, debounceMs);\n };\n const reconcileTimer = setInterval(\n guardedTick(\n async () => requestCycle(false),\n () => stopping\n ),\n reconcileIntervalMs\n );\n\n return await new Promise<void>((resolveShutdown, rejectShutdown) => {\n let shutdownReason: Error | null = null;\n const shutdown = (reason?: Error) => {\n shutdownReason ??= reason ?? null;\n if (stopping) return;\n stopping = true;\n const shutdownTask = (async () => {\n process.off('SIGINT', stopNormally);\n process.off('SIGTERM', stopNormally);\n opts.signal?.removeEventListener('abort', stopNormally);\n clearInterval(reconcileTimer);\n if (debounceTimer) clearTimeout(debounceTimer);\n watcher?.close();\n await running;\n let finalError: unknown;\n try {\n // A normal stop performs one final freshness pass. Ownership loss drains existing work\n // but starts nothing new because the replacement watcher now owns that responsibility.\n if (!shutdownReason) {\n const { store: finalStore } = await openStore(cfg);\n await finalStore.close();\n }\n } catch (err) {\n finalError = err;\n }\n try {\n await closeClaim();\n } catch (err) {\n if (finalError) throw new AggregateError([finalError, err], 'watch shutdown and claim cleanup both failed');\n throw err;\n }\n if (finalError) throw finalError;\n if (shutdownReason) throw shutdownReason;\n })();\n shutdownTask.then(resolveShutdown, rejectShutdown);\n };\n const stopNormally = () => shutdown();\n requestShutdown = shutdown;\n process.once('SIGINT', stopNormally);\n process.once('SIGTERM', stopNormally);\n if (claimFailure) shutdown(claimFailure);\n else if (opts.signal?.aborted) shutdown();\n else {\n opts.signal?.addEventListener('abort', stopNormally, { once: true });\n try {\n onEvent({ type: 'started', rootDir, dbPath });\n if (!stopping) {\n for (const event of startupEvents) {\n if (stopping) break;\n if (event.warnings.length > 0 || event.parsed > 0) onEvent({ type: 'reconciled', ...event });\n }\n }\n } catch (err) {\n shutdown(err instanceof Error ? err : new Error(String(err)));\n }\n }\n });\n } catch (err) {\n watcher?.close();\n if (claimCloseStarted) throw err;\n try {\n await closeClaim();\n } catch (closeError) {\n throw new AggregateError([err, closeError], 'watch setup and claim cleanup both failed');\n }\n throw err;\n } finally {\n requestShutdown = undefined;\n }\n}\n"],"names":["watch","fsWatch","SenseError","guardedTick","shouldReconcileWatchNotification","docCount","openStore","startWatchClaim","WATCH_HEARTBEAT_INTERVAL_MS","DEBOUNCE_MS","runWatch","cfg","opts","onEvent","debounceMs","reconcileIntervalMs","heartbeatIntervalMs","Number","isFinite","claimIntervalMs","Math","min","rootDir","baseDir","configDir","claimFailure","requestShutdown","claim","force","err","claimCloseStarted","closeClaim","close","watcher","startup","startupNotification","scheduleReconcile","stopping","recursive","_event","filename","on","startupEvents","store","initialStore","dbPath","warnings","initialWarnings","parsed","initialParsed","initialTotal","length","push","total","signal","aborted","running","queuedAlways","tick","alwaysEmit","type","message","startCycle","then","again","requestCycle","debounceTimer","clearTimeout","setTimeout","reconcileTimer","setInterval","Promise","resolveShutdown","rejectShutdown","shutdownReason","shutdown","reason","shutdownTask","process","off","stopNormally","removeEventListener","clearInterval","finalError","finalStore","AggregateError","once","addEventListener","event","Error","String","closeError","undefined"],"mappings":"AAAA,SAASA,SAASC,OAAO,QAAQ,UAAU;AAE3C,SAASC,UAAU,QAAQ,cAAc;AACzC,SAASC,WAAW,QAAQ,wBAAwB;AACpD,SAASC,gCAAgC,QAAQ,8BAA8B;AAC/E,SAASC,QAAQ,EAAEC,SAAS,QAAQ,mBAAmB;AACvD,SAASC,eAAe,EAAEC,2BAA2B,QAAQ,mBAAmB;AAEhF,4FAA4F;AAC5F,6DAA6D;AAC7D,MAAMC,cAAc;AAapB,oGAAoG;AACpG,oGAAoG;AACpG,OAAO,eAAeC,SAASC,GAAmB,EAAEC,OAAqB,CAAC,CAAC;QACzDA,eACGA,kBACSA,2BAIZD,cACEA,gBAG6BC;IAV/C,MAAMC,WAAUD,gBAAAA,KAAKC,OAAO,cAAZD,2BAAAA,gBAAiB,KAAO;IACxC,MAAME,cAAaF,mBAAAA,KAAKE,UAAU,cAAfF,8BAAAA,mBAAmBH;IACtC,MAAMM,uBAAsBH,4BAAAA,KAAKI,mBAAmB,cAAxBJ,uCAAAA,4BAA4BJ;IACxD,IAAI,CAACS,OAAOC,QAAQ,CAACH,wBAAwBA,uBAAuB,GAAG,MAAM,IAAIb,WAAW,kBAAkB;IAC9G,gGAAgG;IAChG,MAAMiB,kBAAkBC,KAAKC,GAAG,CAACN,qBAAqBP;IACtD,MAAMc,WAAUX,eAAAA,IAAIW,OAAO,cAAXX,0BAAAA,eAAeA,IAAIY,OAAO;IAC1C,MAAMC,aAAYb,iBAAAA,IAAIa,SAAS,cAAbb,4BAAAA,iBAAiBW;IACnC,IAAIG,eAA6B;IACjC,IAAIC;IACJ,MAAMC,QAAQ,MAAMpB,gBAAgBiB,YAAWZ,cAAAA,KAAKgB,KAAK,cAAVhB,yBAAAA,cAAc,OAAOO,iBAAiB,CAACU;QACpFJ,yBAAAA,0BAAAA,eAAAA,eAAiBI;QACjBH,4BAAAA,sCAAAA,gBAAkBG;IACpB;IACA,IAAIC,oBAAoB;IACxB,MAAMC,aAAa;QACjBD,oBAAoB;QACpB,OAAOH,MAAMK,KAAK;IACpB;IACA,IAAIC;IACJ,IAAIC,UAAU;IACd,IAAIC,sBAAsB;IAC1B,IAAIC;IACJ,IAAIC,WAAW;IACf,IAAI;YAoC+BzB;QAnCjC,2FAA2F;QAC3F,mEAAmE;QACnEqB,UAAUhC,QAAQqB,SAAS;YAAEgB,WAAW;QAAK,GAAG,CAACC,QAAQC;YACvD,IAAI,CAACpC,iCAAiCoC,WAAW;YACjD,IAAIN,SAASC,sBAAsB;iBAC9BC,8BAAAA,wCAAAA;QACP;QACAH,QAAQQ,EAAE,CAAC,SAAS,CAACZ;YACnBJ,yBAAAA,0BAAAA,eAAAA,eAAiBI;YACjBH,4BAAAA,sCAAAA,gBAAkBG;QACpB;QAEA,MAAMa,gBAA8E,EAAE;QACtF,MAAM,EAAEC,OAAOC,YAAY,EAAEC,MAAM,EAAEC,UAAUC,eAAe,EAAEC,QAAQC,aAAa,EAAE,GAAG,MAAM3C,UAAUK;QAC1G,IAAIuC,eAAe;QACnB,IAAI;YACF,mFAAmF;YACnFA,eAAeH,gBAAgBI,MAAM,GAAG,KAAKF,gBAAgB,IAAI,MAAM5C,SAASuC,gBAAgB;QAClG,SAAU;YACR,MAAMA,aAAaZ,KAAK;QAC1B;QACA,IAAIP,cAAc,MAAMA;QAExBiB,cAAcU,IAAI,CAAC;YAAEJ,QAAQC;YAAeI,OAAOH;YAAcJ,UAAUC;QAAgB;QAC3F,4FAA4F;QAC5F,kFAAkF;QAClF,GAAG;YACDZ,sBAAsB;YACtB,MAAM,EAAEQ,KAAK,EAAEK,MAAM,EAAEF,QAAQ,EAAE,GAAG,MAAMxC,UAAUK;YACpD,IAAI;gBACF,IAAIqC,SAAS,KAAKF,SAASK,MAAM,GAAG,GAAGT,cAAcU,IAAI,CAAC;oBAAEJ;oBAAQK,OAAO,MAAMhD,SAASsC;oBAAQG;gBAAS;YAC7G,SAAU;gBACR,MAAMH,MAAMX,KAAK;YACnB;YACA,IAAIP,cAAc,MAAMA;QAC1B,QAASU,uBAAuB,GAACvB,eAAAA,KAAK0C,MAAM,cAAX1C,mCAAAA,aAAa2C,OAAO,EAAE;QACvDrB,UAAU;QAEV,6FAA6F;QAC7F,qFAAqF;QACrF,IAAIsB,UAAgC;QACpC,IAAIC,eAA+B;QAEnC,MAAMC,OAAO,OAAOC;YAClB,IAAI;gBACF,MAAM,EAAEhB,KAAK,EAAEK,MAAM,EAAEF,QAAQ,EAAE,GAAG,MAAMxC,UAAUK;gBACpD,IAAI;oBACF,IAAIgD,cAAcX,SAAS,KAAKF,SAASK,MAAM,GAAG,GAAG;wBACnDtC,QAAQ;4BAAE+C,MAAM;4BAAcZ;4BAAQK,OAAO,MAAMhD,SAASsC;4BAAQG;wBAAS;oBAC/E;gBACF,SAAU;oBACR,MAAMH,MAAMX,KAAK;gBACnB;YACF,EAAE,OAAOH,KAAK;gBACZhB,QAAQ;oBAAE+C,MAAM;oBAAmBC,SAAS,AAAChC,IAAcgC,OAAO;gBAAC;YACrE;QACF;QAEA,MAAMC,aAAa,CAACH;YAClBH,UAAUE,KAAKC,YAAYI,IAAI,CAAC;gBAC9BP,UAAU;gBACV,IAAIC,iBAAiB,QAAQ,CAACpB,UAAU;oBACtC,MAAM2B,QAAQP;oBACdA,eAAe;oBACfK,WAAWE;gBACb;YACF;QACF;QAEA,MAAMC,eAAe,CAACN;YACpB,IAAItB,UAAU;YACd,IAAImB,SAAS;gBACXC,eAAeA,gBAAgBE;gBAC/B;YACF;YACAG,WAAWH;QACb;QAEA,IAAIO,gBAAuC;QAC3C,gGAAgG;QAChG,mGAAmG;QACnG9B,oBAAoB;YAClB,IAAI8B,eAAeC,aAAaD;YAChCA,gBAAgBE,WAAW;gBACzBF,gBAAgB;gBAChBD,aAAa;YACf,GAAGnD;QACL;QACA,MAAMuD,iBAAiBC,YACrBnE,YACE,UAAY8D,aAAa,QACzB,IAAM5B,WAERtB;QAGF,OAAO,MAAM,IAAIwD,QAAc,CAACC,iBAAiBC;gBAyCtC7D;YAxCT,IAAI8D,iBAA+B;YACnC,MAAMC,WAAW,CAACC;gBAChBF,2BAAAA,4BAAAA,iBAAAA,iBAAmBE,mBAAAA,oBAAAA,SAAU;gBAC7B,IAAIvC,UAAU;gBACdA,WAAW;gBACX,MAAMwC,eAAe,AAAC,CAAA;wBAGpBjE;oBAFAkE,QAAQC,GAAG,CAAC,UAAUC;oBACtBF,QAAQC,GAAG,CAAC,WAAWC;qBACvBpE,eAAAA,KAAK0C,MAAM,cAAX1C,mCAAAA,aAAaqE,mBAAmB,CAAC,SAASD;oBAC1CE,cAAcb;oBACd,IAAIH,eAAeC,aAAaD;oBAChCjC,oBAAAA,8BAAAA,QAASD,KAAK;oBACd,MAAMwB;oBACN,IAAI2B;oBACJ,IAAI;wBACF,uFAAuF;wBACvF,uFAAuF;wBACvF,IAAI,CAACT,gBAAgB;4BACnB,MAAM,EAAE/B,OAAOyC,UAAU,EAAE,GAAG,MAAM9E,UAAUK;4BAC9C,MAAMyE,WAAWpD,KAAK;wBACxB;oBACF,EAAE,OAAOH,KAAK;wBACZsD,aAAatD;oBACf;oBACA,IAAI;wBACF,MAAME;oBACR,EAAE,OAAOF,KAAK;wBACZ,IAAIsD,YAAY,MAAM,IAAIE,eAAe;4BAACF;4BAAYtD;yBAAI,EAAE;wBAC5D,MAAMA;oBACR;oBACA,IAAIsD,YAAY,MAAMA;oBACtB,IAAIT,gBAAgB,MAAMA;gBAC5B,CAAA;gBACAG,aAAad,IAAI,CAACS,iBAAiBC;YACrC;YACA,MAAMO,eAAe,IAAML;YAC3BjD,kBAAkBiD;YAClBG,QAAQQ,IAAI,CAAC,UAAUN;YACvBF,QAAQQ,IAAI,CAAC,WAAWN;YACxB,IAAIvD,cAAckD,SAASlD;iBACtB,KAAIb,eAAAA,KAAK0C,MAAM,cAAX1C,mCAAAA,aAAa2C,OAAO,EAAEoB;iBAC1B;oBACH/D;iBAAAA,gBAAAA,KAAK0C,MAAM,cAAX1C,oCAAAA,cAAa2E,gBAAgB,CAAC,SAASP,cAAc;oBAAEM,MAAM;gBAAK;gBAClE,IAAI;oBACFzE,QAAQ;wBAAE+C,MAAM;wBAAWtC;wBAASuB;oBAAO;oBAC3C,IAAI,CAACR,UAAU;wBACb,KAAK,MAAMmD,SAAS9C,cAAe;4BACjC,IAAIL,UAAU;4BACd,IAAImD,MAAM1C,QAAQ,CAACK,MAAM,GAAG,KAAKqC,MAAMxC,MAAM,GAAG,GAAGnC,QAAQ;gCAAE+C,MAAM;gCAAc,GAAG4B,KAAK;4BAAC;wBAC5F;oBACF;gBACF,EAAE,OAAO3D,KAAK;oBACZ8C,SAAS9C,eAAe4D,QAAQ5D,MAAM,IAAI4D,MAAMC,OAAO7D;gBACzD;YACF;QACF;IACF,EAAE,OAAOA,KAAK;QACZI,oBAAAA,8BAAAA,QAASD,KAAK;QACd,IAAIF,mBAAmB,MAAMD;QAC7B,IAAI;YACF,MAAME;QACR,EAAE,OAAO4D,YAAY;YACnB,MAAM,IAAIN,eAAe;gBAACxD;gBAAK8D;aAAW,EAAE;QAC9C;QACA,MAAM9D;IACR,SAAU;QACRH,kBAAkBkE;IACpB;AACF"}
|
|
1
|
+
{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/watch.ts"],"sourcesContent":["import { watch as fsWatch } from 'node:fs';\nimport type { ResolvedConfig } from './config/index.ts';\nimport { SenseError } from './errors.ts';\nimport { guardedTick } from './lib/guarded-tick.ts';\nimport { shouldReconcileWatchNotification } from './lib/watch-notification.ts';\nimport { docCount, openStore } from './store/index.ts';\nimport { startWatchClaim, WATCH_HEARTBEAT_INTERVAL_MS } from './watch-claim.ts';\n\n// Watch prepares the index for snapshot queries; filesystem events request debounced builds\n// and periodic reconciliation recovers missed notifications.\nconst DEBOUNCE_MS = 200;\n\nexport type WatchEvent = { type: 'started'; rootDir: string; dbPath: string } | { type: 'reconciled'; parsed: number; total: number; warnings: string[] } | { type: 'reconcile-error'; message: string };\n\n/** Options for the root {@link runWatch} API. */\nexport interface WatchOptions {\n force?: boolean;\n onEvent?: (event: WatchEvent) => void;\n /** Caller abort cancels provider work, skips final preparation, and resolves after cleanup. */\n signal?: AbortSignal;\n debounceMs?: number;\n heartbeatIntervalMs?: number;\n}\n\nfunction isCallerAbort(signal: AbortSignal | undefined, reason: unknown): boolean {\n return signal?.aborted === true && Object.is(reason, signal.reason);\n}\n\nfunction errorMessage(reason: unknown): string {\n return reason instanceof Error ? reason.message : String(reason);\n}\n\n/**\n * Runs in the foreground. Caller abort waits for active native work and cleanup; SIGINT/SIGTERM\n * also run one final freshness pass. Cleanup failures reject either shutdown path.\n */\nexport async function runWatch(cfg: ResolvedConfig, opts: WatchOptions = {}): Promise<void> {\n if (opts.signal?.aborted) return;\n const onEvent = opts.onEvent ?? (() => {});\n const debounceMs = opts.debounceMs ?? DEBOUNCE_MS;\n const reconcileIntervalMs = opts.heartbeatIntervalMs ?? WATCH_HEARTBEAT_INTERVAL_MS;\n if (!Number.isFinite(reconcileIntervalMs) || reconcileIntervalMs <= 0) throw new SenseError('CONFIG_INVALID', 'watch heartbeatIntervalMs must be a finite positive number');\n // Reconciliation may be parked for a caller, but live ownership must renew before fixed expiry.\n const claimIntervalMs = Math.min(reconcileIntervalMs, WATCH_HEARTBEAT_INTERVAL_MS);\n const rootDir = cfg.rootDir ?? cfg.baseDir;\n const configDir = cfg.configDir ?? rootDir;\n let claimFailure: Error | null = null;\n let requestShutdown: ((reason: Error) => void) | undefined;\n const claim = await startWatchClaim(configDir, opts.force ?? false, claimIntervalMs, (err) => {\n claimFailure ??= err;\n requestShutdown?.(err);\n });\n let claimCloseStarted = false;\n const closeClaim = (): Promise<void> => {\n claimCloseStarted = true;\n return claim.close();\n };\n let watcher: ReturnType<typeof fsWatch> | undefined;\n let startup = true;\n let startupNotification = false;\n let scheduleReconcile: (() => void) | undefined;\n let stopping = false;\n let callerCanceled = false;\n try {\n opts.signal?.throwIfAborted();\n // Subscribe before the first build. Native watchers have no ready barrier, so startup also\n // performs an unconditional catch-up pass after the initial build.\n watcher = fsWatch(rootDir, { recursive: true }, (_event, filename) => {\n if (!shouldReconcileWatchNotification(filename)) return;\n if (startup) startupNotification = true;\n else scheduleReconcile?.();\n });\n watcher.on('error', (err) => {\n claimFailure ??= err;\n requestShutdown?.(err);\n });\n\n const startupEvents: Array<{ parsed: number; total: number; warnings: string[] }> = [];\n const { store: initialStore, dbPath, warnings: initialWarnings, parsed: initialParsed } = await openStore(cfg, { signal: opts.signal });\n let initialTotal = 0;\n try {\n // total is read before close: an event fires only once nothing is held open again.\n initialTotal = initialWarnings.length > 0 || initialParsed > 0 ? await docCount(initialStore) : 0;\n } finally {\n await initialStore.close();\n }\n opts.signal?.throwIfAborted();\n if (claimFailure) throw claimFailure;\n\n startupEvents.push({ parsed: initialParsed, total: initialTotal, warnings: initialWarnings });\n // Always reopen once after subscribing, then drain notifications observed while startup was\n // in progress. Each pass closes before the next one starts, so no stores overlap.\n do {\n startupNotification = false;\n opts.signal?.throwIfAborted();\n const { store, parsed, warnings } = await openStore(cfg, { signal: opts.signal });\n try {\n if (parsed > 0 || warnings.length > 0) startupEvents.push({ parsed, total: await docCount(store), warnings });\n } finally {\n await store.close();\n }\n opts.signal?.throwIfAborted();\n if (claimFailure) throw claimFailure;\n } while (startupNotification && !opts.signal?.aborted);\n startup = false;\n\n // At most one open+reconcile+close cycle runs at a time; a trigger that arrives mid-cycle is\n // coalesced into a single rerun instead of opening a second, overlapping connection.\n let running: Promise<void> | null = null;\n let queuedAlways: boolean | null = null;\n\n const tick = async (alwaysEmit: boolean): Promise<void> => {\n try {\n const { store, parsed, warnings } = await openStore(cfg, { signal: opts.signal });\n try {\n if (alwaysEmit || parsed > 0 || warnings.length > 0) {\n onEvent({ type: 'reconciled', parsed, total: await docCount(store), warnings });\n }\n } finally {\n await store.close();\n }\n } catch (err) {\n if (isCallerAbort(opts.signal, err)) return;\n onEvent({ type: 'reconcile-error', message: errorMessage(err) });\n }\n };\n\n const startCycle = (alwaysEmit: boolean) => {\n running = tick(alwaysEmit).then(() => {\n running = null;\n if (queuedAlways !== null && !stopping) {\n const again = queuedAlways;\n queuedAlways = null;\n startCycle(again);\n }\n });\n };\n\n const requestCycle = (alwaysEmit: boolean) => {\n if (stopping) return;\n if (running) {\n queuedAlways = queuedAlways || alwaysEmit;\n return;\n }\n startCycle(alwaysEmit);\n };\n\n let debounceTimer: NodeJS.Timeout | null = null;\n // Ignore our own state files, or cache and claim writes would retrigger reconciliation forever.\n // An unresolvable filename reconciles because parsing nothing costs less than missing a real edit.\n scheduleReconcile = () => {\n if (debounceTimer) clearTimeout(debounceTimer);\n debounceTimer = setTimeout(() => {\n debounceTimer = null;\n requestCycle(true);\n }, debounceMs);\n };\n const reconcileTimer = setInterval(\n guardedTick(\n async () => requestCycle(false),\n () => stopping\n ),\n reconcileIntervalMs\n );\n\n return await new Promise<void>((resolveShutdown, rejectShutdown) => {\n let shutdownReason: Error | null = null;\n const shutdown = (reason?: Error, canceledByCaller = false) => {\n shutdownReason ??= reason ?? null;\n callerCanceled ||= canceledByCaller;\n if (stopping) return;\n stopping = true;\n const shutdownTask = (async () => {\n process.off('SIGINT', stopNormally);\n process.off('SIGTERM', stopNormally);\n opts.signal?.removeEventListener('abort', stopForCaller);\n clearInterval(reconcileTimer);\n if (debounceTimer) clearTimeout(debounceTimer);\n watcher?.close();\n await running;\n let finalError: unknown;\n try {\n // A normal stop performs one final freshness pass. Ownership loss drains existing work\n // but starts nothing new because the replacement watcher now owns that responsibility.\n if (!shutdownReason && !callerCanceled) {\n const { store: finalStore } = await openStore(cfg);\n await finalStore.close();\n }\n } catch (err) {\n finalError = err;\n }\n try {\n await closeClaim();\n } catch (err) {\n if (finalError) throw new AggregateError([finalError, err], 'watch shutdown and claim cleanup both failed');\n throw err;\n }\n if (finalError) throw finalError;\n if (shutdownReason) throw shutdownReason;\n })();\n shutdownTask.then(resolveShutdown, rejectShutdown);\n };\n const stopNormally = () => shutdown();\n const stopForCaller = () => shutdown(undefined, true);\n requestShutdown = shutdown;\n process.once('SIGINT', stopNormally);\n process.once('SIGTERM', stopNormally);\n if (claimFailure) shutdown(claimFailure);\n else if (opts.signal?.aborted) stopForCaller();\n else {\n opts.signal?.addEventListener('abort', stopForCaller, { once: true });\n try {\n onEvent({ type: 'started', rootDir, dbPath });\n if (!stopping) {\n for (const event of startupEvents) {\n if (stopping) break;\n if (event.warnings.length > 0 || event.parsed > 0) onEvent({ type: 'reconciled', ...event });\n }\n }\n } catch (err) {\n shutdown(err instanceof Error ? err : new Error(String(err)));\n }\n }\n });\n } catch (err) {\n const canceledByCaller = isCallerAbort(opts.signal, err);\n watcher?.close();\n if (claimCloseStarted) throw err;\n try {\n await closeClaim();\n } catch (closeError) {\n throw new AggregateError([err, closeError], 'watch setup and claim cleanup both failed');\n }\n if (canceledByCaller) return;\n throw err;\n } finally {\n requestShutdown = undefined;\n }\n}\n"],"names":["watch","fsWatch","SenseError","guardedTick","shouldReconcileWatchNotification","docCount","openStore","startWatchClaim","WATCH_HEARTBEAT_INTERVAL_MS","DEBOUNCE_MS","isCallerAbort","signal","reason","aborted","Object","is","errorMessage","Error","message","String","runWatch","cfg","opts","onEvent","debounceMs","reconcileIntervalMs","heartbeatIntervalMs","Number","isFinite","claimIntervalMs","Math","min","rootDir","baseDir","configDir","claimFailure","requestShutdown","claim","force","err","claimCloseStarted","closeClaim","close","watcher","startup","startupNotification","scheduleReconcile","stopping","callerCanceled","throwIfAborted","recursive","_event","filename","on","startupEvents","store","initialStore","dbPath","warnings","initialWarnings","parsed","initialParsed","initialTotal","length","push","total","running","queuedAlways","tick","alwaysEmit","type","startCycle","then","again","requestCycle","debounceTimer","clearTimeout","setTimeout","reconcileTimer","setInterval","Promise","resolveShutdown","rejectShutdown","shutdownReason","shutdown","canceledByCaller","shutdownTask","process","off","stopNormally","removeEventListener","stopForCaller","clearInterval","finalError","finalStore","AggregateError","undefined","once","addEventListener","event","closeError"],"mappings":"AAAA,SAASA,SAASC,OAAO,QAAQ,UAAU;AAE3C,SAASC,UAAU,QAAQ,cAAc;AACzC,SAASC,WAAW,QAAQ,wBAAwB;AACpD,SAASC,gCAAgC,QAAQ,8BAA8B;AAC/E,SAASC,QAAQ,EAAEC,SAAS,QAAQ,mBAAmB;AACvD,SAASC,eAAe,EAAEC,2BAA2B,QAAQ,mBAAmB;AAEhF,4FAA4F;AAC5F,6DAA6D;AAC7D,MAAMC,cAAc;AAcpB,SAASC,cAAcC,MAA+B,EAAEC,MAAe;IACrE,OAAOD,CAAAA,mBAAAA,6BAAAA,OAAQE,OAAO,MAAK,QAAQC,OAAOC,EAAE,CAACH,QAAQD,OAAOC,MAAM;AACpE;AAEA,SAASI,aAAaJ,MAAe;IACnC,OAAOA,kBAAkBK,QAAQL,OAAOM,OAAO,GAAGC,OAAOP;AAC3D;AAEA;;;CAGC,GACD,OAAO,eAAeQ,SAASC,GAAmB,EAAEC,OAAqB,CAAC,CAAC;QAEzDA,eACGA,kBACSA,2BAIZD,cACEA,gBAG6BC;QAX3CA;IAAJ,KAAIA,eAAAA,KAAKX,MAAM,cAAXW,mCAAAA,aAAaT,OAAO,EAAE;IAC1B,MAAMU,WAAUD,gBAAAA,KAAKC,OAAO,cAAZD,2BAAAA,gBAAiB,KAAO;IACxC,MAAME,cAAaF,mBAAAA,KAAKE,UAAU,cAAfF,8BAAAA,mBAAmBb;IACtC,MAAMgB,uBAAsBH,4BAAAA,KAAKI,mBAAmB,cAAxBJ,uCAAAA,4BAA4Bd;IACxD,IAAI,CAACmB,OAAOC,QAAQ,CAACH,wBAAwBA,uBAAuB,GAAG,MAAM,IAAIvB,WAAW,kBAAkB;IAC9G,gGAAgG;IAChG,MAAM2B,kBAAkBC,KAAKC,GAAG,CAACN,qBAAqBjB;IACtD,MAAMwB,WAAUX,eAAAA,IAAIW,OAAO,cAAXX,0BAAAA,eAAeA,IAAIY,OAAO;IAC1C,MAAMC,aAAYb,iBAAAA,IAAIa,SAAS,cAAbb,4BAAAA,iBAAiBW;IACnC,IAAIG,eAA6B;IACjC,IAAIC;IACJ,MAAMC,QAAQ,MAAM9B,gBAAgB2B,YAAWZ,cAAAA,KAAKgB,KAAK,cAAVhB,yBAAAA,cAAc,OAAOO,iBAAiB,CAACU;QACpFJ,yBAAAA,0BAAAA,eAAAA,eAAiBI;QACjBH,4BAAAA,sCAAAA,gBAAkBG;IACpB;IACA,IAAIC,oBAAoB;IACxB,MAAMC,aAAa;QACjBD,oBAAoB;QACpB,OAAOH,MAAMK,KAAK;IACpB;IACA,IAAIC;IACJ,IAAIC,UAAU;IACd,IAAIC,sBAAsB;IAC1B,IAAIC;IACJ,IAAIC,WAAW;IACf,IAAIC,iBAAiB;IACrB,IAAI;YACF1B,eAsBAA,eAiBiCA;SAvCjCA,gBAAAA,KAAKX,MAAM,cAAXW,oCAAAA,cAAa2B,cAAc;QAC3B,2FAA2F;QAC3F,mEAAmE;QACnEN,UAAU1C,QAAQ+B,SAAS;YAAEkB,WAAW;QAAK,GAAG,CAACC,QAAQC;YACvD,IAAI,CAAChD,iCAAiCgD,WAAW;YACjD,IAAIR,SAASC,sBAAsB;iBAC9BC,8BAAAA,wCAAAA;QACP;QACAH,QAAQU,EAAE,CAAC,SAAS,CAACd;YACnBJ,yBAAAA,0BAAAA,eAAAA,eAAiBI;YACjBH,4BAAAA,sCAAAA,gBAAkBG;QACpB;QAEA,MAAMe,gBAA8E,EAAE;QACtF,MAAM,EAAEC,OAAOC,YAAY,EAAEC,MAAM,EAAEC,UAAUC,eAAe,EAAEC,QAAQC,aAAa,EAAE,GAAG,MAAMvD,UAAUe,KAAK;YAAEV,QAAQW,KAAKX,MAAM;QAAC;QACrI,IAAImD,eAAe;QACnB,IAAI;YACF,mFAAmF;YACnFA,eAAeH,gBAAgBI,MAAM,GAAG,KAAKF,gBAAgB,IAAI,MAAMxD,SAASmD,gBAAgB;QAClG,SAAU;YACR,MAAMA,aAAad,KAAK;QAC1B;SACApB,gBAAAA,KAAKX,MAAM,cAAXW,oCAAAA,cAAa2B,cAAc;QAC3B,IAAId,cAAc,MAAMA;QAExBmB,cAAcU,IAAI,CAAC;YAAEJ,QAAQC;YAAeI,OAAOH;YAAcJ,UAAUC;QAAgB;QAC3F,4FAA4F;QAC5F,kFAAkF;QAClF,GAAG;gBAEDrC,eAOAA;YARAuB,sBAAsB;aACtBvB,gBAAAA,KAAKX,MAAM,cAAXW,oCAAAA,cAAa2B,cAAc;YAC3B,MAAM,EAAEM,KAAK,EAAEK,MAAM,EAAEF,QAAQ,EAAE,GAAG,MAAMpD,UAAUe,KAAK;gBAAEV,QAAQW,KAAKX,MAAM;YAAC;YAC/E,IAAI;gBACF,IAAIiD,SAAS,KAAKF,SAASK,MAAM,GAAG,GAAGT,cAAcU,IAAI,CAAC;oBAAEJ;oBAAQK,OAAO,MAAM5D,SAASkD;oBAAQG;gBAAS;YAC7G,SAAU;gBACR,MAAMH,MAAMb,KAAK;YACnB;aACApB,gBAAAA,KAAKX,MAAM,cAAXW,oCAAAA,cAAa2B,cAAc;YAC3B,IAAId,cAAc,MAAMA;QAC1B,QAASU,uBAAuB,GAACvB,gBAAAA,KAAKX,MAAM,cAAXW,oCAAAA,cAAaT,OAAO,EAAE;QACvD+B,UAAU;QAEV,6FAA6F;QAC7F,qFAAqF;QACrF,IAAIsB,UAAgC;QACpC,IAAIC,eAA+B;QAEnC,MAAMC,OAAO,OAAOC;YAClB,IAAI;gBACF,MAAM,EAAEd,KAAK,EAAEK,MAAM,EAAEF,QAAQ,EAAE,GAAG,MAAMpD,UAAUe,KAAK;oBAAEV,QAAQW,KAAKX,MAAM;gBAAC;gBAC/E,IAAI;oBACF,IAAI0D,cAAcT,SAAS,KAAKF,SAASK,MAAM,GAAG,GAAG;wBACnDxC,QAAQ;4BAAE+C,MAAM;4BAAcV;4BAAQK,OAAO,MAAM5D,SAASkD;4BAAQG;wBAAS;oBAC/E;gBACF,SAAU;oBACR,MAAMH,MAAMb,KAAK;gBACnB;YACF,EAAE,OAAOH,KAAK;gBACZ,IAAI7B,cAAcY,KAAKX,MAAM,EAAE4B,MAAM;gBACrChB,QAAQ;oBAAE+C,MAAM;oBAAmBpD,SAASF,aAAauB;gBAAK;YAChE;QACF;QAEA,MAAMgC,aAAa,CAACF;YAClBH,UAAUE,KAAKC,YAAYG,IAAI,CAAC;gBAC9BN,UAAU;gBACV,IAAIC,iBAAiB,QAAQ,CAACpB,UAAU;oBACtC,MAAM0B,QAAQN;oBACdA,eAAe;oBACfI,WAAWE;gBACb;YACF;QACF;QAEA,MAAMC,eAAe,CAACL;YACpB,IAAItB,UAAU;YACd,IAAImB,SAAS;gBACXC,eAAeA,gBAAgBE;gBAC/B;YACF;YACAE,WAAWF;QACb;QAEA,IAAIM,gBAAuC;QAC3C,gGAAgG;QAChG,mGAAmG;QACnG7B,oBAAoB;YAClB,IAAI6B,eAAeC,aAAaD;YAChCA,gBAAgBE,WAAW;gBACzBF,gBAAgB;gBAChBD,aAAa;YACf,GAAGlD;QACL;QACA,MAAMsD,iBAAiBC,YACrB5E,YACE,UAAYuE,aAAa,QACzB,IAAM3B,WAERtB;QAGF,OAAO,MAAM,IAAIuD,QAAc,CAACC,iBAAiBC;gBA2CtC5D;YA1CT,IAAI6D,iBAA+B;YACnC,MAAMC,WAAW,CAACxE,QAAgByE,mBAAmB,KAAK;gBACxDF,2BAAAA,4BAAAA,iBAAAA,iBAAmBvE,mBAAAA,oBAAAA,SAAU;gBAC7BoC,mBAAAA,iBAAmBqC;gBACnB,IAAItC,UAAU;gBACdA,WAAW;gBACX,MAAMuC,eAAe,AAAC,CAAA;wBAGpBhE;oBAFAiE,QAAQC,GAAG,CAAC,UAAUC;oBACtBF,QAAQC,GAAG,CAAC,WAAWC;qBACvBnE,eAAAA,KAAKX,MAAM,cAAXW,mCAAAA,aAAaoE,mBAAmB,CAAC,SAASC;oBAC1CC,cAAcd;oBACd,IAAIH,eAAeC,aAAaD;oBAChChC,oBAAAA,8BAAAA,QAASD,KAAK;oBACd,MAAMwB;oBACN,IAAI2B;oBACJ,IAAI;wBACF,uFAAuF;wBACvF,uFAAuF;wBACvF,IAAI,CAACV,kBAAkB,CAACnC,gBAAgB;4BACtC,MAAM,EAAEO,OAAOuC,UAAU,EAAE,GAAG,MAAMxF,UAAUe;4BAC9C,MAAMyE,WAAWpD,KAAK;wBACxB;oBACF,EAAE,OAAOH,KAAK;wBACZsD,aAAatD;oBACf;oBACA,IAAI;wBACF,MAAME;oBACR,EAAE,OAAOF,KAAK;wBACZ,IAAIsD,YAAY,MAAM,IAAIE,eAAe;4BAACF;4BAAYtD;yBAAI,EAAE;wBAC5D,MAAMA;oBACR;oBACA,IAAIsD,YAAY,MAAMA;oBACtB,IAAIV,gBAAgB,MAAMA;gBAC5B,CAAA;gBACAG,aAAad,IAAI,CAACS,iBAAiBC;YACrC;YACA,MAAMO,eAAe,IAAML;YAC3B,MAAMO,gBAAgB,IAAMP,SAASY,WAAW;YAChD5D,kBAAkBgD;YAClBG,QAAQU,IAAI,CAAC,UAAUR;YACvBF,QAAQU,IAAI,CAAC,WAAWR;YACxB,IAAItD,cAAciD,SAASjD;iBACtB,KAAIb,eAAAA,KAAKX,MAAM,cAAXW,mCAAAA,aAAaT,OAAO,EAAE8E;iBAC1B;oBACHrE;iBAAAA,gBAAAA,KAAKX,MAAM,cAAXW,oCAAAA,cAAa4E,gBAAgB,CAAC,SAASP,eAAe;oBAAEM,MAAM;gBAAK;gBACnE,IAAI;oBACF1E,QAAQ;wBAAE+C,MAAM;wBAAWtC;wBAASyB;oBAAO;oBAC3C,IAAI,CAACV,UAAU;wBACb,KAAK,MAAMoD,SAAS7C,cAAe;4BACjC,IAAIP,UAAU;4BACd,IAAIoD,MAAMzC,QAAQ,CAACK,MAAM,GAAG,KAAKoC,MAAMvC,MAAM,GAAG,GAAGrC,QAAQ;gCAAE+C,MAAM;gCAAc,GAAG6B,KAAK;4BAAC;wBAC5F;oBACF;gBACF,EAAE,OAAO5D,KAAK;oBACZ6C,SAAS7C,eAAetB,QAAQsB,MAAM,IAAItB,MAAME,OAAOoB;gBACzD;YACF;QACF;IACF,EAAE,OAAOA,KAAK;QACZ,MAAM8C,mBAAmB3E,cAAcY,KAAKX,MAAM,EAAE4B;QACpDI,oBAAAA,8BAAAA,QAASD,KAAK;QACd,IAAIF,mBAAmB,MAAMD;QAC7B,IAAI;YACF,MAAME;QACR,EAAE,OAAO2D,YAAY;YACnB,MAAM,IAAIL,eAAe;gBAACxD;gBAAK6D;aAAW,EAAE;QAC9C;QACA,IAAIf,kBAAkB;QACtB,MAAM9C;IACR,SAAU;QACRH,kBAAkB4D;IACpB;AACF"}
|
|
@@ -6,8 +6,8 @@ export interface ParseWorkerData {
|
|
|
6
6
|
cfg: Config;
|
|
7
7
|
featureNames: FeatureName[];
|
|
8
8
|
}
|
|
9
|
-
export type ParseTask = FileStat;
|
|
10
|
-
export type
|
|
9
|
+
export type ParseTask = FileStat[];
|
|
10
|
+
export type ParseFileResult = {
|
|
11
11
|
ok: true;
|
|
12
12
|
doc: ParsedDoc;
|
|
13
13
|
warnings: string[];
|
|
@@ -16,4 +16,5 @@ export type ParseTaskResult = {
|
|
|
16
16
|
ok: false;
|
|
17
17
|
error: WorkerErrorPayload;
|
|
18
18
|
};
|
|
19
|
-
export
|
|
19
|
+
export type ParseTaskResult = ParseFileResult[];
|
|
20
|
+
export default function parseTask(files: ParseTask): ParseTaskResult;
|
|
@@ -8,21 +8,23 @@ const { cfg, featureNames } = Tinypool.workerData;
|
|
|
8
8
|
// Filtering the registry (rather than mapping the names) keeps registry order, which is the
|
|
9
9
|
// order `extracted` keys land in on the serial path.
|
|
10
10
|
const selected = FEATURES.filter((feature)=>featureNames.includes(feature.name));
|
|
11
|
-
export default function parseTask(
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
11
|
+
export default function parseTask(files) {
|
|
12
|
+
return files.map((file)=>{
|
|
13
|
+
try {
|
|
14
|
+
const start = process.hrtime.bigint();
|
|
15
|
+
const { doc, warnings } = parseFile(file, featuresForFile(selected, cfg, file), cfg);
|
|
16
|
+
const parseMs = Number(process.hrtime.bigint() - start) / 1e6;
|
|
17
|
+
return {
|
|
18
|
+
ok: true,
|
|
19
|
+
doc,
|
|
20
|
+
warnings,
|
|
21
|
+
parseMs
|
|
22
|
+
};
|
|
23
|
+
} catch (err) {
|
|
24
|
+
return {
|
|
25
|
+
ok: false,
|
|
26
|
+
error: serializeError(err)
|
|
27
|
+
};
|
|
28
|
+
}
|
|
29
|
+
});
|
|
28
30
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/workers/parse.ts"],"sourcesContent":["import Tinypool from 'tinypool';\nimport type { Config, FeatureName } from '../config/index.ts';\nimport { FEATURES } from '../features/index.ts';\nimport type { ParsedDoc } from '../scan/index.ts';\nimport { parseFile } from '../scan/index.ts';\nimport type { FileStat } from '../scan/list.ts';\nimport { featuresForFile } from '../scan/reparse.ts';\nimport type { WorkerErrorPayload } from '../scan/worker-error.ts';\nimport { serializeError } from '../scan/worker-error.ts';\n\n// Constant for the whole dispatch, so it crosses once per worker instead of once per task. A\n// Feature carries closures and cannot cross the thread boundary; its name can, and the registry here resolves it back.\nexport interface ParseWorkerData {\n cfg: Config;\n featureNames: FeatureName[];\n}\n\n//
|
|
1
|
+
{"version":3,"sources":["/Users/kevin/Dev/OpenSource/ai/sensemaking/src/workers/parse.ts"],"sourcesContent":["import Tinypool from 'tinypool';\nimport type { Config, FeatureName } from '../config/index.ts';\nimport { FEATURES } from '../features/index.ts';\nimport type { ParsedDoc } from '../scan/index.ts';\nimport { parseFile } from '../scan/index.ts';\nimport type { FileStat } from '../scan/list.ts';\nimport { featuresForFile } from '../scan/reparse.ts';\nimport type { WorkerErrorPayload } from '../scan/worker-error.ts';\nimport { serializeError } from '../scan/worker-error.ts';\n\n// Constant for the whole dispatch, so it crosses once per worker instead of once per task. A\n// Feature carries closures and cannot cross the thread boundary; its name can, and the registry here resolves it back.\nexport interface ParseWorkerData {\n cfg: Config;\n featureNames: FeatureName[];\n}\n\n// Each task is a small file batch. Results stay per-file so one failure does not stop the\n// remaining files in that batch, and no token tree crosses the thread boundary.\nexport type ParseTask = FileStat[];\n\n// parseMs is the worker's own hrtime for parseFile, excluding dispatch and the return clone --\n// what separates it from the pool's dispatch-to-drain `parse` stage.\nexport type ParseFileResult = { ok: true; doc: ParsedDoc; warnings: string[]; parseMs: number } | { ok: false; error: WorkerErrorPayload };\nexport type ParseTaskResult = ParseFileResult[];\n\n// Read once per worker, not per task.\nconst { cfg, featureNames } = Tinypool.workerData as ParseWorkerData;\n// Filtering the registry (rather than mapping the names) keeps registry order, which is the\n// order `extracted` keys land in on the serial path.\nconst selected = FEATURES.filter((feature) => featureNames.includes(feature.name));\n\nexport default function parseTask(files: ParseTask): ParseTaskResult {\n return files.map((file): ParseFileResult => {\n try {\n const start = process.hrtime.bigint();\n const { doc, warnings } = parseFile(file, featuresForFile(selected, cfg, file), cfg);\n const parseMs = Number(process.hrtime.bigint() - start) / 1e6;\n return { ok: true, doc, warnings, parseMs };\n } catch (err) {\n return { ok: false, error: serializeError(err) };\n }\n });\n}\n"],"names":["Tinypool","FEATURES","parseFile","featuresForFile","serializeError","cfg","featureNames","workerData","selected","filter","feature","includes","name","parseTask","files","map","file","start","process","hrtime","bigint","doc","warnings","parseMs","Number","ok","err","error"],"mappings":"AAAA,OAAOA,cAAc,WAAW;AAEhC,SAASC,QAAQ,QAAQ,uBAAuB;AAEhD,SAASC,SAAS,QAAQ,mBAAmB;AAE7C,SAASC,eAAe,QAAQ,qBAAqB;AAErD,SAASC,cAAc,QAAQ,0BAA0B;AAkBzD,sCAAsC;AACtC,MAAM,EAAEC,GAAG,EAAEC,YAAY,EAAE,GAAGN,SAASO,UAAU;AACjD,4FAA4F;AAC5F,qDAAqD;AACrD,MAAMC,WAAWP,SAASQ,MAAM,CAAC,CAACC,UAAYJ,aAAaK,QAAQ,CAACD,QAAQE,IAAI;AAEhF,eAAe,SAASC,UAAUC,KAAgB;IAChD,OAAOA,MAAMC,GAAG,CAAC,CAACC;QAChB,IAAI;YACF,MAAMC,QAAQC,QAAQC,MAAM,CAACC,MAAM;YACnC,MAAM,EAAEC,GAAG,EAAEC,QAAQ,EAAE,GAAGpB,UAAUc,MAAMb,gBAAgBK,UAAUH,KAAKW,OAAOX;YAChF,MAAMkB,UAAUC,OAAON,QAAQC,MAAM,CAACC,MAAM,KAAKH,SAAS;YAC1D,OAAO;gBAAEQ,IAAI;gBAAMJ;gBAAKC;gBAAUC;YAAQ;QAC5C,EAAE,OAAOG,KAAK;YACZ,OAAO;gBAAED,IAAI;gBAAOE,OAAOvB,eAAesB;YAAK;QACjD;IACF;AACF"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "sensemaking",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.26.0",
|
|
4
4
|
"description": "Query and search your markdown notes with context-aware progressive disclosure: SQL over frontmatter, links, and text, plus semantic search and link-graph ranking. No server, no build step.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"markdown",
|
package/skills/sense/SKILL.md
CHANGED
|
@@ -5,7 +5,7 @@ description: "Query a markdown tree with the sense CLI: filter notes by frontmat
|
|
|
5
5
|
|
|
6
6
|
# sense
|
|
7
7
|
|
|
8
|
-
Use sense to locate evidence in a markdown tree before reading files. With the config default `"build": true`, CLI queries incrementally scan the configured tree for the capabilities they need; vector preparation is limited to the eligible scope. Config `"build": false` or a one-query `--no-build` reads the last completed indexed generation, checks requested readiness and does not scan live files. Explicit build and watch prepare it regardless of that default. Results contain paths, metadata,
|
|
8
|
+
Use sense to locate evidence in a markdown tree before reading files. With the config default `"build": true`, CLI queries incrementally scan the configured tree for the capabilities they need; vector preparation is limited to the eligible scope. Config `"build": false` or a one-query `--no-build` reads the last completed indexed generation, checks requested readiness and does not scan live files. Explicit build and watch prepare it regardless of that default. Results contain paths, metadata, snippets, and line ranges. Read the returned files or ranges when the task needs their prose.
|
|
9
9
|
|
|
10
10
|
Setup, store selection, presets, and note design belong to the `sense-setup` skill. Translating an Obsidian Bases file belongs to `sense-bases`.
|
|
11
11
|
|
|
@@ -73,7 +73,9 @@ Read [search evidence and troubleshooting](references/search.md) when a search w
|
|
|
73
73
|
|
|
74
74
|
## Scope and output
|
|
75
75
|
|
|
76
|
-
Bare commands use the `default` preset. `--preset <name>` chooses another. For `search`, `--include`, `--exclude`, and `--no-exclude` change the query scope for one invocation, but cannot reach files that no preset indexes. `sense status` shows actual coverage.
|
|
76
|
+
Bare commands use the `default` preset. `--preset <name>` chooses another. For `search`, `map`, `peek`, `path`, and `related`, `--include`, `--exclude`, and `--no-exclude` change the query scope for one invocation, but cannot reach files that no preset indexes. `sense status` shows actual coverage. Scope selects results; presets are not a security boundary.
|
|
77
|
+
|
|
78
|
+
`peek` rejects an exact path outside scope before trying a basename, resolves basenames within scope, and shows only scoped resolved links and backlinks; unresolved targets remain as written. `path` resolves its endpoints across the indexed tree and scopes intermediate notes. `related` resolves its seed across the indexed tree and scopes result candidates.
|
|
77
79
|
|
|
78
80
|
`--where` filters search and graph commands against frontmatter alias `f`:
|
|
79
81
|
|
|
@@ -4,7 +4,7 @@ Read this guide when `store` is `duckdb`.
|
|
|
4
4
|
|
|
5
5
|
Sense stores the cache at `.sense/cache.duckdb`. It is a DuckDB database file with frontmatter values in `VARIANT` columns and embeddings in native fixed-width float arrays. Use this store when another DuckDB tool needs to inspect or analyze the cache.
|
|
6
6
|
|
|
7
|
-
The first open installs `@duckdb/node-api` if needed. Its native download is about 110 MB. DuckDB
|
|
7
|
+
The first open installs `@duckdb/node-api` if needed. Its native download is about 110 MB. Under DuckDB's [concurrency model](https://duckdb.org/docs/current/connect/concurrency), Sense opens an existing cache for no-build observation in read-only mode, so several independent reader processes can coexist. A build-enabled command or watcher opens it read-write and excludes every other process until it closes. Contenders wait within Sense's bounded lock budget, then fail with `STORE_BUSY`. This cross-process rule does not promise writer/reader coexistence or unrestricted sharing by independently configured DuckDB instances in one process.
|
|
8
8
|
|
|
9
9
|
## `sense search` grammar
|
|
10
10
|
|
|
@@ -22,7 +22,7 @@ sense status
|
|
|
22
22
|
|
|
23
23
|
By default, the config indexes its own directory. A top-level `root` can point at another markdown tree. Relative roots resolve from the config directory, and `.sense/` remains beside the config. Use this when separate consumers need their own presets, saved queries, or caches over one tree.
|
|
24
24
|
|
|
25
|
-
Globs and indexed paths are relative to the selected root. Run `sense status` and `sense map` after changing the config to confirm preset coverage and the selected store. A change that affects indexed content rebuilds the relevant cache and reports the reason.
|
|
25
|
+
Globs and indexed paths are relative to the selected root. Run `sense status` and `sense map` after changing the config to confirm preset coverage and the selected store. A change that affects indexed content rebuilds the relevant cache and reports the reason. A build-enabled command replaces an incompatible derived cache format and repairs incomplete feature state; config `"build": false` or `--no-build` reports the required build without rebuilding or repairing the Sense index.
|
|
26
26
|
|
|
27
27
|
## Choose the store
|
|
28
28
|
|
|
@@ -86,7 +86,7 @@ Vectors require two choices. The top-level `embed` block names the model and pro
|
|
|
86
86
|
}
|
|
87
87
|
```
|
|
88
88
|
|
|
89
|
-
The config's `"build": true` default lets CLI queries prepare their needed capabilities. Set `"build": false` for manual-build or watch workflows. The first build-enabled vector search downloads a named static model and embeds its eligible scope. `sense build` and `sense watch` prepare all configured capabilities regardless of this setting; `sense download` fetches the model earlier when CI, offline work, or timing makes that useful. A config change that alters the model, vector coverage, or chunking can rebuild vectors. Config `"build": false` or `--no-build` requires the requested vector scope to be ready and does not read newer live files.
|
|
89
|
+
The config's `"build": true` default lets CLI queries prepare their needed capabilities. Set `"build": false` for manual-build or watch workflows. The first build-enabled vector search downloads a named static model and embeds its eligible scope. `sense build` and `sense watch` prepare all configured capabilities regardless of this setting; `sense download` fetches the model earlier when CI, offline work, or timing makes that useful. A config change that alters the provider, model, effective endpoint, vector coverage, or chunking can invalidate or rebuild vectors. Config `"build": false` or `--no-build` requires the requested vector scope to be ready and does not read newer live files.
|
|
90
90
|
|
|
91
91
|
Read [embedding setup](references/embeddings.md) when choosing a provider or model, supporting a non-English tree, changing chunk size, or tuning signal weights.
|
|
92
92
|
|
|
@@ -22,7 +22,7 @@ For a static model, inspect current Model2Vec models and their cards rather than
|
|
|
22
22
|
|
|
23
23
|
Naming a remote static model in the config authorizes its download. `sense download` performs that fetch before a query. A default vector query embeds its eligible scope; `sense build` and `sense watch` prepare all configured vector capabilities, while `--no-build` requires the requested scope to be ready.
|
|
24
24
|
|
|
25
|
-
Changing the model
|
|
25
|
+
Changing the provider, model, or effective endpoint invalidates stored vector values while preserving the indexed chunks. Changing `embed.chunkTokens` rebuilds affected embedding rows, and changing preset coverage can add, remove, or rebuild them. A build-enabled command performs this work; config `"build": false` or `--no-build` reports incompatible or unprepared state without rebuilding or repairing the Sense index. Settle the broad scope before embedding a large tree.
|
|
26
26
|
|
|
27
27
|
## Chunk size
|
|
28
28
|
|
|
@@ -1,15 +1,15 @@
|
|
|
1
|
-
<!-- sense-store-benchmark release=0.
|
|
1
|
+
<!-- sense-store-benchmark release=0.26.0 -->
|
|
2
2
|
# Current store benchmark summary
|
|
3
3
|
|
|
4
|
-
The release assessment generated this file for store selection. Release `0.
|
|
4
|
+
The release assessment generated this file for store selection. Release `0.26.0` passed on 2026-10-01, measured on Apple M1 Pro with Node v26.10.0.
|
|
5
5
|
|
|
6
6
|
The timing rows ran on the same 6,566-note tree. They include CLI startup and each store's complete selected path. Ranked candidates and downstream work can differ by store, so these are current operating measurements rather than an isolated database-engine contest.
|
|
7
7
|
|
|
8
8
|
| Store | Cold index | Warm count | Lexical search | Semantic search | Portable semantic nDCG@10 |
|
|
9
9
|
|---|---|---|---|---|---|
|
|
10
|
-
| sqlite |
|
|
11
|
-
| duckdb |
|
|
12
|
-
| turso |
|
|
10
|
+
| sqlite | 2,158 ms | 256 ms | 351 ms | 569 ms | 0.3306 |
|
|
11
|
+
| duckdb | 3,100 ms | 315 ms | 501 ms | 691 ms | 0.3282 |
|
|
12
|
+
| turso | 3,268 ms | 276 ms | 409 ms | 1,066 ms | 0.3299 |
|
|
13
13
|
|
|
14
14
|
Cold index measures core index creation and a count query, excluding document embeddings. Warm count is a no-change `COUNT(*)` query. Lexical and semantic search are steady-state `sense search` commands. Lower timing is faster. Higher nDCG@10 is better; that quality column uses the same NFCorpus queries, judgments, result count, and model on every store.
|
|
15
15
|
|
|
@@ -18,7 +18,7 @@ Sense's [store benchmark summary](../store-benchmarks.md) measures the current a
|
|
|
18
18
|
|
|
19
19
|
- DuckDB support is experimental.
|
|
20
20
|
- The first open installs `@duckdb/node-api`, with a native download of about 110 MB.
|
|
21
|
-
- DuckDB
|
|
21
|
+
- Under DuckDB's [concurrency model](https://duckdb.org/docs/current/connect/concurrency), Sense opens an existing cache for no-build observation in read-only mode, so several independent reader processes can coexist. A build-enabled command or watcher opens it read-write and excludes every other process until it closes. Contenders wait within Sense's bounded lock budget, then fail with `STORE_BUSY`. This cross-process rule does not promise writer/reader coexistence or unrestricted sharing by independently configured DuckDB instances in one process.
|
|
22
22
|
- `sense search` accepts bare words and quoted phrases. It rejects FTS5 boolean, prefix, `NEAR`, initial-token, grouping, and column-filter syntax.
|
|
23
23
|
- SQLite's raw `MATCH`, `snippet()`, and `bm25()` queries do not port.
|
|
24
24
|
- Mixed-type `VARIANT` fields can require explicit casts in predicates.
|