grepmax 0.26.79 → 0.26.81
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/README.md +5 -5
- package/dist/lib/daemon/daemon.js +97 -1
- package/dist/lib/daemon/ipc-handler.js +15 -14
- package/dist/lib/store/lance-cleanup.js +23 -3
- package/dist/lib/store/maintenance-policy.js +42 -2
- package/dist/lib/store/recovery-admission.js +3 -0
- package/dist/lib/store/vector-db.js +76 -9
- package/dist/lib/store/version-cleanup.js +120 -0
- package/dist/lib/utils/host-resource.js +10 -3
- package/dist/lib/utils/operation-coordinator.js +60 -3
- package/lance-maintenance/prune.py +15 -6
- package/package.json +1 -1
- package/plugins/grepmax/.claude-plugin/plugin.json +1 -1
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
"name": "grepmax",
|
|
11
11
|
"source": "./plugins/grepmax",
|
|
12
12
|
"description": "Semantic code search for Claude Code. Automatically indexes your project and provides intelligent search capabilities.",
|
|
13
|
-
"version": "0.26.
|
|
13
|
+
"version": "0.26.81",
|
|
14
14
|
"author": {
|
|
15
15
|
"name": "Robert Owens",
|
|
16
16
|
"email": "robowens@me.com"
|
package/README.md
CHANGED
|
@@ -524,15 +524,15 @@ To serve bounded reads under an existing startup quarantine, run `gmax watch --d
|
|
|
524
524
|
|
|
525
525
|
Full-table compaction and new FTS/ANN index builds are disabled in the containment release, including forced maintenance and doctor repair. Existing indexes remain readable; missing lexical indexes fall back to available retrieval. Do not repeatedly force repair, use future cutoffs or manually remove index fragments.
|
|
526
526
|
|
|
527
|
-
The package includes a separate prune-only helper with pinned Lance 12 dependencies, plus an exclusively leased Node wrapper. Runtime preparation uses `uv >=0.12.18`, an existing Python interpreter and locked wheels; it does not download Python, build from source, load an embedding model or run during installation. The helper currently supports macOS and Linux.
|
|
527
|
+
The package includes a separate prune-only helper with pinned Lance 12 dependencies, plus an exclusively leased Node wrapper. Runtime preparation uses `uv >=0.12.18`, an existing Python interpreter and locked wheels; it does not download Python, build from source, load an embedding model or run during installation. The helper currently supports macOS and Linux. The daemon runs independent prune-only retention: at most 128 old versions per pass, keeping current, tagged and two-minute recent versions. It retries backlog every minute and checks settled history every five minutes; status polling and watched edits do not prevent cleanup. Runtime setup happens before writer suspension. Older readers and external store owners are drained under an exclusive lease; new read-only queries may use the protected current version while history is deleted. Writers resume and watchers reconcile edits after the pass. Full-table compaction admission never gates this reclamation. Normal configured host policy applies; explicit offline recovery retains its separate admission.
|
|
528
528
|
|
|
529
529
|
`gmax recover --table /path/to/lancedb/chunks.lance --json` reads the durable recovery receipt without starting a daemon or preparing Python. `--check` samples admission without pruning. `--prune --version <current-version> --cutoff <absolute-ISO-timestamp>` explicitly requests one offline prune. A prior uncertain attempt requires `--acknowledge-uncertain <attempt-id>` after inspection. Recovery never creates/clears quarantine, stops other owners, retries automatically or enables rewriting.
|
|
530
530
|
|
|
531
531
|
Production recovery currently requires macOS, a persistent `autostart-disabled` file in the selected store home, and closed store owners. Recovery uses the shared aggregate resource budget and requires measured OS pressure and healthy kernel pressure. The configured `critical-only` policy permits a known OS memory warning while retaining aggregate accounting; the default `strict` policy pauses on warnings. Immediately free RAM is measured for diagnostics but has no fixed minimum: macOS can reclaim cached memory, so a low free-page count alone must not prevent cleanup. The helper reserves 512 MiB of admission capacity; this is not an OS-enforced footprint cap. Recovery samples have an eight-second total ceiling, including a five-second kernel probe allowance, three-second command allowance and three-second footprint allowance, with fresh sampling at deletion admission and five-second heartbeat scheduling. Unknown, stale, critical, kernel-warning or over-budget measurements refuse/interrupt recovery. A prune has a ten-minute wall-time ceiling so rate-limited deletion of thousands of manifests can finish; the 90-second CPU limit and guarded heartbeat remain. Pruning needs at least 1 MiB of measured metadata disk headroom, not full-table rewrite space. Initial wheel setup requires 512 MiB; an existing environment is checked against the lock offline without that download-space requirement.
|
|
532
532
|
|
|
533
|
-
The child waits for launch and deletion admission tokens, with verified helper PID/start identity recorded as the primary exclusive owner. Existing readers retain exclusion if the parent dies while the helper is alive. Current/tagged/post-cutoff metadata is verified after pruning; recovery refuses more than 256 protected versions before deletion to bound verification.
|
|
533
|
+
The child waits for launch and deletion admission tokens, with verified helper PID/start identity recorded as the primary exclusive owner. Existing readers retain exclusion if the parent dies while the helper is alive. Current/tagged/post-cutoff metadata is verified after pruning; recovery refuses more than 256 protected versions before deletion to bound verification. Explicit recovery requires verification and acknowledgement of interrupted attempts. Automatic retention may make a fresh exclusively owned prune-only pass after interruption; it re-verifies retained state and links the prior uncertain attempt without claiming that attempt succeeded. Linux/APFS fixtures exercise native retention and interruption; supported production host admission is tested separately and still runs freshly for each operation.
|
|
534
534
|
|
|
535
|
-
If a short-lived client child exits during memory sampling, confirmed dead processes can leave the sample while all surviving processes retain measured footprints. An uncertain exit, new process or changed identity requires a complete fresh measurement; a
|
|
535
|
+
If a short-lived client child exits during memory sampling, confirmed dead processes can leave the sample while all surviving processes retain measured footprints. An uncertain exit, new process or changed identity requires a complete fresh measurement; a recovery sample may do that once within the original deadline while retaining any observed OS warning for the final policy check. Normal service requires normal pressure for this retry. It never accepts an unmeasured live process. Continued changes, unavailable measurements and critical pressure still refuse admission; warning admission follows the recovery policy above.
|
|
536
536
|
|
|
537
537
|
For an operator-coordinated recovery window:
|
|
538
538
|
|
|
@@ -544,9 +544,9 @@ For an operator-coordinated recovery window:
|
|
|
544
544
|
|
|
545
545
|
Explicit helper calls persist a private `.gmax-prune-state.json` receipt outside the table before deletion. Cancellation, invalid results, lost ownership and abandoned running receipts remain uncertain; a verified explicit retry records the prior uncertain attempt. Successful receipts include before/after file lengths, allocated bytes and filesystem free space. These measurements do not imply zero retained copies or a macOS physical-footprint cap. Current, tagged and post-cutoff versions are protected; unverified files are retained. Full-table rewriting remains disabled regardless of receipt state.
|
|
546
546
|
|
|
547
|
-
`~/.gmax/logs/daemon.log` records containment skips and retained historical compaction outcomes, including duration, logical size, disk size before/after, free space, and bytes reclaimed. `gmax status --json` exposes
|
|
547
|
+
`~/.gmax/logs/daemon.log` records containment skips and retained historical compaction outcomes, including duration, logical size, disk size before/after, free space, and bytes reclaimed. `gmax status --json` exposes historical compaction under `daemon.compaction` and independent retention under `daemon.versionCleanup`; their `at` timestamps identify when each was recorded. Retention reports versions removed, eligible backlog, allocated bytes before/after and signed net reclamation, with `rewritten: false`. Five-minute resource snapshots track footprint, RSS, heap, external/ArrayBuffer memory, Lance cache, workers and pending work for memory investigations. Reading status uses retained snapshots and does not scan the index directory. Logs rotate to `daemon.log.prev`.
|
|
548
548
|
|
|
549
|
-
`gmax doctor --fix` refuses work under quarantine
|
|
549
|
+
`gmax doctor --fix` refuses work under quarantine. With a current daemon it requests prune-only version cleanup; it does not enable full-table rewriting. Without a daemon, the historical full-table optimizer remains disabled. It exits nonzero when a requested optimization did not complete or an older daemon could not verify its outcome. A busy or failed daemon is never retried as a second writer in the CLI process.
|
|
550
550
|
|
|
551
551
|
### Known issues
|
|
552
552
|
|
|
@@ -62,6 +62,7 @@ const maintenance_policy_1 = require("../store/maintenance-policy");
|
|
|
62
62
|
const meta_cache_1 = require("../store/meta-cache");
|
|
63
63
|
const store_lease_1 = require("../store/store-lease");
|
|
64
64
|
const vector_db_1 = require("../store/vector-db");
|
|
65
|
+
const version_cleanup_1 = require("../store/version-cleanup");
|
|
65
66
|
const async_semaphore_1 = require("../utils/async-semaphore");
|
|
66
67
|
const autostart_1 = require("../utils/autostart");
|
|
67
68
|
const blocked_roots_1 = require("../utils/blocked-roots");
|
|
@@ -159,6 +160,9 @@ class Daemon {
|
|
|
159
160
|
this.startTime = Date.now();
|
|
160
161
|
this.heartbeatInterval = null;
|
|
161
162
|
this.idleInterval = null;
|
|
163
|
+
this.cleanupInterval = null;
|
|
164
|
+
this.cleanupPromise = null;
|
|
165
|
+
this.lastCleanupAttempt = 0;
|
|
162
166
|
this.heartbeatTick = 0;
|
|
163
167
|
this.shuttingDown = false;
|
|
164
168
|
this.recycling = false;
|
|
@@ -450,6 +454,7 @@ class Daemon {
|
|
|
450
454
|
this.assertStartupActive();
|
|
451
455
|
// Resources are open — only now may resource-dependent IPC commands run.
|
|
452
456
|
this.ready = true;
|
|
457
|
+
this.startVersionCleanupLoop();
|
|
453
458
|
}
|
|
454
459
|
catch (err) {
|
|
455
460
|
console.error("[daemon] Failed to open shared resources:", err);
|
|
@@ -830,7 +835,7 @@ class Daemon {
|
|
|
830
835
|
const db = this.vectorDb;
|
|
831
836
|
if (!db)
|
|
832
837
|
throw new Error("daemon resources not ready");
|
|
833
|
-
const compaction = yield this.
|
|
838
|
+
const compaction = yield this.runVersionCleanup();
|
|
834
839
|
if (!compaction)
|
|
835
840
|
return { ok: false, error: "maintenance produced no compaction outcome" };
|
|
836
841
|
return Object.assign({ ok: compaction.status === "completed", compaction }, (compaction.status === "completed"
|
|
@@ -838,6 +843,95 @@ class Daemon {
|
|
|
838
843
|
: { error: (_a = compaction.reason) !== null && _a !== void 0 ? _a : compaction.status }));
|
|
839
844
|
});
|
|
840
845
|
}
|
|
846
|
+
versionCleanupStatus() {
|
|
847
|
+
var _a;
|
|
848
|
+
var _b;
|
|
849
|
+
return (_b = (_a = this.vectorDb) === null || _a === void 0 ? void 0 : _a.versionCleanupStatus()) !== null && _b !== void 0 ? _b : null;
|
|
850
|
+
}
|
|
851
|
+
/** Status polling and watched edits cannot starve independent reclamation. */
|
|
852
|
+
startVersionCleanupLoop() {
|
|
853
|
+
if (this.cleanupInterval)
|
|
854
|
+
return;
|
|
855
|
+
this.cleanupInterval = setInterval(() => {
|
|
856
|
+
var _a;
|
|
857
|
+
const interval = ((_a = this.versionCleanupStatus()) === null || _a === void 0 ? void 0 : _a.eligibleVersionsRemaining) === 0
|
|
858
|
+
? 5 * 60000
|
|
859
|
+
: 60000;
|
|
860
|
+
if (!this.ready ||
|
|
861
|
+
this.shuttingDown ||
|
|
862
|
+
this.recycling ||
|
|
863
|
+
this.pausedReason !== null ||
|
|
864
|
+
this.cleanupPromise ||
|
|
865
|
+
this.operations.status !== "open" ||
|
|
866
|
+
Date.now() - this.lastCleanupAttempt < interval)
|
|
867
|
+
return;
|
|
868
|
+
void this.runVersionCleanup().catch((error) => {
|
|
869
|
+
(0, logger_1.log)("daemon", `Version cleanup deferred: ${error instanceof Error ? error.message : String(error)}`);
|
|
870
|
+
});
|
|
871
|
+
}, 60000);
|
|
872
|
+
this.cleanupInterval.unref();
|
|
873
|
+
}
|
|
874
|
+
runVersionCleanup() {
|
|
875
|
+
if (this.cleanupPromise)
|
|
876
|
+
return this.cleanupPromise;
|
|
877
|
+
this.lastCleanupAttempt = Date.now();
|
|
878
|
+
const run = this.performVersionCleanup();
|
|
879
|
+
const tracked = run.finally(() => {
|
|
880
|
+
if (this.cleanupPromise === tracked)
|
|
881
|
+
this.cleanupPromise = null;
|
|
882
|
+
});
|
|
883
|
+
this.cleanupPromise = tracked;
|
|
884
|
+
return tracked;
|
|
885
|
+
}
|
|
886
|
+
performVersionCleanup() {
|
|
887
|
+
return __awaiter(this, void 0, void 0, function* () {
|
|
888
|
+
if (!this.vectorDb ||
|
|
889
|
+
!this.ready ||
|
|
890
|
+
this.shuttingDown ||
|
|
891
|
+
this.recycling ||
|
|
892
|
+
this.pausedReason !== null)
|
|
893
|
+
throw new Error("daemon resources not ready for version cleanup");
|
|
894
|
+
this.assertHeavyOperationAdmission("version-cleanup");
|
|
895
|
+
const deadline = AbortSignal.timeout(60000);
|
|
896
|
+
const runtime = yield (0, version_cleanup_1.prepareVersionCleanup)(config_1.PATHS.lancedbDir, deadline);
|
|
897
|
+
let roots = [];
|
|
898
|
+
try {
|
|
899
|
+
return yield this.operations.runExclusive("version-cleanup", () => __awaiter(this, void 0, void 0, function* () {
|
|
900
|
+
deadline.throwIfAborted();
|
|
901
|
+
this.assertHeavyOperationAdmission("version-cleanup");
|
|
902
|
+
roots = [
|
|
903
|
+
...new Set([
|
|
904
|
+
...this.processors.keys(),
|
|
905
|
+
...this.subscriptions.keys(),
|
|
906
|
+
]),
|
|
907
|
+
];
|
|
908
|
+
yield this.watcherManager.quiesceAll();
|
|
909
|
+
}), (signal) => __awaiter(this, void 0, void 0, function* () {
|
|
910
|
+
this.assertHeavyOperationAdmission("version-cleanup");
|
|
911
|
+
const result = yield this.vectorDb.cleanupVersions(runtime, AbortSignal.any([signal, deadline]), () => this.operations.openVersionCleanupReadWindow((name) => (0, read_verbs_1.getReadVerb)(name) !== undefined ||
|
|
912
|
+
[
|
|
913
|
+
"search",
|
|
914
|
+
"keyword-search",
|
|
915
|
+
"documents",
|
|
916
|
+
"project-stats",
|
|
917
|
+
].includes(name)));
|
|
918
|
+
if (!result)
|
|
919
|
+
return (0, compaction_result_1.skippedCompaction)("no indexed table");
|
|
920
|
+
(0, logger_1.log)("daemon", `Version cleanup completed: ${JSON.stringify(result)}`);
|
|
921
|
+
return result;
|
|
922
|
+
}), { queueShared: true });
|
|
923
|
+
}
|
|
924
|
+
finally {
|
|
925
|
+
if (!this.shuttingDown &&
|
|
926
|
+
this.ready &&
|
|
927
|
+
this.pausedReason === null &&
|
|
928
|
+
roots.length) {
|
|
929
|
+
yield this.watcherManager.resumeAll(roots, { catchup: false });
|
|
930
|
+
yield this.watcherManager.catchupAll(roots);
|
|
931
|
+
}
|
|
932
|
+
}
|
|
933
|
+
});
|
|
934
|
+
}
|
|
841
935
|
operationStatus() {
|
|
842
936
|
return this.operations.status;
|
|
843
937
|
}
|
|
@@ -2468,6 +2562,8 @@ class Daemon {
|
|
|
2468
2562
|
clearInterval(this.heartbeatInterval);
|
|
2469
2563
|
if (this.idleInterval)
|
|
2470
2564
|
clearInterval(this.idleInterval);
|
|
2565
|
+
if (this.cleanupInterval)
|
|
2566
|
+
clearInterval(this.cleanupInterval);
|
|
2471
2567
|
for (const timer of this.pendingIndexRetryTimers.values()) {
|
|
2472
2568
|
clearTimeout(timer);
|
|
2473
2569
|
}
|
|
@@ -125,8 +125,8 @@ function startHeartbeat(conn, intervalMs = HEARTBEAT_INTERVAL_MS) {
|
|
|
125
125
|
*/
|
|
126
126
|
function handleCommand(daemon, cmd, conn) {
|
|
127
127
|
return __awaiter(this, void 0, void 0, function* () {
|
|
128
|
-
var _a, _b, _c, _d;
|
|
129
|
-
var
|
|
128
|
+
var _a, _b, _c, _d, _e;
|
|
129
|
+
var _f, _g, _h, _j, _k, _l, _m, _o, _p, _q, _r;
|
|
130
130
|
try {
|
|
131
131
|
(0, logger_1.debug)("daemon", `ipc cmd=${cmd.cmd}${cmd.root ? ` root=${cmd.root}` : ""}`);
|
|
132
132
|
if (cmd.cmd !== "ping" &&
|
|
@@ -197,13 +197,14 @@ function handleCommand(daemon, cmd, conn) {
|
|
|
197
197
|
diskPressure: daemon.getDiskPressure(),
|
|
198
198
|
mlx: daemon.getMlxStatus(),
|
|
199
199
|
workers: daemon.workerCount(),
|
|
200
|
-
resources: (
|
|
201
|
-
compaction: (
|
|
200
|
+
resources: (_f = (_b = daemon.resourceSnapshot) === null || _b === void 0 ? void 0 : _b.call(daemon)) !== null && _f !== void 0 ? _f : null,
|
|
201
|
+
compaction: (_g = (_c = daemon.compactionStatus) === null || _c === void 0 ? void 0 : _c.call(daemon)) !== null && _g !== void 0 ? _g : null,
|
|
202
|
+
versionCleanup: (_h = (_d = daemon.versionCleanupStatus) === null || _d === void 0 ? void 0 : _d.call(daemon)) !== null && _h !== void 0 ? _h : null,
|
|
202
203
|
workerThreads: config_1.WORKER_THREADS_SETTING,
|
|
203
|
-
service: (
|
|
204
|
+
service: (_e = daemon.serviceStatus) === null || _e === void 0 ? void 0 : _e.call(daemon),
|
|
204
205
|
};
|
|
205
206
|
case "pause":
|
|
206
|
-
daemon.pauseWork(String((
|
|
207
|
+
daemon.pauseWork(String((_j = cmd.reason) !== null && _j !== void 0 ? _j : "read-only service requested"));
|
|
207
208
|
return { ok: true, service: daemon.serviceStatus() };
|
|
208
209
|
case "optimize":
|
|
209
210
|
// Keeps compaction single-writer: see Daemon.runOptimize.
|
|
@@ -218,14 +219,14 @@ function handleCommand(daemon, cmd, conn) {
|
|
|
218
219
|
return Object.assign({ ok: true }, (yield daemon.projectStats(root)));
|
|
219
220
|
}
|
|
220
221
|
case "shutdown": {
|
|
221
|
-
const reason = String((
|
|
222
|
-
const fromPid = (
|
|
223
|
-
const fromPpid = (
|
|
224
|
-
const fromVer = (
|
|
222
|
+
const reason = String((_k = cmd.reason) !== null && _k !== void 0 ? _k : "unknown");
|
|
223
|
+
const fromPid = (_l = cmd.from_pid) !== null && _l !== void 0 ? _l : "?";
|
|
224
|
+
const fromPpid = (_m = cmd.from_ppid) !== null && _m !== void 0 ? _m : "?";
|
|
225
|
+
const fromVer = (_o = cmd.from_version) !== null && _o !== void 0 ? _o : "?";
|
|
225
226
|
const fromArgv = Array.isArray(cmd.from_argv)
|
|
226
227
|
? cmd.from_argv.join(" ")
|
|
227
228
|
: "?";
|
|
228
|
-
const fromParentCmd = (
|
|
229
|
+
const fromParentCmd = (_p = cmd.from_parent_cmd) !== null && _p !== void 0 ? _p : "?";
|
|
229
230
|
console.log(`[daemon] shutdown command received via IPC: reason=${reason} from_pid=${fromPid} from_ppid=${fromPpid} from_version=${fromVer} from_argv=[${fromArgv}] from_parent_cmd=[${fromParentCmd}]`);
|
|
230
231
|
setImmediate(() => daemon.shutdown());
|
|
231
232
|
return { ok: true };
|
|
@@ -272,7 +273,7 @@ function handleCommand(daemon, cmd, conn) {
|
|
|
272
273
|
try {
|
|
273
274
|
maxPerFile = (0, per_file_1.validateMaxPerFile)(cmd.maxPerFile);
|
|
274
275
|
}
|
|
275
|
-
catch (
|
|
276
|
+
catch (_s) {
|
|
276
277
|
return { ok: false, error: "invalid maxPerFile" };
|
|
277
278
|
}
|
|
278
279
|
if (!Number.isInteger(limitRaw) || limitRaw < 1 || limitRaw > 1000) {
|
|
@@ -319,8 +320,8 @@ function handleCommand(daemon, cmd, conn) {
|
|
|
319
320
|
return { ok: false, error: "invalid project_roots" };
|
|
320
321
|
}
|
|
321
322
|
for (const prefix of [
|
|
322
|
-
...((
|
|
323
|
-
...((
|
|
323
|
+
...((_q = filters === null || filters === void 0 ? void 0 : filters.inPrefixes) !== null && _q !== void 0 ? _q : []),
|
|
324
|
+
...((_r = filters === null || filters === void 0 ? void 0 : filters.excludePrefixes) !== null && _r !== void 0 ? _r : []),
|
|
324
325
|
]) {
|
|
325
326
|
(0, path_containment_1.resolveContainedPath)(projectRoot, prefix);
|
|
326
327
|
}
|
|
@@ -157,6 +157,13 @@ function runCleanupProcess(executable, args, options = {}) {
|
|
|
157
157
|
var _a;
|
|
158
158
|
failure !== null && failure !== void 0 ? failure : (failure = new Error(`Cleanup process could not start (${(_a = error.code) !== null && _a !== void 0 ? _a : "unknown"})`));
|
|
159
159
|
});
|
|
160
|
+
child.once("exit", () => {
|
|
161
|
+
// An exited helper no longer consumes host resources. Do not turn a
|
|
162
|
+
// verified result into an uncertain receipt while stdout is draining.
|
|
163
|
+
clearTimeout(timeout);
|
|
164
|
+
if (monitor)
|
|
165
|
+
clearInterval(monitor);
|
|
166
|
+
});
|
|
160
167
|
child.once("close", (code, signal) => {
|
|
161
168
|
var _a, _b;
|
|
162
169
|
var _c;
|
|
@@ -221,7 +228,7 @@ function runCleanupProcess(executable, args, options = {}) {
|
|
|
221
228
|
});
|
|
222
229
|
}
|
|
223
230
|
let preparation;
|
|
224
|
-
/**
|
|
231
|
+
/** Pinned runtime shared by explicit recovery and prune-only maintenance. */
|
|
225
232
|
function prepareCleanupRuntime(signal) {
|
|
226
233
|
return __awaiter(this, void 0, void 0, function* () {
|
|
227
234
|
if (signal === null || signal === void 0 ? void 0 : signal.aborted)
|
|
@@ -304,7 +311,7 @@ function prepareCleanupRuntime(signal) {
|
|
|
304
311
|
return preparation;
|
|
305
312
|
});
|
|
306
313
|
}
|
|
307
|
-
/**
|
|
314
|
+
/** Caller must have drained local work/readers before
|
|
308
315
|
* acquiring the exclusive lease. The lease cannot be released during this call. */
|
|
309
316
|
function pruneVersions(runtime, store, version, cutoff, options) {
|
|
310
317
|
return __awaiter(this, void 0, void 0, function* () {
|
|
@@ -317,6 +324,11 @@ function pruneVersions(runtime, store, version, cutoff, options) {
|
|
|
317
324
|
cutoff.getTime() > Date.now()) {
|
|
318
325
|
throw new Error("Invalid prune version or future cutoff");
|
|
319
326
|
}
|
|
327
|
+
if (options.maxVersions !== undefined &&
|
|
328
|
+
(!Number.isSafeInteger(options.maxVersions) ||
|
|
329
|
+
options.maxVersions < 1 ||
|
|
330
|
+
options.maxVersions > 128))
|
|
331
|
+
throw new Error("Invalid bounded prune version count");
|
|
320
332
|
const table = fs.realpathSync(store);
|
|
321
333
|
if (!table.endsWith(".lance") || !fs.statSync(table).isDirectory())
|
|
322
334
|
throw new Error("Pruning requires a local Lance table");
|
|
@@ -330,7 +342,8 @@ function pruneVersions(runtime, store, version, cutoff, options) {
|
|
|
330
342
|
var _a;
|
|
331
343
|
const previous = (0, prune_state_1.readPruneState)(storeDir);
|
|
332
344
|
if ((previous === null || previous === void 0 ? void 0 : previous.outcome) === "uncertain" &&
|
|
333
|
-
options.acknowledgeUncertain !== previous.attemptId
|
|
345
|
+
options.acknowledgeUncertain !== previous.attemptId &&
|
|
346
|
+
!options.retryUncertain)
|
|
334
347
|
throw new Error(`Previous prune completion is uncertain; explicit acknowledgement required for attempt ${previous.attemptId}`);
|
|
335
348
|
admission = (_a = options.admission) !== null && _a !== void 0 ? _a : (0, recovery_admission_1.admitPrune)(storeDir);
|
|
336
349
|
attempt = {
|
|
@@ -361,6 +374,9 @@ function pruneVersions(runtime, store, version, cutoff, options) {
|
|
|
361
374
|
"--lease-nonce",
|
|
362
375
|
options.lease.owner.nonce,
|
|
363
376
|
"--require-admission",
|
|
377
|
+
...(options.maxVersions === undefined
|
|
378
|
+
? []
|
|
379
|
+
: ["--max-versions", String(options.maxVersions)]),
|
|
364
380
|
], {
|
|
365
381
|
signal: options.signal,
|
|
366
382
|
timeoutMs: MAX_PRUNE_MS,
|
|
@@ -387,6 +403,10 @@ function pruneVersions(runtime, store, version, cutoff, options) {
|
|
|
387
403
|
if (result.engine !== "12.0.0" ||
|
|
388
404
|
result.version !== version ||
|
|
389
405
|
result.rewritten !== false ||
|
|
406
|
+
(options.maxVersions !== undefined &&
|
|
407
|
+
(!Number.isSafeInteger(result.eligibleVersionsRemaining) ||
|
|
408
|
+
result.eligibleVersionsRemaining < 0 ||
|
|
409
|
+
result.versionsRemoved > options.maxVersions)) ||
|
|
390
410
|
![
|
|
391
411
|
"fragments",
|
|
392
412
|
"bytesRemoved",
|
|
@@ -41,13 +41,14 @@ exports.storeMutationDeniedReason = storeMutationDeniedReason;
|
|
|
41
41
|
exports.assertStoreMutationAllowed = assertStoreMutationAllowed;
|
|
42
42
|
exports.maintenancePolicyPath = maintenancePolicyPath;
|
|
43
43
|
exports.recordMaintenanceContainment = recordMaintenanceContainment;
|
|
44
|
+
exports.recordVersionCleanupCompleted = recordVersionCleanupCompleted;
|
|
44
45
|
const node_crypto_1 = require("node:crypto");
|
|
45
46
|
const fs = __importStar(require("node:fs"));
|
|
46
47
|
const path = __importStar(require("node:path"));
|
|
47
48
|
const config_1 = require("../../config");
|
|
48
49
|
const autostart_1 = require("../utils/autostart");
|
|
49
50
|
const safety_latch_1 = require("../utils/safety-latch");
|
|
50
|
-
exports.FULL_TABLE_MAINTENANCE_DISABLED_REASON = "full-table maintenance disabled by host-safety containment;
|
|
51
|
+
exports.FULL_TABLE_MAINTENANCE_DISABLED_REASON = "full-table maintenance disabled by host-safety containment; version cleanup runs independently";
|
|
51
52
|
/** This release has no production override: force, configuration and persisted
|
|
52
53
|
* state cannot enable rewriting. Tests may mock this module to cover the old
|
|
53
54
|
* native algorithm without turning it into a supported production entry point. */
|
|
@@ -122,7 +123,8 @@ function recordMaintenanceContainment(storeDir) {
|
|
|
122
123
|
previous.storeIdentity === storeIdentity &&
|
|
123
124
|
previous.mode === "disabled" &&
|
|
124
125
|
previous.rewriteBudgetBytes === 0 &&
|
|
125
|
-
previous.cleanupPending ===
|
|
126
|
+
typeof previous.cleanupPending === "boolean" &&
|
|
127
|
+
previous.reason === exports.FULL_TABLE_MAINTENANCE_DISABLED_REASON) {
|
|
126
128
|
fs.chmodSync(target, 0o600);
|
|
127
129
|
return exports.FULL_TABLE_MAINTENANCE_DISABLED_REASON;
|
|
128
130
|
}
|
|
@@ -170,3 +172,41 @@ function recordMaintenanceContainment(storeDir) {
|
|
|
170
172
|
}
|
|
171
173
|
}
|
|
172
174
|
}
|
|
175
|
+
/** A completed prune changes retention state, never enables table rewrites. */
|
|
176
|
+
function recordVersionCleanupCompleted(storeDir, at) {
|
|
177
|
+
recordMaintenanceContainment(storeDir);
|
|
178
|
+
const identity = fs.realpathSync(storeDir);
|
|
179
|
+
const target = maintenancePolicyPath(identity);
|
|
180
|
+
const stat = fs.lstatSync(target);
|
|
181
|
+
if (!stat.isFile() || stat.size > 16 * 1024)
|
|
182
|
+
throw new Error("maintenance receipt is unverified");
|
|
183
|
+
const policy = JSON.parse(fs.readFileSync(target, "utf8"));
|
|
184
|
+
if (policy.storeIdentity !== identity ||
|
|
185
|
+
policy.mode !== "disabled" ||
|
|
186
|
+
policy.rewriteBudgetBytes !== 0 ||
|
|
187
|
+
!Number.isSafeInteger(at))
|
|
188
|
+
throw new Error("maintenance receipt is invalid");
|
|
189
|
+
const temporary = path.join(identity, `.gmax-maintenance-${(0, node_crypto_1.randomUUID)()}.tmp`);
|
|
190
|
+
try {
|
|
191
|
+
const descriptor = fs.openSync(temporary, "wx", 0o600);
|
|
192
|
+
try {
|
|
193
|
+
fs.writeFileSync(descriptor, `${JSON.stringify(Object.assign(Object.assign({}, policy), { cleanupPending: false, lastVersionCleanupAt: at }))}\n`);
|
|
194
|
+
fs.fsyncSync(descriptor);
|
|
195
|
+
}
|
|
196
|
+
finally {
|
|
197
|
+
fs.closeSync(descriptor);
|
|
198
|
+
}
|
|
199
|
+
fs.renameSync(temporary, target);
|
|
200
|
+
const directory = fs.openSync(identity, "r");
|
|
201
|
+
try {
|
|
202
|
+
fs.fsyncSync(directory);
|
|
203
|
+
}
|
|
204
|
+
finally {
|
|
205
|
+
fs.closeSync(directory);
|
|
206
|
+
}
|
|
207
|
+
}
|
|
208
|
+
finally {
|
|
209
|
+
if (fs.existsSync(temporary))
|
|
210
|
+
fs.unlinkSync(temporary);
|
|
211
|
+
}
|
|
212
|
+
}
|
|
@@ -60,6 +60,9 @@ function createRecoveryBudget() {
|
|
|
60
60
|
kernelProbeTimeoutMs: 5000,
|
|
61
61
|
commandProbeTimeoutMs: 3000,
|
|
62
62
|
footprintProbeTimeoutMs: 3000,
|
|
63
|
+
// Re-measure a changing cohort while retaining warning severity. The
|
|
64
|
+
// fresh policy check above still decides whether that warning is allowed.
|
|
65
|
+
retryWarningPressure: true,
|
|
63
66
|
},
|
|
64
67
|
sampleMaxAgeMs: 10000,
|
|
65
68
|
});
|
|
@@ -69,6 +69,7 @@ const compaction_result_1 = require("./compaction-result");
|
|
|
69
69
|
const lancedb = __importStar(require("./lance-sdk"));
|
|
70
70
|
const maintenance_policy_1 = require("./maintenance-policy");
|
|
71
71
|
const store_lease_1 = require("./store-lease");
|
|
72
|
+
const version_cleanup_1 = require("./version-cleanup");
|
|
72
73
|
/**
|
|
73
74
|
* Find abandoned Lance scratch files under `<lancedbDir>/ *.lance/data`.
|
|
74
75
|
*
|
|
@@ -239,6 +240,9 @@ class VectorDB {
|
|
|
239
240
|
this.indexOwner = false;
|
|
240
241
|
this.lastOptimizeDidWork = false;
|
|
241
242
|
this.lastCompactionResult = null;
|
|
243
|
+
this.lastVersionCleanupResult = null;
|
|
244
|
+
this.versionCleanupActive = false;
|
|
245
|
+
this.versionCleanupPromise = null;
|
|
242
246
|
/** Compaction rate limiter — see COMPACTION_MIN_INTERVAL_MS. */
|
|
243
247
|
this.lastCompactionMs = 0;
|
|
244
248
|
this.compactionIntervalMs = COMPACTION_MIN_INTERVAL_MS;
|
|
@@ -354,7 +358,7 @@ class VectorDB {
|
|
|
354
358
|
}
|
|
355
359
|
}
|
|
356
360
|
isMaintenanceActive() {
|
|
357
|
-
return this.maintenancePromise !== null;
|
|
361
|
+
return this.maintenancePromise !== null || this.versionCleanupActive;
|
|
358
362
|
}
|
|
359
363
|
maintenanceDue() {
|
|
360
364
|
return (this.writeEpoch !== this.maintainedEpoch ||
|
|
@@ -375,6 +379,65 @@ class VectorDB {
|
|
|
375
379
|
resumeMaintenanceLoop() {
|
|
376
380
|
this.startMaintenanceLoop();
|
|
377
381
|
}
|
|
382
|
+
/** The daemon has drained operations and quiesced watchers before calling.
|
|
383
|
+
* Close native handles before the helper owns the store; keep this instance
|
|
384
|
+
* and worker/searcher references available after shared ownership resumes. */
|
|
385
|
+
versionCleanupStatus() {
|
|
386
|
+
return this.lastVersionCleanupResult
|
|
387
|
+
? Object.assign({}, this.lastVersionCleanupResult) : null;
|
|
388
|
+
}
|
|
389
|
+
cleanupVersions(runtime, signal, allowCurrentReads) {
|
|
390
|
+
if (this.versionCleanupPromise)
|
|
391
|
+
return this.versionCleanupPromise;
|
|
392
|
+
const run = this.performVersionCleanup(runtime, signal, allowCurrentReads);
|
|
393
|
+
const tracked = run.finally(() => {
|
|
394
|
+
if (this.versionCleanupPromise === tracked)
|
|
395
|
+
this.versionCleanupPromise = null;
|
|
396
|
+
});
|
|
397
|
+
this.versionCleanupPromise = tracked;
|
|
398
|
+
return tracked;
|
|
399
|
+
}
|
|
400
|
+
performVersionCleanup(runtime, signal, allowCurrentReads) {
|
|
401
|
+
return __awaiter(this, void 0, void 0, function* () {
|
|
402
|
+
var _a, _b;
|
|
403
|
+
this.assertMutationAllowed();
|
|
404
|
+
this.versionCleanupActive = true;
|
|
405
|
+
try {
|
|
406
|
+
const lease = yield this.upgradeStoreLease(signal, 750);
|
|
407
|
+
yield Promise.all([this.drainWrites(), this.drainCompactions()]);
|
|
408
|
+
const table = yield this.openExistingTableUnsafe();
|
|
409
|
+
if (!table)
|
|
410
|
+
return null;
|
|
411
|
+
let version;
|
|
412
|
+
try {
|
|
413
|
+
version = yield table.version();
|
|
414
|
+
}
|
|
415
|
+
finally {
|
|
416
|
+
table.close();
|
|
417
|
+
}
|
|
418
|
+
(_a = this.db) === null || _a === void 0 ? void 0 : _a.close();
|
|
419
|
+
this.db = null;
|
|
420
|
+
this.session = null;
|
|
421
|
+
(_b = this.resourceReservation) === null || _b === void 0 ? void 0 : _b.release();
|
|
422
|
+
this.resourceReservation = null;
|
|
423
|
+
// All older readers are drained. New queries can only open the protected
|
|
424
|
+
// latest version while writers stay excluded for this prune-only pass.
|
|
425
|
+
allowCurrentReads === null || allowCurrentReads === void 0 ? void 0 : allowCurrentReads();
|
|
426
|
+
const result = yield (0, version_cleanup_1.runVersionCleanup)(this.lancedbDir, lease, version, runtime, signal);
|
|
427
|
+
this.lastVersionCleanupResult = result;
|
|
428
|
+
return result;
|
|
429
|
+
}
|
|
430
|
+
finally {
|
|
431
|
+
try {
|
|
432
|
+
if (!this.closed)
|
|
433
|
+
yield this.downgradeStoreLease();
|
|
434
|
+
}
|
|
435
|
+
finally {
|
|
436
|
+
this.versionCleanupActive = false;
|
|
437
|
+
}
|
|
438
|
+
}
|
|
439
|
+
});
|
|
440
|
+
}
|
|
378
441
|
getDb() {
|
|
379
442
|
return __awaiter(this, void 0, void 0, function* () {
|
|
380
443
|
var _a;
|
|
@@ -461,7 +524,7 @@ class VectorDB {
|
|
|
461
524
|
});
|
|
462
525
|
}
|
|
463
526
|
/** Upgrade and retain this instance's current lease for transfer to a new DB. */
|
|
464
|
-
upgradeStoreLease(signal) {
|
|
527
|
+
upgradeStoreLease(signal, timeoutMs) {
|
|
465
528
|
return __awaiter(this, void 0, void 0, function* () {
|
|
466
529
|
this.assertMutationAllowed();
|
|
467
530
|
if (this.closed)
|
|
@@ -472,7 +535,7 @@ class VectorDB {
|
|
|
472
535
|
const current = yield this.getLease();
|
|
473
536
|
if (current.mode === "exclusive")
|
|
474
537
|
return current;
|
|
475
|
-
const upgrade = current.upgrade({ signal });
|
|
538
|
+
const upgrade = current.upgrade({ signal, timeoutMs });
|
|
476
539
|
this.leasePromise = upgrade;
|
|
477
540
|
try {
|
|
478
541
|
const upgraded = yield upgrade;
|
|
@@ -560,6 +623,8 @@ class VectorDB {
|
|
|
560
623
|
*/
|
|
561
624
|
withWriteGate(fn) {
|
|
562
625
|
return __awaiter(this, void 0, void 0, function* () {
|
|
626
|
+
if (this.versionCleanupActive)
|
|
627
|
+
throw new Error("writes paused for version cleanup");
|
|
563
628
|
this.assertMutationAllowed();
|
|
564
629
|
this.ensureDiskOk();
|
|
565
630
|
while (this.exclusiveMutationPromise || this.compactingPromise) {
|
|
@@ -807,7 +872,8 @@ class VectorDB {
|
|
|
807
872
|
}
|
|
808
873
|
ensureTable() {
|
|
809
874
|
return __awaiter(this, void 0, void 0, function* () {
|
|
810
|
-
if (this.
|
|
875
|
+
if (this.versionCleanupActive ||
|
|
876
|
+
this.options.readOnly ||
|
|
811
877
|
(0, maintenance_policy_1.storeMutationDeniedReason)() !== null ||
|
|
812
878
|
this.checkDiskPressure(true) === "critical") {
|
|
813
879
|
const existing = yield this.openExistingTableUnsafe();
|
|
@@ -1913,8 +1979,9 @@ class VectorDB {
|
|
|
1913
1979
|
}
|
|
1914
1980
|
finishClose() {
|
|
1915
1981
|
return __awaiter(this, void 0, void 0, function* () {
|
|
1916
|
-
var _a, _b, _c;
|
|
1917
|
-
yield (_a = this.
|
|
1982
|
+
var _a, _b, _c, _d;
|
|
1983
|
+
yield (_a = this.versionCleanupPromise) === null || _a === void 0 ? void 0 : _a.catch(() => { });
|
|
1984
|
+
yield (_b = this.connecting) === null || _b === void 0 ? void 0 : _b.catch(() => { });
|
|
1918
1985
|
if (this.maintenanceTimer) {
|
|
1919
1986
|
clearInterval(this.maintenanceTimer);
|
|
1920
1987
|
this.maintenanceTimer = null;
|
|
@@ -1933,11 +2000,11 @@ class VectorDB {
|
|
|
1933
2000
|
]);
|
|
1934
2001
|
this.maintenancePromise = null;
|
|
1935
2002
|
}
|
|
1936
|
-
(
|
|
2003
|
+
(_c = this.unregisterCleanup) === null || _c === void 0 ? void 0 : _c.call(this);
|
|
1937
2004
|
this.unregisterCleanup = undefined;
|
|
1938
2005
|
let closeError;
|
|
1939
2006
|
let nativeClosed = !this.db;
|
|
1940
|
-
if ((
|
|
2007
|
+
if ((_d = this.db) === null || _d === void 0 ? void 0 : _d.close) {
|
|
1941
2008
|
try {
|
|
1942
2009
|
const closing = this.db.close();
|
|
1943
2010
|
void Promise.resolve(closing).then(() => {
|
|
@@ -1978,7 +2045,7 @@ class VectorDB {
|
|
|
1978
2045
|
try {
|
|
1979
2046
|
lease = yield this.leasePromise;
|
|
1980
2047
|
}
|
|
1981
|
-
catch (
|
|
2048
|
+
catch (_e) {
|
|
1982
2049
|
this.leasePromise = null;
|
|
1983
2050
|
}
|
|
1984
2051
|
if (lease) {
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
|
3
|
+
if (k2 === undefined) k2 = k;
|
|
4
|
+
var desc = Object.getOwnPropertyDescriptor(m, k);
|
|
5
|
+
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
|
6
|
+
desc = { enumerable: true, get: function() { return m[k]; } };
|
|
7
|
+
}
|
|
8
|
+
Object.defineProperty(o, k2, desc);
|
|
9
|
+
}) : (function(o, m, k, k2) {
|
|
10
|
+
if (k2 === undefined) k2 = k;
|
|
11
|
+
o[k2] = m[k];
|
|
12
|
+
}));
|
|
13
|
+
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
|
14
|
+
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
|
15
|
+
}) : function(o, v) {
|
|
16
|
+
o["default"] = v;
|
|
17
|
+
});
|
|
18
|
+
var __importStar = (this && this.__importStar) || (function () {
|
|
19
|
+
var ownKeys = function(o) {
|
|
20
|
+
ownKeys = Object.getOwnPropertyNames || function (o) {
|
|
21
|
+
var ar = [];
|
|
22
|
+
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
|
23
|
+
return ar;
|
|
24
|
+
};
|
|
25
|
+
return ownKeys(o);
|
|
26
|
+
};
|
|
27
|
+
return function (mod) {
|
|
28
|
+
if (mod && mod.__esModule) return mod;
|
|
29
|
+
var result = {};
|
|
30
|
+
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
|
31
|
+
__setModuleDefault(result, mod);
|
|
32
|
+
return result;
|
|
33
|
+
};
|
|
34
|
+
})();
|
|
35
|
+
var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
|
|
36
|
+
function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
|
|
37
|
+
return new (P || (P = Promise))(function (resolve, reject) {
|
|
38
|
+
function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
|
|
39
|
+
function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
|
|
40
|
+
function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
|
|
41
|
+
step((generator = generator.apply(thisArg, _arguments || [])).next());
|
|
42
|
+
});
|
|
43
|
+
};
|
|
44
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
45
|
+
exports.MAX_CLEANUP_VERSIONS = exports.VERSION_RETENTION_MS = void 0;
|
|
46
|
+
exports.prepareVersionCleanup = prepareVersionCleanup;
|
|
47
|
+
exports.runVersionCleanup = runVersionCleanup;
|
|
48
|
+
const path = __importStar(require("node:path"));
|
|
49
|
+
const resource_budget_1 = require("../utils/resource-budget");
|
|
50
|
+
const lance_cleanup_1 = require("./lance-cleanup");
|
|
51
|
+
const maintenance_policy_1 = require("./maintenance-policy");
|
|
52
|
+
exports.VERSION_RETENTION_MS = 2 * 60000;
|
|
53
|
+
exports.MAX_CLEANUP_VERSIONS = 128;
|
|
54
|
+
function check(storeDir, signal) {
|
|
55
|
+
signal === null || signal === void 0 ? void 0 : signal.throwIfAborted();
|
|
56
|
+
(0, maintenance_policy_1.assertStoreMutationAllowed)();
|
|
57
|
+
const free = (0, maintenance_policy_1.availableStoreDiskBytes)(storeDir);
|
|
58
|
+
// Reclamation needs metadata space, never room for a second copy of the table.
|
|
59
|
+
if (!Number.isFinite(free) || free < Math.pow(1024, 2))
|
|
60
|
+
throw new Error("version cleanup metadata disk headroom unavailable");
|
|
61
|
+
resource_budget_1.resourceBudget.check();
|
|
62
|
+
}
|
|
63
|
+
/** Prepare before pausing service. Runtime downloads never hold store ownership. */
|
|
64
|
+
function prepareVersionCleanup(storeDir, signal) {
|
|
65
|
+
return __awaiter(this, void 0, void 0, function* () {
|
|
66
|
+
check(storeDir, signal);
|
|
67
|
+
const reservation = resource_budget_1.resourceBudget.reserve(512, "cleanup-runtime");
|
|
68
|
+
try {
|
|
69
|
+
return yield (0, lance_cleanup_1.prepareCleanupRuntime)(signal);
|
|
70
|
+
}
|
|
71
|
+
finally {
|
|
72
|
+
reservation.release();
|
|
73
|
+
}
|
|
74
|
+
});
|
|
75
|
+
}
|
|
76
|
+
/** Independent retention: no optimizer, compaction plan or rewrite admission. */
|
|
77
|
+
function runVersionCleanup(storeDir, lease, version, runtime, signal) {
|
|
78
|
+
return __awaiter(this, void 0, void 0, function* () {
|
|
79
|
+
var _a;
|
|
80
|
+
const started = Date.now();
|
|
81
|
+
check(storeDir, signal);
|
|
82
|
+
const reservation = resource_budget_1.resourceBudget.reserve(512, "version-cleanup");
|
|
83
|
+
try {
|
|
84
|
+
const result = yield (0, lance_cleanup_1.pruneVersions)(runtime, path.join(storeDir, "chunks.lance"), version, new Date(Date.now() - exports.VERSION_RETENTION_MS), {
|
|
85
|
+
lease,
|
|
86
|
+
signal,
|
|
87
|
+
maxVersions: exports.MAX_CLEANUP_VERSIONS,
|
|
88
|
+
retryUncertain: true,
|
|
89
|
+
admission: {
|
|
90
|
+
start: (pid) => reservation.attach(pid),
|
|
91
|
+
approve: () => check(storeDir, signal),
|
|
92
|
+
check: () => check(storeDir, signal),
|
|
93
|
+
close: () => reservation.release(),
|
|
94
|
+
},
|
|
95
|
+
});
|
|
96
|
+
const outcome = {
|
|
97
|
+
status: "completed",
|
|
98
|
+
reason: "old versions reclaimed without rewriting data",
|
|
99
|
+
at: Date.now(),
|
|
100
|
+
attempts: 1,
|
|
101
|
+
elapsedMs: Date.now() - started,
|
|
102
|
+
rewritten: false,
|
|
103
|
+
versionsRemoved: result.versionsRemoved,
|
|
104
|
+
eligibleVersionsRemaining: (_a = result.eligibleVersionsRemaining) !== null && _a !== void 0 ? _a : 0,
|
|
105
|
+
diskBytesBefore: result.allocatedBytesBefore,
|
|
106
|
+
diskBytesAfter: result.allocatedBytesAfter,
|
|
107
|
+
freeBytesBefore: result.freeBytesBefore,
|
|
108
|
+
freeBytesAfter: result.freeBytesAfter,
|
|
109
|
+
bytesReclaimed: result.bytesRemoved,
|
|
110
|
+
netBytesReclaimed: result.allocatedBytesBefore - result.allocatedBytesAfter,
|
|
111
|
+
cleanupPasses: 1,
|
|
112
|
+
};
|
|
113
|
+
(0, maintenance_policy_1.recordVersionCleanupCompleted)(storeDir, outcome.at);
|
|
114
|
+
return outcome;
|
|
115
|
+
}
|
|
116
|
+
finally {
|
|
117
|
+
reservation.release();
|
|
118
|
+
}
|
|
119
|
+
});
|
|
120
|
+
}
|
|
@@ -134,13 +134,20 @@ function sampleHostResources(extraPids = [], overrides = {}, extraGroups = []) {
|
|
|
134
134
|
const monotonic = (_a = overrides.monotonic) !== null && _a !== void 0 ? _a : (() => performance.now());
|
|
135
135
|
const deadline = monotonic() + ((_b = overrides.sampleTimeoutMs) !== null && _b !== void 0 ? _b : exports.HOST_SAMPLE_TIMEOUT_MS);
|
|
136
136
|
let snapshot;
|
|
137
|
+
let observedWarning = false;
|
|
137
138
|
for (let attempt = 0; attempt < 2; attempt++) {
|
|
138
139
|
snapshot = sampleHostResourcesOnce(extraPids, Object.assign(Object.assign({}, overrides), { monotonic, sampleTimeoutMs: Math.max(0, deadline - monotonic()) }), extraGroups);
|
|
140
|
+
observedWarning || (observedWarning = snapshot.memoryPressure === "warn");
|
|
141
|
+
// A fresh complete cohort may retain an observed warning; retrying must
|
|
142
|
+
// never downgrade it to normal or weaken the caller's final policy check.
|
|
143
|
+
if (observedWarning && snapshot.memoryPressure === "normal")
|
|
144
|
+
snapshot.memoryPressure = "warn";
|
|
139
145
|
// Short-lived client children routinely exit during footprint sampling.
|
|
140
146
|
// Re-measure the entire cohort once, within the original deadline. Never
|
|
141
|
-
// accept a partial total or
|
|
147
|
+
// accept a partial total or downgrade observed warning/critical pressure.
|
|
142
148
|
if (attempt === 1 ||
|
|
143
|
-
snapshot.memoryPressure !== "normal"
|
|
149
|
+
(snapshot.memoryPressure !== "normal" &&
|
|
150
|
+
!(overrides.retryWarningPressure && snapshot.memoryPressure === "warn")) ||
|
|
144
151
|
snapshot.kernelPressure !== "ok" ||
|
|
145
152
|
snapshot.incompleteReasons.length !== 1 ||
|
|
146
153
|
snapshot.incompleteReasons[0] !==
|
|
@@ -163,7 +170,7 @@ function sampleHostResourcesOnce(extraPids, overrides, extraGroups) {
|
|
|
163
170
|
return false;
|
|
164
171
|
throw new Error("process liveness unavailable");
|
|
165
172
|
}
|
|
166
|
-
}, sampleTimeoutMs: exports.HOST_SAMPLE_TIMEOUT_MS, kernelProbeTimeoutMs: 500, commandProbeTimeoutMs: 500, footprintProbeTimeoutMs: 1500, run: (command, args, timeoutMs) => (0, node_child_process_1.execFileSync)(command, args, {
|
|
173
|
+
}, sampleTimeoutMs: exports.HOST_SAMPLE_TIMEOUT_MS, kernelProbeTimeoutMs: 500, commandProbeTimeoutMs: 500, footprintProbeTimeoutMs: 1500, retryWarningPressure: false, run: (command, args, timeoutMs) => (0, node_child_process_1.execFileSync)(command, args, {
|
|
167
174
|
encoding: "utf8",
|
|
168
175
|
timeout: timeoutMs,
|
|
169
176
|
maxBuffer: 512 * 1024,
|
|
@@ -41,6 +41,8 @@ class OperationCoordinator {
|
|
|
41
41
|
this.sharedTasks = new Set();
|
|
42
42
|
this.taskNames = new Map();
|
|
43
43
|
this.closePromise = null;
|
|
44
|
+
this.queueShared = false;
|
|
45
|
+
this.waiters = new Set();
|
|
44
46
|
}
|
|
45
47
|
get status() {
|
|
46
48
|
return this.state.kind;
|
|
@@ -53,8 +55,40 @@ class OperationCoordinator {
|
|
|
53
55
|
return [...this.activeTasks].map((task) => { var _a; return (_a = this.taskNames.get(task)) !== null && _a !== void 0 ? _a : "?"; });
|
|
54
56
|
}
|
|
55
57
|
runShared(name, signal, fn) {
|
|
58
|
+
var _a;
|
|
59
|
+
if (this.queueShared &&
|
|
60
|
+
(this.state.kind === "exclusive" ||
|
|
61
|
+
this.state.kind === "exclusive-pending") &&
|
|
62
|
+
!((_a = this.readWindow) === null || _a === void 0 ? void 0 : _a.call(this, name)) &&
|
|
63
|
+
!name.startsWith("watch") &&
|
|
64
|
+
name !== "store-maintenance") {
|
|
65
|
+
if (this.waiters.size >= 64)
|
|
66
|
+
return Promise.reject(new OperationBusyError(this.state.name));
|
|
67
|
+
if (signal === null || signal === void 0 ? void 0 : signal.aborted)
|
|
68
|
+
return Promise.reject(abortError(signal.reason));
|
|
69
|
+
return new Promise((resolve, reject) => {
|
|
70
|
+
const cleanup = () => {
|
|
71
|
+
this.waiters.delete(waiter);
|
|
72
|
+
signal === null || signal === void 0 ? void 0 : signal.removeEventListener("abort", abort);
|
|
73
|
+
};
|
|
74
|
+
const waiter = {
|
|
75
|
+
name,
|
|
76
|
+
resume: () => {
|
|
77
|
+
cleanup();
|
|
78
|
+
resolve(this.runShared(name, signal, fn));
|
|
79
|
+
},
|
|
80
|
+
reject: (error) => {
|
|
81
|
+
cleanup();
|
|
82
|
+
reject(error);
|
|
83
|
+
},
|
|
84
|
+
};
|
|
85
|
+
const abort = () => waiter.reject(abortError(signal === null || signal === void 0 ? void 0 : signal.reason));
|
|
86
|
+
this.waiters.add(waiter);
|
|
87
|
+
signal === null || signal === void 0 ? void 0 : signal.addEventListener("abort", abort, { once: true });
|
|
88
|
+
});
|
|
89
|
+
}
|
|
56
90
|
try {
|
|
57
|
-
this.assertSharedAdmission();
|
|
91
|
+
this.assertSharedAdmission(name);
|
|
58
92
|
}
|
|
59
93
|
catch (error) {
|
|
60
94
|
return Promise.reject(error);
|
|
@@ -81,7 +115,8 @@ class OperationCoordinator {
|
|
|
81
115
|
void task.then(cleanup, cleanup);
|
|
82
116
|
return task;
|
|
83
117
|
}
|
|
84
|
-
runExclusive(name, quiesce, fn) {
|
|
118
|
+
runExclusive(name, quiesce, fn, options = {}) {
|
|
119
|
+
var _a;
|
|
85
120
|
if (this.state.kind === "closing" || this.state.kind === "closed") {
|
|
86
121
|
return Promise.reject(new OperationClosedError());
|
|
87
122
|
}
|
|
@@ -90,6 +125,7 @@ class OperationCoordinator {
|
|
|
90
125
|
return Promise.reject(new OperationBusyError(current));
|
|
91
126
|
}
|
|
92
127
|
this.state = { kind: "exclusive-pending", name };
|
|
128
|
+
this.queueShared = (_a = options.queueShared) !== null && _a !== void 0 ? _a : false;
|
|
93
129
|
const controller = new AbortController();
|
|
94
130
|
this.controllers.add(controller);
|
|
95
131
|
const task = (() => __awaiter(this, void 0, void 0, function* () {
|
|
@@ -111,6 +147,10 @@ class OperationCoordinator {
|
|
|
111
147
|
this.state.kind === "exclusive-pending") {
|
|
112
148
|
this.state = { kind: "open" };
|
|
113
149
|
}
|
|
150
|
+
this.queueShared = false;
|
|
151
|
+
this.readWindow = undefined;
|
|
152
|
+
for (const waiter of [...this.waiters])
|
|
153
|
+
waiter.resume();
|
|
114
154
|
}
|
|
115
155
|
}))();
|
|
116
156
|
this.activeTasks.add(task);
|
|
@@ -122,6 +162,18 @@ class OperationCoordinator {
|
|
|
122
162
|
void task.then(cleanup, cleanup);
|
|
123
163
|
return task;
|
|
124
164
|
}
|
|
165
|
+
/** Only prune-only retention may reopen current-version reads after draining
|
|
166
|
+
* old readers and closing their native handles. Writers remain excluded. */
|
|
167
|
+
openVersionCleanupReadWindow(allowed) {
|
|
168
|
+
if (this.state.kind !== "exclusive" ||
|
|
169
|
+
this.state.name !== "version-cleanup" ||
|
|
170
|
+
!this.queueShared)
|
|
171
|
+
throw new Error("current-version reads require exclusive prune-only cleanup");
|
|
172
|
+
this.readWindow = allowed;
|
|
173
|
+
for (const waiter of [...this.waiters])
|
|
174
|
+
if (allowed(waiter.name))
|
|
175
|
+
waiter.resume();
|
|
176
|
+
}
|
|
125
177
|
/** Cancel existing work without closing admission for subsequent bounded reads. */
|
|
126
178
|
abortAndDrain(reason) {
|
|
127
179
|
for (const controller of this.controllers)
|
|
@@ -132,6 +184,8 @@ class OperationCoordinator {
|
|
|
132
184
|
if (this.closePromise)
|
|
133
185
|
return this.closePromise;
|
|
134
186
|
this.state = { kind: "closing" };
|
|
187
|
+
for (const waiter of [...this.waiters])
|
|
188
|
+
waiter.reject(reason);
|
|
135
189
|
for (const controller of this.controllers)
|
|
136
190
|
controller.abort(reason);
|
|
137
191
|
const tasks = [...this.activeTasks];
|
|
@@ -140,12 +194,15 @@ class OperationCoordinator {
|
|
|
140
194
|
});
|
|
141
195
|
return this.closePromise;
|
|
142
196
|
}
|
|
143
|
-
assertSharedAdmission() {
|
|
197
|
+
assertSharedAdmission(name) {
|
|
198
|
+
var _a;
|
|
144
199
|
if (this.state.kind === "closing" || this.state.kind === "closed") {
|
|
145
200
|
throw new OperationClosedError();
|
|
146
201
|
}
|
|
147
202
|
if (this.state.kind === "exclusive" ||
|
|
148
203
|
this.state.kind === "exclusive-pending") {
|
|
204
|
+
if (this.state.kind === "exclusive" && ((_a = this.readWindow) === null || _a === void 0 ? void 0 : _a.call(this, name)))
|
|
205
|
+
return;
|
|
149
206
|
throw new OperationBusyError(this.state.name);
|
|
150
207
|
}
|
|
151
208
|
}
|
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
"""
|
|
1
|
+
"""Exclusively leased Lance 12 prune-only cleanup.
|
|
2
2
|
|
|
3
3
|
Never compacts, writes rows, builds indexes or starts a server. This helper is
|
|
4
|
-
|
|
5
|
-
|
|
4
|
+
used by recovery and bounded automatic retention. Native host admission and
|
|
5
|
+
exclusive ownership are required before deleting files.
|
|
6
6
|
"""
|
|
7
7
|
|
|
8
8
|
import argparse
|
|
@@ -107,7 +107,7 @@ def open_version(lance, root, version):
|
|
|
107
107
|
metadata_cache_size_bytes=CACHE_BYTES)
|
|
108
108
|
|
|
109
109
|
|
|
110
|
-
def prune(store, expected_version, cutoff_ms, owner_file, nonce, admission=None):
|
|
110
|
+
def prune(store, expected_version, cutoff_ms, owner_file, nonce, admission=None, max_versions=None):
|
|
111
111
|
root = Path(store).resolve(strict=True)
|
|
112
112
|
if not root.is_dir() or root.suffix != ".lance":
|
|
113
113
|
raise ValueError("Cleanup requires a local Lance table directory")
|
|
@@ -116,6 +116,8 @@ def prune(store, expected_version, cutoff_ms, owner_file, nonce, admission=None)
|
|
|
116
116
|
raise ValueError("Cleanup requires a positive current version")
|
|
117
117
|
if cutoff_ms > time.time_ns() // 1_000_000:
|
|
118
118
|
raise ValueError("Cleanup cutoff must not be in the future")
|
|
119
|
+
if max_versions is not None and (type(max_versions) is not int or not 1 <= max_versions <= 128):
|
|
120
|
+
raise ValueError("Invalid bounded prune version count")
|
|
119
121
|
if importlib.metadata.version("pylance") != "12.0.0":
|
|
120
122
|
raise ValueError("Cleanup requires pylance 12.0.0")
|
|
121
123
|
before_bytes, before_allocated, before_free = measure(root)
|
|
@@ -134,6 +136,10 @@ def prune(store, expected_version, cutoff_ms, owner_file, nonce, admission=None)
|
|
|
134
136
|
protected.add(version["version"])
|
|
135
137
|
if version["version"] not in protected and version["timestamp"].timestamp() * 1000 < cutoff_ms:
|
|
136
138
|
versions.append(version["version"])
|
|
139
|
+
eligible_count = len(versions)
|
|
140
|
+
if max_versions is not None:
|
|
141
|
+
versions = sorted(versions)[:max_versions]
|
|
142
|
+
retained = {version["version"] for version in all_versions} - set(versions)
|
|
137
143
|
if dataset.latest_version != expected_version or verify_lease(root, owner_file, nonce) != owner:
|
|
138
144
|
raise ValueError("Table or exclusive ownership changed during cleanup admission")
|
|
139
145
|
if len(protected) > MAX_PROTECTED_VERSIONS:
|
|
@@ -152,7 +158,7 @@ def prune(store, expected_version, cutoff_ms, owner_file, nonce, admission=None)
|
|
|
152
158
|
fresh = open_dataset(lance, root)
|
|
153
159
|
remaining = {version["version"] for version in fresh.versions()}
|
|
154
160
|
if (fresh.version != expected_version or fresh.latest_version != expected_version
|
|
155
|
-
or snapshot(fresh) != current or
|
|
161
|
+
or snapshot(fresh) != current or remaining != retained
|
|
156
162
|
or set(versions).intersection(remaining)
|
|
157
163
|
or verify_lease(root, owner_file, nonce) != owner):
|
|
158
164
|
raise ValueError("Current/protected state or exclusive ownership changed; no retry")
|
|
@@ -166,6 +172,7 @@ def prune(store, expected_version, cutoff_ms, owner_file, nonce, admission=None)
|
|
|
166
172
|
"bytesRemoved": stats.bytes_removed if stats else 0,
|
|
167
173
|
"versionsRemoved": stats.old_versions if stats else 0,
|
|
168
174
|
"rewritten": False,
|
|
175
|
+
"eligibleVersionsRemaining": eligible_count - len(versions),
|
|
169
176
|
"fileBytesBefore": before_bytes, "fileBytesAfter": after_bytes,
|
|
170
177
|
"allocatedBytesBefore": before_allocated, "allocatedBytesAfter": after_allocated,
|
|
171
178
|
"freeBytesBefore": before_free, "freeBytesAfter": after_free,
|
|
@@ -180,6 +187,7 @@ def main():
|
|
|
180
187
|
parser.add_argument("--lease-owner", required=True)
|
|
181
188
|
parser.add_argument("--lease-nonce", required=True)
|
|
182
189
|
parser.add_argument("--require-admission", action="store_true")
|
|
190
|
+
parser.add_argument("--max-versions", type=int)
|
|
183
191
|
args = parser.parse_args()
|
|
184
192
|
if not args.require_admission:
|
|
185
193
|
parser.error("Prune helper requires guarded gmax recovery admission")
|
|
@@ -193,7 +201,8 @@ def main():
|
|
|
193
201
|
if sys.stdin.readline(256).strip() != args.lease_nonce:
|
|
194
202
|
raise ValueError("Deletion admission unavailable; no cleanup")
|
|
195
203
|
print(json.dumps(prune(args.store, args.version, args.cutoff_ms, args.lease_owner,
|
|
196
|
-
args.lease_nonce, admission if args.require_admission else None
|
|
204
|
+
args.lease_nonce, admission if args.require_admission else None,
|
|
205
|
+
args.max_versions)))
|
|
197
206
|
|
|
198
207
|
|
|
199
208
|
if __name__ == "__main__":
|
package/package.json
CHANGED