grepmax 0.26.58 → 0.26.60
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/README.md +6 -1
- package/dist/commands/doctor.js +10 -10
- package/dist/commands/mcp.js +13 -0
- package/dist/commands/search-output.js +4 -2
- package/dist/commands/search-run.js +3 -0
- package/dist/commands/search.js +0 -2
- package/dist/lib/daemon/daemon.js +53 -5
- package/dist/lib/daemon/mlx-server-manager.js +41 -6
- package/dist/lib/llm/server.js +40 -6
- package/dist/lib/store/vector-db.js +51 -5
- package/dist/lib/utils/host-resource.js +248 -0
- package/dist/lib/utils/resource-budget.js +409 -0
- package/dist/lib/utils/resource-snapshot.js +1 -14
- package/dist/lib/workers/pool.js +49 -10
- package/package.json +1 -1
- package/plugins/grepmax/.claude-plugin/plugin.json +1 -1
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
"name": "grepmax",
|
|
11
11
|
"source": "./plugins/grepmax",
|
|
12
12
|
"description": "Semantic code search for Claude Code. Automatically indexes your project and provides intelligent search capabilities.",
|
|
13
|
-
"version": "0.26.
|
|
13
|
+
"version": "0.26.60",
|
|
14
14
|
"author": {
|
|
15
15
|
"name": "Robert Owens",
|
|
16
16
|
"email": "robowens@me.com"
|
package/README.md
CHANGED
|
@@ -453,6 +453,7 @@ fixtures/
|
|
|
453
453
|
| --- | --- | --- |
|
|
454
454
|
| `GMAX_EMBED_MODE` | Force `cpu` or `gpu` | Auto-detect |
|
|
455
455
|
| `GMAX_WORKER_THREADS` | Worker processes for embedding; overrides `gmax config --worker-threads` | `min(cores, 4, max(2, floor(cores/2)))` |
|
|
456
|
+
| `GMAX_RESOURCE_BUDGET_MB` | macOS aggregate admission budget across clients, stores, workers and models; accepts 256–6144 MiB | `6144` |
|
|
456
457
|
| `GMAX_WORKER_RSS_RECYCLE_MB` | Recycle workers that remain above this RSS; `0` disables the check | `1536` |
|
|
457
458
|
| `GMAX_DEBUG` | Debug logging | Off |
|
|
458
459
|
| `GMAX_RERANK` | Force ColBERT rerank (`1`); concentrated candidates can also enable it automatically ([why](docs/known-limitations.md)) | Off |
|
|
@@ -474,7 +475,7 @@ gmax watch restart # Restart daemon
|
|
|
474
475
|
|
|
475
476
|
`gmax doctor` and `gmax status` report persistent safety stops and daemon startup quarantine. `doctor` exits with code 2 when startup is blocked; `status --json` includes `daemon.startupBlockedReason`. Installing an update preserves these stops. macOS kernel-zone probes use `zprint -L` to omit the wired-memory report and kernel symbolication while retaining the same zone counters and pressure thresholds.
|
|
476
477
|
|
|
477
|
-
When heavy work is paused, the daemon keeps metadata, bounded symbol/file lookups, stored skeletons by exact file path, and keyword search available. Search uses the existing FTS index and excludes unindexed rows, loads no embedding model, returns at most 20 results, and reports keyword-only scores and retained-index freshness. Native queries have a two-second deadline and run one at a time with 48 MiB of combined Lance cache capacity. Indexing, watchers, vector/graph scans, and model launches remain paused. Missing FTS indexes return an error; paused search never rebuilds an index or falls back to a corpus scan.
|
|
478
|
+
When heavy work is paused, the daemon keeps metadata, bounded symbol/file lookups, stored skeletons by exact file path, and keyword search available. Search uses the existing FTS index and excludes unindexed rows, loads no embedding model, returns at most 20 results, and reports keyword-only scores and retained-index freshness. CLI agent output preserves the keyword-only warning, and daemon reads skip client-side model setup. Native queries have a two-second deadline and run one at a time with 48 MiB of combined Lance cache capacity. Indexing, watchers, vector/graph scans, and model launches remain paused. Missing FTS indexes return an error; paused search never rebuilds an index or falls back to a corpus scan.
|
|
478
479
|
|
|
479
480
|
To serve bounded reads under an existing startup quarantine, run `gmax watch --daemon --read-only -b`. This preserves the safety markers. `status --json` reports `daemon.service.mode` and its reason; `doctor` reports paused availability and skips native corpus diagnostics. Pausing is sticky for that daemon process. Full service requires a deliberate restart after host recovery and quarantine review. Known critical OS or kernel pressure still stops even the read-only service.
|
|
480
481
|
|
|
@@ -552,3 +553,7 @@ grepmax is built upon the foundation of [mgrep](https://github.com/mixedbread-ai
|
|
|
552
553
|
## License
|
|
553
554
|
|
|
554
555
|
Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE).
|
|
556
|
+
|
|
557
|
+
On macOS, heavy work uses a shared resource ledger and a bounded batch footprint probe covering every gmax process and its children. Native stores, workers and model launches reserve headroom before allocating it. Warning or unknown measurements pause heavy work; known critical pressure persists a stop. The 6 GiB admission budget is a conservative scheduling threshold, not an OS-enforced memory cap. It may be lowered with `GMAX_RESOURCE_BUDGET_MB`. Existing safety markers remain in place after upgrades.
|
|
558
|
+
|
|
559
|
+
MCP sessions started before this resource policy must reconnect after upgrading before heavy work can resume. Keyword search through the bounded read service stays available while those sessions are running. Cached resource snapshots in `gmax status --json` distinguish aggregate physical footprint from daemon RSS.
|
package/dist/commands/doctor.js
CHANGED
|
@@ -358,16 +358,6 @@ exports.doctor = new commander_1.Command("doctor")
|
|
|
358
358
|
}
|
|
359
359
|
if (!opts.agent)
|
|
360
360
|
console.log("gmax Doctor\n");
|
|
361
|
-
const startupBlockedReason = (0, autostart_1.daemonStartDeniedReason)();
|
|
362
|
-
if (startupBlockedReason !== null) {
|
|
363
|
-
const reason = startupBlockedReason.replace(/[\r\n\t]/g, " ");
|
|
364
|
-
console.log(opts.agent
|
|
365
|
-
? `daemon_startup\tblocked=true\treason=${reason}`
|
|
366
|
-
: `FAIL Daemon startup blocked: ${reason}`);
|
|
367
|
-
// Inspect the retained index, but do not report a healthy installation
|
|
368
|
-
// while the persistent host stop or autostart quarantine prevents use.
|
|
369
|
-
process.exitCode = 2;
|
|
370
|
-
}
|
|
371
361
|
// Metadata-only diagnostics while paused: never open a second native
|
|
372
362
|
// reader or scan the retained corpus on a pressured host.
|
|
373
363
|
const { sendDaemonCommand } = yield Promise.resolve().then(() => __importStar(require("../lib/utils/daemon-client")));
|
|
@@ -384,6 +374,16 @@ exports.doctor = new commander_1.Command("doctor")
|
|
|
384
374
|
process.exitCode = 2;
|
|
385
375
|
return;
|
|
386
376
|
}
|
|
377
|
+
const startupBlockedReason = (0, autostart_1.daemonStartDeniedReason)();
|
|
378
|
+
if (startupBlockedReason !== null) {
|
|
379
|
+
const reason = startupBlockedReason.replace(/[\r\n\t]/g, " ");
|
|
380
|
+
console.log(opts.agent
|
|
381
|
+
? `daemon_startup\tblocked=true\treason=${reason}`
|
|
382
|
+
: `FAIL Daemon startup blocked: ${reason}`);
|
|
383
|
+
// Inspect the retained index, but do not report a healthy installation
|
|
384
|
+
// while the persistent host stop or autostart quarantine prevents use.
|
|
385
|
+
process.exitCode = 2;
|
|
386
|
+
}
|
|
387
387
|
const root = config_1.PATHS.globalRoot;
|
|
388
388
|
const models = config_1.PATHS.models;
|
|
389
389
|
const grammars = config_1.PATHS.grammars;
|
package/dist/commands/mcp.js
CHANGED
|
@@ -101,6 +101,7 @@ const mcp_execution_1 = require("../lib/utils/mcp-execution");
|
|
|
101
101
|
const path_containment_1 = require("../lib/utils/path-containment");
|
|
102
102
|
const project_registry_1 = require("../lib/utils/project-registry");
|
|
103
103
|
const project_root_1 = require("../lib/utils/project-root");
|
|
104
|
+
const resource_budget_1 = require("../lib/utils/resource-budget");
|
|
104
105
|
const scope_filter_1 = require("../lib/utils/scope-filter");
|
|
105
106
|
const store_access_1 = require("../lib/utils/store-access");
|
|
106
107
|
const store_context_1 = require("../lib/utils/store-context");
|
|
@@ -368,6 +369,18 @@ exports.mcp = new commander_1.Command("mcp")
|
|
|
368
369
|
.description("Start MCP server (stdio, auto-started by plugins)")
|
|
369
370
|
.action((_optsArg, _cmd) => __awaiter(void 0, void 0, void 0, function* () {
|
|
370
371
|
process.title = "gmax-mcp";
|
|
372
|
+
try {
|
|
373
|
+
const client = resource_budget_1.resourceBudget.registerClient();
|
|
374
|
+
process.once("exit", () => {
|
|
375
|
+
try {
|
|
376
|
+
client.release();
|
|
377
|
+
}
|
|
378
|
+
catch (_a) { }
|
|
379
|
+
});
|
|
380
|
+
}
|
|
381
|
+
catch (error) {
|
|
382
|
+
console.error(`gmax: resource coordination unavailable; heavy work paused (${error instanceof Error ? error.message : "unknown"})`);
|
|
383
|
+
}
|
|
371
384
|
// --- Lifecycle ---
|
|
372
385
|
// No per-session VectorDB. Every read tool goes through the daemon's read
|
|
373
386
|
// verbs, and the only store this process may open is the short-lived one
|
|
@@ -210,9 +210,11 @@ function renderSearchOutput(params) {
|
|
|
210
210
|
var _a, _b, _c;
|
|
211
211
|
var _d, _e, _f, _g, _h, _j;
|
|
212
212
|
const { searchResult, options, minScore, crossProject, pattern, effectiveRoot, projectRoot, vectorDb, precomputedSkeletons, precomputedGraph, indexState, } = params;
|
|
213
|
-
if (
|
|
213
|
+
if ((_a = searchResult.warnings) === null || _a === void 0 ? void 0 : _a.length) {
|
|
214
214
|
for (const w of searchResult.warnings) {
|
|
215
|
-
console.warn(
|
|
215
|
+
console.warn(options.agent
|
|
216
|
+
? `warn\t${w.replace(/[\r\n\t]/g, " ")}`
|
|
217
|
+
: `Warning: ${w}`);
|
|
216
218
|
}
|
|
217
219
|
}
|
|
218
220
|
// Partial-index signal (Phase 6): when the index is mid-catchup, results
|
|
@@ -53,6 +53,7 @@ const sync_helpers_1 = require("../lib/index/sync-helpers");
|
|
|
53
53
|
const syncer_1 = require("../lib/index/syncer");
|
|
54
54
|
const per_file_1 = require("../lib/search/per-file");
|
|
55
55
|
const searcher_1 = require("../lib/search/searcher");
|
|
56
|
+
const setup_helpers_1 = require("../lib/setup/setup-helpers");
|
|
56
57
|
const vector_db_1 = require("../lib/store/vector-db");
|
|
57
58
|
const lock_1 = require("../lib/utils/lock");
|
|
58
59
|
const project_registry_1 = require("../lib/utils/project-registry");
|
|
@@ -160,6 +161,8 @@ function runSearch(params) {
|
|
|
160
161
|
// In-process fallback: open VectorDB, ensure index, run Searcher.
|
|
161
162
|
// Only entered when the daemon path didn't produce results.
|
|
162
163
|
if (!searchResult) {
|
|
164
|
+
// A daemon read needs no client-side model download or loading.
|
|
165
|
+
yield (0, setup_helpers_1.ensureSetup)();
|
|
163
166
|
const vectorDb = new vector_db_1.VectorDB(paths.lancedbDir);
|
|
164
167
|
openedDb = vectorDb;
|
|
165
168
|
// Check for active indexing lock and warn if present
|
package/dist/commands/search.js
CHANGED
|
@@ -46,7 +46,6 @@ exports.search = void 0;
|
|
|
46
46
|
const path = __importStar(require("node:path"));
|
|
47
47
|
const commander_1 = require("commander");
|
|
48
48
|
const per_file_1 = require("../lib/search/per-file");
|
|
49
|
-
const setup_helpers_1 = require("../lib/setup/setup-helpers");
|
|
50
49
|
const cross_project_1 = require("../lib/utils/cross-project");
|
|
51
50
|
const exit_1 = require("../lib/utils/exit");
|
|
52
51
|
const path_containment_1 = require("../lib/utils/path-containment");
|
|
@@ -238,7 +237,6 @@ Examples:
|
|
|
238
237
|
return;
|
|
239
238
|
}
|
|
240
239
|
try {
|
|
241
|
-
yield (0, setup_helpers_1.ensureSetup)();
|
|
242
240
|
let checkRoot = cwdProjectRoot;
|
|
243
241
|
if (options.root) {
|
|
244
242
|
const resolved = resolvedOptionRoot !== undefined
|
|
@@ -67,6 +67,7 @@ const autostart_1 = require("../utils/autostart");
|
|
|
67
67
|
const blocked_roots_1 = require("../utils/blocked-roots");
|
|
68
68
|
const daemon_client_1 = require("../utils/daemon-client");
|
|
69
69
|
const daemon_launcher_1 = require("../utils/daemon-launcher");
|
|
70
|
+
const host_resource_1 = require("../utils/host-resource");
|
|
70
71
|
const kernel_zone_1 = require("../utils/kernel-zone");
|
|
71
72
|
const keyed_mutex_1 = require("../utils/keyed-mutex");
|
|
72
73
|
const log_rotate_1 = require("../utils/log-rotate");
|
|
@@ -76,6 +77,7 @@ const process_1 = require("../utils/process");
|
|
|
76
77
|
const process_footprint_1 = require("../utils/process-footprint");
|
|
77
78
|
const project_registry_1 = require("../utils/project-registry");
|
|
78
79
|
const query_timeout_1 = require("../utils/query-timeout");
|
|
80
|
+
const resource_budget_1 = require("../utils/resource-budget");
|
|
79
81
|
const resource_snapshot_1 = require("../utils/resource-snapshot");
|
|
80
82
|
const root_availability_1 = require("../utils/root-availability");
|
|
81
83
|
const safety_latch_1 = require("../utils/safety-latch");
|
|
@@ -165,6 +167,7 @@ class Daemon {
|
|
|
165
167
|
// resources must be gated on this to avoid hitting null stores mid-startup.
|
|
166
168
|
this.ready = false;
|
|
167
169
|
this.pausedReason = null;
|
|
170
|
+
this.lastHostResources = null;
|
|
168
171
|
this.pausePromise = null;
|
|
169
172
|
this.pausedReadGate = new async_semaphore_1.AsyncSemaphore(1);
|
|
170
173
|
this.processManager = new process_manager_1.ProcessManager({
|
|
@@ -246,7 +249,7 @@ class Daemon {
|
|
|
246
249
|
}
|
|
247
250
|
start() {
|
|
248
251
|
return __awaiter(this, arguments, void 0, function* (options = {}) {
|
|
249
|
-
var _a, _b;
|
|
252
|
+
var _a, _b, _c;
|
|
250
253
|
const denied = (0, autostart_1.daemonStartDeniedReason)();
|
|
251
254
|
if (denied !== null && !options.readOnly)
|
|
252
255
|
throw new Error(`gmax: ${denied}; preserving containment`);
|
|
@@ -260,6 +263,19 @@ class Daemon {
|
|
|
260
263
|
this.pausedReason = (_a = denied !== null && denied !== void 0 ? denied : pressure === null || pressure === void 0 ? void 0 : pressure.reason) !== null && _a !== void 0 ? _a : "read-only service requested";
|
|
261
264
|
}
|
|
262
265
|
process.title = "gmax-daemon";
|
|
266
|
+
if (this.pausedReason === null) {
|
|
267
|
+
try {
|
|
268
|
+
this.lastHostResources = resource_budget_1.resourceBudget.check();
|
|
269
|
+
}
|
|
270
|
+
catch (error) {
|
|
271
|
+
if (error instanceof resource_budget_1.ResourceAdmissionError && error.critical)
|
|
272
|
+
throw error;
|
|
273
|
+
this.pausedReason =
|
|
274
|
+
error instanceof Error
|
|
275
|
+
? error.message
|
|
276
|
+
: "aggregate resource admission unavailable";
|
|
277
|
+
}
|
|
278
|
+
}
|
|
263
279
|
// Read verbs register here rather than at read-verbs.ts module scope: the
|
|
264
280
|
// registry is asserted on exactly in tests/ipc-read-verbs.test.ts, and a
|
|
265
281
|
// module-scope registration would make it non-empty on import. Register
|
|
@@ -311,7 +327,7 @@ class Daemon {
|
|
|
311
327
|
try {
|
|
312
328
|
fs.unlinkSync(config_1.PATHS.daemonSocket);
|
|
313
329
|
}
|
|
314
|
-
catch (
|
|
330
|
+
catch (_d) { }
|
|
315
331
|
this.server = net.createServer((conn) => {
|
|
316
332
|
(0, logger_1.debug)("daemon", "client connected");
|
|
317
333
|
this.connections.add(conn);
|
|
@@ -405,9 +421,13 @@ class Daemon {
|
|
|
405
421
|
}
|
|
406
422
|
catch (error) {
|
|
407
423
|
const freshPressure = this.hostPressureDeniedReason();
|
|
408
|
-
if (freshPressure === null || freshPressure.critical)
|
|
424
|
+
if (freshPressure === null || freshPressure === void 0 ? void 0 : freshPressure.critical)
|
|
425
|
+
throw error;
|
|
426
|
+
if (freshPressure === null &&
|
|
427
|
+
!(error instanceof resource_budget_1.ResourceAdmissionError) &&
|
|
428
|
+
(error === null || error === void 0 ? void 0 : error.code) !== "HOST_SAFETY")
|
|
409
429
|
throw error;
|
|
410
|
-
this.pauseWork(freshPressure.reason);
|
|
430
|
+
this.pauseWork((_b = freshPressure === null || freshPressure === void 0 ? void 0 : freshPressure.reason) !== null && _b !== void 0 ? _b : error.message);
|
|
411
431
|
yield this.pausePromise;
|
|
412
432
|
}
|
|
413
433
|
if (this.pausedReason === null) {
|
|
@@ -427,7 +447,7 @@ class Daemon {
|
|
|
427
447
|
// 6. LLM server manager (constructed, not started — starts on first request)
|
|
428
448
|
this.llmServer = new server_1.LlmServer();
|
|
429
449
|
// 6b. MLX embed server — start if GPU mode is active
|
|
430
|
-
const globalConfig = (
|
|
450
|
+
const globalConfig = (_c = this.activeConfig) !== null && _c !== void 0 ? _c : (0, index_config_1.readGlobalConfig)();
|
|
431
451
|
const isAppleSilicon = process.arch === "arm64" && process.platform === "darwin";
|
|
432
452
|
if (this.pausedReason === null &&
|
|
433
453
|
isAppleSilicon &&
|
|
@@ -510,6 +530,8 @@ class Daemon {
|
|
|
510
530
|
this.logResourceSnapshot("paused-heartbeat", null);
|
|
511
531
|
return;
|
|
512
532
|
}
|
|
533
|
+
if (!this.checkAggregateResources())
|
|
534
|
+
return;
|
|
513
535
|
// Every 5 ticks (5 min), probe the MLX embed server and respawn if
|
|
514
536
|
// it's gone zombie (port held but /health unresponsive). Closes the
|
|
515
537
|
// 42h-degradation window where workers silently fell back to ONNX CPU
|
|
@@ -758,6 +780,7 @@ class Daemon {
|
|
|
758
780
|
});
|
|
759
781
|
}
|
|
760
782
|
assertHeavyOperationAdmission(name) {
|
|
783
|
+
var _a;
|
|
761
784
|
const denied = (0, autostart_1.daemonStartDeniedReason)();
|
|
762
785
|
if (denied !== null)
|
|
763
786
|
throw new Error(`${name} refused: ${denied}`);
|
|
@@ -765,6 +788,8 @@ class Daemon {
|
|
|
765
788
|
if (!this.checkKernelZonePressure()) {
|
|
766
789
|
throw new Error(`${name} refused: host pressure is unsafe or unknown${this.pausedReason ? `: ${this.pausedReason}` : ""}`);
|
|
767
790
|
}
|
|
791
|
+
if (!this.checkAggregateResources())
|
|
792
|
+
throw new Error(`${name} refused: ${(_a = this.pausedReason) !== null && _a !== void 0 ? _a : "aggregate resources unavailable"}`);
|
|
768
793
|
// Another process can quarantine the machine while the probes run.
|
|
769
794
|
const finalDenied = (0, autostart_1.daemonStartDeniedReason)();
|
|
770
795
|
if (finalDenied !== null)
|
|
@@ -2037,6 +2062,8 @@ class Daemon {
|
|
|
2037
2062
|
// Both ordinary probes and deferred-recycle ticks check safety first.
|
|
2038
2063
|
if (!this.checkKernelZonePressure())
|
|
2039
2064
|
return;
|
|
2065
|
+
if (sampleFootprint && !this.checkAggregateResources())
|
|
2066
|
+
return;
|
|
2040
2067
|
this.maybeRecycle(sampleFootprint);
|
|
2041
2068
|
if (this.shuttingDown || this.recycling || !sampleFootprint)
|
|
2042
2069
|
return;
|
|
@@ -2048,6 +2075,8 @@ class Daemon {
|
|
|
2048
2075
|
logResourceSnapshot(reason, footprintMb) {
|
|
2049
2076
|
var _a, _b;
|
|
2050
2077
|
var _c, _d;
|
|
2078
|
+
if (reason.startsWith("paused-"))
|
|
2079
|
+
this.lastHostResources = (0, host_resource_1.sampleHostResources)();
|
|
2051
2080
|
let lanceCacheBytes = null;
|
|
2052
2081
|
try {
|
|
2053
2082
|
lanceCacheBytes = (_c = (_a = this.vectorDb) === null || _a === void 0 ? void 0 : _a.cacheSizeBytes()) !== null && _c !== void 0 ? _c : null;
|
|
@@ -2061,6 +2090,7 @@ class Daemon {
|
|
|
2061
2090
|
pendingFiles: [...this.processors.values()].reduce((sum, processor) => sum + processor.progress.pendingFiles, 0),
|
|
2062
2091
|
operations: this.operations.activeCount,
|
|
2063
2092
|
maintenance: (_d = (_b = this.vectorDb) === null || _b === void 0 ? void 0 : _b.isMaintenanceActive()) !== null && _d !== void 0 ? _d : false,
|
|
2093
|
+
host: this.lastHostResources,
|
|
2064
2094
|
});
|
|
2065
2095
|
console.log(`[daemon] Resource snapshot: ${JSON.stringify(this.lastResourceSnapshot)}`);
|
|
2066
2096
|
}
|
|
@@ -2165,6 +2195,24 @@ class Daemon {
|
|
|
2165
2195
|
}
|
|
2166
2196
|
return this.pausedReason === null;
|
|
2167
2197
|
}
|
|
2198
|
+
checkAggregateResources() {
|
|
2199
|
+
if (this.shuttingDown || this.recycling || this.pausedReason !== null)
|
|
2200
|
+
return false;
|
|
2201
|
+
try {
|
|
2202
|
+
this.lastHostResources = resource_budget_1.resourceBudget.check();
|
|
2203
|
+
return true;
|
|
2204
|
+
}
|
|
2205
|
+
catch (error) {
|
|
2206
|
+
const reason = error instanceof Error
|
|
2207
|
+
? error.message
|
|
2208
|
+
: "aggregate resource admission unavailable";
|
|
2209
|
+
if (error instanceof resource_budget_1.ResourceAdmissionError && error.critical)
|
|
2210
|
+
this.stopForSafety(reason);
|
|
2211
|
+
else
|
|
2212
|
+
this.pauseWork(reason);
|
|
2213
|
+
return false;
|
|
2214
|
+
}
|
|
2215
|
+
}
|
|
2168
2216
|
/** Unknown/warning pauses resource expansion; only known critical shuts down. */
|
|
2169
2217
|
hostPressureDeniedReason() {
|
|
2170
2218
|
if (process.platform !== "darwin")
|
|
@@ -52,6 +52,7 @@ const config_1 = require("../../config");
|
|
|
52
52
|
const log_rotate_1 = require("../utils/log-rotate");
|
|
53
53
|
const mlx_hf_cache_1 = require("../utils/mlx-hf-cache");
|
|
54
54
|
const process_1 = require("../utils/process");
|
|
55
|
+
const resource_budget_1 = require("../utils/resource-budget");
|
|
55
56
|
const STARTUP_ATTEMPTS = 30;
|
|
56
57
|
const STARTUP_POLL_MS = 1000;
|
|
57
58
|
const MAX_HEALTH_BODY_BYTES = 16 * 1024;
|
|
@@ -151,11 +152,18 @@ class MlxServerManager {
|
|
|
151
152
|
terminateOwned(server) {
|
|
152
153
|
return __awaiter(this, void 0, void 0, function* () {
|
|
153
154
|
var _a;
|
|
154
|
-
|
|
155
|
+
var _b;
|
|
156
|
+
const terminate = (_b = this.deps.terminateGroup) !== null && _b !== void 0 ? _b : process_1.terminateProcessGroup;
|
|
155
157
|
const stopped = yield terminate(server.pid);
|
|
156
158
|
if (!stopped) {
|
|
157
159
|
console.error(`[daemon] Failed to fully stop owned MLX process group ${server.pid}`);
|
|
158
160
|
}
|
|
161
|
+
else {
|
|
162
|
+
try {
|
|
163
|
+
(_a = server.reservation) === null || _a === void 0 ? void 0 : _a.release();
|
|
164
|
+
}
|
|
165
|
+
catch (_c) { }
|
|
166
|
+
}
|
|
159
167
|
server.child.removeAllListeners("error");
|
|
160
168
|
server.child.removeAllListeners("exit");
|
|
161
169
|
if (this.stable === server)
|
|
@@ -204,7 +212,7 @@ class MlxServerManager {
|
|
|
204
212
|
ensureInternal(model) {
|
|
205
213
|
return __awaiter(this, void 0, void 0, function* () {
|
|
206
214
|
var _a, _b, _c, _d, _e;
|
|
207
|
-
var _f, _g, _h, _j, _k;
|
|
215
|
+
var _f, _g, _h, _j, _k, _l;
|
|
208
216
|
this.phase = "probing";
|
|
209
217
|
const health = yield this.probeHealth(this.port);
|
|
210
218
|
if (this.shouldStop())
|
|
@@ -223,6 +231,7 @@ class MlxServerManager {
|
|
|
223
231
|
this.stable.lastHealthyAt = Date.now();
|
|
224
232
|
return;
|
|
225
233
|
}
|
|
234
|
+
resource_budget_1.resourceBudget.check(this.getPortPid(this.port));
|
|
226
235
|
if (this.stable.kind === "owned")
|
|
227
236
|
yield this.terminateOwned(this.stable);
|
|
228
237
|
this.stable = { kind: "adopted", model, lastHealthyAt: Date.now() };
|
|
@@ -278,17 +287,41 @@ class MlxServerManager {
|
|
|
278
287
|
const logFd = openLog(path.join(config_1.PATHS.logsDir, "mlx-embed-server.log"));
|
|
279
288
|
const env = Object.assign(Object.assign({}, process.env), { MLX_EMBED_MODEL: model, GMAX_EMBED_OWNER_TOKEN: owner, HF_HOME: (0, mlx_hf_cache_1.resolveMlxHfHome)(model) });
|
|
280
289
|
this.phase = "starting";
|
|
281
|
-
let child;
|
|
290
|
+
let child = null;
|
|
291
|
+
let reservation;
|
|
282
292
|
try {
|
|
293
|
+
reservation = resource_budget_1.resourceBudget.reserve(resource_budget_1.EMBEDDING_RESOURCE_RESERVE_MB, "embedding");
|
|
283
294
|
child = ((_j = this.deps.spawn) !== null && _j !== void 0 ? _j : node_child_process_1.spawn)("uv", ["run", "python", "server.py"], {
|
|
284
295
|
cwd: serverDir,
|
|
285
296
|
detached: true,
|
|
286
297
|
stdio: ["ignore", logFd, logFd],
|
|
287
298
|
env,
|
|
288
299
|
});
|
|
300
|
+
if (!child.pid) {
|
|
301
|
+
reservation.release();
|
|
302
|
+
throw new Error("spawned MLX process has no PID");
|
|
303
|
+
}
|
|
304
|
+
try {
|
|
305
|
+
reservation.attach(child.pid, true);
|
|
306
|
+
}
|
|
307
|
+
catch (error) {
|
|
308
|
+
const stopped = yield ((_k = this.deps.terminateGroup) !== null && _k !== void 0 ? _k : process_1.terminateProcessGroup)(child.pid);
|
|
309
|
+
if (stopped)
|
|
310
|
+
reservation.release();
|
|
311
|
+
throw error;
|
|
312
|
+
}
|
|
289
313
|
}
|
|
290
314
|
catch (error) {
|
|
315
|
+
// If no child was created, there is no native group to keep charged.
|
|
316
|
+
if (!child) {
|
|
317
|
+
try {
|
|
318
|
+
reservation === null || reservation === void 0 ? void 0 : reservation.release();
|
|
319
|
+
}
|
|
320
|
+
catch (_m) { }
|
|
321
|
+
}
|
|
291
322
|
this.lastError = error instanceof Error ? error.message : String(error);
|
|
323
|
+
if (error instanceof resource_budget_1.ResourceAdmissionError)
|
|
324
|
+
this.enabled = false;
|
|
292
325
|
this.phase = "failed";
|
|
293
326
|
console.error(`[daemon] MLX embed server failed to spawn: ${this.lastError} — falling back to CPU embeddings`);
|
|
294
327
|
return;
|
|
@@ -297,9 +330,9 @@ class MlxServerManager {
|
|
|
297
330
|
try {
|
|
298
331
|
closeFd(logFd);
|
|
299
332
|
}
|
|
300
|
-
catch (
|
|
333
|
+
catch (_o) { }
|
|
301
334
|
}
|
|
302
|
-
if (!child.pid) {
|
|
335
|
+
if (!(child === null || child === void 0 ? void 0 : child.pid)) {
|
|
303
336
|
this.lastError = "spawned MLX process has no PID";
|
|
304
337
|
this.phase = "failed";
|
|
305
338
|
return;
|
|
@@ -312,6 +345,7 @@ class MlxServerManager {
|
|
|
312
345
|
model,
|
|
313
346
|
owner,
|
|
314
347
|
lastHealthyAt: 0,
|
|
348
|
+
reservation,
|
|
315
349
|
};
|
|
316
350
|
const startup = { error: null, exited: false };
|
|
317
351
|
const onError = (error) => {
|
|
@@ -338,6 +372,7 @@ class MlxServerManager {
|
|
|
338
372
|
child.off("exit", onExit);
|
|
339
373
|
if (readiness.owner !== owner) {
|
|
340
374
|
yield this.terminateOwned(candidate);
|
|
375
|
+
resource_budget_1.resourceBudget.check(this.getPortPid(this.port));
|
|
341
376
|
this.stable = {
|
|
342
377
|
kind: "adopted",
|
|
343
378
|
model,
|
|
@@ -356,7 +391,7 @@ class MlxServerManager {
|
|
|
356
391
|
child.off("error", onError);
|
|
357
392
|
child.off("exit", onExit);
|
|
358
393
|
yield this.terminateOwned(candidate);
|
|
359
|
-
this.lastError = (
|
|
394
|
+
this.lastError = (_l = (_e = startup.error) === null || _e === void 0 ? void 0 : _e.message) !== null && _l !== void 0 ? _l : (this.shouldStop() ? "MLX startup cancelled" : "MLX startup timed out");
|
|
360
395
|
if (!this.shouldStop()) {
|
|
361
396
|
this.phase = "failed";
|
|
362
397
|
console.error(`[daemon] ${this.lastError} — falling back to CPU embeddings`);
|
package/dist/lib/llm/server.js
CHANGED
|
@@ -50,6 +50,8 @@ const path = __importStar(require("node:path"));
|
|
|
50
50
|
const config_1 = require("../../config");
|
|
51
51
|
const index_config_1 = require("../index/index-config");
|
|
52
52
|
const log_rotate_1 = require("../utils/log-rotate");
|
|
53
|
+
const process_1 = require("../utils/process");
|
|
54
|
+
const resource_budget_1 = require("../utils/resource-budget");
|
|
53
55
|
const config_2 = require("./config");
|
|
54
56
|
// ⛔ HARD STOP FOR AI AGENTS: This starts llama-server with a multi-GB local
|
|
55
57
|
// LLM (Qwen3.5-35B GGUF ≈21GB by default). DO NOT start it — via start(),
|
|
@@ -124,6 +126,7 @@ class LlmServer {
|
|
|
124
126
|
throw new Error("LLM is disabled. Run `gmax llm on` to enable.");
|
|
125
127
|
}
|
|
126
128
|
if (yield this.healthy()) {
|
|
129
|
+
resource_budget_1.resourceBudget.check(this.readPid());
|
|
127
130
|
// Adopt an existing server (e.g. after daemon crash + restart)
|
|
128
131
|
this.lastRequestTime = Date.now();
|
|
129
132
|
this.startTime = Date.now();
|
|
@@ -148,7 +151,17 @@ class LlmServer {
|
|
|
148
151
|
if (!fs.existsSync(this.config.model)) {
|
|
149
152
|
throw new Error(`Model file not found: "${this.config.model}". Set GMAX_LLM_MODEL to a valid .gguf path`);
|
|
150
153
|
}
|
|
151
|
-
|
|
154
|
+
// Reserve the model's on-disk size plus runtime headroom before starting
|
|
155
|
+
// it. The default multi-GB model cannot fit the containment budget.
|
|
156
|
+
const reservation = resource_budget_1.resourceBudget.reserve(Math.ceil(fs.statSync(this.config.model).size / 1048576) + 512, "llm");
|
|
157
|
+
let logFd;
|
|
158
|
+
try {
|
|
159
|
+
logFd = (0, log_rotate_1.openRotatedLog)(config_1.PATHS.llmLogFile);
|
|
160
|
+
}
|
|
161
|
+
catch (error) {
|
|
162
|
+
reservation.release();
|
|
163
|
+
throw error;
|
|
164
|
+
}
|
|
152
165
|
const args = [
|
|
153
166
|
"-m",
|
|
154
167
|
this.config.model,
|
|
@@ -164,16 +177,36 @@ class LlmServer {
|
|
|
164
177
|
if (this.config.reasoningFormat) {
|
|
165
178
|
args.push("--reasoning-format", this.config.reasoningFormat);
|
|
166
179
|
}
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
180
|
+
let child;
|
|
181
|
+
try {
|
|
182
|
+
child = (0, node_child_process_1.spawn)(binary, args, {
|
|
183
|
+
detached: true,
|
|
184
|
+
stdio: ["ignore", logFd, logFd],
|
|
185
|
+
});
|
|
186
|
+
}
|
|
187
|
+
catch (error) {
|
|
188
|
+
reservation.release();
|
|
189
|
+
throw error;
|
|
190
|
+
}
|
|
191
|
+
finally {
|
|
192
|
+
fs.closeSync(logFd);
|
|
193
|
+
}
|
|
171
194
|
child.unref();
|
|
172
|
-
fs.closeSync(logFd);
|
|
173
195
|
const pid = child.pid;
|
|
174
196
|
if (!pid) {
|
|
197
|
+
reservation.release();
|
|
175
198
|
throw new Error("Failed to spawn llama-server — no PID returned");
|
|
176
199
|
}
|
|
200
|
+
// Group reservations are reaped only after the whole detached group is
|
|
201
|
+
// confirmed dead. A launcher exit cannot make surviving children free.
|
|
202
|
+
try {
|
|
203
|
+
reservation.attach(pid, true);
|
|
204
|
+
}
|
|
205
|
+
catch (error) {
|
|
206
|
+
if (yield (0, process_1.terminateProcessGroup)(pid))
|
|
207
|
+
reservation.release();
|
|
208
|
+
throw error;
|
|
209
|
+
}
|
|
177
210
|
fs.writeFileSync(config_1.PATHS.llmPidFile, String(pid));
|
|
178
211
|
console.log(`[llm] Starting llama-server (PID: ${pid}, port: ${this.config.port})`);
|
|
179
212
|
// Poll until ready
|
|
@@ -257,6 +290,7 @@ class LlmServer {
|
|
|
257
290
|
throw new Error("LLM is disabled. Run `gmax llm on` to enable.");
|
|
258
291
|
}
|
|
259
292
|
if (yield this.healthy()) {
|
|
293
|
+
resource_budget_1.resourceBudget.check(this.readPid());
|
|
260
294
|
this.touchIdle();
|
|
261
295
|
return;
|
|
262
296
|
}
|
|
@@ -63,6 +63,7 @@ const cleanup_1 = require("../utils/cleanup");
|
|
|
63
63
|
const filter_builder_1 = require("../utils/filter-builder");
|
|
64
64
|
const logger_1 = require("../utils/logger");
|
|
65
65
|
const query_timeout_1 = require("../utils/query-timeout");
|
|
66
|
+
const resource_budget_1 = require("../utils/resource-budget");
|
|
66
67
|
const ann_config_1 = require("./ann-config");
|
|
67
68
|
const compaction_result_1 = require("./compaction-result");
|
|
68
69
|
const lancedb = __importStar(require("./lance-sdk"));
|
|
@@ -252,6 +253,8 @@ class VectorDB {
|
|
|
252
253
|
this.lastDiskCheckMs = 0;
|
|
253
254
|
this.lastLoggedPressure = "ok";
|
|
254
255
|
this.leasePromise = null;
|
|
256
|
+
this.resourceReservation = null;
|
|
257
|
+
this.connecting = null;
|
|
255
258
|
this.leaseAbort = new AbortController();
|
|
256
259
|
this.closePromise = null;
|
|
257
260
|
this.releaseLeaseOnClose = true;
|
|
@@ -374,6 +377,8 @@ class VectorDB {
|
|
|
374
377
|
}
|
|
375
378
|
getDb() {
|
|
376
379
|
return __awaiter(this, void 0, void 0, function* () {
|
|
380
|
+
var _a;
|
|
381
|
+
var _b, _c;
|
|
377
382
|
if (this.closed) {
|
|
378
383
|
throw new Error("VectorDB connection is closed");
|
|
379
384
|
}
|
|
@@ -387,17 +392,38 @@ class VectorDB {
|
|
|
387
392
|
throw new Error("VectorDB connection is closed");
|
|
388
393
|
}
|
|
389
394
|
if (!this.db) {
|
|
395
|
+
if (this.connecting)
|
|
396
|
+
return this.connecting;
|
|
390
397
|
if (creatingStore || !fs.existsSync(this.lancedbDir)) {
|
|
391
398
|
this.assertMutationAllowed();
|
|
392
399
|
(0, maintenance_policy_1.assertFreshDiskMutationAllowed)(this.lancedbDir);
|
|
393
400
|
}
|
|
394
401
|
fs.mkdirSync(this.lancedbDir, { recursive: true });
|
|
402
|
+
// The explicit bounded reader remains available under warning/unknown
|
|
403
|
+
// pressure. All ordinary clients reserve their native cache capacity in
|
|
404
|
+
// one shared ledger before opening a connection, including offline MCP.
|
|
405
|
+
if (!this.options.readOnly) {
|
|
406
|
+
this.resourceReservation = resource_budget_1.resourceBudget.reserve(((_b = this.options.indexCacheMb) !== null && _b !== void 0 ? _b : LANCE_INDEX_CACHE_MB) +
|
|
407
|
+
((_c = this.options.metadataCacheMb) !== null && _c !== void 0 ? _c : LANCE_METADATA_CACHE_MB), "native-store");
|
|
408
|
+
}
|
|
395
409
|
this.session = createLanceSession(this.options);
|
|
396
410
|
// 0.38 accepts the legacy third argument in its types but drops it in
|
|
397
411
|
// the JS wrapper. Put the session in native ConnectionOptions instead.
|
|
398
|
-
this.
|
|
412
|
+
this.connecting = lancedb.connect(this.lancedbDir, {
|
|
399
413
|
session: this.session,
|
|
400
414
|
});
|
|
415
|
+
try {
|
|
416
|
+
this.db = yield this.connecting;
|
|
417
|
+
}
|
|
418
|
+
catch (error) {
|
|
419
|
+
(_a = this.resourceReservation) === null || _a === void 0 ? void 0 : _a.release();
|
|
420
|
+
this.resourceReservation = null;
|
|
421
|
+
this.session = null;
|
|
422
|
+
throw error;
|
|
423
|
+
}
|
|
424
|
+
finally {
|
|
425
|
+
this.connecting = null;
|
|
426
|
+
}
|
|
401
427
|
}
|
|
402
428
|
return this.db;
|
|
403
429
|
});
|
|
@@ -1874,7 +1900,8 @@ class VectorDB {
|
|
|
1874
1900
|
}
|
|
1875
1901
|
finishClose() {
|
|
1876
1902
|
return __awaiter(this, void 0, void 0, function* () {
|
|
1877
|
-
var _a, _b;
|
|
1903
|
+
var _a, _b, _c;
|
|
1904
|
+
yield (_a = this.connecting) === null || _a === void 0 ? void 0 : _a.catch(() => { });
|
|
1878
1905
|
if (this.maintenanceTimer) {
|
|
1879
1906
|
clearInterval(this.maintenanceTimer);
|
|
1880
1907
|
this.maintenanceTimer = null;
|
|
@@ -1893,12 +1920,22 @@ class VectorDB {
|
|
|
1893
1920
|
]);
|
|
1894
1921
|
this.maintenancePromise = null;
|
|
1895
1922
|
}
|
|
1896
|
-
(
|
|
1923
|
+
(_b = this.unregisterCleanup) === null || _b === void 0 ? void 0 : _b.call(this);
|
|
1897
1924
|
this.unregisterCleanup = undefined;
|
|
1898
1925
|
let closeError;
|
|
1899
|
-
|
|
1926
|
+
let nativeClosed = !this.db;
|
|
1927
|
+
if ((_c = this.db) === null || _c === void 0 ? void 0 : _c.close) {
|
|
1900
1928
|
try {
|
|
1901
1929
|
const closing = this.db.close();
|
|
1930
|
+
void Promise.resolve(closing).then(() => {
|
|
1931
|
+
var _a;
|
|
1932
|
+
nativeClosed = true;
|
|
1933
|
+
try {
|
|
1934
|
+
(_a = this.resourceReservation) === null || _a === void 0 ? void 0 : _a.release();
|
|
1935
|
+
this.resourceReservation = null;
|
|
1936
|
+
}
|
|
1937
|
+
catch (_b) { }
|
|
1938
|
+
}, () => { });
|
|
1902
1939
|
if (this.requireClosedOnClose)
|
|
1903
1940
|
yield closing;
|
|
1904
1941
|
else {
|
|
@@ -1914,12 +1951,21 @@ class VectorDB {
|
|
|
1914
1951
|
}
|
|
1915
1952
|
this.db = null;
|
|
1916
1953
|
this.session = null;
|
|
1954
|
+
if (nativeClosed && this.resourceReservation) {
|
|
1955
|
+
try {
|
|
1956
|
+
this.resourceReservation.release();
|
|
1957
|
+
this.resourceReservation = null;
|
|
1958
|
+
}
|
|
1959
|
+
catch (error) {
|
|
1960
|
+
closeError !== null && closeError !== void 0 ? closeError : (closeError = error);
|
|
1961
|
+
}
|
|
1962
|
+
}
|
|
1917
1963
|
if (this.leasePromise) {
|
|
1918
1964
|
let lease = null;
|
|
1919
1965
|
try {
|
|
1920
1966
|
lease = yield this.leasePromise;
|
|
1921
1967
|
}
|
|
1922
|
-
catch (
|
|
1968
|
+
catch (_d) {
|
|
1923
1969
|
this.leasePromise = null;
|
|
1924
1970
|
}
|
|
1925
1971
|
if (lease) {
|