doc2vec 2.13.0 → 2.14.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -3
- package/dist/code-chunker.js +146 -5
- package/dist/content-processor.js +41 -8
- package/dist/controller/job-runner.js +6 -3
- package/dist/controller/migrations.js +5 -0
- package/dist/controller/server.js +25 -1
- package/dist/controller/store.js +4 -4
- package/dist/doc2vec.js +58 -9
- package/dist/ui/assets/index-Cf1xdzH_.css +1 -0
- package/dist/ui/assets/{index-zutKK2b4.js → index-JQ3zqQ_O.js} +62 -62
- package/dist/ui/index.html +2 -2
- package/package.json +1 -1
- package/dist/ui/assets/index-C-uNubcD.css +0 -1
package/README.md
CHANGED
|
@@ -8,7 +8,7 @@ The primary goal is to prepare documentation content for Retrieval-Augmented Gen
|
|
|
8
8
|
|
|
9
9
|
Run it two ways:
|
|
10
10
|
|
|
11
|
-
* **One-shot sync** — `doc2vec run config.yaml` processes every source in a config file and exits. Ideal for a cron job or CI step.
|
|
11
|
+
* **One-shot sync** — `doc2vec run config.yaml` processes every source in a config file and exits. Ideal for a cron job or CI step. Pass `--source <product_name>` (all entries with that name) or `--source-index <n>` (a single entry, by its 0-based position in the config) to sync only selected sources; both flags are repeatable.
|
|
12
12
|
* **[Controller mode](#controller-mode)** — `doc2vec controller ./configs/` stays running: it schedules each config on its own cron expression, keeps run history and per-source statistics in Postgres, and serves a web UI with live log streaming, searchable run logs, and chunk inspection. Deploy it once and manage all your sources from the browser.
|
|
13
13
|
|
|
14
14
|
[](#controller-mode)
|
|
@@ -451,7 +451,7 @@ Besides the one-shot sync, doc2vec can run as a **long-lived controller** that s
|
|
|
451
451
|
doc2vec controller --database-url postgres://user:pass@host:5432/doc2vec ./configs/
|
|
452
452
|
```
|
|
453
453
|
|
|
454
|
-
Open http://localhost:8080 to see the dashboard ([screenshot above](#doc2vec)): every config with its schedule, next run, and the outcome of its last run, plus a **Run now** button.
|
|
454
|
+
Open http://localhost:8080 to see the dashboard ([screenshot above](#doc2vec)): every config with its schedule, next run, and the outcome of its last run, plus a **Run now** button. The **▾** next to it opens a source picker to sync only a subset of the config's source entries — each entry is selectable individually, even when several share a product name (e.g. istio as github + code + website) — and the run history marks such partial runs with the selected sources.
|
|
455
455
|
|
|
456
456
|
Each config gets its own page with the full run history, per-source results, and the raw YAML:
|
|
457
457
|
|
|
@@ -477,7 +477,7 @@ sources:
|
|
|
477
477
|
...
|
|
478
478
|
```
|
|
479
479
|
|
|
480
|
-
Configs without a `schedule` are still listed in the UI and can be triggered manually with **Run now
|
|
480
|
+
Configs without a `schedule` are still listed in the UI and can be triggered manually with **Run now** (optionally for a subset of sources via the **▾** picker). Each run executes `doc2vec run <config.yaml>` as a child process (with `--source-index <n>` flags when a subset was selected); overlapping runs of the same config are skipped, and `--max-parallel` (default 1) caps how many sync jobs run at once — keep it low, since website sources launch a headless Chromium.
|
|
481
481
|
|
|
482
482
|
### Logs
|
|
483
483
|
|
|
@@ -767,6 +767,12 @@ If you don't specify a config path, it will look for config.yaml in the current
|
|
|
767
767
|
|
|
768
768
|
## Recent Changes
|
|
769
769
|
|
|
770
|
+
### Code Scans: Dangling Symlinks and Incomplete Walks
|
|
771
|
+
- **Dangling symlinks no longer abort a directory:** `statSync` follows symlinks, so a link whose target isn't in the clone (common in vendored forks, e.g. `crates/cel-fork/cel/README.md`) threw `ENOENT` and abandoned every remaining entry in that directory. Broken entries are now skipped with a warning and the walk continues
|
|
772
|
+
- **Incomplete scans never delete chunks:** if a directory genuinely can't be listed, the scan is reported as incomplete — obsolete-file cleanup, the tracked-file list, the last-mtime marker, and the last-synced git SHA are all skipped, so unscanned files keep their chunks and the next run rescans them
|
|
773
|
+
- **Incomplete scans fail the source:** a partial ingest is reported as a failed source instead of a silent success
|
|
774
|
+
- **Quieter crawls:** planned browser restarts (initial launch, the every-50-pages recycle) log at `info`; only unplanned restarts (disconnected or degraded browser) warn
|
|
775
|
+
|
|
770
776
|
### Searchable Controller Run Logs
|
|
771
777
|
- **Server-side log filtering:** Level chips and the keyword filter now query Postgres instead of the browser's in-memory buffer, so errors and warnings from the start of a long run are reachable (`GET /api/runs/:id/logs?levels=error,warn&q=...`, paged with **Load 2,000 more**)
|
|
772
778
|
- **Whole-run level counts:** New `GET /api/runs/:id/logs/counts` endpoint backs the level chips, which previously only counted the lines still held in the browser
|
package/dist/code-chunker.js
CHANGED
|
@@ -65,15 +65,39 @@ class CodeChunker {
|
|
|
65
65
|
if (!text.trim()) {
|
|
66
66
|
return [];
|
|
67
67
|
}
|
|
68
|
+
if (CodeChunker.moduleDamaged) {
|
|
69
|
+
throw new Error('tree-sitter module is damaged by an earlier parse failure; refusing to parse');
|
|
70
|
+
}
|
|
68
71
|
const parser = await CodeChunker.getParser(this.lang);
|
|
69
|
-
const
|
|
70
|
-
|
|
72
|
+
const source = Buffer.from(text, 'utf-8').toString();
|
|
73
|
+
// A grammar that traps inside parse unwinds out of WASM without letting
|
|
74
|
+
// tree-sitter clean up, and the damage is cumulative: after enough of
|
|
75
|
+
// them every later parse fails with "memory access out of bounds", on
|
|
76
|
+
// files and languages that are perfectly fine. One clear error beats
|
|
77
|
+
// thousands of misleading ones, so stop using the module.
|
|
78
|
+
let tree;
|
|
79
|
+
try {
|
|
80
|
+
tree = parser.parse(source);
|
|
81
|
+
}
|
|
82
|
+
catch (error) {
|
|
83
|
+
CodeChunker.moduleDamaged = true;
|
|
84
|
+
throw error;
|
|
85
|
+
}
|
|
71
86
|
if (!tree) {
|
|
72
87
|
throw new Error('Failed to parse code');
|
|
73
88
|
}
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
89
|
+
// The syntax tree lives in the tree-sitter WASM heap, which the JS
|
|
90
|
+
// garbage collector cannot reach. Without this delete the heap grows by
|
|
91
|
+
// the size of every tree we ever parse, until an allocation fails with
|
|
92
|
+
// "memory access out of bounds" on some unrelated file.
|
|
93
|
+
try {
|
|
94
|
+
const chunks = [];
|
|
95
|
+
await this.recursiveChunk(tree.rootNode, source, chunks);
|
|
96
|
+
return this.mergeChunks(chunks);
|
|
97
|
+
}
|
|
98
|
+
finally {
|
|
99
|
+
tree.delete();
|
|
100
|
+
}
|
|
77
101
|
}
|
|
78
102
|
static async getParser(lang) {
|
|
79
103
|
const formattedLang = lang.toLowerCase().replace(/-/g, '_');
|
|
@@ -93,6 +117,7 @@ class CodeChunker {
|
|
|
93
117
|
}
|
|
94
118
|
const wasmPath = CodeChunker.resolveWasmPath(formattedLang);
|
|
95
119
|
const wasmBuffer = fs.readFileSync(wasmPath);
|
|
120
|
+
CodeChunker.assertGrammarLinks(formattedLang, wasmBuffer);
|
|
96
121
|
const language = await web_tree_sitter_1.Language.load(wasmBuffer);
|
|
97
122
|
const parser = new web_tree_sitter_1.Parser();
|
|
98
123
|
parser.setLanguage(language);
|
|
@@ -101,6 +126,110 @@ class CodeChunker {
|
|
|
101
126
|
this.parserCache.set(formattedLang, parserPromise);
|
|
102
127
|
return parserPromise;
|
|
103
128
|
}
|
|
129
|
+
/**
|
|
130
|
+
* Refuse a grammar whose imports the tree-sitter runtime cannot satisfy.
|
|
131
|
+
*
|
|
132
|
+
* The grammars ship separately from web-tree-sitter, so a grammar built
|
|
133
|
+
* against a different runtime can import a C symbol this runtime does not
|
|
134
|
+
* export. Emscripten binds side-module imports lazily, so the mismatch
|
|
135
|
+
* stays invisible until the grammar calls the symbol mid-parse — then it
|
|
136
|
+
* throws out of WASM and damages the shared module for every language.
|
|
137
|
+
* Checking the import table up front turns that into a clean load failure,
|
|
138
|
+
* and chunkCode falls back to token chunking as it does for any other
|
|
139
|
+
* unsupported language.
|
|
140
|
+
*/
|
|
141
|
+
static assertGrammarLinks(formattedLang, wasmBuffer) {
|
|
142
|
+
if (!CodeChunker.hostExports) {
|
|
143
|
+
const hostWasm = path.join(path.dirname(require.resolve('web-tree-sitter')), 'tree-sitter.wasm');
|
|
144
|
+
CodeChunker.hostExports = CodeChunker.readWasmTables(fs.readFileSync(hostWasm)).exports;
|
|
145
|
+
}
|
|
146
|
+
// Emscripten resolves an import from the host's exports first, then
|
|
147
|
+
// from the side module's own exports.
|
|
148
|
+
const grammar = CodeChunker.readWasmTables(wasmBuffer);
|
|
149
|
+
const unresolved = grammar.functionImports.filter(name => !CodeChunker.hostExports.has(name) &&
|
|
150
|
+
!grammar.exports.has(name) &&
|
|
151
|
+
!CodeChunker.ABORT_ONLY_SYMBOLS.has(name));
|
|
152
|
+
if (unresolved.length > 0) {
|
|
153
|
+
throw new Error(`Tree-sitter grammar "${formattedLang}" is incompatible with the installed ` +
|
|
154
|
+
`web-tree-sitter runtime: unresolved imports ${unresolved.join(', ')}.`);
|
|
155
|
+
}
|
|
156
|
+
}
|
|
157
|
+
/**
|
|
158
|
+
* Read a module's function imports and its export names straight out of the
|
|
159
|
+
* WASM binary.
|
|
160
|
+
*
|
|
161
|
+
* WebAssembly.Module.imports() would be shorter, but building a
|
|
162
|
+
* WebAssembly.Module compiles the whole binary, and doing that on top of
|
|
163
|
+
* the compile Language.load already does exhausts V8's compiler zone once a
|
|
164
|
+
* process loads a dozen grammars. Walking the two sections costs nothing.
|
|
165
|
+
*/
|
|
166
|
+
static readWasmTables(buffer) {
|
|
167
|
+
const functionImports = [];
|
|
168
|
+
const exports = new Set();
|
|
169
|
+
let offset = 8; // magic number and version
|
|
170
|
+
const readVarUint = () => {
|
|
171
|
+
let result = 0;
|
|
172
|
+
let shift = 0;
|
|
173
|
+
let byte;
|
|
174
|
+
do {
|
|
175
|
+
byte = buffer[offset++];
|
|
176
|
+
result |= (byte & 0x7f) << shift;
|
|
177
|
+
shift += 7;
|
|
178
|
+
} while (byte & 0x80);
|
|
179
|
+
return result >>> 0;
|
|
180
|
+
};
|
|
181
|
+
const readName = () => {
|
|
182
|
+
const length = readVarUint();
|
|
183
|
+
const name = buffer.toString('utf8', offset, offset + length);
|
|
184
|
+
offset += length;
|
|
185
|
+
return name;
|
|
186
|
+
};
|
|
187
|
+
const skipLimits = () => {
|
|
188
|
+
const flags = readVarUint();
|
|
189
|
+
readVarUint(); // minimum
|
|
190
|
+
if (flags & 0x01)
|
|
191
|
+
readVarUint(); // maximum
|
|
192
|
+
};
|
|
193
|
+
while (offset < buffer.length) {
|
|
194
|
+
const sectionId = readVarUint();
|
|
195
|
+
const sectionSize = readVarUint();
|
|
196
|
+
const sectionEnd = offset + sectionSize;
|
|
197
|
+
if (sectionId === 2) { // import section
|
|
198
|
+
const count = readVarUint();
|
|
199
|
+
for (let i = 0; i < count; i++) {
|
|
200
|
+
readName(); // module
|
|
201
|
+
const field = readName();
|
|
202
|
+
const kind = buffer[offset++];
|
|
203
|
+
switch (kind) {
|
|
204
|
+
case 0x00: // function
|
|
205
|
+
readVarUint(); // type index
|
|
206
|
+
functionImports.push(field);
|
|
207
|
+
break;
|
|
208
|
+
case 0x01: // table
|
|
209
|
+
offset++; // element type
|
|
210
|
+
skipLimits();
|
|
211
|
+
break;
|
|
212
|
+
case 0x02: // memory
|
|
213
|
+
skipLimits();
|
|
214
|
+
break;
|
|
215
|
+
default: // global: value type + mutability
|
|
216
|
+
offset += 2;
|
|
217
|
+
break;
|
|
218
|
+
}
|
|
219
|
+
}
|
|
220
|
+
}
|
|
221
|
+
else if (sectionId === 7) { // export section
|
|
222
|
+
const count = readVarUint();
|
|
223
|
+
for (let i = 0; i < count; i++) {
|
|
224
|
+
exports.add(readName());
|
|
225
|
+
offset++; // kind
|
|
226
|
+
readVarUint(); // index
|
|
227
|
+
}
|
|
228
|
+
}
|
|
229
|
+
offset = sectionEnd;
|
|
230
|
+
}
|
|
231
|
+
return { functionImports, exports };
|
|
232
|
+
}
|
|
104
233
|
static resolveWasmPath(formattedLang) {
|
|
105
234
|
const nodeModulesPath = CodeChunker.findNearestNodeModules(__dirname);
|
|
106
235
|
if (!nodeModulesPath) {
|
|
@@ -177,3 +306,15 @@ class CodeChunker {
|
|
|
177
306
|
exports.CodeChunker = CodeChunker;
|
|
178
307
|
CodeChunker.treeSitterInitialized = false;
|
|
179
308
|
CodeChunker.parserCache = new Map();
|
|
309
|
+
CodeChunker.hostExports = null;
|
|
310
|
+
CodeChunker.moduleDamaged = false;
|
|
311
|
+
/**
|
|
312
|
+
* Symbols a grammar only calls when it is already aborting. Emscripten
|
|
313
|
+
* resolves side-module imports lazily, so an import that never gets called
|
|
314
|
+
* costs nothing. Ignoring these two keeps python, html, cpp, ruby, php and
|
|
315
|
+
* vue usable — every one of them imports __assert_fail and none of them
|
|
316
|
+
* ever calls it. Any other unresolved import is a normal-operation symbol
|
|
317
|
+
* (bash calls isalpha on real scripts, yaml calls operator new), and those
|
|
318
|
+
* are the ones that bring the whole module down.
|
|
319
|
+
*/
|
|
320
|
+
CodeChunker.ABORT_ONLY_SYMBOLS = new Set(['__assert_fail', 'abort']);
|
|
@@ -462,8 +462,14 @@ class ContentProcessor {
|
|
|
462
462
|
let pagesSinceRestart = 0;
|
|
463
463
|
// Restart the browser every N pages to prevent memory accumulation
|
|
464
464
|
const PAGES_BETWEEN_RESTARTS = 50;
|
|
465
|
-
|
|
466
|
-
|
|
465
|
+
// Planned restarts (first launch, the every-N-pages recycle) are routine
|
|
466
|
+
// and would otherwise dominate a crawl's warning count; only unplanned
|
|
467
|
+
// ones — a disconnected or degraded browser — deserve a warning.
|
|
468
|
+
const restartBrowser = async (reason, planned = false) => {
|
|
469
|
+
if (planned)
|
|
470
|
+
logger.info(`Restarting browser: ${reason}`);
|
|
471
|
+
else
|
|
472
|
+
logger.warn(`Restarting browser: ${reason}`);
|
|
467
473
|
if (browser) {
|
|
468
474
|
try {
|
|
469
475
|
await browser.close();
|
|
@@ -485,11 +491,11 @@ class ContentProcessor {
|
|
|
485
491
|
const reason = needsBrowserRestart
|
|
486
492
|
? `protocol errors persisting after page recreation (${consecutiveProtocolErrors} consecutive)`
|
|
487
493
|
: browser ? 'browser disconnected' : 'initial launch';
|
|
488
|
-
await restartBrowser(reason);
|
|
494
|
+
await restartBrowser(reason, !browser && !needsBrowserRestart);
|
|
489
495
|
}
|
|
490
496
|
else if (pagesSinceRestart >= PAGES_BETWEEN_RESTARTS) {
|
|
491
497
|
// Periodic restart to prevent memory accumulation from long crawls
|
|
492
|
-
await restartBrowser(`periodic restart after ${pagesSinceRestart} pages
|
|
498
|
+
await restartBrowser(`periodic restart after ${pagesSinceRestart} pages`, true);
|
|
493
499
|
}
|
|
494
500
|
else if (pageNeedsRecreation) {
|
|
495
501
|
// The previous page.evaluate or navigation timed out / errored.
|
|
@@ -1539,7 +1545,17 @@ class ContentProcessor {
|
|
|
1539
1545
|
let skippedFiles = 0;
|
|
1540
1546
|
for (const file of files) {
|
|
1541
1547
|
const filePath = path.join(dirPath, file);
|
|
1542
|
-
|
|
1548
|
+
// A dangling symlink makes statSync throw ENOENT; skip the
|
|
1549
|
+
// entry rather than abandoning the rest of the directory
|
|
1550
|
+
let stat;
|
|
1551
|
+
try {
|
|
1552
|
+
stat = fs.statSync(filePath);
|
|
1553
|
+
}
|
|
1554
|
+
catch (error) {
|
|
1555
|
+
logger.warn(`Skipping unreadable entry ${filePath}:`, error);
|
|
1556
|
+
skippedFiles++;
|
|
1557
|
+
continue;
|
|
1558
|
+
}
|
|
1543
1559
|
// Skip already visited paths
|
|
1544
1560
|
if (visitedPaths.has(filePath)) {
|
|
1545
1561
|
logger.debug(`Skipping already visited path: ${filePath}`);
|
|
@@ -1629,6 +1645,7 @@ class ContentProcessor {
|
|
|
1629
1645
|
const excludeExtensions = config.exclude_extensions || [];
|
|
1630
1646
|
const encoding = config.encoding || 'utf8';
|
|
1631
1647
|
let maxMtime = 0;
|
|
1648
|
+
let incomplete = false;
|
|
1632
1649
|
try {
|
|
1633
1650
|
const files = fs.readdirSync(dirPath);
|
|
1634
1651
|
let processedFiles = 0;
|
|
@@ -1638,7 +1655,19 @@ class ContentProcessor {
|
|
|
1638
1655
|
const trackFiles = options?.trackFiles;
|
|
1639
1656
|
for (const file of files) {
|
|
1640
1657
|
const filePath = path.join(dirPath, file);
|
|
1641
|
-
|
|
1658
|
+
// statSync follows symlinks, so a dangling one (common in
|
|
1659
|
+
// vendored forks) throws ENOENT. Skipping the entry keeps the
|
|
1660
|
+
// rest of the directory — letting it throw used to abandon
|
|
1661
|
+
// every sibling after it, and cleanup then deleted their chunks.
|
|
1662
|
+
let stat;
|
|
1663
|
+
try {
|
|
1664
|
+
stat = fs.statSync(filePath);
|
|
1665
|
+
}
|
|
1666
|
+
catch (error) {
|
|
1667
|
+
logger.warn(`Skipping unreadable entry ${filePath}:`, error);
|
|
1668
|
+
skippedFiles++;
|
|
1669
|
+
continue;
|
|
1670
|
+
}
|
|
1642
1671
|
if (visitedPaths.has(filePath)) {
|
|
1643
1672
|
logger.debug(`Skipping already visited path: ${filePath}`);
|
|
1644
1673
|
continue;
|
|
@@ -1650,6 +1679,7 @@ class ContentProcessor {
|
|
|
1650
1679
|
processedFiles += childResult.processedFiles;
|
|
1651
1680
|
skippedFiles += childResult.skippedFiles;
|
|
1652
1681
|
maxMtime = Math.max(maxMtime, childResult.maxMtime);
|
|
1682
|
+
incomplete = incomplete || childResult.incomplete;
|
|
1653
1683
|
}
|
|
1654
1684
|
else {
|
|
1655
1685
|
logger.debug(`Skipping directory ${filePath} (recursive=false)`);
|
|
@@ -1694,11 +1724,14 @@ class ContentProcessor {
|
|
|
1694
1724
|
}
|
|
1695
1725
|
}
|
|
1696
1726
|
logger.info(`Code directory processed. Processed: ${processedFiles}, Skipped: ${skippedFiles}`);
|
|
1697
|
-
return { processedFiles, skippedFiles, maxMtime };
|
|
1727
|
+
return { processedFiles, skippedFiles, maxMtime, incomplete };
|
|
1698
1728
|
}
|
|
1699
1729
|
catch (error) {
|
|
1730
|
+
// The directory could not be listed at all — report the scan as
|
|
1731
|
+
// incomplete so the caller skips cleanup instead of treating every
|
|
1732
|
+
// file under this path as deleted
|
|
1700
1733
|
logger.error(`Error reading code directory ${dirPath}:`, error);
|
|
1701
|
-
return { processedFiles: 0, skippedFiles: 0, maxMtime };
|
|
1734
|
+
return { processedFiles: 0, skippedFiles: 0, maxMtime, incomplete: true };
|
|
1702
1735
|
}
|
|
1703
1736
|
}
|
|
1704
1737
|
async getTokenChunker(chunkSize) {
|
|
@@ -66,7 +66,7 @@ class JobRunner {
|
|
|
66
66
|
runningCount() {
|
|
67
67
|
return this.active.size;
|
|
68
68
|
}
|
|
69
|
-
async enqueue(config, trigger) {
|
|
69
|
+
async enqueue(config, trigger, sources) {
|
|
70
70
|
if (this.shuttingDown) {
|
|
71
71
|
throw new types_1.ConflictError('controller is shutting down');
|
|
72
72
|
}
|
|
@@ -83,9 +83,9 @@ class JobRunner {
|
|
|
83
83
|
if (config.parse_error) {
|
|
84
84
|
throw new types_1.ConflictError(`config is invalid: ${config.parse_error}`);
|
|
85
85
|
}
|
|
86
|
-
const run = await this.store.createRun(config.id, config.content_hash, trigger, 'queued');
|
|
86
|
+
const run = await this.store.createRun(config.id, config.content_hash, trigger, 'queued', undefined, sources);
|
|
87
87
|
this.queuedConfigs.add(config.id);
|
|
88
|
-
this.queue.push({ runId: run.id, configId: config.id, configPath: config.path });
|
|
88
|
+
this.queue.push({ runId: run.id, configId: config.id, configPath: config.path, sources });
|
|
89
89
|
this.events.emitRunUpdate(run);
|
|
90
90
|
this.pump();
|
|
91
91
|
return run;
|
|
@@ -117,6 +117,9 @@ class JobRunner {
|
|
|
117
117
|
}
|
|
118
118
|
async start(job) {
|
|
119
119
|
const { cmd, args } = this.commandFor(job.configPath);
|
|
120
|
+
for (const source of job.sources ?? []) {
|
|
121
|
+
args.push('--source-index', String(source.index));
|
|
122
|
+
}
|
|
120
123
|
this.logger.info(`Starting run ${job.runId}: ${cmd} ${args.join(' ')}`);
|
|
121
124
|
const child = (0, child_process_1.spawn)(cmd, args, {
|
|
122
125
|
env: { ...process.env, DOC2VEC_STRUCTURED_LOGS: '1' },
|
|
@@ -57,4 +57,9 @@ exports.MIGRATIONS = [
|
|
|
57
57
|
`
|
|
58
58
|
CREATE INDEX IF NOT EXISTS d2v_run_logs_level_idx ON d2v_run_logs (run_id, level, seq);
|
|
59
59
|
`,
|
|
60
|
+
// 3: manual runs can target a subset of a config's source entries (stored as
|
|
61
|
+
// [{index, product_name, type, version}]); NULL means the run covers all sources
|
|
62
|
+
`
|
|
63
|
+
ALTER TABLE d2v_runs ADD COLUMN IF NOT EXISTS requested_sources JSONB;
|
|
64
|
+
`,
|
|
60
65
|
];
|
|
@@ -152,7 +152,31 @@ function createServer(deps) {
|
|
|
152
152
|
const config = registry.get(parseId(String(req.params.id)));
|
|
153
153
|
if (!config)
|
|
154
154
|
throw new types_1.NotFoundError('config not found');
|
|
155
|
-
|
|
155
|
+
// Optional source selection: 0-based positions in the config's sources
|
|
156
|
+
// list (exact — product names can repeat). Default: all sources.
|
|
157
|
+
let sources;
|
|
158
|
+
const { sources: rawSources, baseHash } = (req.body ?? {});
|
|
159
|
+
if (rawSources !== undefined) {
|
|
160
|
+
if (!Array.isArray(rawSources) || rawSources.some(i => !Number.isInteger(i))) {
|
|
161
|
+
throw new types_1.ValidationError('sources must be an array of source indices (integers)');
|
|
162
|
+
}
|
|
163
|
+
// Indices are only meaningful against the config revision the client
|
|
164
|
+
// saw — reject the run if the config changed underneath it.
|
|
165
|
+
if (baseHash !== undefined && baseHash !== config.content_hash) {
|
|
166
|
+
throw new types_1.ConflictError('config changed since the source list was loaded — reload and retry');
|
|
167
|
+
}
|
|
168
|
+
const total = config.source_summary.length;
|
|
169
|
+
const outOfRange = rawSources.filter((i) => i < 0 || i >= total);
|
|
170
|
+
if (outOfRange.length > 0) {
|
|
171
|
+
throw new types_1.ValidationError(`source index out of range: ${outOfRange.join(', ')} (config has ${total} sources)`);
|
|
172
|
+
}
|
|
173
|
+
// Deduplicate, keep config order; empty or full selection = full run
|
|
174
|
+
const indices = [...new Set(rawSources)].sort((a, b) => a - b);
|
|
175
|
+
if (indices.length > 0 && indices.length < total) {
|
|
176
|
+
sources = indices.map(index => ({ index, ...config.source_summary[index] }));
|
|
177
|
+
}
|
|
178
|
+
}
|
|
179
|
+
const run = await runner.enqueue(config, 'manual', sources);
|
|
156
180
|
res.status(202).json(run);
|
|
157
181
|
});
|
|
158
182
|
app.get('/api/configs/:id/stats', async (req, res) => {
|
package/dist/controller/store.js
CHANGED
|
@@ -88,11 +88,11 @@ class ControllerStore {
|
|
|
88
88
|
return rows[0] ?? null;
|
|
89
89
|
}
|
|
90
90
|
// ------------------------------------------------------------------ runs
|
|
91
|
-
async createRun(configId, configHash, trigger, status, error) {
|
|
91
|
+
async createRun(configId, configHash, trigger, status, error, requestedSources) {
|
|
92
92
|
const terminal = status === 'skipped';
|
|
93
|
-
const { rows } = await this.pool.query(`INSERT INTO d2v_runs (config_id, config_hash, trigger, status, error, finished_at)
|
|
94
|
-
VALUES ($1, $2, $3, $4, $5, ${terminal ? 'NOW()' : 'NULL'})
|
|
95
|
-
RETURNING *`, [configId, configHash, trigger, status, error ?? null]);
|
|
93
|
+
const { rows } = await this.pool.query(`INSERT INTO d2v_runs (config_id, config_hash, trigger, status, error, requested_sources, finished_at)
|
|
94
|
+
VALUES ($1, $2, $3, $4, $5, $6, ${terminal ? 'NOW()' : 'NULL'})
|
|
95
|
+
RETURNING *`, [configId, configHash, trigger, status, error ?? null, requestedSources?.length ? JSON.stringify(requestedSources) : null]);
|
|
96
96
|
return rows[0];
|
|
97
97
|
}
|
|
98
98
|
async markRunStarted(runId, pid) {
|
package/dist/doc2vec.js
CHANGED
|
@@ -174,14 +174,40 @@ class Doc2Vec {
|
|
|
174
174
|
}
|
|
175
175
|
return parsedValue;
|
|
176
176
|
}
|
|
177
|
-
|
|
177
|
+
/**
|
|
178
|
+
* Sync sources sequentially. An optional selection restricts the run to a
|
|
179
|
+
* subset of the config's sources: `indices` are 0-based positions in the
|
|
180
|
+
* config's sources list (exact — the only unambiguous identifier, since
|
|
181
|
+
* product names can repeat), and `names` match every source with that
|
|
182
|
+
* product_name. The union of both is processed, in config order.
|
|
183
|
+
*/
|
|
184
|
+
async run(selection) {
|
|
185
|
+
let sources = this.config.sources;
|
|
186
|
+
const wantedIndices = selection?.indices ?? [];
|
|
187
|
+
const wantedNames = selection?.names ?? [];
|
|
188
|
+
if (wantedIndices.length > 0 || wantedNames.length > 0) {
|
|
189
|
+
const badIndices = wantedIndices.filter(i => !Number.isInteger(i) || i < 0 || i >= this.config.sources.length);
|
|
190
|
+
if (badIndices.length > 0) {
|
|
191
|
+
throw new Error(`Source index out of range: ${badIndices.join(', ')} (config has ${this.config.sources.length} sources)`);
|
|
192
|
+
}
|
|
193
|
+
const known = new Set(this.config.sources.map(s => s.product_name));
|
|
194
|
+
const unknown = wantedNames.filter(name => !known.has(name));
|
|
195
|
+
if (unknown.length > 0) {
|
|
196
|
+
throw new Error(`Unknown source(s): ${unknown.join(', ')}. Available: ${[...known].join(', ')}`);
|
|
197
|
+
}
|
|
198
|
+
const nameSet = new Set(wantedNames);
|
|
199
|
+
const indexSet = new Set(wantedIndices);
|
|
200
|
+
sources = this.config.sources.filter((s, i) => indexSet.has(i) || nameSet.has(s.product_name));
|
|
201
|
+
const label = sources.map(s => `${s.product_name} (${s.type})`).join(', ');
|
|
202
|
+
this.logger.info(`Source filter active: syncing ${sources.length} of ${this.config.sources.length} sources: ${label}`);
|
|
203
|
+
}
|
|
178
204
|
// Initialize Postgres markdown store table if configured
|
|
179
205
|
if (this.markdownStore) {
|
|
180
206
|
await this.markdownStore.init();
|
|
181
207
|
}
|
|
182
208
|
this.logger.section('PROCESSING SOURCES');
|
|
183
209
|
const runStats = [];
|
|
184
|
-
for (const sourceConfig of
|
|
210
|
+
for (const sourceConfig of sources) {
|
|
185
211
|
const sourceLogger = this.logger.child(`source:${sourceConfig.product_name}`);
|
|
186
212
|
sourceLogger.info(`Processing ${sourceConfig.type} source for ${sourceConfig.product_name}@${sourceConfig.version}`);
|
|
187
213
|
const startTime = Date.now();
|
|
@@ -1051,6 +1077,10 @@ class Doc2Vec {
|
|
|
1051
1077
|
let lastMtimeKey;
|
|
1052
1078
|
let trackedFiles;
|
|
1053
1079
|
let maxObservedMtime = 0;
|
|
1080
|
+
// Set when a directory could not be listed: the files we saw are only a
|
|
1081
|
+
// subset of what is on disk, so nothing may be treated as deleted and no
|
|
1082
|
+
// "last scanned" marker may advance past this run
|
|
1083
|
+
let scanIncomplete = false;
|
|
1054
1084
|
if (config.source === 'local_directory') {
|
|
1055
1085
|
if (!config.path) {
|
|
1056
1086
|
logger.error('Code source type local_directory requires a path.');
|
|
@@ -1132,13 +1162,20 @@ class Doc2Vec {
|
|
|
1132
1162
|
mtimeCutoff,
|
|
1133
1163
|
trackFiles: trackedFiles
|
|
1134
1164
|
});
|
|
1165
|
+
scanIncomplete = scanResult.incomplete;
|
|
1166
|
+
if (scanIncomplete) {
|
|
1167
|
+
logger.error('Directory scan was incomplete — skipping cleanup so unscanned files keep their chunks');
|
|
1168
|
+
}
|
|
1135
1169
|
if (trackedFiles) {
|
|
1136
1170
|
maxObservedMtime = scanResult.maxMtime;
|
|
1137
1171
|
}
|
|
1138
1172
|
}
|
|
1139
1173
|
finally {
|
|
1140
1174
|
logger.section('CLEANUP');
|
|
1141
|
-
if (
|
|
1175
|
+
if (scanIncomplete) {
|
|
1176
|
+
logger.warn('Skipping cleanup and last-scanned markers: the scan did not cover the whole tree');
|
|
1177
|
+
}
|
|
1178
|
+
else if (incrementalMode) {
|
|
1142
1179
|
if (deleteUrls.length > 0) {
|
|
1143
1180
|
logger.info(`Cleaning up ${deleteUrls.length} deleted/renamed files`);
|
|
1144
1181
|
for (const url of deleteUrls) {
|
|
@@ -1197,7 +1234,9 @@ class Doc2Vec {
|
|
|
1197
1234
|
this.counters.items_deleted += removed.items;
|
|
1198
1235
|
this.counters.chunks_deleted += removed.chunks;
|
|
1199
1236
|
}
|
|
1200
|
-
|
|
1237
|
+
// Storing the SHA after an incomplete scan would make the next run
|
|
1238
|
+
// diff from it and never revisit the files this run missed
|
|
1239
|
+
if (config.source === 'github' && basePath && repoBranch && !scanIncomplete) {
|
|
1201
1240
|
const headSha = await this.getRepoHeadSha(basePath, logger);
|
|
1202
1241
|
if (headSha) {
|
|
1203
1242
|
const shaKey = this.buildCodeShaMetadataKey(config.repo, repoBranch);
|
|
@@ -1214,6 +1253,11 @@ class Doc2Vec {
|
|
|
1214
1253
|
}
|
|
1215
1254
|
}
|
|
1216
1255
|
}
|
|
1256
|
+
// Fail the source rather than reporting a partial ingest as a success —
|
|
1257
|
+
// the collection is still serving whatever it had for the unscanned files
|
|
1258
|
+
if (scanIncomplete) {
|
|
1259
|
+
throw new Error(`Code scan of ${basePath} was incomplete: at least one directory could not be read`);
|
|
1260
|
+
}
|
|
1217
1261
|
logger.info(`Finished processing code source (${config.source})`);
|
|
1218
1262
|
}
|
|
1219
1263
|
buildCodeShaMetadataKey(repo, branch) {
|
|
@@ -1820,15 +1864,15 @@ exports.Doc2Vec = Doc2Vec;
|
|
|
1820
1864
|
Doc2Vec.MAX_EMBEDDING_TOKENS = 8191;
|
|
1821
1865
|
Doc2Vec.CHARS_PER_TOKEN = 4;
|
|
1822
1866
|
Doc2Vec.MAX_EMBEDDING_CHARS = Doc2Vec.MAX_EMBEDDING_TOKENS * Doc2Vec.CHARS_PER_TOKEN;
|
|
1823
|
-
function runOneShot(configPath) {
|
|
1867
|
+
function runOneShot(configPath, selection) {
|
|
1824
1868
|
if (!fs.existsSync(configPath)) {
|
|
1825
1869
|
console.error('Please provide a valid path to a YAML config file.');
|
|
1826
1870
|
process.exit(1);
|
|
1827
1871
|
}
|
|
1828
1872
|
const doc2Vec = new Doc2Vec(configPath);
|
|
1829
|
-
doc2Vec.run()
|
|
1873
|
+
doc2Vec.run(selection)
|
|
1830
1874
|
.then((stats) => process.exit(stats.some(s => !s.ok) ? 1 : 0))
|
|
1831
|
-
.catch((err) => { console.error(err); process.exit(1); });
|
|
1875
|
+
.catch((err) => { console.error(err instanceof Error ? err.message : err); process.exit(1); });
|
|
1832
1876
|
}
|
|
1833
1877
|
if (require.main === module) {
|
|
1834
1878
|
const { Command } = require('commander');
|
|
@@ -1836,13 +1880,18 @@ if (require.main === module) {
|
|
|
1836
1880
|
program
|
|
1837
1881
|
.name('doc2vec')
|
|
1838
1882
|
.description('Crawl documentation sources and store embeddings in vector databases')
|
|
1839
|
-
// Legacy invocation: `doc2vec [config.yaml]` runs a one-shot sync
|
|
1883
|
+
// Legacy invocation: `doc2vec [config.yaml]` runs a one-shot sync.
|
|
1884
|
+
// The source-filter flags live only on the `run` subcommand: declaring
|
|
1885
|
+
// them here too would swallow them at the program level before the
|
|
1886
|
+
// subcommand's parser ever sees them.
|
|
1840
1887
|
.argument('[config]', 'path to a YAML config file', 'config.yaml')
|
|
1841
1888
|
.action((configPath) => runOneShot(configPath));
|
|
1842
1889
|
program
|
|
1843
1890
|
.command('run <config>')
|
|
1844
1891
|
.description('Run a one-shot sync for a config file (what the controller spawns)')
|
|
1845
|
-
.
|
|
1892
|
+
.option('--source <product_name>', 'only sync sources with this product_name (repeatable)', (value, previous) => [...previous, value], [])
|
|
1893
|
+
.option('--source-index <n>', 'only sync the source at this 0-based position in the config (repeatable, exact)', (value, previous) => [...previous, parseInt(value, 10)], [])
|
|
1894
|
+
.action((configPath, options) => runOneShot(configPath, { names: options.source, indices: options.sourceIndex }));
|
|
1846
1895
|
program
|
|
1847
1896
|
.command('controller [configs...]')
|
|
1848
1897
|
.description('Run as a long-lived controller: schedule sync jobs per config file, persist runs in Postgres, serve a web UI')
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
@layer properties{@supports (((-webkit-hyphens:none)) and (not (margin-trim:inline))) or ((-moz-orient:inline) and (not (color:rgb(from red r g b)))){*,:before,:after,::backdrop{--tw-space-y-reverse:0;--tw-border-style:solid;--tw-leading:initial;--tw-font-weight:initial;--tw-tracking:initial;--tw-ordinal:initial;--tw-slashed-zero:initial;--tw-numeric-figure:initial;--tw-numeric-spacing:initial;--tw-numeric-fraction:initial;--tw-shadow:0 0 #0000;--tw-shadow-color:initial;--tw-shadow-alpha:100%;--tw-inset-shadow:0 0 #0000;--tw-inset-shadow-color:initial;--tw-inset-shadow-alpha:100%;--tw-ring-color:initial;--tw-ring-shadow:0 0 #0000;--tw-inset-ring-color:initial;--tw-inset-ring-shadow:0 0 #0000;--tw-ring-inset:initial;--tw-ring-offset-width:0px;--tw-ring-offset-color:#fff;--tw-ring-offset-shadow:0 0 #0000;--tw-blur:initial;--tw-brightness:initial;--tw-contrast:initial;--tw-grayscale:initial;--tw-hue-rotate:initial;--tw-invert:initial;--tw-opacity:initial;--tw-saturate:initial;--tw-sepia:initial;--tw-drop-shadow:initial;--tw-drop-shadow-color:initial;--tw-drop-shadow-alpha:100%;--tw-drop-shadow-size:initial}}}@layer theme{:root,:host{--font-mono:ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace;--color-white:#fff;--spacing:.25rem;--container-xs:20rem;--container-md:28rem;--container-2xl:42rem;--container-6xl:72rem;--text-xs:.75rem;--text-xs--line-height:calc(1 / .75);--text-sm:.875rem;--text-sm--line-height:calc(1.25 / .875);--text-lg:1.125rem;--text-lg--line-height:calc(1.75 / 1.125);--text-xl:1.25rem;--text-xl--line-height:calc(1.75 / 1.25);--text-2xl:1.5rem;--text-2xl--line-height:calc(2 / 1.5);--font-weight-normal:400;--font-weight-medium:500;--font-weight-semibold:600;--tracking-tight:-.025em;--tracking-wide:.025em;--radius-md:.375rem;--radius-lg:.5rem;--animate-pulse:pulse 2s cubic-bezier(.4, 0, .6, 1) infinite;--default-transition-duration:.15s;--default-transition-timing-function:cubic-bezier(.4, 0, .2, 1);--default-font-family:system-ui, -apple-system, "Segoe UI", sans-serif;--default-mono-font-family:var(--font-mono)}}@layer base{*,:after,:before,::backdrop{box-sizing:border-box;border:0 solid;margin:0;padding:0}::file-selector-button{box-sizing:border-box;border:0 solid;margin:0;padding:0}html,:host{-webkit-text-size-adjust:100%;tab-size:4;line-height:1.5;font-family:var(--default-font-family,ui-sans-serif, system-ui, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji");font-feature-settings:var(--default-font-feature-settings,normal);font-variation-settings:var(--default-font-variation-settings,normal);-webkit-tap-highlight-color:transparent}hr{height:0;color:inherit;border-top-width:1px}abbr:where([title]){-webkit-text-decoration:underline dotted;text-decoration:underline dotted}h1,h2,h3,h4,h5,h6{font-size:inherit;font-weight:inherit}a{color:inherit;-webkit-text-decoration:inherit;text-decoration:inherit}b,strong{font-weight:bolder}code,kbd,samp,pre{font-family:var(--default-mono-font-family,ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace);font-feature-settings:var(--default-mono-font-feature-settings,normal);font-variation-settings:var(--default-mono-font-variation-settings,normal);font-size:1em}small{font-size:80%}sub,sup{vertical-align:baseline;font-size:75%;line-height:0;position:relative}sub{bottom:-.25em}sup{top:-.5em}table{text-indent:0;border-color:inherit;border-collapse:collapse}:-moz-focusring{outline:auto}progress{vertical-align:baseline}summary{display:list-item}ol,ul,menu{list-style:none}img,svg,video,canvas,audio,iframe,embed,object{vertical-align:middle;display:block}img,video{max-width:100%;height:auto}button,input,select,optgroup,textarea{font:inherit;font-feature-settings:inherit;font-variation-settings:inherit;letter-spacing:inherit;color:inherit;opacity:1;background-color:#0000;border-radius:0}::file-selector-button{font:inherit;font-feature-settings:inherit;font-variation-settings:inherit;letter-spacing:inherit;color:inherit;opacity:1;background-color:#0000;border-radius:0}:where(select:is([multiple],[size])) optgroup{font-weight:bolder}:where(select:is([multiple],[size])) optgroup option{padding-inline-start:20px}::file-selector-button{margin-inline-end:4px}::placeholder{opacity:1}@supports (not ((-webkit-appearance:-apple-pay-button))) or (contain-intrinsic-size:1px){::placeholder{color:currentColor}@supports (color:color-mix(in lab,red,red)){::placeholder{color:color-mix(in oklab,currentcolor 50%,transparent)}}}textarea{resize:vertical}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-date-and-time-value{min-height:1lh;text-align:inherit}::-webkit-datetime-edit{display:inline-flex}::-webkit-datetime-edit-fields-wrapper{padding:0}::-webkit-datetime-edit{padding-block:0}::-webkit-datetime-edit-year-field{padding-block:0}::-webkit-datetime-edit-month-field{padding-block:0}::-webkit-datetime-edit-day-field{padding-block:0}::-webkit-datetime-edit-hour-field{padding-block:0}::-webkit-datetime-edit-minute-field{padding-block:0}::-webkit-datetime-edit-second-field{padding-block:0}::-webkit-datetime-edit-millisecond-field{padding-block:0}::-webkit-datetime-edit-meridiem-field{padding-block:0}::-webkit-calendar-picker-indicator{line-height:1}:-moz-ui-invalid{box-shadow:none}button,input:where([type=button],[type=reset],[type=submit]){appearance:button}::file-selector-button{appearance:button}::-webkit-inner-spin-button{height:auto}::-webkit-outer-spin-button{height:auto}[hidden]:where(:not([hidden=until-found])){display:none!important}}@layer components;@layer utilities{.visible{visibility:visible}.absolute{position:absolute}.fixed{position:fixed}.relative{position:relative}.inset-0{inset:0}.top-full{top:100%}.right-0{right:0}.z-10{z-index:10}.z-20{z-index:20}.mx-auto{margin-inline:auto}.mt-0\.5{margin-top:calc(var(--spacing) * .5)}.mt-1{margin-top:var(--spacing)}.mt-1\.5{margin-top:calc(var(--spacing) * 1.5)}.mt-2{margin-top:calc(var(--spacing) * 2)}.mt-3{margin-top:calc(var(--spacing) * 3)}.mt-4{margin-top:calc(var(--spacing) * 4)}.-mb-px{margin-bottom:-1px}.mb-1{margin-bottom:var(--spacing)}.mb-2{margin-bottom:calc(var(--spacing) * 2)}.mb-3{margin-bottom:calc(var(--spacing) * 3)}.mb-4{margin-bottom:calc(var(--spacing) * 4)}.ml-1{margin-left:var(--spacing)}.ml-1\.5{margin-left:calc(var(--spacing) * 1.5)}.ml-2{margin-left:calc(var(--spacing) * 2)}.ml-auto{margin-left:auto}.block{display:block}.flex{display:flex}.grid{display:grid}.hidden{display:none}.inline-block{display:inline-block}.inline-flex{display:inline-flex}.table{display:table}.h-2{height:calc(var(--spacing) * 2)}.h-4{height:calc(var(--spacing) * 4)}.max-h-64{max-height:calc(var(--spacing) * 64)}.max-h-96{max-height:calc(var(--spacing) * 96)}.max-h-\[560px\]{max-height:560px}.min-h-screen{min-height:100vh}.w-2{width:calc(var(--spacing) * 2)}.w-4{width:calc(var(--spacing) * 4)}.w-10{width:calc(var(--spacing) * 10)}.w-44{width:calc(var(--spacing) * 44)}.w-72{width:calc(var(--spacing) * 72)}.w-80{width:calc(var(--spacing) * 80)}.w-full{width:100%}.max-w-2xl{max-width:var(--container-2xl)}.max-w-6xl{max-width:var(--container-6xl)}.max-w-md{max-width:var(--container-md)}.max-w-xs{max-width:var(--container-xs)}.min-w-0{min-width:0}.shrink-0{flex-shrink:0}.animate-pulse{animation:var(--animate-pulse)}.cursor-pointer{cursor:pointer}.grid-cols-2{grid-template-columns:repeat(2,minmax(0,1fr))}.flex-col{flex-direction:column}.flex-wrap{flex-wrap:wrap}.items-baseline{align-items:baseline}.items-center{align-items:center}.justify-between{justify-content:space-between}.gap-1{gap:var(--spacing)}.gap-1\.5{gap:calc(var(--spacing) * 1.5)}.gap-2{gap:calc(var(--spacing) * 2)}.gap-3{gap:calc(var(--spacing) * 3)}.gap-4{gap:calc(var(--spacing) * 4)}.gap-6{gap:calc(var(--spacing) * 6)}:where(.space-y-2>:not(:last-child)){--tw-space-y-reverse:0;margin-block-start:calc(calc(var(--spacing) * 2) * var(--tw-space-y-reverse));margin-block-end:calc(calc(var(--spacing) * 2) * calc(1 - var(--tw-space-y-reverse)))}:where(.space-y-3>:not(:last-child)){--tw-space-y-reverse:0;margin-block-start:calc(calc(var(--spacing) * 3) * var(--tw-space-y-reverse));margin-block-end:calc(calc(var(--spacing) * 3) * calc(1 - var(--tw-space-y-reverse)))}:where(.space-y-4>:not(:last-child)){--tw-space-y-reverse:0;margin-block-start:calc(calc(var(--spacing) * 4) * var(--tw-space-y-reverse));margin-block-end:calc(calc(var(--spacing) * 4) * calc(1 - var(--tw-space-y-reverse)))}:where(.space-y-6>:not(:last-child)){--tw-space-y-reverse:0;margin-block-start:calc(calc(var(--spacing) * 6) * var(--tw-space-y-reverse));margin-block-end:calc(calc(var(--spacing) * 6) * calc(1 - var(--tw-space-y-reverse)))}.gap-x-4{column-gap:calc(var(--spacing) * 4)}.gap-x-8{column-gap:calc(var(--spacing) * 8)}.gap-y-1{row-gap:var(--spacing)}.gap-y-2{row-gap:calc(var(--spacing) * 2)}.gap-y-3{row-gap:calc(var(--spacing) * 3)}.truncate{text-overflow:ellipsis;white-space:nowrap;overflow:hidden}.overflow-auto{overflow:auto}.overflow-hidden{overflow:hidden}.overflow-x-auto{overflow-x:auto}.overflow-y-auto{overflow-y:auto}.rounded{border-radius:.25rem}.rounded-full{border-radius:3.40282e38px}.rounded-lg{border-radius:var(--radius-lg)}.rounded-md{border-radius:var(--radius-md)}.rounded-l-md{border-top-left-radius:var(--radius-md);border-bottom-left-radius:var(--radius-md)}.rounded-r-md{border-top-right-radius:var(--radius-md);border-bottom-right-radius:var(--radius-md)}.border{border-style:var(--tw-border-style);border-width:1px}.border-t{border-top-style:var(--tw-border-style);border-top-width:1px}.border-b{border-bottom-style:var(--tw-border-style);border-bottom-width:1px}.border-b-2{border-bottom-style:var(--tw-border-style);border-bottom-width:2px}.border-l{border-left-style:var(--tw-border-style);border-left-width:1px}.border-l-0{border-left-style:var(--tw-border-style);border-left-width:0}.border-dashed{--tw-border-style:dashed;border-style:dashed}.border-accent,.border-accent\/30{border-color:var(--accent)}@supports (color:color-mix(in lab,red,red)){.border-accent\/30{border-color:color-mix(in oklab,var(--accent) 30%,transparent)}}.border-accent\/40{border-color:var(--accent)}@supports (color:color-mix(in lab,red,red)){.border-accent\/40{border-color:color-mix(in oklab,var(--accent) 40%,transparent)}}.border-critical\/40{border-color:var(--status-critical)}@supports (color:color-mix(in lab,red,red)){.border-critical\/40{border-color:color-mix(in oklab,var(--status-critical) 40%,transparent)}}.border-critical\/50{border-color:var(--status-critical)}@supports (color:color-mix(in lab,red,red)){.border-critical\/50{border-color:color-mix(in oklab,var(--status-critical) 50%,transparent)}}.border-current{border-color:currentColor}.border-edge,.border-edge\/60{border-color:var(--border)}@supports (color:color-mix(in lab,red,red)){.border-edge\/60{border-color:color-mix(in oklab,var(--border) 60%,transparent)}}.border-good\/40{border-color:var(--status-good)}@supports (color:color-mix(in lab,red,red)){.border-good\/40{border-color:color-mix(in oklab,var(--status-good) 40%,transparent)}}.border-transparent{border-color:#0000}.border-warning\/40{border-color:var(--status-warning)}@supports (color:color-mix(in lab,red,red)){.border-warning\/40{border-color:color-mix(in oklab,var(--status-warning) 40%,transparent)}}.border-warning\/50{border-color:var(--status-warning)}@supports (color:color-mix(in lab,red,red)){.border-warning\/50{border-color:color-mix(in oklab,var(--status-warning) 50%,transparent)}}.border-white\/25{border-color:#ffffff40}@supports (color:color-mix(in lab,red,red)){.border-white\/25{border-color:color-mix(in oklab,var(--color-white) 25%,transparent)}}.bg-accent{background-color:var(--accent)}.bg-edge\/10{background-color:var(--border)}@supports (color:color-mix(in lab,red,red)){.bg-edge\/10{background-color:color-mix(in oklab,var(--border) 10%,transparent)}}.bg-page{background-color:var(--page)}.bg-surface{background-color:var(--surface-1)}.p-3{padding:calc(var(--spacing) * 3)}.p-4{padding:calc(var(--spacing) * 4)}.px-1{padding-inline:var(--spacing)}.px-1\.5{padding-inline:calc(var(--spacing) * 1.5)}.px-2{padding-inline:calc(var(--spacing) * 2)}.px-2\.5{padding-inline:calc(var(--spacing) * 2.5)}.px-3{padding-inline:calc(var(--spacing) * 3)}.px-3\.5{padding-inline:calc(var(--spacing) * 3.5)}.px-4{padding-inline:calc(var(--spacing) * 4)}.px-6{padding-inline:calc(var(--spacing) * 6)}.py-0\.5{padding-block:calc(var(--spacing) * .5)}.py-1{padding-block:var(--spacing)}.py-1\.5{padding-block:calc(var(--spacing) * 1.5)}.py-2{padding-block:calc(var(--spacing) * 2)}.py-2\.5{padding-block:calc(var(--spacing) * 2.5)}.py-3{padding-block:calc(var(--spacing) * 3)}.py-4{padding-block:calc(var(--spacing) * 4)}.py-8{padding-block:calc(var(--spacing) * 8)}.py-10{padding-block:calc(var(--spacing) * 10)}.pr-4{padding-right:calc(var(--spacing) * 4)}.pb-2{padding-bottom:calc(var(--spacing) * 2)}.text-center{text-align:center}.text-left{text-align:left}.text-right{text-align:right}.align-middle{vertical-align:middle}.font-mono{font-family:var(--font-mono)}.text-2xl{font-size:var(--text-2xl);line-height:var(--tw-leading,var(--text-2xl--line-height))}.text-lg{font-size:var(--text-lg);line-height:var(--tw-leading,var(--text-lg--line-height))}.text-sm{font-size:var(--text-sm);line-height:var(--tw-leading,var(--text-sm--line-height))}.text-xl{font-size:var(--text-xl);line-height:var(--tw-leading,var(--text-xl--line-height))}.text-xs{font-size:var(--text-xs);line-height:var(--tw-leading,var(--text-xs--line-height))}.text-\[10px\]{font-size:10px}.leading-5{--tw-leading:calc(var(--spacing) * 5);line-height:calc(var(--spacing) * 5)}.font-medium{--tw-font-weight:var(--font-weight-medium);font-weight:var(--font-weight-medium)}.font-normal{--tw-font-weight:var(--font-weight-normal);font-weight:var(--font-weight-normal)}.font-semibold{--tw-font-weight:var(--font-weight-semibold);font-weight:var(--font-weight-semibold)}.tracking-tight{--tw-tracking:var(--tracking-tight);letter-spacing:var(--tracking-tight)}.tracking-wide{--tw-tracking:var(--tracking-wide);letter-spacing:var(--tracking-wide)}.break-all{word-break:break-all}.whitespace-nowrap{white-space:nowrap}.whitespace-pre-wrap{white-space:pre-wrap}.text-accent{color:var(--accent)}.text-critical{color:var(--status-critical)}.text-good-text{color:var(--good-text)}.text-ink{color:var(--text-primary)}.text-ink-muted{color:var(--text-muted)}.text-ink-secondary{color:var(--text-secondary)}.text-warning{color:var(--status-warning)}.text-white{color:var(--color-white)}.capitalize{text-transform:capitalize}.uppercase{text-transform:uppercase}.tabular-nums{--tw-numeric-spacing:tabular-nums;font-variant-numeric:var(--tw-ordinal,) var(--tw-slashed-zero,) var(--tw-numeric-figure,) var(--tw-numeric-spacing,) var(--tw-numeric-fraction,)}.underline-offset-2{text-underline-offset:2px}.accent-\(--accent\),.accent-accent{accent-color:var(--accent)}.opacity-30{opacity:.3}.shadow-lg{--tw-shadow:0 10px 15px -3px var(--tw-shadow-color,#0000001a), 0 4px 6px -4px var(--tw-shadow-color,#0000001a);box-shadow:var(--tw-inset-shadow),var(--tw-inset-ring-shadow),var(--tw-ring-offset-shadow),var(--tw-ring-shadow),var(--tw-shadow)}.shadow-sm{--tw-shadow:0 1px 3px 0 var(--tw-shadow-color,#0000001a), 0 1px 2px -1px var(--tw-shadow-color,#0000001a);box-shadow:var(--tw-inset-shadow),var(--tw-inset-ring-shadow),var(--tw-ring-offset-shadow),var(--tw-ring-shadow),var(--tw-shadow)}.filter{filter:var(--tw-blur,) var(--tw-brightness,) var(--tw-contrast,) var(--tw-grayscale,) var(--tw-hue-rotate,) var(--tw-invert,) var(--tw-saturate,) var(--tw-sepia,) var(--tw-drop-shadow,)}.transition{transition-property:color,background-color,border-color,outline-color,text-decoration-color,fill,stroke,--tw-gradient-from,--tw-gradient-via,--tw-gradient-to,opacity,box-shadow,transform,translate,scale,rotate,filter,-webkit-backdrop-filter,backdrop-filter,display,content-visibility,overlay,pointer-events;transition-timing-function:var(--tw-ease,var(--default-transition-timing-function));transition-duration:var(--tw-duration,var(--default-transition-duration))}.outline-none{--tw-outline-style:none;outline-style:none}.select-none{-webkit-user-select:none;user-select:none}.placeholder\:text-ink-muted::placeholder,.placeholder\:text-ink-muted\/60::placeholder{color:var(--text-muted)}@supports (color:color-mix(in lab,red,red)){.placeholder\:text-ink-muted\/60::placeholder{color:color-mix(in oklab,var(--text-muted) 60%,transparent)}}.last\:border-0:last-child{border-style:var(--tw-border-style);border-width:0}@media(hover:hover){.hover\:border-accent:hover,.hover\:border-accent\/50:hover{border-color:var(--accent)}@supports (color:color-mix(in lab,red,red)){.hover\:border-accent\/50:hover{border-color:color-mix(in oklab,var(--accent) 50%,transparent)}}.hover\:bg-critical\/10:hover{background-color:var(--status-critical)}@supports (color:color-mix(in lab,red,red)){.hover\:bg-critical\/10:hover{background-color:color-mix(in oklab,var(--status-critical) 10%,transparent)}}.hover\:bg-edge\/20:hover{background-color:var(--border)}@supports (color:color-mix(in lab,red,red)){.hover\:bg-edge\/20:hover{background-color:color-mix(in oklab,var(--border) 20%,transparent)}}.hover\:bg-ink\/\[0\.03\]:hover{background-color:var(--text-primary)}@supports (color:color-mix(in lab,red,red)){.hover\:bg-ink\/\[0\.03\]:hover{background-color:color-mix(in oklab,var(--text-primary) 3%,transparent)}}.hover\:text-accent:hover{color:var(--accent)}.hover\:text-critical:hover{color:var(--status-critical)}.hover\:text-ink-secondary:hover{color:var(--text-secondary)}.hover\:underline:hover{text-decoration-line:underline}.hover\:opacity-90:hover{opacity:.9}}.focus\:border-accent:focus{border-color:var(--accent)}.focus\:outline-none:focus{--tw-outline-style:none;outline-style:none}.disabled\:cursor-not-allowed:disabled{cursor:not-allowed}.disabled\:opacity-40:disabled{opacity:.4}.disabled\:opacity-50:disabled{opacity:.5}@media(min-width:40rem){.sm\:col-span-2{grid-column:span 2/span 2}.sm\:grid-cols-2{grid-template-columns:repeat(2,minmax(0,1fr))}.sm\:grid-cols-3{grid-template-columns:repeat(3,minmax(0,1fr))}}@media(min-width:48rem){.md\:grid{display:grid}.md\:grid-cols-4{grid-template-columns:repeat(4,minmax(0,1fr))}.md\:grid-cols-\[minmax\(9rem\,1fr\)_12rem_15rem_8rem_auto\]{grid-template-columns:minmax(9rem,1fr) 12rem 15rem 8rem auto}.md\:items-start{align-items:flex-start}.md\:gap-x-6{column-gap:calc(var(--spacing) * 6)}.md\:self-center{align-self:center}}@media(min-width:64rem){.lg\:grid-cols-6{grid-template-columns:repeat(6,minmax(0,1fr))}}}:root{--page:#f9f9f7;--surface-1:#fcfcfb;--text-primary:#0b0b0b;--text-secondary:#52514e;--text-muted:#898781;--gridline:#e1e0d9;--baseline:#c3c2b7;--border:#0b0b0b1a;--series-1:#2a78d6;--status-good:#0ca30c;--status-warning:#fab219;--status-critical:#d03b3b;--good-text:#006300;--accent:#2a78d6}@media(prefers-color-scheme:dark){:root{--page:#0d0d0d;--surface-1:#1a1a19;--text-primary:#fff;--text-secondary:#c3c2b7;--text-muted:#898781;--gridline:#2c2c2a;--baseline:#383835;--border:#ffffff1a;--series-1:#3987e5;--status-good:#0ca30c;--status-warning:#fab219;--status-critical:#d03b3b;--good-text:#0ca30c;--accent:#3987e5}}html{background:var(--page);color:var(--text-primary);font-family:system-ui,-apple-system,Segoe UI,sans-serif}.log-scroll::-webkit-scrollbar{width:10px;height:10px}.log-scroll::-webkit-scrollbar-thumb{background:var(--baseline);border-radius:5px}@property --tw-space-y-reverse{syntax:"*";inherits:false;initial-value:0}@property --tw-border-style{syntax:"*";inherits:false;initial-value:solid}@property --tw-leading{syntax:"*";inherits:false}@property --tw-font-weight{syntax:"*";inherits:false}@property --tw-tracking{syntax:"*";inherits:false}@property --tw-ordinal{syntax:"*";inherits:false}@property --tw-slashed-zero{syntax:"*";inherits:false}@property --tw-numeric-figure{syntax:"*";inherits:false}@property --tw-numeric-spacing{syntax:"*";inherits:false}@property --tw-numeric-fraction{syntax:"*";inherits:false}@property --tw-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-shadow-color{syntax:"*";inherits:false}@property --tw-shadow-alpha{syntax:"<percentage>";inherits:false;initial-value:100%}@property --tw-inset-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-inset-shadow-color{syntax:"*";inherits:false}@property --tw-inset-shadow-alpha{syntax:"<percentage>";inherits:false;initial-value:100%}@property --tw-ring-color{syntax:"*";inherits:false}@property --tw-ring-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-inset-ring-color{syntax:"*";inherits:false}@property --tw-inset-ring-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-ring-inset{syntax:"*";inherits:false}@property --tw-ring-offset-width{syntax:"<length>";inherits:false;initial-value:0}@property --tw-ring-offset-color{syntax:"*";inherits:false;initial-value:#fff}@property --tw-ring-offset-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-blur{syntax:"*";inherits:false}@property --tw-brightness{syntax:"*";inherits:false}@property --tw-contrast{syntax:"*";inherits:false}@property --tw-grayscale{syntax:"*";inherits:false}@property --tw-hue-rotate{syntax:"*";inherits:false}@property --tw-invert{syntax:"*";inherits:false}@property --tw-opacity{syntax:"*";inherits:false}@property --tw-saturate{syntax:"*";inherits:false}@property --tw-sepia{syntax:"*";inherits:false}@property --tw-drop-shadow{syntax:"*";inherits:false}@property --tw-drop-shadow-color{syntax:"*";inherits:false}@property --tw-drop-shadow-alpha{syntax:"<percentage>";inherits:false;initial-value:100%}@property --tw-drop-shadow-size{syntax:"*";inherits:false}@keyframes pulse{50%{opacity:.5}}
|