doc2vec 2.13.0 → 2.14.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -8,7 +8,7 @@ The primary goal is to prepare documentation content for Retrieval-Augmented Gen
8
8
 
9
9
  Run it two ways:
10
10
 
11
- * **One-shot sync** — `doc2vec run config.yaml` processes every source in a config file and exits. Ideal for a cron job or CI step.
11
+ * **One-shot sync** — `doc2vec run config.yaml` processes every source in a config file and exits. Ideal for a cron job or CI step. Pass `--source <product_name>` (all entries with that name) or `--source-index <n>` (a single entry, by its 0-based position in the config) to sync only selected sources; both flags are repeatable.
12
12
  * **[Controller mode](#controller-mode)** — `doc2vec controller ./configs/` stays running: it schedules each config on its own cron expression, keeps run history and per-source statistics in Postgres, and serves a web UI with live log streaming, searchable run logs, and chunk inspection. Deploy it once and manage all your sources from the browser.
13
13
 
14
14
  [![doc2vec controller dashboard](docs/images/controller-dashboard.png)](#controller-mode)
@@ -451,7 +451,7 @@ Besides the one-shot sync, doc2vec can run as a **long-lived controller** that s
451
451
  doc2vec controller --database-url postgres://user:pass@host:5432/doc2vec ./configs/
452
452
  ```
453
453
 
454
- Open http://localhost:8080 to see the dashboard ([screenshot above](#doc2vec)): every config with its schedule, next run, and the outcome of its last run, plus a **Run now** button.
454
+ Open http://localhost:8080 to see the dashboard ([screenshot above](#doc2vec)): every config with its schedule, next run, and the outcome of its last run, plus a **Run now** button. The **▾** next to it opens a source picker to sync only a subset of the config's source entries — each entry is selectable individually, even when several share a product name (e.g. istio as github + code + website) — and the run history marks such partial runs with the selected sources.
455
455
 
456
456
  Each config gets its own page with the full run history, per-source results, and the raw YAML:
457
457
 
@@ -477,7 +477,7 @@ sources:
477
477
  ...
478
478
  ```
479
479
 
480
- Configs without a `schedule` are still listed in the UI and can be triggered manually with **Run now**. Each run executes `doc2vec run <config.yaml>` as a child process; overlapping runs of the same config are skipped, and `--max-parallel` (default 1) caps how many sync jobs run at once — keep it low, since website sources launch a headless Chromium.
480
+ Configs without a `schedule` are still listed in the UI and can be triggered manually with **Run now** (optionally for a subset of sources via the **▾** picker). Each run executes `doc2vec run <config.yaml>` as a child process (with `--source-index <n>` flags when a subset was selected); overlapping runs of the same config are skipped, and `--max-parallel` (default 1) caps how many sync jobs run at once — keep it low, since website sources launch a headless Chromium.
481
481
 
482
482
  ### Logs
483
483
 
@@ -767,6 +767,12 @@ If you don't specify a config path, it will look for config.yaml in the current
767
767
 
768
768
  ## Recent Changes
769
769
 
770
+ ### Code Scans: Dangling Symlinks and Incomplete Walks
771
+ - **Dangling symlinks no longer abort a directory:** `statSync` follows symlinks, so a link whose target isn't in the clone (common in vendored forks, e.g. `crates/cel-fork/cel/README.md`) threw `ENOENT` and abandoned every remaining entry in that directory. Broken entries are now skipped with a warning and the walk continues
772
+ - **Incomplete scans never delete chunks:** if a directory genuinely can't be listed, the scan is reported as incomplete — obsolete-file cleanup, the tracked-file list, the last-mtime marker, and the last-synced git SHA are all skipped, so unscanned files keep their chunks and the next run rescans them
773
+ - **Incomplete scans fail the source:** a partial ingest is reported as a failed source instead of a silent success
774
+ - **Quieter crawls:** planned browser restarts (initial launch, the every-50-pages recycle) log at `info`; only unplanned restarts (disconnected or degraded browser) warn
775
+
770
776
  ### Searchable Controller Run Logs
771
777
  - **Server-side log filtering:** Level chips and the keyword filter now query Postgres instead of the browser's in-memory buffer, so errors and warnings from the start of a long run are reachable (`GET /api/runs/:id/logs?levels=error,warn&q=...`, paged with **Load 2,000 more**)
772
778
  - **Whole-run level counts:** New `GET /api/runs/:id/logs/counts` endpoint backs the level chips, which previously only counted the lines still held in the browser
@@ -65,15 +65,39 @@ class CodeChunker {
65
65
  if (!text.trim()) {
66
66
  return [];
67
67
  }
68
+ if (CodeChunker.moduleDamaged) {
69
+ throw new Error('tree-sitter module is damaged by an earlier parse failure; refusing to parse');
70
+ }
68
71
  const parser = await CodeChunker.getParser(this.lang);
69
- const originalTextBytes = Buffer.from(text, 'utf-8');
70
- const tree = parser.parse(originalTextBytes.toString());
72
+ const source = Buffer.from(text, 'utf-8').toString();
73
+ // A grammar that traps inside parse unwinds out of WASM without letting
74
+ // tree-sitter clean up, and the damage is cumulative: after enough of
75
+ // them every later parse fails with "memory access out of bounds", on
76
+ // files and languages that are perfectly fine. One clear error beats
77
+ // thousands of misleading ones, so stop using the module.
78
+ let tree;
79
+ try {
80
+ tree = parser.parse(source);
81
+ }
82
+ catch (error) {
83
+ CodeChunker.moduleDamaged = true;
84
+ throw error;
85
+ }
71
86
  if (!tree) {
72
87
  throw new Error('Failed to parse code');
73
88
  }
74
- const chunks = [];
75
- await this.recursiveChunk(tree.rootNode, originalTextBytes.toString(), chunks);
76
- return this.mergeChunks(chunks);
89
+ // The syntax tree lives in the tree-sitter WASM heap, which the JS
90
+ // garbage collector cannot reach. Without this delete the heap grows by
91
+ // the size of every tree we ever parse, until an allocation fails with
92
+ // "memory access out of bounds" on some unrelated file.
93
+ try {
94
+ const chunks = [];
95
+ await this.recursiveChunk(tree.rootNode, source, chunks);
96
+ return this.mergeChunks(chunks);
97
+ }
98
+ finally {
99
+ tree.delete();
100
+ }
77
101
  }
78
102
  static async getParser(lang) {
79
103
  const formattedLang = lang.toLowerCase().replace(/-/g, '_');
@@ -93,6 +117,7 @@ class CodeChunker {
93
117
  }
94
118
  const wasmPath = CodeChunker.resolveWasmPath(formattedLang);
95
119
  const wasmBuffer = fs.readFileSync(wasmPath);
120
+ CodeChunker.assertGrammarLinks(formattedLang, wasmBuffer);
96
121
  const language = await web_tree_sitter_1.Language.load(wasmBuffer);
97
122
  const parser = new web_tree_sitter_1.Parser();
98
123
  parser.setLanguage(language);
@@ -101,6 +126,110 @@ class CodeChunker {
101
126
  this.parserCache.set(formattedLang, parserPromise);
102
127
  return parserPromise;
103
128
  }
129
+ /**
130
+ * Refuse a grammar whose imports the tree-sitter runtime cannot satisfy.
131
+ *
132
+ * The grammars ship separately from web-tree-sitter, so a grammar built
133
+ * against a different runtime can import a C symbol this runtime does not
134
+ * export. Emscripten binds side-module imports lazily, so the mismatch
135
+ * stays invisible until the grammar calls the symbol mid-parse — then it
136
+ * throws out of WASM and damages the shared module for every language.
137
+ * Checking the import table up front turns that into a clean load failure,
138
+ * and chunkCode falls back to token chunking as it does for any other
139
+ * unsupported language.
140
+ */
141
+ static assertGrammarLinks(formattedLang, wasmBuffer) {
142
+ if (!CodeChunker.hostExports) {
143
+ const hostWasm = path.join(path.dirname(require.resolve('web-tree-sitter')), 'tree-sitter.wasm');
144
+ CodeChunker.hostExports = CodeChunker.readWasmTables(fs.readFileSync(hostWasm)).exports;
145
+ }
146
+ // Emscripten resolves an import from the host's exports first, then
147
+ // from the side module's own exports.
148
+ const grammar = CodeChunker.readWasmTables(wasmBuffer);
149
+ const unresolved = grammar.functionImports.filter(name => !CodeChunker.hostExports.has(name) &&
150
+ !grammar.exports.has(name) &&
151
+ !CodeChunker.ABORT_ONLY_SYMBOLS.has(name));
152
+ if (unresolved.length > 0) {
153
+ throw new Error(`Tree-sitter grammar "${formattedLang}" is incompatible with the installed ` +
154
+ `web-tree-sitter runtime: unresolved imports ${unresolved.join(', ')}.`);
155
+ }
156
+ }
157
+ /**
158
+ * Read a module's function imports and its export names straight out of the
159
+ * WASM binary.
160
+ *
161
+ * WebAssembly.Module.imports() would be shorter, but building a
162
+ * WebAssembly.Module compiles the whole binary, and doing that on top of
163
+ * the compile Language.load already does exhausts V8's compiler zone once a
164
+ * process loads a dozen grammars. Walking the two sections costs nothing.
165
+ */
166
+ static readWasmTables(buffer) {
167
+ const functionImports = [];
168
+ const exports = new Set();
169
+ let offset = 8; // magic number and version
170
+ const readVarUint = () => {
171
+ let result = 0;
172
+ let shift = 0;
173
+ let byte;
174
+ do {
175
+ byte = buffer[offset++];
176
+ result |= (byte & 0x7f) << shift;
177
+ shift += 7;
178
+ } while (byte & 0x80);
179
+ return result >>> 0;
180
+ };
181
+ const readName = () => {
182
+ const length = readVarUint();
183
+ const name = buffer.toString('utf8', offset, offset + length);
184
+ offset += length;
185
+ return name;
186
+ };
187
+ const skipLimits = () => {
188
+ const flags = readVarUint();
189
+ readVarUint(); // minimum
190
+ if (flags & 0x01)
191
+ readVarUint(); // maximum
192
+ };
193
+ while (offset < buffer.length) {
194
+ const sectionId = readVarUint();
195
+ const sectionSize = readVarUint();
196
+ const sectionEnd = offset + sectionSize;
197
+ if (sectionId === 2) { // import section
198
+ const count = readVarUint();
199
+ for (let i = 0; i < count; i++) {
200
+ readName(); // module
201
+ const field = readName();
202
+ const kind = buffer[offset++];
203
+ switch (kind) {
204
+ case 0x00: // function
205
+ readVarUint(); // type index
206
+ functionImports.push(field);
207
+ break;
208
+ case 0x01: // table
209
+ offset++; // element type
210
+ skipLimits();
211
+ break;
212
+ case 0x02: // memory
213
+ skipLimits();
214
+ break;
215
+ default: // global: value type + mutability
216
+ offset += 2;
217
+ break;
218
+ }
219
+ }
220
+ }
221
+ else if (sectionId === 7) { // export section
222
+ const count = readVarUint();
223
+ for (let i = 0; i < count; i++) {
224
+ exports.add(readName());
225
+ offset++; // kind
226
+ readVarUint(); // index
227
+ }
228
+ }
229
+ offset = sectionEnd;
230
+ }
231
+ return { functionImports, exports };
232
+ }
104
233
  static resolveWasmPath(formattedLang) {
105
234
  const nodeModulesPath = CodeChunker.findNearestNodeModules(__dirname);
106
235
  if (!nodeModulesPath) {
@@ -177,3 +306,15 @@ class CodeChunker {
177
306
  exports.CodeChunker = CodeChunker;
178
307
  CodeChunker.treeSitterInitialized = false;
179
308
  CodeChunker.parserCache = new Map();
309
+ CodeChunker.hostExports = null;
310
+ CodeChunker.moduleDamaged = false;
311
+ /**
312
+ * Symbols a grammar only calls when it is already aborting. Emscripten
313
+ * resolves side-module imports lazily, so an import that never gets called
314
+ * costs nothing. Ignoring these two keeps python, html, cpp, ruby, php and
315
+ * vue usable — every one of them imports __assert_fail and none of them
316
+ * ever calls it. Any other unresolved import is a normal-operation symbol
317
+ * (bash calls isalpha on real scripts, yaml calls operator new), and those
318
+ * are the ones that bring the whole module down.
319
+ */
320
+ CodeChunker.ABORT_ONLY_SYMBOLS = new Set(['__assert_fail', 'abort']);
@@ -462,8 +462,14 @@ class ContentProcessor {
462
462
  let pagesSinceRestart = 0;
463
463
  // Restart the browser every N pages to prevent memory accumulation
464
464
  const PAGES_BETWEEN_RESTARTS = 50;
465
- const restartBrowser = async (reason) => {
466
- logger.warn(`Restarting browser: ${reason}`);
465
+ // Planned restarts (first launch, the every-N-pages recycle) are routine
466
+ // and would otherwise dominate a crawl's warning count; only unplanned
467
+ // ones — a disconnected or degraded browser — deserve a warning.
468
+ const restartBrowser = async (reason, planned = false) => {
469
+ if (planned)
470
+ logger.info(`Restarting browser: ${reason}`);
471
+ else
472
+ logger.warn(`Restarting browser: ${reason}`);
467
473
  if (browser) {
468
474
  try {
469
475
  await browser.close();
@@ -485,11 +491,11 @@ class ContentProcessor {
485
491
  const reason = needsBrowserRestart
486
492
  ? `protocol errors persisting after page recreation (${consecutiveProtocolErrors} consecutive)`
487
493
  : browser ? 'browser disconnected' : 'initial launch';
488
- await restartBrowser(reason);
494
+ await restartBrowser(reason, !browser && !needsBrowserRestart);
489
495
  }
490
496
  else if (pagesSinceRestart >= PAGES_BETWEEN_RESTARTS) {
491
497
  // Periodic restart to prevent memory accumulation from long crawls
492
- await restartBrowser(`periodic restart after ${pagesSinceRestart} pages`);
498
+ await restartBrowser(`periodic restart after ${pagesSinceRestart} pages`, true);
493
499
  }
494
500
  else if (pageNeedsRecreation) {
495
501
  // The previous page.evaluate or navigation timed out / errored.
@@ -1539,7 +1545,17 @@ class ContentProcessor {
1539
1545
  let skippedFiles = 0;
1540
1546
  for (const file of files) {
1541
1547
  const filePath = path.join(dirPath, file);
1542
- const stat = fs.statSync(filePath);
1548
+ // A dangling symlink makes statSync throw ENOENT; skip the
1549
+ // entry rather than abandoning the rest of the directory
1550
+ let stat;
1551
+ try {
1552
+ stat = fs.statSync(filePath);
1553
+ }
1554
+ catch (error) {
1555
+ logger.warn(`Skipping unreadable entry ${filePath}:`, error);
1556
+ skippedFiles++;
1557
+ continue;
1558
+ }
1543
1559
  // Skip already visited paths
1544
1560
  if (visitedPaths.has(filePath)) {
1545
1561
  logger.debug(`Skipping already visited path: ${filePath}`);
@@ -1629,6 +1645,7 @@ class ContentProcessor {
1629
1645
  const excludeExtensions = config.exclude_extensions || [];
1630
1646
  const encoding = config.encoding || 'utf8';
1631
1647
  let maxMtime = 0;
1648
+ let incomplete = false;
1632
1649
  try {
1633
1650
  const files = fs.readdirSync(dirPath);
1634
1651
  let processedFiles = 0;
@@ -1638,7 +1655,19 @@ class ContentProcessor {
1638
1655
  const trackFiles = options?.trackFiles;
1639
1656
  for (const file of files) {
1640
1657
  const filePath = path.join(dirPath, file);
1641
- const stat = fs.statSync(filePath);
1658
+ // statSync follows symlinks, so a dangling one (common in
1659
+ // vendored forks) throws ENOENT. Skipping the entry keeps the
1660
+ // rest of the directory — letting it throw used to abandon
1661
+ // every sibling after it, and cleanup then deleted their chunks.
1662
+ let stat;
1663
+ try {
1664
+ stat = fs.statSync(filePath);
1665
+ }
1666
+ catch (error) {
1667
+ logger.warn(`Skipping unreadable entry ${filePath}:`, error);
1668
+ skippedFiles++;
1669
+ continue;
1670
+ }
1642
1671
  if (visitedPaths.has(filePath)) {
1643
1672
  logger.debug(`Skipping already visited path: ${filePath}`);
1644
1673
  continue;
@@ -1650,6 +1679,7 @@ class ContentProcessor {
1650
1679
  processedFiles += childResult.processedFiles;
1651
1680
  skippedFiles += childResult.skippedFiles;
1652
1681
  maxMtime = Math.max(maxMtime, childResult.maxMtime);
1682
+ incomplete = incomplete || childResult.incomplete;
1653
1683
  }
1654
1684
  else {
1655
1685
  logger.debug(`Skipping directory ${filePath} (recursive=false)`);
@@ -1694,11 +1724,14 @@ class ContentProcessor {
1694
1724
  }
1695
1725
  }
1696
1726
  logger.info(`Code directory processed. Processed: ${processedFiles}, Skipped: ${skippedFiles}`);
1697
- return { processedFiles, skippedFiles, maxMtime };
1727
+ return { processedFiles, skippedFiles, maxMtime, incomplete };
1698
1728
  }
1699
1729
  catch (error) {
1730
+ // The directory could not be listed at all — report the scan as
1731
+ // incomplete so the caller skips cleanup instead of treating every
1732
+ // file under this path as deleted
1700
1733
  logger.error(`Error reading code directory ${dirPath}:`, error);
1701
- return { processedFiles: 0, skippedFiles: 0, maxMtime };
1734
+ return { processedFiles: 0, skippedFiles: 0, maxMtime, incomplete: true };
1702
1735
  }
1703
1736
  }
1704
1737
  async getTokenChunker(chunkSize) {
@@ -66,7 +66,7 @@ class JobRunner {
66
66
  runningCount() {
67
67
  return this.active.size;
68
68
  }
69
- async enqueue(config, trigger) {
69
+ async enqueue(config, trigger, sources) {
70
70
  if (this.shuttingDown) {
71
71
  throw new types_1.ConflictError('controller is shutting down');
72
72
  }
@@ -83,9 +83,9 @@ class JobRunner {
83
83
  if (config.parse_error) {
84
84
  throw new types_1.ConflictError(`config is invalid: ${config.parse_error}`);
85
85
  }
86
- const run = await this.store.createRun(config.id, config.content_hash, trigger, 'queued');
86
+ const run = await this.store.createRun(config.id, config.content_hash, trigger, 'queued', undefined, sources);
87
87
  this.queuedConfigs.add(config.id);
88
- this.queue.push({ runId: run.id, configId: config.id, configPath: config.path });
88
+ this.queue.push({ runId: run.id, configId: config.id, configPath: config.path, sources });
89
89
  this.events.emitRunUpdate(run);
90
90
  this.pump();
91
91
  return run;
@@ -117,6 +117,9 @@ class JobRunner {
117
117
  }
118
118
  async start(job) {
119
119
  const { cmd, args } = this.commandFor(job.configPath);
120
+ for (const source of job.sources ?? []) {
121
+ args.push('--source-index', String(source.index));
122
+ }
120
123
  this.logger.info(`Starting run ${job.runId}: ${cmd} ${args.join(' ')}`);
121
124
  const child = (0, child_process_1.spawn)(cmd, args, {
122
125
  env: { ...process.env, DOC2VEC_STRUCTURED_LOGS: '1' },
@@ -57,4 +57,9 @@ exports.MIGRATIONS = [
57
57
  `
58
58
  CREATE INDEX IF NOT EXISTS d2v_run_logs_level_idx ON d2v_run_logs (run_id, level, seq);
59
59
  `,
60
+ // 3: manual runs can target a subset of a config's source entries (stored as
61
+ // [{index, product_name, type, version}]); NULL means the run covers all sources
62
+ `
63
+ ALTER TABLE d2v_runs ADD COLUMN IF NOT EXISTS requested_sources JSONB;
64
+ `,
60
65
  ];
@@ -152,7 +152,31 @@ function createServer(deps) {
152
152
  const config = registry.get(parseId(String(req.params.id)));
153
153
  if (!config)
154
154
  throw new types_1.NotFoundError('config not found');
155
- const run = await runner.enqueue(config, 'manual');
155
+ // Optional source selection: 0-based positions in the config's sources
156
+ // list (exact — product names can repeat). Default: all sources.
157
+ let sources;
158
+ const { sources: rawSources, baseHash } = (req.body ?? {});
159
+ if (rawSources !== undefined) {
160
+ if (!Array.isArray(rawSources) || rawSources.some(i => !Number.isInteger(i))) {
161
+ throw new types_1.ValidationError('sources must be an array of source indices (integers)');
162
+ }
163
+ // Indices are only meaningful against the config revision the client
164
+ // saw — reject the run if the config changed underneath it.
165
+ if (baseHash !== undefined && baseHash !== config.content_hash) {
166
+ throw new types_1.ConflictError('config changed since the source list was loaded — reload and retry');
167
+ }
168
+ const total = config.source_summary.length;
169
+ const outOfRange = rawSources.filter((i) => i < 0 || i >= total);
170
+ if (outOfRange.length > 0) {
171
+ throw new types_1.ValidationError(`source index out of range: ${outOfRange.join(', ')} (config has ${total} sources)`);
172
+ }
173
+ // Deduplicate, keep config order; empty or full selection = full run
174
+ const indices = [...new Set(rawSources)].sort((a, b) => a - b);
175
+ if (indices.length > 0 && indices.length < total) {
176
+ sources = indices.map(index => ({ index, ...config.source_summary[index] }));
177
+ }
178
+ }
179
+ const run = await runner.enqueue(config, 'manual', sources);
156
180
  res.status(202).json(run);
157
181
  });
158
182
  app.get('/api/configs/:id/stats', async (req, res) => {
@@ -88,11 +88,11 @@ class ControllerStore {
88
88
  return rows[0] ?? null;
89
89
  }
90
90
  // ------------------------------------------------------------------ runs
91
- async createRun(configId, configHash, trigger, status, error) {
91
+ async createRun(configId, configHash, trigger, status, error, requestedSources) {
92
92
  const terminal = status === 'skipped';
93
- const { rows } = await this.pool.query(`INSERT INTO d2v_runs (config_id, config_hash, trigger, status, error, finished_at)
94
- VALUES ($1, $2, $3, $4, $5, ${terminal ? 'NOW()' : 'NULL'})
95
- RETURNING *`, [configId, configHash, trigger, status, error ?? null]);
93
+ const { rows } = await this.pool.query(`INSERT INTO d2v_runs (config_id, config_hash, trigger, status, error, requested_sources, finished_at)
94
+ VALUES ($1, $2, $3, $4, $5, $6, ${terminal ? 'NOW()' : 'NULL'})
95
+ RETURNING *`, [configId, configHash, trigger, status, error ?? null, requestedSources?.length ? JSON.stringify(requestedSources) : null]);
96
96
  return rows[0];
97
97
  }
98
98
  async markRunStarted(runId, pid) {
package/dist/doc2vec.js CHANGED
@@ -174,14 +174,40 @@ class Doc2Vec {
174
174
  }
175
175
  return parsedValue;
176
176
  }
177
- async run() {
177
+ /**
178
+ * Sync sources sequentially. An optional selection restricts the run to a
179
+ * subset of the config's sources: `indices` are 0-based positions in the
180
+ * config's sources list (exact — the only unambiguous identifier, since
181
+ * product names can repeat), and `names` match every source with that
182
+ * product_name. The union of both is processed, in config order.
183
+ */
184
+ async run(selection) {
185
+ let sources = this.config.sources;
186
+ const wantedIndices = selection?.indices ?? [];
187
+ const wantedNames = selection?.names ?? [];
188
+ if (wantedIndices.length > 0 || wantedNames.length > 0) {
189
+ const badIndices = wantedIndices.filter(i => !Number.isInteger(i) || i < 0 || i >= this.config.sources.length);
190
+ if (badIndices.length > 0) {
191
+ throw new Error(`Source index out of range: ${badIndices.join(', ')} (config has ${this.config.sources.length} sources)`);
192
+ }
193
+ const known = new Set(this.config.sources.map(s => s.product_name));
194
+ const unknown = wantedNames.filter(name => !known.has(name));
195
+ if (unknown.length > 0) {
196
+ throw new Error(`Unknown source(s): ${unknown.join(', ')}. Available: ${[...known].join(', ')}`);
197
+ }
198
+ const nameSet = new Set(wantedNames);
199
+ const indexSet = new Set(wantedIndices);
200
+ sources = this.config.sources.filter((s, i) => indexSet.has(i) || nameSet.has(s.product_name));
201
+ const label = sources.map(s => `${s.product_name} (${s.type})`).join(', ');
202
+ this.logger.info(`Source filter active: syncing ${sources.length} of ${this.config.sources.length} sources: ${label}`);
203
+ }
178
204
  // Initialize Postgres markdown store table if configured
179
205
  if (this.markdownStore) {
180
206
  await this.markdownStore.init();
181
207
  }
182
208
  this.logger.section('PROCESSING SOURCES');
183
209
  const runStats = [];
184
- for (const sourceConfig of this.config.sources) {
210
+ for (const sourceConfig of sources) {
185
211
  const sourceLogger = this.logger.child(`source:${sourceConfig.product_name}`);
186
212
  sourceLogger.info(`Processing ${sourceConfig.type} source for ${sourceConfig.product_name}@${sourceConfig.version}`);
187
213
  const startTime = Date.now();
@@ -1051,6 +1077,10 @@ class Doc2Vec {
1051
1077
  let lastMtimeKey;
1052
1078
  let trackedFiles;
1053
1079
  let maxObservedMtime = 0;
1080
+ // Set when a directory could not be listed: the files we saw are only a
1081
+ // subset of what is on disk, so nothing may be treated as deleted and no
1082
+ // "last scanned" marker may advance past this run
1083
+ let scanIncomplete = false;
1054
1084
  if (config.source === 'local_directory') {
1055
1085
  if (!config.path) {
1056
1086
  logger.error('Code source type local_directory requires a path.');
@@ -1132,13 +1162,20 @@ class Doc2Vec {
1132
1162
  mtimeCutoff,
1133
1163
  trackFiles: trackedFiles
1134
1164
  });
1165
+ scanIncomplete = scanResult.incomplete;
1166
+ if (scanIncomplete) {
1167
+ logger.error('Directory scan was incomplete — skipping cleanup so unscanned files keep their chunks');
1168
+ }
1135
1169
  if (trackedFiles) {
1136
1170
  maxObservedMtime = scanResult.maxMtime;
1137
1171
  }
1138
1172
  }
1139
1173
  finally {
1140
1174
  logger.section('CLEANUP');
1141
- if (incrementalMode) {
1175
+ if (scanIncomplete) {
1176
+ logger.warn('Skipping cleanup and last-scanned markers: the scan did not cover the whole tree');
1177
+ }
1178
+ else if (incrementalMode) {
1142
1179
  if (deleteUrls.length > 0) {
1143
1180
  logger.info(`Cleaning up ${deleteUrls.length} deleted/renamed files`);
1144
1181
  for (const url of deleteUrls) {
@@ -1197,7 +1234,9 @@ class Doc2Vec {
1197
1234
  this.counters.items_deleted += removed.items;
1198
1235
  this.counters.chunks_deleted += removed.chunks;
1199
1236
  }
1200
- if (config.source === 'github' && basePath && repoBranch) {
1237
+ // Storing the SHA after an incomplete scan would make the next run
1238
+ // diff from it and never revisit the files this run missed
1239
+ if (config.source === 'github' && basePath && repoBranch && !scanIncomplete) {
1201
1240
  const headSha = await this.getRepoHeadSha(basePath, logger);
1202
1241
  if (headSha) {
1203
1242
  const shaKey = this.buildCodeShaMetadataKey(config.repo, repoBranch);
@@ -1214,6 +1253,11 @@ class Doc2Vec {
1214
1253
  }
1215
1254
  }
1216
1255
  }
1256
+ // Fail the source rather than reporting a partial ingest as a success —
1257
+ // the collection is still serving whatever it had for the unscanned files
1258
+ if (scanIncomplete) {
1259
+ throw new Error(`Code scan of ${basePath} was incomplete: at least one directory could not be read`);
1260
+ }
1217
1261
  logger.info(`Finished processing code source (${config.source})`);
1218
1262
  }
1219
1263
  buildCodeShaMetadataKey(repo, branch) {
@@ -1820,15 +1864,15 @@ exports.Doc2Vec = Doc2Vec;
1820
1864
  Doc2Vec.MAX_EMBEDDING_TOKENS = 8191;
1821
1865
  Doc2Vec.CHARS_PER_TOKEN = 4;
1822
1866
  Doc2Vec.MAX_EMBEDDING_CHARS = Doc2Vec.MAX_EMBEDDING_TOKENS * Doc2Vec.CHARS_PER_TOKEN;
1823
- function runOneShot(configPath) {
1867
+ function runOneShot(configPath, selection) {
1824
1868
  if (!fs.existsSync(configPath)) {
1825
1869
  console.error('Please provide a valid path to a YAML config file.');
1826
1870
  process.exit(1);
1827
1871
  }
1828
1872
  const doc2Vec = new Doc2Vec(configPath);
1829
- doc2Vec.run()
1873
+ doc2Vec.run(selection)
1830
1874
  .then((stats) => process.exit(stats.some(s => !s.ok) ? 1 : 0))
1831
- .catch((err) => { console.error(err); process.exit(1); });
1875
+ .catch((err) => { console.error(err instanceof Error ? err.message : err); process.exit(1); });
1832
1876
  }
1833
1877
  if (require.main === module) {
1834
1878
  const { Command } = require('commander');
@@ -1836,13 +1880,18 @@ if (require.main === module) {
1836
1880
  program
1837
1881
  .name('doc2vec')
1838
1882
  .description('Crawl documentation sources and store embeddings in vector databases')
1839
- // Legacy invocation: `doc2vec [config.yaml]` runs a one-shot sync
1883
+ // Legacy invocation: `doc2vec [config.yaml]` runs a one-shot sync.
1884
+ // The source-filter flags live only on the `run` subcommand: declaring
1885
+ // them here too would swallow them at the program level before the
1886
+ // subcommand's parser ever sees them.
1840
1887
  .argument('[config]', 'path to a YAML config file', 'config.yaml')
1841
1888
  .action((configPath) => runOneShot(configPath));
1842
1889
  program
1843
1890
  .command('run <config>')
1844
1891
  .description('Run a one-shot sync for a config file (what the controller spawns)')
1845
- .action((configPath) => runOneShot(configPath));
1892
+ .option('--source <product_name>', 'only sync sources with this product_name (repeatable)', (value, previous) => [...previous, value], [])
1893
+ .option('--source-index <n>', 'only sync the source at this 0-based position in the config (repeatable, exact)', (value, previous) => [...previous, parseInt(value, 10)], [])
1894
+ .action((configPath, options) => runOneShot(configPath, { names: options.source, indices: options.sourceIndex }));
1846
1895
  program
1847
1896
  .command('controller [configs...]')
1848
1897
  .description('Run as a long-lived controller: schedule sync jobs per config file, persist runs in Postgres, serve a web UI')
@@ -0,0 +1 @@
1
+ @layer properties{@supports (((-webkit-hyphens:none)) and (not (margin-trim:inline))) or ((-moz-orient:inline) and (not (color:rgb(from red r g b)))){*,:before,:after,::backdrop{--tw-space-y-reverse:0;--tw-border-style:solid;--tw-leading:initial;--tw-font-weight:initial;--tw-tracking:initial;--tw-ordinal:initial;--tw-slashed-zero:initial;--tw-numeric-figure:initial;--tw-numeric-spacing:initial;--tw-numeric-fraction:initial;--tw-shadow:0 0 #0000;--tw-shadow-color:initial;--tw-shadow-alpha:100%;--tw-inset-shadow:0 0 #0000;--tw-inset-shadow-color:initial;--tw-inset-shadow-alpha:100%;--tw-ring-color:initial;--tw-ring-shadow:0 0 #0000;--tw-inset-ring-color:initial;--tw-inset-ring-shadow:0 0 #0000;--tw-ring-inset:initial;--tw-ring-offset-width:0px;--tw-ring-offset-color:#fff;--tw-ring-offset-shadow:0 0 #0000;--tw-blur:initial;--tw-brightness:initial;--tw-contrast:initial;--tw-grayscale:initial;--tw-hue-rotate:initial;--tw-invert:initial;--tw-opacity:initial;--tw-saturate:initial;--tw-sepia:initial;--tw-drop-shadow:initial;--tw-drop-shadow-color:initial;--tw-drop-shadow-alpha:100%;--tw-drop-shadow-size:initial}}}@layer theme{:root,:host{--font-mono:ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace;--color-white:#fff;--spacing:.25rem;--container-xs:20rem;--container-md:28rem;--container-2xl:42rem;--container-6xl:72rem;--text-xs:.75rem;--text-xs--line-height:calc(1 / .75);--text-sm:.875rem;--text-sm--line-height:calc(1.25 / .875);--text-lg:1.125rem;--text-lg--line-height:calc(1.75 / 1.125);--text-xl:1.25rem;--text-xl--line-height:calc(1.75 / 1.25);--text-2xl:1.5rem;--text-2xl--line-height:calc(2 / 1.5);--font-weight-normal:400;--font-weight-medium:500;--font-weight-semibold:600;--tracking-tight:-.025em;--tracking-wide:.025em;--radius-md:.375rem;--radius-lg:.5rem;--animate-pulse:pulse 2s cubic-bezier(.4, 0, .6, 1) infinite;--default-transition-duration:.15s;--default-transition-timing-function:cubic-bezier(.4, 0, .2, 1);--default-font-family:system-ui, -apple-system, "Segoe UI", sans-serif;--default-mono-font-family:var(--font-mono)}}@layer base{*,:after,:before,::backdrop{box-sizing:border-box;border:0 solid;margin:0;padding:0}::file-selector-button{box-sizing:border-box;border:0 solid;margin:0;padding:0}html,:host{-webkit-text-size-adjust:100%;tab-size:4;line-height:1.5;font-family:var(--default-font-family,ui-sans-serif, system-ui, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji");font-feature-settings:var(--default-font-feature-settings,normal);font-variation-settings:var(--default-font-variation-settings,normal);-webkit-tap-highlight-color:transparent}hr{height:0;color:inherit;border-top-width:1px}abbr:where([title]){-webkit-text-decoration:underline dotted;text-decoration:underline dotted}h1,h2,h3,h4,h5,h6{font-size:inherit;font-weight:inherit}a{color:inherit;-webkit-text-decoration:inherit;text-decoration:inherit}b,strong{font-weight:bolder}code,kbd,samp,pre{font-family:var(--default-mono-font-family,ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace);font-feature-settings:var(--default-mono-font-feature-settings,normal);font-variation-settings:var(--default-mono-font-variation-settings,normal);font-size:1em}small{font-size:80%}sub,sup{vertical-align:baseline;font-size:75%;line-height:0;position:relative}sub{bottom:-.25em}sup{top:-.5em}table{text-indent:0;border-color:inherit;border-collapse:collapse}:-moz-focusring{outline:auto}progress{vertical-align:baseline}summary{display:list-item}ol,ul,menu{list-style:none}img,svg,video,canvas,audio,iframe,embed,object{vertical-align:middle;display:block}img,video{max-width:100%;height:auto}button,input,select,optgroup,textarea{font:inherit;font-feature-settings:inherit;font-variation-settings:inherit;letter-spacing:inherit;color:inherit;opacity:1;background-color:#0000;border-radius:0}::file-selector-button{font:inherit;font-feature-settings:inherit;font-variation-settings:inherit;letter-spacing:inherit;color:inherit;opacity:1;background-color:#0000;border-radius:0}:where(select:is([multiple],[size])) optgroup{font-weight:bolder}:where(select:is([multiple],[size])) optgroup option{padding-inline-start:20px}::file-selector-button{margin-inline-end:4px}::placeholder{opacity:1}@supports (not ((-webkit-appearance:-apple-pay-button))) or (contain-intrinsic-size:1px){::placeholder{color:currentColor}@supports (color:color-mix(in lab,red,red)){::placeholder{color:color-mix(in oklab,currentcolor 50%,transparent)}}}textarea{resize:vertical}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-date-and-time-value{min-height:1lh;text-align:inherit}::-webkit-datetime-edit{display:inline-flex}::-webkit-datetime-edit-fields-wrapper{padding:0}::-webkit-datetime-edit{padding-block:0}::-webkit-datetime-edit-year-field{padding-block:0}::-webkit-datetime-edit-month-field{padding-block:0}::-webkit-datetime-edit-day-field{padding-block:0}::-webkit-datetime-edit-hour-field{padding-block:0}::-webkit-datetime-edit-minute-field{padding-block:0}::-webkit-datetime-edit-second-field{padding-block:0}::-webkit-datetime-edit-millisecond-field{padding-block:0}::-webkit-datetime-edit-meridiem-field{padding-block:0}::-webkit-calendar-picker-indicator{line-height:1}:-moz-ui-invalid{box-shadow:none}button,input:where([type=button],[type=reset],[type=submit]){appearance:button}::file-selector-button{appearance:button}::-webkit-inner-spin-button{height:auto}::-webkit-outer-spin-button{height:auto}[hidden]:where(:not([hidden=until-found])){display:none!important}}@layer components;@layer utilities{.visible{visibility:visible}.absolute{position:absolute}.fixed{position:fixed}.relative{position:relative}.inset-0{inset:0}.top-full{top:100%}.right-0{right:0}.z-10{z-index:10}.z-20{z-index:20}.mx-auto{margin-inline:auto}.mt-0\.5{margin-top:calc(var(--spacing) * .5)}.mt-1{margin-top:var(--spacing)}.mt-1\.5{margin-top:calc(var(--spacing) * 1.5)}.mt-2{margin-top:calc(var(--spacing) * 2)}.mt-3{margin-top:calc(var(--spacing) * 3)}.mt-4{margin-top:calc(var(--spacing) * 4)}.-mb-px{margin-bottom:-1px}.mb-1{margin-bottom:var(--spacing)}.mb-2{margin-bottom:calc(var(--spacing) * 2)}.mb-3{margin-bottom:calc(var(--spacing) * 3)}.mb-4{margin-bottom:calc(var(--spacing) * 4)}.ml-1{margin-left:var(--spacing)}.ml-1\.5{margin-left:calc(var(--spacing) * 1.5)}.ml-2{margin-left:calc(var(--spacing) * 2)}.ml-auto{margin-left:auto}.block{display:block}.flex{display:flex}.grid{display:grid}.hidden{display:none}.inline-block{display:inline-block}.inline-flex{display:inline-flex}.table{display:table}.h-2{height:calc(var(--spacing) * 2)}.h-4{height:calc(var(--spacing) * 4)}.max-h-64{max-height:calc(var(--spacing) * 64)}.max-h-96{max-height:calc(var(--spacing) * 96)}.max-h-\[560px\]{max-height:560px}.min-h-screen{min-height:100vh}.w-2{width:calc(var(--spacing) * 2)}.w-4{width:calc(var(--spacing) * 4)}.w-10{width:calc(var(--spacing) * 10)}.w-44{width:calc(var(--spacing) * 44)}.w-72{width:calc(var(--spacing) * 72)}.w-80{width:calc(var(--spacing) * 80)}.w-full{width:100%}.max-w-2xl{max-width:var(--container-2xl)}.max-w-6xl{max-width:var(--container-6xl)}.max-w-md{max-width:var(--container-md)}.max-w-xs{max-width:var(--container-xs)}.min-w-0{min-width:0}.shrink-0{flex-shrink:0}.animate-pulse{animation:var(--animate-pulse)}.cursor-pointer{cursor:pointer}.grid-cols-2{grid-template-columns:repeat(2,minmax(0,1fr))}.flex-col{flex-direction:column}.flex-wrap{flex-wrap:wrap}.items-baseline{align-items:baseline}.items-center{align-items:center}.justify-between{justify-content:space-between}.gap-1{gap:var(--spacing)}.gap-1\.5{gap:calc(var(--spacing) * 1.5)}.gap-2{gap:calc(var(--spacing) * 2)}.gap-3{gap:calc(var(--spacing) * 3)}.gap-4{gap:calc(var(--spacing) * 4)}.gap-6{gap:calc(var(--spacing) * 6)}:where(.space-y-2>:not(:last-child)){--tw-space-y-reverse:0;margin-block-start:calc(calc(var(--spacing) * 2) * var(--tw-space-y-reverse));margin-block-end:calc(calc(var(--spacing) * 2) * calc(1 - var(--tw-space-y-reverse)))}:where(.space-y-3>:not(:last-child)){--tw-space-y-reverse:0;margin-block-start:calc(calc(var(--spacing) * 3) * var(--tw-space-y-reverse));margin-block-end:calc(calc(var(--spacing) * 3) * calc(1 - var(--tw-space-y-reverse)))}:where(.space-y-4>:not(:last-child)){--tw-space-y-reverse:0;margin-block-start:calc(calc(var(--spacing) * 4) * var(--tw-space-y-reverse));margin-block-end:calc(calc(var(--spacing) * 4) * calc(1 - var(--tw-space-y-reverse)))}:where(.space-y-6>:not(:last-child)){--tw-space-y-reverse:0;margin-block-start:calc(calc(var(--spacing) * 6) * var(--tw-space-y-reverse));margin-block-end:calc(calc(var(--spacing) * 6) * calc(1 - var(--tw-space-y-reverse)))}.gap-x-4{column-gap:calc(var(--spacing) * 4)}.gap-x-8{column-gap:calc(var(--spacing) * 8)}.gap-y-1{row-gap:var(--spacing)}.gap-y-2{row-gap:calc(var(--spacing) * 2)}.gap-y-3{row-gap:calc(var(--spacing) * 3)}.truncate{text-overflow:ellipsis;white-space:nowrap;overflow:hidden}.overflow-auto{overflow:auto}.overflow-hidden{overflow:hidden}.overflow-x-auto{overflow-x:auto}.overflow-y-auto{overflow-y:auto}.rounded{border-radius:.25rem}.rounded-full{border-radius:3.40282e38px}.rounded-lg{border-radius:var(--radius-lg)}.rounded-md{border-radius:var(--radius-md)}.rounded-l-md{border-top-left-radius:var(--radius-md);border-bottom-left-radius:var(--radius-md)}.rounded-r-md{border-top-right-radius:var(--radius-md);border-bottom-right-radius:var(--radius-md)}.border{border-style:var(--tw-border-style);border-width:1px}.border-t{border-top-style:var(--tw-border-style);border-top-width:1px}.border-b{border-bottom-style:var(--tw-border-style);border-bottom-width:1px}.border-b-2{border-bottom-style:var(--tw-border-style);border-bottom-width:2px}.border-l{border-left-style:var(--tw-border-style);border-left-width:1px}.border-l-0{border-left-style:var(--tw-border-style);border-left-width:0}.border-dashed{--tw-border-style:dashed;border-style:dashed}.border-accent,.border-accent\/30{border-color:var(--accent)}@supports (color:color-mix(in lab,red,red)){.border-accent\/30{border-color:color-mix(in oklab,var(--accent) 30%,transparent)}}.border-accent\/40{border-color:var(--accent)}@supports (color:color-mix(in lab,red,red)){.border-accent\/40{border-color:color-mix(in oklab,var(--accent) 40%,transparent)}}.border-critical\/40{border-color:var(--status-critical)}@supports (color:color-mix(in lab,red,red)){.border-critical\/40{border-color:color-mix(in oklab,var(--status-critical) 40%,transparent)}}.border-critical\/50{border-color:var(--status-critical)}@supports (color:color-mix(in lab,red,red)){.border-critical\/50{border-color:color-mix(in oklab,var(--status-critical) 50%,transparent)}}.border-current{border-color:currentColor}.border-edge,.border-edge\/60{border-color:var(--border)}@supports (color:color-mix(in lab,red,red)){.border-edge\/60{border-color:color-mix(in oklab,var(--border) 60%,transparent)}}.border-good\/40{border-color:var(--status-good)}@supports (color:color-mix(in lab,red,red)){.border-good\/40{border-color:color-mix(in oklab,var(--status-good) 40%,transparent)}}.border-transparent{border-color:#0000}.border-warning\/40{border-color:var(--status-warning)}@supports (color:color-mix(in lab,red,red)){.border-warning\/40{border-color:color-mix(in oklab,var(--status-warning) 40%,transparent)}}.border-warning\/50{border-color:var(--status-warning)}@supports (color:color-mix(in lab,red,red)){.border-warning\/50{border-color:color-mix(in oklab,var(--status-warning) 50%,transparent)}}.border-white\/25{border-color:#ffffff40}@supports (color:color-mix(in lab,red,red)){.border-white\/25{border-color:color-mix(in oklab,var(--color-white) 25%,transparent)}}.bg-accent{background-color:var(--accent)}.bg-edge\/10{background-color:var(--border)}@supports (color:color-mix(in lab,red,red)){.bg-edge\/10{background-color:color-mix(in oklab,var(--border) 10%,transparent)}}.bg-page{background-color:var(--page)}.bg-surface{background-color:var(--surface-1)}.p-3{padding:calc(var(--spacing) * 3)}.p-4{padding:calc(var(--spacing) * 4)}.px-1{padding-inline:var(--spacing)}.px-1\.5{padding-inline:calc(var(--spacing) * 1.5)}.px-2{padding-inline:calc(var(--spacing) * 2)}.px-2\.5{padding-inline:calc(var(--spacing) * 2.5)}.px-3{padding-inline:calc(var(--spacing) * 3)}.px-3\.5{padding-inline:calc(var(--spacing) * 3.5)}.px-4{padding-inline:calc(var(--spacing) * 4)}.px-6{padding-inline:calc(var(--spacing) * 6)}.py-0\.5{padding-block:calc(var(--spacing) * .5)}.py-1{padding-block:var(--spacing)}.py-1\.5{padding-block:calc(var(--spacing) * 1.5)}.py-2{padding-block:calc(var(--spacing) * 2)}.py-2\.5{padding-block:calc(var(--spacing) * 2.5)}.py-3{padding-block:calc(var(--spacing) * 3)}.py-4{padding-block:calc(var(--spacing) * 4)}.py-8{padding-block:calc(var(--spacing) * 8)}.py-10{padding-block:calc(var(--spacing) * 10)}.pr-4{padding-right:calc(var(--spacing) * 4)}.pb-2{padding-bottom:calc(var(--spacing) * 2)}.text-center{text-align:center}.text-left{text-align:left}.text-right{text-align:right}.align-middle{vertical-align:middle}.font-mono{font-family:var(--font-mono)}.text-2xl{font-size:var(--text-2xl);line-height:var(--tw-leading,var(--text-2xl--line-height))}.text-lg{font-size:var(--text-lg);line-height:var(--tw-leading,var(--text-lg--line-height))}.text-sm{font-size:var(--text-sm);line-height:var(--tw-leading,var(--text-sm--line-height))}.text-xl{font-size:var(--text-xl);line-height:var(--tw-leading,var(--text-xl--line-height))}.text-xs{font-size:var(--text-xs);line-height:var(--tw-leading,var(--text-xs--line-height))}.text-\[10px\]{font-size:10px}.leading-5{--tw-leading:calc(var(--spacing) * 5);line-height:calc(var(--spacing) * 5)}.font-medium{--tw-font-weight:var(--font-weight-medium);font-weight:var(--font-weight-medium)}.font-normal{--tw-font-weight:var(--font-weight-normal);font-weight:var(--font-weight-normal)}.font-semibold{--tw-font-weight:var(--font-weight-semibold);font-weight:var(--font-weight-semibold)}.tracking-tight{--tw-tracking:var(--tracking-tight);letter-spacing:var(--tracking-tight)}.tracking-wide{--tw-tracking:var(--tracking-wide);letter-spacing:var(--tracking-wide)}.break-all{word-break:break-all}.whitespace-nowrap{white-space:nowrap}.whitespace-pre-wrap{white-space:pre-wrap}.text-accent{color:var(--accent)}.text-critical{color:var(--status-critical)}.text-good-text{color:var(--good-text)}.text-ink{color:var(--text-primary)}.text-ink-muted{color:var(--text-muted)}.text-ink-secondary{color:var(--text-secondary)}.text-warning{color:var(--status-warning)}.text-white{color:var(--color-white)}.capitalize{text-transform:capitalize}.uppercase{text-transform:uppercase}.tabular-nums{--tw-numeric-spacing:tabular-nums;font-variant-numeric:var(--tw-ordinal,) var(--tw-slashed-zero,) var(--tw-numeric-figure,) var(--tw-numeric-spacing,) var(--tw-numeric-fraction,)}.underline-offset-2{text-underline-offset:2px}.accent-\(--accent\),.accent-accent{accent-color:var(--accent)}.opacity-30{opacity:.3}.shadow-lg{--tw-shadow:0 10px 15px -3px var(--tw-shadow-color,#0000001a), 0 4px 6px -4px var(--tw-shadow-color,#0000001a);box-shadow:var(--tw-inset-shadow),var(--tw-inset-ring-shadow),var(--tw-ring-offset-shadow),var(--tw-ring-shadow),var(--tw-shadow)}.shadow-sm{--tw-shadow:0 1px 3px 0 var(--tw-shadow-color,#0000001a), 0 1px 2px -1px var(--tw-shadow-color,#0000001a);box-shadow:var(--tw-inset-shadow),var(--tw-inset-ring-shadow),var(--tw-ring-offset-shadow),var(--tw-ring-shadow),var(--tw-shadow)}.filter{filter:var(--tw-blur,) var(--tw-brightness,) var(--tw-contrast,) var(--tw-grayscale,) var(--tw-hue-rotate,) var(--tw-invert,) var(--tw-saturate,) var(--tw-sepia,) var(--tw-drop-shadow,)}.transition{transition-property:color,background-color,border-color,outline-color,text-decoration-color,fill,stroke,--tw-gradient-from,--tw-gradient-via,--tw-gradient-to,opacity,box-shadow,transform,translate,scale,rotate,filter,-webkit-backdrop-filter,backdrop-filter,display,content-visibility,overlay,pointer-events;transition-timing-function:var(--tw-ease,var(--default-transition-timing-function));transition-duration:var(--tw-duration,var(--default-transition-duration))}.outline-none{--tw-outline-style:none;outline-style:none}.select-none{-webkit-user-select:none;user-select:none}.placeholder\:text-ink-muted::placeholder,.placeholder\:text-ink-muted\/60::placeholder{color:var(--text-muted)}@supports (color:color-mix(in lab,red,red)){.placeholder\:text-ink-muted\/60::placeholder{color:color-mix(in oklab,var(--text-muted) 60%,transparent)}}.last\:border-0:last-child{border-style:var(--tw-border-style);border-width:0}@media(hover:hover){.hover\:border-accent:hover,.hover\:border-accent\/50:hover{border-color:var(--accent)}@supports (color:color-mix(in lab,red,red)){.hover\:border-accent\/50:hover{border-color:color-mix(in oklab,var(--accent) 50%,transparent)}}.hover\:bg-critical\/10:hover{background-color:var(--status-critical)}@supports (color:color-mix(in lab,red,red)){.hover\:bg-critical\/10:hover{background-color:color-mix(in oklab,var(--status-critical) 10%,transparent)}}.hover\:bg-edge\/20:hover{background-color:var(--border)}@supports (color:color-mix(in lab,red,red)){.hover\:bg-edge\/20:hover{background-color:color-mix(in oklab,var(--border) 20%,transparent)}}.hover\:bg-ink\/\[0\.03\]:hover{background-color:var(--text-primary)}@supports (color:color-mix(in lab,red,red)){.hover\:bg-ink\/\[0\.03\]:hover{background-color:color-mix(in oklab,var(--text-primary) 3%,transparent)}}.hover\:text-accent:hover{color:var(--accent)}.hover\:text-critical:hover{color:var(--status-critical)}.hover\:text-ink-secondary:hover{color:var(--text-secondary)}.hover\:underline:hover{text-decoration-line:underline}.hover\:opacity-90:hover{opacity:.9}}.focus\:border-accent:focus{border-color:var(--accent)}.focus\:outline-none:focus{--tw-outline-style:none;outline-style:none}.disabled\:cursor-not-allowed:disabled{cursor:not-allowed}.disabled\:opacity-40:disabled{opacity:.4}.disabled\:opacity-50:disabled{opacity:.5}@media(min-width:40rem){.sm\:col-span-2{grid-column:span 2/span 2}.sm\:grid-cols-2{grid-template-columns:repeat(2,minmax(0,1fr))}.sm\:grid-cols-3{grid-template-columns:repeat(3,minmax(0,1fr))}}@media(min-width:48rem){.md\:grid{display:grid}.md\:grid-cols-4{grid-template-columns:repeat(4,minmax(0,1fr))}.md\:grid-cols-\[minmax\(9rem\,1fr\)_12rem_15rem_8rem_auto\]{grid-template-columns:minmax(9rem,1fr) 12rem 15rem 8rem auto}.md\:items-start{align-items:flex-start}.md\:gap-x-6{column-gap:calc(var(--spacing) * 6)}.md\:self-center{align-self:center}}@media(min-width:64rem){.lg\:grid-cols-6{grid-template-columns:repeat(6,minmax(0,1fr))}}}:root{--page:#f9f9f7;--surface-1:#fcfcfb;--text-primary:#0b0b0b;--text-secondary:#52514e;--text-muted:#898781;--gridline:#e1e0d9;--baseline:#c3c2b7;--border:#0b0b0b1a;--series-1:#2a78d6;--status-good:#0ca30c;--status-warning:#fab219;--status-critical:#d03b3b;--good-text:#006300;--accent:#2a78d6}@media(prefers-color-scheme:dark){:root{--page:#0d0d0d;--surface-1:#1a1a19;--text-primary:#fff;--text-secondary:#c3c2b7;--text-muted:#898781;--gridline:#2c2c2a;--baseline:#383835;--border:#ffffff1a;--series-1:#3987e5;--status-good:#0ca30c;--status-warning:#fab219;--status-critical:#d03b3b;--good-text:#0ca30c;--accent:#3987e5}}html{background:var(--page);color:var(--text-primary);font-family:system-ui,-apple-system,Segoe UI,sans-serif}.log-scroll::-webkit-scrollbar{width:10px;height:10px}.log-scroll::-webkit-scrollbar-thumb{background:var(--baseline);border-radius:5px}@property --tw-space-y-reverse{syntax:"*";inherits:false;initial-value:0}@property --tw-border-style{syntax:"*";inherits:false;initial-value:solid}@property --tw-leading{syntax:"*";inherits:false}@property --tw-font-weight{syntax:"*";inherits:false}@property --tw-tracking{syntax:"*";inherits:false}@property --tw-ordinal{syntax:"*";inherits:false}@property --tw-slashed-zero{syntax:"*";inherits:false}@property --tw-numeric-figure{syntax:"*";inherits:false}@property --tw-numeric-spacing{syntax:"*";inherits:false}@property --tw-numeric-fraction{syntax:"*";inherits:false}@property --tw-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-shadow-color{syntax:"*";inherits:false}@property --tw-shadow-alpha{syntax:"<percentage>";inherits:false;initial-value:100%}@property --tw-inset-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-inset-shadow-color{syntax:"*";inherits:false}@property --tw-inset-shadow-alpha{syntax:"<percentage>";inherits:false;initial-value:100%}@property --tw-ring-color{syntax:"*";inherits:false}@property --tw-ring-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-inset-ring-color{syntax:"*";inherits:false}@property --tw-inset-ring-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-ring-inset{syntax:"*";inherits:false}@property --tw-ring-offset-width{syntax:"<length>";inherits:false;initial-value:0}@property --tw-ring-offset-color{syntax:"*";inherits:false;initial-value:#fff}@property --tw-ring-offset-shadow{syntax:"*";inherits:false;initial-value:0 0 #0000}@property --tw-blur{syntax:"*";inherits:false}@property --tw-brightness{syntax:"*";inherits:false}@property --tw-contrast{syntax:"*";inherits:false}@property --tw-grayscale{syntax:"*";inherits:false}@property --tw-hue-rotate{syntax:"*";inherits:false}@property --tw-invert{syntax:"*";inherits:false}@property --tw-opacity{syntax:"*";inherits:false}@property --tw-saturate{syntax:"*";inherits:false}@property --tw-sepia{syntax:"*";inherits:false}@property --tw-drop-shadow{syntax:"*";inherits:false}@property --tw-drop-shadow-color{syntax:"*";inherits:false}@property --tw-drop-shadow-alpha{syntax:"<percentage>";inherits:false;initial-value:100%}@property --tw-drop-shadow-size{syntax:"*";inherits:false}@keyframes pulse{50%{opacity:.5}}