doc2vec 2.10.6 → 2.12.0

package/README.md

@@ -52,6 +52,7 @@ The primary goal is to prepare documentation content for Retrieval-Augmented Gen
 *   **Incremental Updates:** For GitHub and Zendesk sources, tracks the last run date to only fetch new or updated issues/tickets.
 *   **Cleanup:** Removes obsolete chunks from the database corresponding to pages or files that are no longer found during processing.
 *   **Configuration:** Driven by a YAML configuration file (`config.yaml`) specifying sites, repositories, local directories, Zendesk instances, database types, metadata, and other parameters.
+*   **[Controller Mode](#controller-mode):** Optionally runs as a long-lived controller that schedules sync jobs from multiple config files (cron `schedule` field), persists run history and statistics in Postgres, and serves a web UI with live log streaming — read-only (configs from files/ConfigMap) or read-write (create/edit configs from the UI).
 *   **Structured Logging:** Uses a custom logger (`logger.ts`) with levels, timestamps, colors, progress bars, and child loggers for clear execution monitoring.
 
 ## Chunk Metadata & Page Reconstruction
@@ -437,6 +438,103 @@ The script will then:
 7.  Cleanup obsolete chunks.
 8.  Output detailed logs.
 
+## Controller Mode
+
+Besides the one-shot sync, doc2vec can run as a **long-lived controller** that schedules sync jobs, records run history and statistics in Postgres, and serves a web UI for monitoring and managing configs:
+
+```bash
+doc2vec controller --database-url postgres://user:pass@host:5432/doc2vec ./configs/
+```
+
+Open http://localhost:8080 to see the dashboard: configs with their schedule and next run, run history with per-source statistics, and live log streaming for running jobs.
+
+### Scheduling
+
+Add a top-level cron `schedule` (and optionally a display `name`) to a config file, and the controller runs it automatically:
+
+```yaml
+name: product-docs
+schedule: "0 2 * * *"   # every day at 02:00
+sources:
+  - type: website
+    ...
+```
+
+Configs without a `schedule` are still listed in the UI and can be triggered manually with **Run now**. Each run executes `doc2vec run <config.yaml>` as a child process; overlapping runs of the same config are skipped, and `--max-parallel` (default 1) caps how many sync jobs run at once — keep it low, since website sources launch a headless Chromium.
+
+### Flags
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--database-url <url>` | `DATABASE_URL` env | Postgres connection string (required) |
+| `--port <n>` | `8080` (or `PORT`) | HTTP port for the API and UI |
+| `--read-write` | off (read-only) | Allow creating/editing/deleting configs from the UI |
+| `--config-dir <dir>` | — | Where UI-created configs are written (required with `--read-write`) |
+| `--max-parallel <n>` | `1` | Max concurrent sync jobs |
+| `--reload-interval <s>` | `30` | How often config files are re-read for changes |
+| `--log-retention-days <n>` | `14` | Run logs older than this are pruned (run history is kept) |
+| `--slack-webhook-url <url>` | `SLACK_WEBHOOK_URL` env | Slack incoming webhook — post a message when a run finishes |
+| `--slack-notify <mode>` | `all` | `all` or `failures` (failures also covers canceled runs) |
+| `--public-url <url>` | `PUBLIC_URL` env | Externally reachable base URL, used for "view run" links in notifications |
+
+Positional arguments are config files and/or directories (directories are scanned for `*.yaml`/`*.yml`, and re-scanned on every reload).
+
+### Slack notifications
+
+With `--slack-webhook-url` (or `SLACK_WEBHOOK_URL`) set, the controller posts to a [Slack incoming webhook](https://api.slack.com/messaging/webhooks) whenever a run reaches a terminal state — ✅ succeeded, ❌ failed (naming the failing sources and their errors), or ⚠️ canceled. Overlap-`skipped` runs are not notified. Set `--slack-notify failures` to silence successes, and `--public-url https://doc2vec.example.com` to get clickable "view run" links.
+
+### Read-only vs read-write
+
+- **Read-only** (default): configs come solely from the files passed on the command line — ideal for Kubernetes, where they live in a ConfigMap. The UI can view configs, trigger runs, and browse history/stats, but not modify anything. ConfigMap updates are picked up automatically via content-hash polling.
+- **Read-write** (`--read-write --config-dir ./configs`): the UI can also create, edit, and delete config YAML files. Files stay the source of truth on disk; concurrent edits are protected by a content-hash check.
+
+In both modes the UI always shows raw YAML: `${ENV_VAR}` secret placeholders are **never** resolved outside the sync job itself.
+
+### Kubernetes example
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: doc2vec-controller
+spec:
+  replicas: 1        # keep a single replica: the scheduler is not distributed
+  selector:
+    matchLabels: { app: doc2vec-controller }
+  template:
+    metadata:
+      labels: { app: doc2vec-controller }
+    spec:
+      containers:
+        - name: controller
+          image: ghcr.io/kagent-dev/doc2vec:latest
+          command: ["node", "dist/doc2vec.js", "controller", "/etc/doc2vec"]
+          ports:
+            - containerPort: 8080
+          env:
+            - name: DATABASE_URL
+              valueFrom:
+                secretKeyRef: { name: doc2vec-secrets, key: database-url }
+            - name: OPENAI_API_KEY
+              valueFrom:
+                secretKeyRef: { name: doc2vec-secrets, key: openai-api-key }
+          volumeMounts:
+            - name: configs
+              mountPath: /etc/doc2vec
+          resources:
+            requests: { memory: 1Gi }
+            limits: { memory: 4Gi }   # website sources launch headless Chromium
+      volumes:
+        - name: configs
+          configMap: { name: doc2vec-configs }
+```
+
+The controller detects ConfigMap updates without a restart. Health endpoint: `GET /api/health`.
+
+### API
+
+The UI is backed by a small REST API you can also use directly: `GET /api/configs`, `POST /api/configs/:id/run`, `GET /api/runs?configId=`, `GET /api/runs/:id/logs`, `POST /api/runs/:id/cancel`, `GET /api/configs/:id/stats?days=30`, plus server-sent events at `GET /api/events` and `GET /api/runs/:id/logs/stream`.
+
 ## Database Options
 
 ### SQLite (`database_config.type: 'sqlite'`)

package/dist/content-processor.js

@@ -36,7 +36,8 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
     return (mod && mod.__esModule) ? mod : { "default": mod };
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.ContentProcessor = void 0;
+exports.ContentProcessor = exports.BrowserLaunchError = void 0;
+exports.resolveBrowserExecutablePath = resolveBrowserExecutablePath;
 const readability_1 = require("@mozilla/readability");
 const axios_1 = __importDefault(require("axios"));
 const cheerio_1 = require("cheerio");
@@ -48,6 +49,97 @@ const fs = __importStar(require("fs"));
 const path = __importStar(require("path"));
 const utils_1 = require("./utils");
 const code_chunker_1 = require("./code-chunker");
+/**
+ * Thrown when the browser cannot be launched at all (even after a retry).
+ * Without a browser no page in the crawl can be processed, so this aborts the
+ * whole website source — the run is then reported as failed instead of
+ * "succeeding" with an error on every URL.
+ */
+class BrowserLaunchError extends Error {
+    constructor(message) {
+        super(message);
+        this.name = 'BrowserLaunchError';
+    }
+}
+exports.BrowserLaunchError = BrowserLaunchError;
+/**
+ * Resolve which browser binary to launch, in order of preference:
+ *  1. PUPPETEER_EXECUTABLE_PATH (explicit override)
+ *  2. Puppeteer's own downloaded Chrome for Testing (version-matched, the
+ *     reliable choice — Debian's chromium package has shipped builds that
+ *     crash at startup in containers, e.g. 150.0.7871 SIGTRAPs immediately)
+ *  3. A system chromium, as the fallback (also covers linux/arm64, where
+ *     Chrome for Testing is not published)
+ *  4. undefined — let puppeteer decide
+ */
+function resolveBrowserExecutablePath() {
+    const override = process.env.PUPPETEER_EXECUTABLE_PATH;
+    if (override)
+        return override;
+    try {
+        const bundled = puppeteer_1.default.executablePath();
+        if (bundled && fs.existsSync(bundled))
+            return bundled;
+    }
+    catch {
+        // no browser downloaded for this platform
+    }
+    if (fs.existsSync('/usr/bin/chromium'))
+        return '/usr/bin/chromium';
+    if (fs.existsSync('/usr/bin/chromium-browser'))
+        return '/usr/bin/chromium-browser';
+    return undefined;
+}
+/**
+ * Best-effort snapshot of process + container memory, for diagnosing browser
+ * launch failures (the usual cause in Kubernetes is the pod's memory cgroup —
+ * Chromium forks, then its renderers are immediately OOM-killed).
+ */
+function memoryDiagnostics() {
+    const gib = (bytes) => `${(bytes / 1024 ** 3).toFixed(2)}GiB`;
+    const parts = [`node rss ${gib(process.memoryUsage().rss)}`];
+    try {
+        // cgroup v2, else v1 — absent entirely outside Linux/containers
+        let current;
+        let max;
+        if (fs.existsSync('/sys/fs/cgroup/memory.current')) {
+            current = fs.readFileSync('/sys/fs/cgroup/memory.current', 'utf8').trim();
+            max = fs.readFileSync('/sys/fs/cgroup/memory.max', 'utf8').trim();
+        }
+        else if (fs.existsSync('/sys/fs/cgroup/memory/memory.usage_in_bytes')) {
+            current = fs.readFileSync('/sys/fs/cgroup/memory/memory.usage_in_bytes', 'utf8').trim();
+            max = fs.readFileSync('/sys/fs/cgroup/memory/memory.limit_in_bytes', 'utf8').trim();
+        }
+        if (current !== undefined && max !== undefined) {
+            const unlimited = max === 'max' || Number(max) > 2 ** 60;
+            parts.push(`cgroup ${gib(Number(current))} used / ${unlimited ? 'no limit' : gib(Number(max))}`);
+        }
+    }
+    catch {
+        // diagnostics only — never fail because of them
+    }
+    try {
+        // /proc/meminfo reflects the NODE in Kubernetes — catches host-level pressure
+        const meminfo = fs.readFileSync('/proc/meminfo', 'utf8');
+        const available = meminfo.match(/^MemAvailable:\s+(\d+) kB/m);
+        if (available) {
+            parts.push(`node available ${gib(Number(available[1]) * 1024)}`);
+        }
+    }
+    catch {
+        // not Linux
+    }
+    try {
+        // Chromium writes its profile and (with --disable-dev-shm-usage) its
+        // shared memory under /tmp — a full disk breaks launches
+        const stat = fs.statfsSync('/tmp');
+        parts.push(`/tmp free ${gib(stat.bavail * stat.bsize)}`);
+    }
+    catch {
+        // statfs unavailable
+    }
+    return parts.join(', ');
+}
 class ContentProcessor {
     constructor(logger) {
         this.logger = logger;
@@ -329,16 +421,8 @@ class ContentProcessor {
         let browser = null;
         let page = null;
         const launchBrowser = async () => {
-            let executablePath = process.env.PUPPETEER_EXECUTABLE_PATH;
-            if (!executablePath) {
-                if (fs.existsSync('/usr/bin/chromium')) {
-                    executablePath = '/usr/bin/chromium';
-                }
-                else if (fs.existsSync('/usr/bin/chromium-browser')) {
-                    executablePath = '/usr/bin/chromium-browser';
-                }
-            }
-            const b = await puppeteer_1.default.launch({
+            const executablePath = resolveBrowserExecutablePath();
+            const launchOptions = {
                 executablePath,
                 args: [
                     '--no-sandbox',
@@ -348,7 +432,25 @@ class ContentProcessor {
                     '--disable-extensions',
                 ],
                 protocolTimeout: 60000,
-            });
+            };
+            let b;
+            try {
+                b = await puppeteer_1.default.launch(launchOptions);
+            }
+            catch (firstError) {
+                // One retry after a pause — the failure may be transient (e.g.
+                // memory pressure from a page that just closed). If the browser
+                // cannot launch at all, no page in this crawl can be processed,
+                // so escalate as fatal instead of failing every URL one by one.
+                logger.warn(`Browser launch failed (${memoryDiagnostics()}), retrying once in 5s...`, firstError);
+                await new Promise(resolve => setTimeout(resolve, 5000));
+                try {
+                    b = await puppeteer_1.default.launch(launchOptions);
+                }
+                catch (secondError) {
+                    throw new BrowserLaunchError(`browser failed to launch twice (${memoryDiagnostics()}): ${secondError instanceof Error ? secondError.message : String(secondError)}`);
+                }
+            }
             const p = await b.newPage();
             return { browser: b, page: p };
         };
@@ -682,6 +784,13 @@ class ContentProcessor {
                     }
                 }
                 catch (error) {
+                    // A browser that cannot launch is fatal for the whole crawl —
+                    // rethrow so the source (and the run) is reported as failed
+                    // rather than logging an error per URL and finishing "green".
+                    if (error instanceof BrowserLaunchError) {
+                        logger.error(`Aborting crawl of ${baseUrl}: ${error.message}`);
+                        throw error;
+                    }
                     const status = this.getHttpStatus(error);
                     // ── Handle 429 (Too Many Requests) with retry ──
                     if (status === 429) {
@@ -846,17 +955,8 @@ class ContentProcessor {
             }
             else {
                 // Standalone mode: launch a browser for this single page
-                let executablePath = process.env.PUPPETEER_EXECUTABLE_PATH;
-                if (!executablePath) {
-                    if (fs.existsSync('/usr/bin/chromium')) {
-                        executablePath = '/usr/bin/chromium';
-                    }
-                    else if (fs.existsSync('/usr/bin/chromium-browser')) {
-                        executablePath = '/usr/bin/chromium-browser';
-                    }
-                }
                 browser = await puppeteer_1.default.launch({
-                    executablePath,
+                    executablePath: resolveBrowserExecutablePath(),
                     args: [
                         '--no-sandbox',
                         '--disable-setuid-sandbox',

package/dist/controller/chunk-inspector.js

@@ -0,0 +1,209 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || (function () {
+    var ownKeys = function(o) {
+        ownKeys = Object.getOwnPropertyNames || function (o) {
+            var ar = [];
+            for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
+            return ar;
+        };
+        return ownKeys(o);
+    };
+    return function (mod) {
+        if (mod && mod.__esModule) return mod;
+        var result = {};
+        if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
+        __setModuleDefault(result, mod);
+        return result;
+    };
+})();
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.lookupChunks = lookupChunks;
+const better_sqlite3_1 = __importDefault(require("better-sqlite3"));
+const path = __importStar(require("path"));
+const sqliteVec = __importStar(require("sqlite-vec"));
+const yaml = __importStar(require("js-yaml"));
+const js_client_rest_1 = require("@qdrant/js-client-rest");
+const types_1 = require("./types");
+const MAX_CHUNKS = 500;
+/**
+ * Read-only lookup of the chunks stored for a URL in a source's vector store.
+ * Resolves the store the same way a sync job would (same defaults as
+ * DatabaseManager.initDatabase, same ${ENV_VAR} substitution as Doc2Vec's
+ * config loader), but never creates tables or collections.
+ */
+async function lookupChunks(configContent, selector, url) {
+    const source = findSource(configContent, selector);
+    const dbConfig = source.database_config ?? {};
+    const params = dbConfig.params ?? {};
+    if (dbConfig.type === 'sqlite') {
+        const dbPath = params.db_path
+            || path.join(process.cwd(), `${String(source.product_name).replace(/\s+/g, '_')}-${source.version}.db`);
+        return {
+            source: { product_name: source.product_name, type: source.type, version: source.version },
+            database: { type: 'sqlite', path: dbPath },
+            chunks: lookupSqlite(dbPath, url),
+        };
+    }
+    if (dbConfig.type === 'qdrant') {
+        const qdrantUrl = params.qdrant_url || 'http://localhost:6333';
+        const qdrantPort = params.qdrant_port || 443;
+        const collectionName = params.collection_name
+            || `${String(source.product_name).toLowerCase().replace(/\s+/g, '_')}_${source.version}`;
+        const client = new js_client_rest_1.QdrantClient({ url: qdrantUrl, apiKey: process.env.QDRANT_API_KEY, port: qdrantPort });
+        return {
+            source: { product_name: source.product_name, type: source.type, version: source.version },
+            database: { type: 'qdrant', url: qdrantUrl, collection: collectionName },
+            chunks: await lookupQdrant(client, collectionName, url),
+        };
+    }
+    throw new types_1.ValidationError(`source has unsupported database type: ${dbConfig.type}`);
+}
+function findSource(configContent, selector) {
+    // Same ${VAR} substitution the sync job applies when it loads the config
+    const substituted = configContent.replace(/\$\{([^}]+)\}/g, (match, varName) => process.env[varName] !== undefined ? process.env[varName] : match);
+    let parsed;
+    try {
+        parsed = yaml.load(substituted);
+    }
+    catch (err) {
+        throw new types_1.ValidationError(`config does not parse: ${err instanceof Error ? err.message : String(err)}`);
+    }
+    const sources = Array.isArray(parsed?.sources) ? parsed.sources : [];
+    const source = sources.find(s => s?.product_name === selector.product_name
+        && (selector.type === undefined || s?.type === selector.type)
+        && (selector.version === undefined || String(effectiveVersion(s)) === selector.version));
+    if (!source) {
+        throw new types_1.NotFoundError(`source '${selector.product_name}' not found in config`);
+    }
+    return { ...source, version: effectiveVersion(source) };
+}
+// Mirrors the version defaulting in Doc2Vec.loadConfig for code sources
+function effectiveVersion(source) {
+    if (source?.version && String(source.version).trim().length > 0)
+        return String(source.version);
+    if (source?.type === 'code') {
+        if (source.branch && String(source.branch).trim().length > 0)
+            return String(source.branch);
+        return 'local';
+    }
+    return String(source?.version ?? '');
+}
+function lookupSqlite(dbPath, url) {
+    let db;
+    try {
+        db = new better_sqlite3_1.default(dbPath, { readonly: true, fileMustExist: true, allowExtension: true });
+    }
+    catch {
+        throw new types_1.NotFoundError(`SQLite database not found at ${dbPath} (has this source ever been synced?)`);
+    }
+    try {
+        sqliteVec.load(db);
+        const hasDates = db
+            .prepare(`SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'vec_chunk_dates'`)
+            .get() !== undefined;
+        const rows = db.prepare(`
+            SELECT v.chunk_id, v.url, v.product_name, v.version, v.section, v.heading_hierarchy,
+                   v.chunk_index, v.total_chunks, v.hash, v.content
+                   ${hasDates ? ', d.created_at AS created_at' : ', NULL AS created_at'}
+            FROM vec_items v
+            ${hasDates ? 'LEFT JOIN vec_chunk_dates d ON d.chunk_id = v.chunk_id' : ''}
+            WHERE v.url = ?
+            LIMIT ${MAX_CHUNKS}
+        `).all(url);
+        return sortChunks(rows.map(row => ({
+            chunk_id: String(row.chunk_id),
+            url: String(row.url),
+            product_name: row.product_name ?? null,
+            version: row.version ?? null,
+            section: row.section ?? null,
+            heading_hierarchy: parseHierarchy(row.heading_hierarchy),
+            chunk_index: row.chunk_index === null || row.chunk_index === undefined ? null : Number(row.chunk_index),
+            total_chunks: row.total_chunks === null || row.total_chunks === undefined ? null : Number(row.total_chunks),
+            hash: row.hash ?? null,
+            created_at: row.created_at ?? null,
+            content: String(row.content ?? ''),
+        })));
+    }
+    finally {
+        db.close();
+    }
+}
+async function lookupQdrant(client, collectionName, url) {
+    const chunks = [];
+    let offset = undefined;
+    do {
+        const response = await client.scroll(collectionName, {
+            limit: 200,
+            offset,
+            with_payload: true,
+            with_vector: false,
+            filter: {
+                must: [{ key: 'url', match: { value: url } }],
+                must_not: [{ key: 'is_metadata', match: { value: true } }],
+            },
+        });
+        for (const point of response.points) {
+            const p = point.payload ?? {};
+            chunks.push({
+                chunk_id: String(p.original_chunk_id ?? point.id),
+                url: String(p.url ?? url),
+                product_name: p.product_name ?? null,
+                version: p.version ?? null,
+                section: p.section ?? null,
+                heading_hierarchy: parseHierarchy(p.heading_hierarchy),
+                chunk_index: p.chunk_index === null || p.chunk_index === undefined ? null : Number(p.chunk_index),
+                total_chunks: p.total_chunks === null || p.total_chunks === undefined ? null : Number(p.total_chunks),
+                hash: p.hash ?? null,
+                created_at: p.created_at ?? null,
+                content: String(p.content ?? ''),
+            });
+            if (chunks.length >= MAX_CHUNKS)
+                return sortChunks(chunks);
+        }
+        offset = response.next_page_offset;
+    } while (offset !== null && offset !== undefined);
+    return sortChunks(chunks);
+}
+function parseHierarchy(value) {
+    if (Array.isArray(value))
+        return value.map(String);
+    if (typeof value === 'string' && value.trim().startsWith('[')) {
+        try {
+            const parsed = JSON.parse(value);
+            if (Array.isArray(parsed))
+                return parsed.map(String);
+        }
+        catch { /* fall through */ }
+    }
+    return value ? [String(value)] : [];
+}
+// Default order: position within the page. The UI offers per-column sorting
+// (including by creation date) on top of this.
+function sortChunks(chunks) {
+    return chunks.sort((a, b) => {
+        const ai = a.chunk_index ?? Number.MAX_SAFE_INTEGER;
+        const bi = b.chunk_index ?? Number.MAX_SAFE_INTEGER;
+        if (ai !== bi)
+            return ai - bi;
+        return a.chunk_id.localeCompare(b.chunk_id);
+    });
+}

package/dist/controller/server.js

@@ -40,6 +40,7 @@ exports.createServer = createServer;
 const express_1 = __importDefault(require("express"));
 const fs = __importStar(require("fs"));
 const path = __importStar(require("path"));
+const chunk_inspector_1 = require("./chunk-inspector");
 const config_registry_1 = require("./config-registry");
 const types_1 = require("./types");
 const SSE_HEARTBEAT_MS = 15000;
@@ -151,6 +152,34 @@ function createServer(deps) {
         const days = Number(req.query.days) || 30;
         res.json(await store.getConfigStats(id, days));
     });
+    // Inspect the chunks currently stored for a URL in one source's vector store.
+    // This reads the live store, so results reflect the present state, not a
+    // historical run.
+    app.get('/api/configs/:id/chunks', async (req, res) => {
+        const config = registry.get(parseId(String(req.params.id)));
+        if (!config)
+            throw new types_1.NotFoundError('config not found');
+        const url = typeof req.query.url === 'string' ? req.query.url.trim() : '';
+        const productName = typeof req.query.product_name === 'string' ? req.query.product_name : '';
+        if (!url)
+            throw new types_1.ValidationError('url query parameter is required');
+        if (!productName)
+            throw new types_1.ValidationError('product_name query parameter is required');
+        try {
+            res.json(await (0, chunk_inspector_1.lookupChunks)(config.content, {
+                product_name: productName,
+                type: typeof req.query.type === 'string' && req.query.type ? req.query.type : undefined,
+                version: typeof req.query.version === 'string' && req.query.version ? req.query.version : undefined,
+            }, url));
+        }
+        catch (err) {
+            if (err instanceof types_1.NotFoundError || err instanceof types_1.ValidationError)
+                throw err;
+            // Vector store unreachable (e.g. Qdrant down) — surface the reason
+            logger.error('Chunk lookup failed:', err);
+            res.status(502).json({ error: `chunk lookup failed: ${err instanceof Error ? err.message : String(err)}` });
+        }
+    });
     // ------------------------------------------------------------------ runs
     app.get('/api/runs', async (req, res) => {
         res.json(await store.listRuns({