kiri-mcp-server 0.9.6 → 0.9.9

package/README.md

@@ -2,7 +2,7 @@
 
 > Intelligent code context extraction for LLMs via Model Context Protocol
 
-[![Version](https://img.shields.io/badge/version-0.9.6-blue.svg)](package.json)
+[![Version](https://img.shields.io/badge/version-0.9.9-blue.svg)](package.json)
 [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.6-blue.svg)](https://www.typescriptlang.org/)
 [![MCP](https://img.shields.io/badge/MCP-Compatible-green.svg)](https://modelcontextprotocol.io/)
@@ -18,6 +18,13 @@
 - **👁️ Auto-Sync**: Watch mode automatically re-indexes when files change
 - **🛡️ Reliable**: Degrade-first architecture works without optional extensions
 - **📝 Phrase-Aware**: Recognizes compound terms (kebab-case, snake_case) for precise matching
+- **🔒 Concurrency-Safe** _(v0.9.7+)_: Per-database queues, canonicalized DuckDB paths, and bootstrap-safe locking prevent FTS rebuild conflicts and keep locks consistent across symlinks—even on first run
+
+## 🆕 What’s New in v0.9.8
+
+- First-time bootstrap now skips redundant DuckDB locking, so `kiri` / `kiri-server` can index fresh repositories without deadlocking on their own locks.
+- Added regression coverage (`tests/server/indexBootstrap.spec.ts`) to ensure repeated `ensureDatabaseIndexed` calls reuse the lock cleanly.
+- All reliability upgrades from v0.9.7 (automatic ILIKE degrade during FTS rebuilds, canonicalized DB paths, expanded verify suite) remain in effect.
 
 ## ⚙️ Prerequisites
 
@@ -104,9 +111,31 @@ For very large repositories (10,000+ files), you may need to increase the timeou
 
 > **Note**: The example shows `480` seconds (8 minutes) for very large repositories (>20,000 files). The default `240` seconds (4 minutes) is sufficient for most projects with <10,000 files.
 
-| Variable                    | Default | Description                                                                    |
-| --------------------------- | ------- | ------------------------------------------------------------------------------ |
-| `KIRI_DAEMON_READY_TIMEOUT` | `240`   | Daemon initialization timeout in seconds. Increase for very large repositories |
+| Variable                    | Default          | Description                                                                                                           |
+| --------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------- |
+| `KIRI_DAEMON_READY_TIMEOUT` | `240`            | Daemon initialization timeout in seconds. Increase for very large repositories                                        |
+| `KIRI_SOCKET_DIR`           | OS tmp directory | Directory for Unix socket fallback when repo paths are too long (e.g., `/var/run/kiri`). Keeps worktree sockets short |
+
+> **Tip**: If you encounter `listen EINVAL` on deep worktrees, set `export KIRI_SOCKET_DIR=/var/run/kiri` (or any short 0700 directory) before launching `kiri`. This fallback ships in v0.9.9+, and an explicit path keeps logs and cleanup predictable.
+
+**Dart Analysis Server Configuration:**
+
+For projects containing Dart code, KIRI uses the Dart Analysis Server to extract accurate symbol information. The following environment variables control the Dart analyzer behavior:
+
+| Variable                       | Default | Description                                                                                       |
+| ------------------------------ | ------- | ------------------------------------------------------------------------------------------------- |
+| `DART_SDK_DETECT_TIMEOUT_MS`   | `5000`  | Timeout in milliseconds for SDK detection (prevents hanging on network issues)                    |
+| `DART_ANALYSIS_MAX_CLIENTS`    | `8`     | Maximum concurrent Dart Analysis Server processes (prevents memory exhaustion on large monorepos) |
+| `DART_ANALYSIS_CLIENT_WAIT_MS` | `10000` | Max wait time in milliseconds for available analysis server slot                                  |
+| `DART_ANALYSIS_IDLE_MS`        | `60000` | Idle time in milliseconds before disposing unused analysis server (60s default)                   |
+| `DART_FILE_QUEUE_TTL_MS`       | `30000` | TTL in milliseconds for file-level request queues (prevents memory leaks)                         |
+
+**When to adjust these values:**
+
+- **Large Dart projects (>500 files)**: Increase `DART_ANALYSIS_MAX_CLIENTS` to 16 or 32
+- **Network/UNC path issues**: Decrease `DART_SDK_DETECT_TIMEOUT_MS` to 2000 for faster failure
+- **Memory constraints**: Decrease `DART_ANALYSIS_MAX_CLIENTS` to 4 and increase `DART_ANALYSIS_IDLE_MS` to 30000
+- **Monorepo with many workspaces**: Increase `DART_ANALYSIS_CLIENT_WAIT_MS` to 30000
 
 #### For Codex CLI
 
@@ -594,6 +623,45 @@ KIRI automatically filters sensitive files and masks sensitive values:
    - Medium project (1,000-10,000 files): 10-100 MB
    - Large project (>10,000 files): 100-500 MB
 
+#### DuckDB Native Binding Errors
+
+**Problem**: Error message like `Cannot find module '.../duckdb.node'` when running from a cloned repository
+
+**Root Cause**: Using `npm link` with pnpm-installed packages causes native module path resolution issues
+
+**Solutions**:
+
+1. **Use pnpm link instead of npm link**:
+
+   ```bash
+   # Remove existing npm link (if any)
+   npm unlink -g kiri-mcp-server 2>/dev/null || true
+
+   # Clean and reinstall
+   rm -rf node_modules pnpm-lock.yaml
+   pnpm install --frozen-lockfile
+
+   # Verify native binding exists
+   ls -la node_modules/.pnpm/duckdb@*/node_modules/duckdb/lib/binding/duckdb.node
+
+   # If missing, rebuild DuckDB
+   pnpm rebuild duckdb
+
+   # Build and link (use pnpm, not npm!)
+   pnpm run build
+   pnpm link --global
+   ```
+
+2. **Prerequisites for building DuckDB**:
+   - **macOS**: Install Xcode Command Line Tools: `xcode-select --install`
+   - **Node.js**: Version 20 or higher: `node -v`
+   - **Network**: Access to `npm.duckdb.org` for prebuilt binaries
+
+3. **Unlink when done**:
+   ```bash
+   pnpm unlink --global kiri-mcp-server
+   ```
+
 ### Getting Help
 
 If you encounter issues not covered here:
@@ -612,8 +680,18 @@ KIRI provides AST-based symbol extraction for the following languages:
 | **TypeScript** | `.ts`, `.tsx` | `class`, `interface`, `enum`, `function`, `method`                                       | TypeScript Compiler API             |
 | **Swift**      | `.swift`      | `class`, `struct`, `protocol`, `enum`, `extension`, `func`, `init`, `property`           | tree-sitter-swift                   |
 | **PHP**        | `.php`        | `class`, `interface`, `trait`, `function`, `method`, `property`, `constant`, `namespace` | tree-sitter-php (pure & HTML-mixed) |
+| **Java**       | `.java`       | `class`, `interface`, `enum`, `annotation`, `method`, `constructor`, `field`             | tree-sitter-java                    |
+| **Dart**       | `.dart`       | `class`, `mixin`, `enum`, `extension`, `function`, `method`, `getter`, `setter`          | Dart Analysis Server                |
+
+**Dart Integration Features:**
+
+- Full IDE-quality symbol extraction via official Dart Analysis Server
+- Automatic SDK detection from PATH or `DART_SDK` environment variable
+- Memory-efficient client pooling with configurable limits
+- Windows path normalization for case-insensitive filesystems
+- Graceful degradation when Dart SDK is unavailable
 
-Other languages are detected and indexed but use full-file snippets instead of symbol-level extraction. Support for additional languages (Rust, Go, Python, Java, etc.) is planned.
+Other languages are detected and indexed but use full-file snippets instead of symbol-level extraction. Support for additional languages (Rust, Go, Python, etc.) is planned.
 
 ## 🏗️ How It Works
 
@@ -685,14 +763,23 @@ pnpm install
 # Build
 pnpm run build
 
-# Link globally for testing
-npm link
+# Link globally for testing (IMPORTANT: use pnpm link, not npm link)
+pnpm link --global
+
+# Verify DuckDB native binding is installed
+ls -la node_modules/.pnpm/duckdb@*/node_modules/duckdb/lib/binding/duckdb.node
+
+# If duckdb.node is missing, rebuild it
+pnpm rebuild duckdb
 
 # Run tests
 pnpm run test
 
 # Start in development mode (HTTP server on :8765)
 pnpm run dev
+
+# Unlink when done
+pnpm unlink --global kiri-mcp-server
 ```
 
 ### Commands Reference

package/dist/client/cli.js

@@ -0,0 +1,68 @@
+#!/usr/bin/env node
+import process from "node:process";
+import { bootstrapServer } from "../server/bootstrap.js";
+import { evaluateSecurityStatus, updateSecurityLock } from "../shared/security/config.js";
+function printUsage() {
+    console.info(`Usage: pnpm exec tsx src/client/cli.ts <command> [options]\n`);
+    console.info(`Commands:`);
+    console.info(`  security verify [--write-lock]    Verify security baseline matches lock file`);
+}
+function formatStatus() {
+    const status = evaluateSecurityStatus();
+    const lockInfo = status.lockHash ? `hash=${status.lockHash}` : "missing";
+    const matchState = status.matches ? "MATCH" : "MISMATCH";
+    return [
+        `config: ${status.configPath}`,
+        `lock: ${status.lockPath} (${lockInfo})`,
+        `state: ${matchState}`,
+    ].join("\n");
+}
+function handleSecurityVerify(argv) {
+    const writeLock = argv.includes("--write-lock");
+    const status = evaluateSecurityStatus();
+    if (!status.lockHash && writeLock) {
+        updateSecurityLock(status.hash);
+        console.info("Security lock created.");
+        const refreshed = evaluateSecurityStatus();
+        console.info([
+            `config: ${refreshed.configPath}`,
+            `lock: ${refreshed.lockPath} (hash=${refreshed.lockHash})`,
+            "state: MATCH",
+        ].join("\n"));
+        return 0;
+    }
+    try {
+        bootstrapServer({ allowWriteLock: writeLock });
+        console.info("Security baseline verified.");
+        console.info(formatStatus());
+        return 0;
+    }
+    catch (error) {
+        console.error(error instanceof Error ? error.message : String(error));
+        console.info(formatStatus());
+        return 1;
+    }
+}
+export function main(argv = process.argv.slice(2)) {
+    const [command, ...rest] = argv;
+    switch (command) {
+        case "security": {
+            const [subcommand, ...subArgs] = rest;
+            if (subcommand === "verify") {
+                return handleSecurityVerify(subArgs);
+            }
+            break;
+        }
+        case undefined:
+            printUsage();
+            return 1;
+        default:
+            break;
+    }
+    printUsage();
+    return 1;
+}
+if (process.argv[1] && new URL(import.meta.url).pathname === process.argv[1]) {
+    process.exitCode = main();
+}
+//# sourceMappingURL=cli.js.map

package/dist/client/cli.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"cli.js","sourceRoot":"","sources":["../../src/client/cli.ts"],"names":[],"mappings":";AACA,OAAO,OAAO,MAAM,cAAc,CAAC;AAEnC,OAAO,EAAE,eAAe,EAAE,MAAM,wBAAwB,CAAC;AACzD,OAAO,EAAE,sBAAsB,EAAE,kBAAkB,EAAE,MAAM,8BAA8B,CAAC;AAE1F,SAAS,UAAU;IACjB,OAAO,CAAC,IAAI,CAAC,8DAA8D,CAAC,CAAC;IAC7E,OAAO,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC;IAC1B,OAAO,CAAC,IAAI,CAAC,gFAAgF,CAAC,CAAC;AACjG,CAAC;AAED,SAAS,YAAY;IACnB,MAAM,MAAM,GAAG,sBAAsB,EAAE,CAAC;IACxC,MAAM,QAAQ,GAAG,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,QAAQ,MAAM,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC;IACzE,MAAM,UAAU,GAAG,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,UAAU,CAAC;IACzD,OAAO;QACL,WAAW,MAAM,CAAC,UAAU,EAAE;QAC9B,SAAS,MAAM,CAAC,QAAQ,KAAK,QAAQ,GAAG;QACxC,UAAU,UAAU,EAAE;KACvB,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACf,CAAC;AAED,SAAS,oBAAoB,CAAC,IAAc;IAC1C,MAAM,SAAS,GAAG,IAAI,CAAC,QAAQ,CAAC,cAAc,CAAC,CAAC;IAChD,MAAM,MAAM,GAAG,sBAAsB,EAAE,CAAC;IACxC,IAAI,CAAC,MAAM,CAAC,QAAQ,IAAI,SAAS,EAAE,CAAC;QAClC,kBAAkB,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QAChC,OAAO,CAAC,IAAI,CAAC,wBAAwB,CAAC,CAAC;QACvC,MAAM,SAAS,GAAG,sBAAsB,EAAE,CAAC;QAC3C,OAAO,CAAC,IAAI,CACV;YACE,WAAW,SAAS,CAAC,UAAU,EAAE;YACjC,SAAS,SAAS,CAAC,QAAQ,UAAU,SAAS,CAAC,QAAQ,GAAG;YAC1D,cAAc;SACf,CAAC,IAAI,CAAC,IAAI,CAAC,CACb,CAAC;QACF,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,CAAC;QACH,eAAe,CAAC,EAAE,cAAc,EAAE,SAAS,EAAE,CAAC,CAAC;QAC/C,OAAO,CAAC,IAAI,CAAC,6BAA6B,CAAC,CAAC;QAC5C,OAAO,CAAC,IAAI,CAAC,YAAY,EAAE,CAAC,CAAC;QAC7B,OAAO,CAAC,CAAC;IACX,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,OAAO,CAAC,KAAK,CAAC,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC;QACtE,OAAO,CAAC,IAAI,CAAC,YAAY,EAAE,CAAC,CAAC;QAC7B,OAAO,CAAC,CAAC;IACX,CAAC;AACH,CAAC;AAED,MAAM,UAAU,IAAI,CAAC,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC;IAC/C,MAAM,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC,GAAG,IAAI,CAAC;IAChC,QAAQ,OAAO,EAAE,CAAC;QAChB,KAAK,UAAU,CAAC,CAAC,CAAC;YAChB,MAAM,CAAC,UAAU,EAAE,GAAG,OAAO,CAAC,GAAG,IAAI,CAAC;YACtC,IAAI,UAAU,KAAK,QAAQ,EAAE,CAAC;gBAC5B,OAAO,oBAAoB,CAAC,OAAO,CAAC,CAAC;YACvC,CAAC;YACD,MAAM;QACR,CAAC;QACD,KAAK,SAAS;YACZ,UAAU,EAAE,CAAC;YACb,OAAO,CAAC,CAAC;QACX;YACE,MAAM;IACV,CAAC;IACD,UAAU,EAAE,CAAC;IACb,OAAO,CAAC,CAAC;AACX,CAAC;AAED,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,IAAI,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,QAAQ,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC;IAC7E,OAAO,CAAC,QAAQ,GAAG,IAAI,EAAE,CAAC;AAC5B,CAAC"}

package/dist/client/index.js

@@ -0,0 +1,5 @@
+export function buildContextBundleRequest(snippets) {
+    // TODO: align with MCP client schema
+    return JSON.stringify({ snippets }, null, 2);
+}
+//# sourceMappingURL=index.js.map

package/dist/client/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/client/index.ts"],"names":[],"mappings":"AAAA,MAAM,UAAU,yBAAyB,CACvC,QAAkD;IAElD,qCAAqC;IACrC,OAAO,IAAI,CAAC,SAAS,CAAC,EAAE,QAAQ,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;AAC/C,CAAC"}

package/dist/eval/metrics.js

@@ -0,0 +1,47 @@
+export function precisionAtK(retrievedIds, relevantIds, k) {
+    if (k <= 0 || retrievedIds.length === 0) {
+        return 0;
+    }
+    const relevantSet = new Set(relevantIds);
+    if (relevantSet.size === 0) {
+        return 0;
+    }
+    const limit = Math.min(k, retrievedIds.length);
+    let hits = 0;
+    for (let index = 0; index < limit; index += 1) {
+        const id = retrievedIds[index];
+        if (id !== undefined && relevantSet.has(id)) {
+            hits += 1;
+        }
+    }
+    return hits / limit;
+}
+export function timeToFirstUseful(events, options = {}) {
+    if (events.length === 0) {
+        return Number.POSITIVE_INFINITY;
+    }
+    const sorted = [...events].sort((a, b) => a.timestampMs - b.timestampMs);
+    const baseline = typeof options.startTimestampMs === "number"
+        ? options.startTimestampMs
+        : (sorted[0]?.timestampMs ?? 0);
+    for (const event of sorted) {
+        if (event.relevant) {
+            const deltaMs = event.timestampMs - baseline;
+            return Math.max(0, deltaMs) / 1000;
+        }
+    }
+    return Number.POSITIVE_INFINITY;
+}
+export function evaluateRetrieval(options) {
+    const { items, relevant, k } = options;
+    const ids = items.map((item) => item.id);
+    const precision = precisionAtK(ids, relevant, k);
+    const relevantSet = new Set(relevant);
+    const latencyEvents = items.map((item) => ({
+        timestampMs: item.timestampMs,
+        relevant: relevantSet.has(item.id),
+    }));
+    const ttff = timeToFirstUseful(latencyEvents);
+    return { precisionAtK: precision, timeToFirstUseful: ttff };
+}
+//# sourceMappingURL=metrics.js.map

package/dist/eval/metrics.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"metrics.js","sourceRoot":"","sources":["../../src/eval/metrics.ts"],"names":[],"mappings":"AAKA,MAAM,UAAU,YAAY,CAC1B,YAAsB,EACtB,WAA6B,EAC7B,CAAS;IAET,IAAI,CAAC,IAAI,CAAC,IAAI,YAAY,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACxC,OAAO,CAAC,CAAC;IACX,CAAC;IACD,MAAM,WAAW,GAAG,IAAI,GAAG,CAAC,WAAW,CAAC,CAAC;IACzC,IAAI,WAAW,CAAC,IAAI,KAAK,CAAC,EAAE,CAAC;QAC3B,OAAO,CAAC,CAAC;IACX,CAAC;IACD,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,YAAY,CAAC,MAAM,CAAC,CAAC;IAC/C,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,KAAK,EAAE,KAAK,IAAI,CAAC,EAAE,CAAC;QAC9C,MAAM,EAAE,GAAG,YAAY,CAAC,KAAK,CAAC,CAAC;QAC/B,IAAI,EAAE,KAAK,SAAS,IAAI,WAAW,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE,CAAC;YAC5C,IAAI,IAAI,CAAC,CAAC;QACZ,CAAC;IACH,CAAC;IACD,OAAO,IAAI,GAAG,KAAK,CAAC;AACtB,CAAC;AAOD,MAAM,UAAU,iBAAiB,CAC/B,MAAsB,EACtB,UAAyC,EAAE;IAE3C,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACxB,OAAO,MAAM,CAAC,iBAAiB,CAAC;IAClC,CAAC;IACD,MAAM,MAAM,GAAG,CAAC,GAAG,MAAM,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,WAAW,GAAG,CAAC,CAAC,WAAW,CAAC,CAAC;IACzE,MAAM,QAAQ,GACZ,OAAO,OAAO,CAAC,gBAAgB,KAAK,QAAQ;QAC1C,CAAC,CAAC,OAAO,CAAC,gBAAgB;QAC1B,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,WAAW,IAAI,CAAC,CAAC,CAAC;IACpC,KAAK,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;QAC3B,IAAI,KAAK,CAAC,QAAQ,EAAE,CAAC;YACnB,MAAM,OAAO,GAAG,KAAK,CAAC,WAAW,GAAG,QAAQ,CAAC;YAC7C,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,OAAO,CAAC,GAAG,IAAI,CAAC;QACrC,CAAC;IACH,CAAC;IACD,OAAO,MAAM,CAAC,iBAAiB,CAAC;AAClC,CAAC;AAaD,MAAM,UAAU,iBAAiB,CAAC,OAAiC;IACjE,MAAM,EAAE,KAAK,EAAE,QAAQ,EAAE,CAAC,EAAE,GAAG,OAAO,CAAC;IACvC,MAAM,GAAG,GAAG,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACzC,MAAM,SAAS,GAAG,YAAY,CAAC,GAAG,EAAE,QAAQ,EAAE,CAAC,CAAC,CAAC;IACjD,MAAM,WAAW,GAAG,IAAI,GAAG,CAAC,QAAQ,CAAC,CAAC;IACtC,MAAM,aAAa,GAAmB,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;QACzD,WAAW,EAAE,IAAI,CAAC,WAAW;QAC7B,QAAQ,EAAE,WAAW,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC;KACnC,CAAC,CAAC,CAAC;IACJ,MAAM,IAAI,GAAG,iBAAiB,CAAC,aAAa,CAAC,CAAC;IAC9C,OAAO,EAAE,YAAY,EAAE,SAAS,EAAE,iBAAiB,EAAE,IAAI,EAAE,CAAC;AAC9D,CAAC"}

package/dist/indexer/cli.js

@@ -0,0 +1,362 @@
+import { createHash } from "node:crypto";
+import { readFile, stat } from "node:fs/promises";
+import { join, resolve, extname } from "node:path";
+import { pathToFileURL } from "node:url";
+import { DuckDBClient } from "../shared/duckdb.js";
+import { generateEmbedding } from "../shared/embedding.js";
+import { analyzeSource, buildFallbackSnippet } from "./codeintel.js";
+import { getDefaultBranch, getHeadCommit, gitLsFiles } from "./git.js";
+import { detectLanguage } from "./language.js";
+import { ensureBaseSchema } from "./schema.js";
+const MAX_SAMPLE_BYTES = 32_768;
+const MAX_FILE_BYTES = 32 * 1024 * 1024; // 32MB limit to prevent memory exhaustion
+const SCAN_BATCH_SIZE = 100; // Process files in batches to limit memory usage
+function countLines(content) {
+    if (content.length === 0) {
+        return 0;
+    }
+    return content.split(/\r?\n/).length;
+}
+function isBinaryBuffer(buffer) {
+    const sample = buffer.subarray(0, Math.min(buffer.length, MAX_SAMPLE_BYTES));
+    if (sample.includes(0)) {
+        return true;
+    }
+    const decoded = sample.toString("utf8");
+    return decoded.includes("\uFFFD");
+}
+/**
+ * Ensures a repository record exists in the database, creating it if necessary.
+ * Uses ON CONFLICT with auto-increment to prevent race conditions in concurrent scenarios.
+ *
+ * @param db - Database client instance
+ * @param repoRoot - Absolute path to the repository root
+ * @param defaultBranch - Default branch name (e.g., "main", "master"), or null if unknown
+ * @returns The repository ID (auto-generated on first insert, reused thereafter)
+ */
+async function ensureRepo(db, repoRoot, defaultBranch) {
+    // Atomically insert or update using ON CONFLICT to leverage auto-increment
+    // This eliminates the TOCTOU race condition present in manual ID generation
+    await db.run(`INSERT INTO repo (root, default_branch, indexed_at)
+     VALUES (?, ?, CURRENT_TIMESTAMP)
+     ON CONFLICT(root) DO UPDATE SET
+       default_branch = COALESCE(excluded.default_branch, repo.default_branch)`, [repoRoot, defaultBranch]);
+    // Fetch the ID of the existing or newly created repo
+    const rows = await db.all("SELECT id FROM repo WHERE root = ?", [repoRoot]);
+    if (rows.length === 0) {
+        throw new Error("Failed to create or find repository record. Check database constraints and schema.");
+    }
+    const row = rows[0];
+    if (!row) {
+        throw new Error("Failed to retrieve repository record. Database returned empty result.");
+    }
+    return row.id;
+}
+async function persistBlobs(db, blobs) {
+    if (blobs.size === 0)
+        return;
+    // Use bulk insert for better performance
+    const blobArray = Array.from(blobs.values());
+    const placeholders = blobArray.map(() => "(?, ?, ?, ?)").join(", ");
+    const sql = `INSERT OR REPLACE INTO blob (hash, size_bytes, line_count, content) VALUES ${placeholders}`;
+    const params = [];
+    for (const blob of blobArray) {
+        params.push(blob.hash, blob.sizeBytes, blob.lineCount, blob.content);
+    }
+    await db.run(sql, params);
+}
+async function persistTrees(db, repoId, commitHash, records) {
+    if (records.length === 0)
+        return;
+    // Use bulk insert for better performance
+    const placeholders = records.map(() => "(?, ?, ?, ?, ?, ?, ?, ?)").join(", ");
+    const sql = `INSERT OR REPLACE INTO tree (repo_id, commit_hash, path, blob_hash, ext, lang, is_binary, mtime) VALUES ${placeholders}`;
+    const params = [];
+    for (const record of records) {
+        params.push(repoId, commitHash, record.path, record.blobHash, record.ext, record.lang, record.isBinary, record.mtimeIso);
+    }
+    await db.run(sql, params);
+}
+async function persistFiles(db, repoId, records) {
+    if (records.length === 0)
+        return;
+    // Use bulk insert for better performance
+    const placeholders = records.map(() => "(?, ?, ?, ?, ?, ?, ?)").join(", ");
+    const sql = `INSERT OR REPLACE INTO file (repo_id, path, blob_hash, ext, lang, is_binary, mtime) VALUES ${placeholders}`;
+    const params = [];
+    for (const record of records) {
+        params.push(repoId, record.path, record.blobHash, record.ext, record.lang, record.isBinary, record.mtimeIso);
+    }
+    await db.run(sql, params);
+}
+async function persistSymbols(db, repoId, records) {
+    if (records.length === 0)
+        return;
+    const placeholders = records.map(() => "(?, ?, ?, ?, ?, ?, ?, ?, ?)").join(", ");
+    const sql = `
+    INSERT OR REPLACE INTO symbol (
+      repo_id, path, symbol_id, name, kind, range_start_line, range_end_line, signature, doc
+    ) VALUES ${placeholders}
+  `;
+    const params = [];
+    for (const record of records) {
+        params.push(repoId, record.path, record.symbolId, record.name, record.kind, record.rangeStartLine, record.rangeEndLine, record.signature, record.doc);
+    }
+    await db.run(sql, params);
+}
+async function persistSnippets(db, repoId, records) {
+    if (records.length === 0)
+        return;
+    const placeholders = records.map(() => "(?, ?, ?, ?, ?, ?)").join(", ");
+    const sql = `
+    INSERT OR REPLACE INTO snippet (
+      repo_id, path, snippet_id, start_line, end_line, symbol_id
+    ) VALUES ${placeholders}
+  `;
+    const params = [];
+    for (const record of records) {
+        params.push(repoId, record.path, record.snippetId, record.startLine, record.endLine, record.symbolId);
+    }
+    await db.run(sql, params);
+}
+async function persistDependencies(db, repoId, records) {
+    if (records.length === 0)
+        return;
+    const placeholders = records.map(() => "(?, ?, ?, ?, ?)").join(", ");
+    const sql = `
+    INSERT OR REPLACE INTO dependency (
+      repo_id, src_path, dst_kind, dst, rel
+    ) VALUES ${placeholders}
+  `;
+    const params = [];
+    for (const record of records) {
+        params.push(repoId, record.srcPath, record.dstKind, record.dst, record.rel);
+    }
+    await db.run(sql, params);
+}
+async function persistEmbeddings(db, repoId, records) {
+    if (records.length === 0)
+        return;
+    const placeholders = records.map(() => "(?, ?, ?, ?, CURRENT_TIMESTAMP)").join(", ");
+    const sql = `
+    INSERT OR REPLACE INTO file_embedding (
+      repo_id, path, dims, vector_json, updated_at
+    ) VALUES ${placeholders}
+  `;
+    const params = [];
+    for (const record of records) {
+        params.push(repoId, record.path, record.dims, JSON.stringify(record.vector));
+    }
+    await db.run(sql, params);
+}
+function buildCodeIntel(files, blobs) {
+    const fileSet = new Set(files.map((file) => file.path));
+    const symbols = [];
+    const snippets = [];
+    const dependencies = new Map();
+    for (const file of files) {
+        if (file.isBinary) {
+            continue;
+        }
+        const blob = blobs.get(file.blobHash);
+        if (!blob || blob.content === null) {
+            continue;
+        }
+        const analysis = analyzeSource(file.path, file.lang, blob.content, fileSet);
+        for (const symbol of analysis.symbols) {
+            symbols.push({
+                path: file.path,
+                symbolId: symbol.symbolId,
+                name: symbol.name,
+                kind: symbol.kind,
+                rangeStartLine: symbol.rangeStartLine,
+                rangeEndLine: symbol.rangeEndLine,
+                signature: symbol.signature,
+                doc: symbol.doc,
+            });
+        }
+        if (analysis.snippets.length > 0) {
+            analysis.snippets.forEach((snippet, index) => {
+                snippets.push({
+                    path: file.path,
+                    snippetId: index + 1,
+                    startLine: snippet.startLine,
+                    endLine: snippet.endLine,
+                    symbolId: snippet.symbolId,
+                });
+            });
+        }
+        else if (blob.lineCount !== null) {
+            const fallback = buildFallbackSnippet(blob.lineCount);
+            snippets.push({
+                path: file.path,
+                snippetId: 1,
+                startLine: fallback.startLine,
+                endLine: fallback.endLine,
+                symbolId: fallback.symbolId,
+            });
+        }
+        for (const dependency of analysis.dependencies) {
+            const key = `${file.path}::${dependency.dstKind}::${dependency.dst}::${dependency.rel}`;
+            if (!dependencies.has(key)) {
+                dependencies.set(key, {
+                    srcPath: file.path,
+                    dstKind: dependency.dstKind,
+                    dst: dependency.dst,
+                    rel: dependency.rel,
+                });
+            }
+        }
+    }
+    return { symbols, snippets, dependencies: Array.from(dependencies.values()) };
+}
+/**
+ * scanFilesのバッチ処理版
+ * メモリ枯渇を防ぐため、ファイルをバッチで処理する
+ */
+async function scanFilesInBatches(repoRoot, paths) {
+    const allBlobs = new Map();
+    const allFiles = [];
+    const allEmbeddings = [];
+    for (let i = 0; i < paths.length; i += SCAN_BATCH_SIZE) {
+        const batch = paths.slice(i, i + SCAN_BATCH_SIZE);
+        const { blobs, files, embeddings } = await scanFiles(repoRoot, batch);
+        // マージ: blobはhashでユニークなので重複排除
+        for (const [hash, blob] of blobs) {
+            if (!allBlobs.has(hash)) {
+                allBlobs.set(hash, blob);
+            }
+        }
+        allFiles.push(...files);
+        allEmbeddings.push(...embeddings);
+        // バッチデータを明示的にクリアしてGCを促す
+        blobs.clear();
+    }
+    return { blobs: allBlobs, files: allFiles, embeddings: allEmbeddings };
+}
+async function scanFiles(repoRoot, paths) {
+    const blobs = new Map();
+    const files = [];
+    const embeddings = [];
+    for (const relativePath of paths) {
+        const absolutePath = join(repoRoot, relativePath);
+        try {
+            const fileStat = await stat(absolutePath);
+            if (!fileStat.isFile()) {
+                continue;
+            }
+            // Check file size before reading to prevent memory exhaustion
+            if (fileStat.size > MAX_FILE_BYTES) {
+                console.warn(`File ${relativePath} exceeds size limit (${fileStat.size} bytes). Increase MAX_FILE_BYTES constant to include it.`);
+                continue;
+            }
+            const buffer = await readFile(absolutePath);
+            const isBinary = isBinaryBuffer(buffer);
+            const hash = createHash("sha1").update(buffer).digest("hex");
+            const ext = extname(relativePath) || null;
+            const lang = ext ? detectLanguage(ext) : null;
+            const mtimeIso = fileStat.mtime.toISOString();
+            let content = null;
+            let lineCount = null;
+            if (!isBinary) {
+                content = buffer.toString("utf8");
+                lineCount = countLines(content);
+            }
+            if (!blobs.has(hash)) {
+                blobs.set(hash, {
+                    hash,
+                    sizeBytes: buffer.length,
+                    lineCount,
+                    content,
+                });
+            }
+            files.push({
+                path: relativePath,
+                blobHash: hash,
+                ext,
+                lang,
+                isBinary,
+                mtimeIso,
+            });
+            if (!isBinary && content) {
+                const embedding = generateEmbedding(content);
+                if (embedding) {
+                    embeddings.push({ path: relativePath, dims: embedding.dims, vector: embedding.values });
+                }
+            }
+        }
+        catch (error) {
+            console.warn(`Cannot read ${relativePath} due to filesystem error. Fix file permissions or remove the file.`);
+            console.warn(error);
+        }
+    }
+    return { blobs, files, embeddings };
+}
+export async function runIndexer(options) {
+    if (!options.full && options.since) {
+        console.warn("Incremental indexing is not yet supported. Falling back to full reindex.");
+    }
+    const repoRoot = resolve(options.repoRoot);
+    const databasePath = resolve(options.databasePath);
+    const [paths, headCommit, defaultBranch] = await Promise.all([
+        gitLsFiles(repoRoot),
+        getHeadCommit(repoRoot),
+        getDefaultBranch(repoRoot),
+    ]);
+    const { blobs, files, embeddings } = await scanFilesInBatches(repoRoot, paths);
+    const codeIntel = buildCodeIntel(files, blobs);
+    const db = await DuckDBClient.connect({ databasePath, ensureDirectory: true });
+    try {
+        await ensureBaseSchema(db);
+        const repoId = await ensureRepo(db, repoRoot, defaultBranch);
+        await db.transaction(async () => {
+            await db.run("DELETE FROM tree WHERE repo_id = ?", [repoId]);
+            await db.run("DELETE FROM file WHERE repo_id = ?", [repoId]);
+            await db.run("DELETE FROM symbol WHERE repo_id = ?", [repoId]);
+            await db.run("DELETE FROM snippet WHERE repo_id = ?", [repoId]);
+            await db.run("DELETE FROM dependency WHERE repo_id = ?", [repoId]);
+            await db.run("DELETE FROM file_embedding WHERE repo_id = ?", [repoId]);
+            await persistBlobs(db, blobs);
+            await persistTrees(db, repoId, headCommit, files);
+            await persistFiles(db, repoId, files);
+            await persistSymbols(db, repoId, codeIntel.symbols);
+            await persistSnippets(db, repoId, codeIntel.snippets);
+            await persistDependencies(db, repoId, codeIntel.dependencies);
+            await persistEmbeddings(db, repoId, embeddings);
+            // Update timestamp inside transaction to ensure atomicity
+            if (defaultBranch) {
+                await db.run("UPDATE repo SET indexed_at = CURRENT_TIMESTAMP, default_branch = ? WHERE id = ?", [defaultBranch, repoId]);
+            }
+            else {
+                await db.run("UPDATE repo SET indexed_at = CURRENT_TIMESTAMP WHERE id = ?", [repoId]);
+            }
+        });
+    }
+    finally {
+        await db.close();
+    }
+    console.info(`Indexed ${files.length} files for repo ${repoRoot} at ${databasePath} (commit ${headCommit.slice(0, 12)})`);
+}
+function parseArg(flag) {
+    const index = process.argv.indexOf(flag);
+    if (index >= 0) {
+        return process.argv[index + 1];
+    }
+    return undefined;
+}
+if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) {
+    const repoRoot = resolve(parseArg("--repo") ?? ".");
+    const databasePath = resolve(parseArg("--db") ?? "var/index.duckdb");
+    const full = process.argv.includes("--full");
+    const since = parseArg("--since");
+    const options = { repoRoot, databasePath, full: full || !since };
+    if (since) {
+        options.since = since;
+    }
+    runIndexer(options).catch((error) => {
+        console.error("Failed to index repository. Retry after resolving the logged error.");
+        console.error(error);
+        process.exitCode = 1;
+    });
+}
+//# sourceMappingURL=cli.js.map