know-cli 0.4.1__tar.gz → 0.4.2__tar.gz

{know_cli-0.4.1 → know_cli-0.4.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: know-cli
-Version: 0.4.1
+Version: 0.4.2
 Summary: Context Intelligence for AI Coding Agents — smart, token-budgeted code context
 Project-URL: Homepage, https://github.com/sushilk1991/know-cli
 Project-URL: Repository, https://github.com/sushilk1991/know-cli

know_cli-0.4.2/docs/plans/2026-02-16-fix-testing-issues-search-graph-performance-plan.md

@@ -0,0 +1,100 @@
+---
+title: "fix: Search fallback, empty import graph, and slow daemon startup"
+type: fix
+date: 2026-02-16
+---
+
+# fix: Search fallback, empty import graph, and slow daemon startup
+
+## Overview
+
+Four issues discovered when testing know-cli v0.4.1 in a real project plus two cross-cutting concerns identified by 4-reviewer analysis.
+
+## Problem Statement
+
+1. **`know search` crashes without numpy/fastembed** — crashes with `No module named 'numpy'` instead of falling back to BM25.
+2. **`know next-file` returns "No more relevant files found"** — BM25 exact-phrase quoting (`daemon_db.py:212`) can't match semantic queries.
+3. **`know related src/know/daemon.py` returns empty** — import graph only stores modules with outgoing edges (`import_graph.py:137-138`).
+4. **~21 second first-run latency** — daemon runs `_full_index()` before accepting connections (`daemon.py:118`).
+5. **`recall_memories` has same exact-phrase bug** as `search_chunks` (`daemon_db.py:419`).
+6. **Thread-unsafe shared SQLite connection** — background indexing thread + event loop queries share one `sqlite3.Connection`.
+
+## Proposed Solution
+
+### Fix 1: BM25 fallback for `know search`
+
+**File:** `src/know/cli/search.py`
+
+- [x] Wrap `from know.semantic_search import SemanticSearcher` in `try/except ImportError`
+- [x] Fallback: use daemon-first pattern (`_get_daemon_client` → direct `DaemonDB`)
+- [x] Mirror all three output branches: `--json`, `--quiet`, and rich
+- [x] Always print hint: `[dim]Tip: pip install know-cli[search] for semantic search[/dim]`
+- [x] Track stats in fallback path via `StatsTracker.record_search()`
+
+### Fix 2: OR-based BM25 query parsing + shared helper
+
+**File:** `src/know/daemon_db.py`
+
+- [x] Add `_build_fts_query()` static method
+- [x] Cap at 12 terms to bound FTS5 OR query complexity
+- [x] Update `search_chunks()` to use `_build_fts_query()`
+- [x] Update `recall_memories()` to use `_build_fts_query()` (same exact-phrase bug)
+
+### Fix 3: Store all modules in import graph
+
+**File:** `src/know/import_graph.py:137-138`
+
+- [x] Remove the `if mod_edges:` guard — store even empty edge lists
+
+### Fix 4: Non-blocking daemon startup
+
+**File:** `src/know/daemon.py`
+
+- [x] Move `start_unix_server()` before `_full_index()`
+- [x] Change `await self._full_index()` to `self._index_task = asyncio.create_task(self._full_index())`
+- [x] Store task reference to prevent GC and enable status checks
+- [x] No `_indexing` flag or `_background_index` wrapper needed (YAGNI)
+
+### Fix 5: Thread-safe SQLite connections
+
+**File:** `src/know/daemon_db.py`
+
+- [x] Replace shared `self._conn` with `threading.local()` per-thread connections
+- [x] Add `PRAGMA busy_timeout=5000` to prevent `SQLITE_BUSY` errors
+- [x] Each thread gets its own connection; WAL mode handles concurrent reads + single writer
+- [x] Update `close()` to close thread-local connections
+
+## Acceptance Criteria
+
+- [x] `know search "daemon socket"` works without fastembed (uses BM25)
+- [x] `know search "daemon socket"` prints fastembed hint
+- [x] `know search "daemon socket" --json` returns valid JSON in fallback
+- [x] `know next-file "message framing protocol"` returns `src/know/daemon.py` (verified via direct DB)
+- [x] `know next-file "knowledge base"` returns relevant file (OR-based BM25 matches partial terms)
+- [x] `know related src/know/daemon.py` shows imports and imported-by
+- [x] `know related src/know/daemon_db.py` shows both directions
+- [x] `know recall "project architecture"` matches even without exact phrase
+- [x] Daemon accepts first query within 2 seconds of startup
+- [x] Full index completes in background
+- [x] All 169 existing tests pass (excluding pre-existing test_week4 failures)
+- [x] All 27 v2 tests pass
+
+## Execution Order
+
+1. Fix 5 (thread-safe connections) — foundational for Fix 4
+2. Fix 2 (BM25 query helper) — foundational for Fix 1
+3. Fix 1 (search fallback) — depends on Fix 2
+4. Fix 3 (import graph) — independent
+5. Fix 4 (non-blocking startup) — depends on Fix 5
+
+## References
+
+- Previous plan: `docs/plans/2026-02-15-refactor-know-cli-v2-daemon-architecture-plan.md`
+- Solution doc: `docs/solutions/architecture/p2-p3-architectural-improvements.md`
+- Key files:
+  - `src/know/cli/search.py:34` — search command
+  - `src/know/daemon_db.py:208-225` — `search_chunks()` BM25 method
+  - `src/know/daemon_db.py:415-432` — `recall_memories()` BM25 method
+  - `src/know/import_graph.py:137-138` — edge storage logic
+  - `src/know/daemon.py:106-135` — daemon serve/startup
+  - `src/know/cli/agent.py:32-71` — next-file command

{know_cli-0.4.1 → know_cli-0.4.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "know-cli"
-version = "0.4.1"
+version = "0.4.2"
 description = "Context Intelligence for AI Coding Agents — smart, token-budgeted code context"
 readme = "README.md"
 license = "MIT"

{know_cli-0.4.1 → know_cli-0.4.2}/src/know/__init__.py

@@ -1,6 +1,6 @@
 """know - Living documentation generator for codebases."""
 
-__version__ = "0.4.1"
+__version__ = "0.4.2"
 __author__ = "Sushil Kumar"
 
 from know.cli import main

{know_cli-0.4.1 → know_cli-0.4.2}/src/know/cli/search.py

@@ -32,13 +32,22 @@ from know.scanner import CodebaseScanner
 )
 @click.pass_context
 def search(ctx: click.Context, query: str, top_k: int, index: bool, chunk: bool) -> None:
-    """Search code semantically using embeddings."""
+    """Search code semantically using embeddings.
+
+    Falls back to BM25 keyword search if fastembed/numpy not installed.
+    """
     config = ctx.obj["config"]
 
-    from know.semantic_search import SemanticSearcher
+    try:
+        from know.semantic_search import SemanticSearcher
+        searcher = SemanticSearcher(project_root=config.root)
+        _search_semantic(ctx, config, searcher, query, top_k, index, chunk)
+    except ImportError:
+        _search_bm25_fallback(ctx, config, query, top_k)
 
-    searcher = SemanticSearcher(project_root=config.root)
 
+def _search_semantic(ctx, config, searcher, query, top_k, index, chunk):
+    """Semantic search using fastembed embeddings."""
     if index:
         if not ctx.obj.get("quiet"):
             console.print(f"[dim]Indexing {config.root}...[/dim]")
@@ -93,6 +102,60 @@ def search(ctx: click.Context, query: str, top_k: int, index: bool, chunk: bool)
             console.print()
 
 
+def _search_bm25_fallback(ctx, config, query, top_k):
+    """BM25 keyword search fallback when fastembed/numpy not installed."""
+    from know.cli.agent import _get_daemon_client, _get_db_fallback
+
+    if not ctx.obj.get("quiet") and not ctx.obj.get("json"):
+        console.print("[dim]Using BM25 keyword search (fastembed not installed)[/dim]")
+
+    import time as _time
+    t0 = _time.monotonic()
+
+    client = _get_daemon_client(config)
+    if client:
+        try:
+            result = client.call_sync("search", {"query": query, "limit": top_k})
+            results = result.get("results", [])
+        except Exception:
+            client = None
+
+    if not client:
+        db = _get_db_fallback(config)
+        results = db.search_chunks(query, top_k)
+
+    duration_ms = int((_time.monotonic() - t0) * 1000)
+
+    # Track stats
+    try:
+        from know.stats import StatsTracker
+        StatsTracker(config).record_search(query, len(results), duration_ms)
+    except Exception as e:
+        logger.debug(f"Stats tracking (search) failed: {e}")
+
+    if ctx.obj.get("json"):
+        import json
+        click.echo(json.dumps({"results": results}))
+    elif ctx.obj.get("quiet"):
+        for r in results:
+            click.echo(f"{r.get('file_path', '')}:{r.get('chunk_name', '')}")
+    else:
+        if not results:
+            console.print("[yellow]No results found[/yellow]")
+            console.print("[dim]Tip: pip install know-cli[search] for semantic search[/dim]")
+            sys.exit(2)
+
+        console.print(f"\n[bold]Top {len(results)} results (BM25):[/bold]\n")
+        for i, r in enumerate(results, 1):
+            label = f"{r.get('file_path', '')}:{r.get('chunk_name', '')}"
+            console.print(f"{i}. {label}")
+            if r.get('signature'):
+                console.print(f"   [dim]{r['signature'][:200]}[/dim]")
+            console.print()
+
+        console.print("[dim]Tip: pip install know-cli[search] for semantic search[/dim]")
+
+
 @click.command()
 @click.argument("query", required=False, default=None)
 @click.option(

{know_cli-0.4.1 → know_cli-0.4.2}/src/know/daemon.py

@@ -114,16 +114,16 @@ class KnowDaemon:
         pf.parent.mkdir(parents=True, exist_ok=True)
         pf.write_text(str(os.getpid()))
 
-        # Initial indexing
-        await self._full_index()
-
+        # Start server FIRST — accept connections immediately
         self._server = await asyncio.start_unix_server(
             self._handle_connection, path=str(sock)
         )
-        # Restrict socket access to owner only
         os.chmod(str(sock), 0o600)
         logger.info(f"Daemon listening on {sock}")
 
+        # Index in background — queries use stale/cached data until done
+        self._index_task = asyncio.create_task(self._full_index())
+
         # Set up idle timeout check
         try:
             while True:

{know_cli-0.4.1 → know_cli-0.4.2}/src/know/daemon_db.py

@@ -120,26 +120,44 @@ CREATE TABLE IF NOT EXISTS file_index (
 """
 
 
+MAX_SEARCH_TERMS = 12
+
+
 class DaemonDB:
     """Unified project database with FTS5 search."""
 
     def __init__(self, project_root: Path):
         self.root = project_root
         self.db_path = project_root / ".know" / "daemon.db"
-        self._conn: Optional[sqlite3.Connection] = None
+        self._local = threading.local()
         self._lock = threading.Lock()
         self._init_db()
 
     def _get_conn(self) -> sqlite3.Connection:
-        if self._conn is None:
+        """Get a thread-local database connection."""
+        conn = getattr(self._local, 'conn', None)
+        if conn is None:
             with self._lock:
-                if self._conn is None:
-                    self.db_path.parent.mkdir(parents=True, exist_ok=True)
-                    self._conn = sqlite3.connect(str(self.db_path), check_same_thread=False)
-                    self._conn.row_factory = sqlite3.Row
-                    self._conn.execute("PRAGMA journal_mode=WAL")
-                    self._conn.execute("PRAGMA synchronous=NORMAL")
-        return self._conn
+                self.db_path.parent.mkdir(parents=True, exist_ok=True)
+            conn = sqlite3.connect(str(self.db_path))
+            conn.row_factory = sqlite3.Row
+            conn.execute("PRAGMA journal_mode=WAL")
+            conn.execute("PRAGMA synchronous=NORMAL")
+            conn.execute("PRAGMA busy_timeout=5000")
+            self._local.conn = conn
+        return conn
+
+    @staticmethod
+    def _build_fts_query(query: str) -> str:
+        """Build OR-based FTS5 query from natural language string.
+
+        Each term is double-quoted to disable FTS5 operator interpretation.
+        BM25 ranking naturally boosts chunks matching more terms.
+        """
+        terms = query.strip().split()[:MAX_SEARCH_TERMS]
+        if not terms:
+            return ""
+        return " OR ".join('"' + t.replace('"', '""') + '"' for t in terms)
 
     def _init_db(self):
         conn = self._get_conn()
@@ -157,9 +175,10 @@ class DaemonDB:
             conn.commit()
 
     def close(self):
-        if self._conn:
-            self._conn.close()
-            self._conn = None
+        conn = getattr(self._local, 'conn', None)
+        if conn is not None:
+            conn.close()
+            self._local.conn = None
 
     def __enter__(self):
         return self
@@ -208,8 +227,9 @@ class DaemonDB:
     def search_chunks(self, query: str, limit: int = 20) -> List[Dict[str, Any]]:
         """BM25 full-text search over code chunks."""
         conn = self._get_conn()
-        # Escape FTS5 special characters by quoting the query
-        safe_query = '"' + query.replace('"', '""') + '"'
+        safe_query = self._build_fts_query(query)
+        if not safe_query:
+            return []
         try:
             rows = conn.execute(
                 """SELECT c.*, rank
@@ -415,8 +435,9 @@ class DaemonDB:
     def recall_memories(self, query: str, limit: int = 10) -> List[Dict[str, Any]]:
         """Search memories using FTS5 BM25."""
         conn = self._get_conn()
-        # Escape FTS5 special characters by quoting the query
-        safe_query = '"' + query.replace('"', '""') + '"'
+        safe_query = self._build_fts_query(query)
+        if not safe_query:
+            return []
         try:
             rows = conn.execute(
                 """SELECT m.*, rank

{know_cli-0.4.1 → know_cli-0.4.2}/src/know/import_graph.py

@@ -134,8 +134,7 @@ class ImportGraph:
                 for resolved, imp_type in targets:
                     mod_edges.append((resolved, imp_type))
 
-            if mod_edges:
-                edges_by_source[mod_name] = mod_edges
+            edges_by_source[mod_name] = mod_edges
 
         # Persist via DaemonDB
         total_edges = 0