codecortex 0.2.2__py3-none-any.whl → 0.3.0__py3-none-any.whl

{codecortex-0.2.2.dist-info → codecortex-0.3.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codecortex
-Version: 0.2.2
+Version: 0.3.0
 Summary: Local-first, MCP-native code-intelligence server — graph, LSP, and semantic search behind one safe code.query tool for coding agents.
 Author: Shammai Hamilton
 License-Expression: MIT
@@ -183,7 +183,7 @@ Full system docs live in [`docs/`](docs/) — start with the index:
 | `codeintel setup [project_root] [--index] [--warm] [--install-uv]` | Check backends + optionally index this repo; ends with a health report |
 | `codeintel index [project_root]` | Index a project for semantic search |
 | `codeintel serve` | Start the MCP server (stdio transport) |
-| `codeintel serve-http [--host HOST] [--port 8766] [--allow-remote]` | Start the HTTP transport (loopback-only unless `--allow-remote`) |
+| `codeintel serve-http [--host HOST] [--port 8766] [--allow-remote] [--token TOKEN]` | Start the HTTP transport (loopback-only unless `--allow-remote`; `--token` requires a bearer token on every request) |
 | `codeintel query --op OP --target TARGET [--engine auto]` | Run a single query and print the result |
 | `codeintel status [project_root]` | Show engine availability and index age |
 | `codeintel doctor [project_root] [--deep] [--json]` | Diagnose per-engine health + repo index status, with a fix for each gap |
@@ -197,17 +197,28 @@ Human-facing commands (`doctor`, `status`, `query`, `setup`, `reset`) honor `--n
 Create `.codeintel.toml` at your project root to override defaults:
 
 ```toml
-backend      = "auto"                   # auto | graph | lsp | semantic
-semantic     = "on"                     # on | off
-reindex      = "on-demand"              # on-demand | never
-cosine_floor = 0.25                     # minimum similarity score for semantic hits
-max_chunks   = 500                      # max chunks to embed per file
-model        = "BAAI/bge-small-en-v1.5" # fastembed embedding model
+backend          = "auto"                   # auto | graph | lsp | semantic
+semantic         = "on"                     # on | off
+reindex          = "on-demand"              # on-demand | never
+cosine_floor     = 0.25                     # minimum similarity score for semantic hits (0–1)
+max_chunks       = 500                      # max chunks to embed per file
+max_total_chunks = 100000                   # safety ceiling on chunks embedded in one index pass
+model            = "BAAI/bge-small-en-v1.5" # fastembed embedding model
 ```
 
+Config is **validated on load** — an out-of-range number, a misspelled enum, or a wrong type falls back to that key's default (with a logged warning) instead of breaking every query.
+
+**Environment variables:**
+
+| Variable | Effect |
+|---|---|
+| `CODEINTEL_HTTP_TOKEN` | Bearer token required by `serve-http` (equivalent to `--token`) |
+| `CODEINTEL_DEBUG=1` | Log the full traceback of any error the never-throw contract swallows (silent by default) — the switch for diagnosing an unexpected `null` |
+| `CODEINTEL_REINDEX=off` | Disable the background reindexer; queries then index inline to stay fresh |
+
 ## Privacy & dependencies
 
-**codeintel is local-first** — one local process, no cloud service, no API keys, no telemetry, and no per-query network. Its own code makes zero outbound HTTP calls, and the HTTP transport binds to `127.0.0.1` only.
+**codeintel is local-first** — one local process, no cloud service, no API keys, no telemetry, and no per-query network. Its own code makes zero outbound HTTP calls, and the HTTP transport binds to `127.0.0.1` only by default — binding a non-loopback host requires `--allow-remote`, and `--token` (or `CODEINTEL_HTTP_TOKEN`) then gates every request behind a bearer token. The server bounds concurrent connections, but for exposure to a hostile network you should still front it with a reverse proxy (TLS, rate-limiting) — the built-in `http.server` is not hardened for the open internet.
 
 **Bundled (installed with the package, run locally):** `mcp` (the tool interface) · `sqlite-vec` (the semantic index, a local DB file) · `fastembed` (the local embedding model).
 
@@ -240,6 +251,8 @@ Over MCP the agent calls `code.query` directly. Over HTTP, start the server and
 codeintel serve-http &   # listens on 127.0.0.1:8766 by default
 ```
 
+For a shared or remote deployment, start it with `--allow-remote --token "$CODEINTEL_HTTP_TOKEN"` and send `Authorization: Bearer <token>` on each request — a missing or wrong token gets a clean `401`. Requests are handled concurrently, so one slow query never blocks another.
+
 ```python
 import urllib.request, json
 

codecortex-0.3.0.dist-info/RECORD

@@ -0,0 +1,31 @@
+codecortex-0.3.0.dist-info/licenses/LICENSE,sha256=DIRvhlH8EulEUOYQxKDFFiimJrlVT5-kCKxQIgi2m7c,1073
+codeintel/__init__.py,sha256=VrXpHDu3erkzwl_WXrqINBm9xWkcyUy53IQOj042dOs,22
+codeintel/__main__.py,sha256=m0MQ0qKFD6VAfrkXP__d_8rbVb1RtUykuzMaa1Wzq6E,15573
+codeintel/cache.py,sha256=UESsUtVqzuIqNQOnXG78ckQfbAz8pz_w1AE9gzeHOww,3158
+codeintel/config.py,sha256=r8tsX4yeiqBgqpnFmUkuMczs6Hbwt8iasEuYSceW200,3761
+codeintel/doctor.py,sha256=jiPuZzS801xfWwTyeVO9UHUxQqAJ-YqTlX-_epZfoyI,6659
+codeintel/gateway.py,sha256=3MEuD5SC2bIB3MLeZWj4q9FJYGfkVfRjmB1n8ISTp_U,9209
+codeintel/http_server.py,sha256=c1IpKMhfLmujS7yKFzOGHQjYebphFtm8p_5cQYjevi0,7862
+codeintel/indexer.py,sha256=brvvoCSdjO5v6VZrRe-8_VrakzQpp1ByyANFhUGTjQQ,10243
+codeintel/injector.py,sha256=IHZTLajQURE2LLxL8dFE9pvJV1RMOAbE7pVRQbFVvgs,2755
+codeintel/installer.py,sha256=PSgK6KA14VEQs2pQKCmjf9JDxJ2mpoEsaWrpFGWSy68,3488
+codeintel/mapper.py,sha256=JN-lMO1CXIyDKSwHfzp0JuK396FR41XhFgtAip77G0E,7493
+codeintel/onboarding.py,sha256=UxNJnlENxEAzG3JnEXu_Yue-4GZiEX2HG2BZVVA6jMI,8925
+codeintel/policy.py,sha256=mUAlk9oYoszQLEcvoroFLRd9DSTzho7YcbM7xb6RS-k,935
+codeintel/provider.py,sha256=GYNhOpJteq3zTp4R_vTp1Wz2k6TPjx6PicGhsq_CT9s,2013
+codeintel/reindexer.py,sha256=sgCvb65scbmTtRd5LKhqxQtWolPOAmwKXJbmXWxvmVc,5667
+codeintel/reset.py,sha256=8Mpak4Tj5e-yTcM3_EnZJzbJPwPV2o8xpxGjv-wk9EE,3675
+codeintel/searcher.py,sha256=R9PV7jSx-GO--qmh65FXlYxpbKbjCnxV-EF9hGe8hZU,4588
+codeintel/semantic_db.py,sha256=9umQ8UUUtHHy7Ngi0B-e8eGCy62XFIOkcWnd3wc3MMQ,3503
+codeintel/server.py,sha256=evo3aZcQYOvSTK7UcfqZg-pRER487oePpLQwCA5BvQc,7497
+codeintel/term.py,sha256=csbSYD8euC13FnnpThy8kylDgZeejQ6j6Ol6Gu21Clo,6819
+codeintel/providers/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+codeintel/providers/graph.py,sha256=nsVsSQueFENSlyXlRf9r4-5aU-f18AnMM_9xOGb8rok,20180
+codeintel/providers/lsp.py,sha256=w4xCPmZcuro4W9MIGU-4pEtuVA2YrvEapXHTjVVd0gA,16840
+codeintel/providers/none.py,sha256=VI84XQkT7opQm6rBAEn0CdM8CNVlKGWYmjAhyeWbHeg,783
+codeintel/providers/semantic.py,sha256=LGIbJLt5zkSv1frbloEeAfy7oplY1xl-FmhxujTEmk0,6311
+codecortex-0.3.0.dist-info/METADATA,sha256=42huRhrKjKhhiUyxr7LKyG1FnI5wRl4dHWL2BO-6czU,16842
+codecortex-0.3.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+codecortex-0.3.0.dist-info/entry_points.txt,sha256=zrYfo95-8JkK0G2e5f9d94sYqqim7ZrdCW8knIfSy2U,54
+codecortex-0.3.0.dist-info/top_level.txt,sha256=DF3TH1hHLWrrnX-7HhfMX0OnS2EXgqPRhPbiL_oSekw,10
+codecortex-0.3.0.dist-info/RECORD,,

codeintel/__init__.py

@@ -1 +1 @@
-__version__ = "0.2.2"
+__version__ = "0.3.0"

codeintel/__main__.py

@@ -50,7 +50,8 @@ def main() -> None:
     http_parser = subparsers.add_parser("serve-http", help="Start the HTTP transport server")
     http_parser.add_argument("--port", type=int, default=8766, help="Port to listen on (default: 8766)")
     http_parser.add_argument("--host", default="127.0.0.1", help="Host to bind to (default: 127.0.0.1)")
-    http_parser.add_argument("--allow-remote", action="store_true", help="Permit binding a non-loopback host (exposes an UNAUTHENTICATED endpoint)")
+    http_parser.add_argument("--allow-remote", action="store_true", help="Permit binding a non-loopback host (use with --token, or the endpoint is UNAUTHENTICATED)")
+    http_parser.add_argument("--token", default=None, help="Require this bearer token on every request (or set CODEINTEL_HTTP_TOKEN). Strongly recommended with --allow-remote.")
 
     # install subcommand
     install_parser = subparsers.add_parser("install", help="Register codeintel with AI agents")
@@ -120,6 +121,7 @@ def main() -> None:
                 window=int(cfg.get("window", 20)),
                 stride=int(cfg.get("stride", 10)),
                 max_chunks=int(cfg.get("max_chunks", 500)),
+                max_total_chunks=int(cfg.get("max_total_chunks", 100000)),
             ).index(project_root)
             if count > 0:
                 print(f"Indexed {count} chunks")
@@ -225,7 +227,7 @@ def main() -> None:
             from codeintel import server
 
             project_root = args.project_root or os.getcwd()
-            status = server.code_status_handler({})
+            status = server.code_status_handler({"project_root": project_root})
 
             print("Engine status:")
             for engine in ["graph", "lsp", "semantic"]:
@@ -318,7 +320,8 @@ def main() -> None:
     elif args.command == "serve-http":
         try:
             from codeintel.http_server import run
-            run(host=args.host, port=args.port, allow_remote=args.allow_remote)
+            token = args.token or os.environ.get("CODEINTEL_HTTP_TOKEN")
+            run(host=args.host, port=args.port, allow_remote=args.allow_remote, token=token)
         except KeyboardInterrupt:
             pass
         except Exception as exc:

codeintel/cache.py

@@ -13,7 +13,7 @@ def _compute_hash(target: str, project_root: str) -> str:
     try:
         root = os.path.realpath(project_root) if project_root else ""
         path = os.path.realpath(target)
-        if root and path.startswith(root + os.sep) or path == root:
+        if (root and path.startswith(root + os.sep)) or path == root:
             if os.path.isfile(path):
                 with open(path, "rb") as fh:
                     return hashlib.sha256(fh.read()).hexdigest()

codeintel/config.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import logging
 import pathlib
 import sys
 
@@ -11,17 +12,28 @@ else:  # pragma: no cover
     except ImportError:
         import tomli as tomllib  # type: ignore[no-redef]
 
+logger = logging.getLogger("codeintel")
+
 _DEFAULTS: dict = {
     "backend": "auto",
     "semantic": "on",
     "reindex": "on-demand",
     "window": 20,
     "stride": 10,
-    "max_chunks": 500,
+    "max_chunks": 500,          # per file
+    "max_total_chunks": 100000,  # safety ceiling on chunks embedded in one index pass
     "cosine_floor": 0.25,
     "model": "BAAI/bge-small-en-v1.5",
 }
 
+# Values restricted to a fixed set — anything else falls back to the default.
+_ENUMS: dict = {
+    "backend": {"auto", "graph", "lsp", "semantic"},
+    "semantic": {"on", "off"},
+    "reindex": {"on-demand", "never"},
+}
+_POSITIVE_INTS = ("window", "stride", "max_chunks", "max_total_chunks")
+
 
 def _read_toml(path: pathlib.Path) -> dict:
     try:
@@ -31,12 +43,55 @@ def _read_toml(path: pathlib.Path) -> dict:
         return {}
 
 
+def _coerce(cfg: dict) -> dict:
+    """Coerce/clamp a merged config to safe values, warning on and dropping bad ones. Never raises:
+    a malformed ``.codeintel.toml`` (a string where a number belongs, an out-of-range floor, an
+    unknown enum) must degrade to the default for that key, not break every query that loads it."""
+    out = dict(_DEFAULTS)
+    for key, default in _DEFAULTS.items():
+        if key not in cfg:
+            continue
+        val = cfg[key]
+        try:
+            if key in _ENUMS:
+                s = str(val).strip().lower()
+                if s in _ENUMS[key]:
+                    out[key] = s
+                else:
+                    logger.warning("config: %s=%r invalid (expected %s) — using %r",
+                                   key, val, sorted(_ENUMS[key]), default)
+            elif key == "model":
+                out[key] = str(val).strip() or default
+            elif key == "cosine_floor":
+                f = float(val)
+                if f != f:  # NaN (TOML allows `nan`) — reject so it can't silently disable the floor
+                    raise ValueError("nan")
+                out[key] = min(1.0, max(0.0, f))
+            elif key in _POSITIVE_INTS:
+                n = int(val)
+                if n > 0:
+                    out[key] = n
+                else:
+                    logger.warning("config: %s=%r must be > 0 — using %r", key, val, default)
+        except (TypeError, ValueError, OverflowError):
+            # int(float('inf')) raises OverflowError (TOML allows `inf`); float("x") → ValueError;
+            # int([]) → TypeError. Any non-usable value logs and keeps the default, so the
+            # docstring's never-raise promise holds even for the CLI paths that don't wrap this.
+            logger.warning("config: %s=%r not usable — using default %r", key, val, default)
+    # Keep any extra keys the user set (forward-compat) without validating them.
+    for k, v in cfg.items():
+        if k not in _DEFAULTS:
+            out[k] = v
+    return out
+
+
 def load_config(project_root: str | None = None) -> dict:
-    """Return merged config: defaults < global < project."""
+    """Return the merged, validated config: defaults < global < project. Values that fail
+    validation fall back to their default (logged), so a bad config file never breaks a query."""
     root = pathlib.Path(project_root) if project_root is not None else pathlib.Path.cwd()
 
     global_cfg = _read_toml(pathlib.Path.home() / ".codeintel" / "config.toml")
     project_cfg = _read_toml(root / ".codeintel.toml")
 
     merged = {**_DEFAULTS, **global_cfg, **project_cfg}
-    return merged
+    return _coerce(merged)

codeintel/gateway.py

@@ -5,7 +5,7 @@ from typing import Any
 
 from codeintel.cache import ContentHashCache
 from codeintel.policy import TieringPolicy
-from codeintel.provider import Result, safe_null_result
+from codeintel.provider import Result, log_swallowed, safe_null_result
 from codeintel.providers.none import NoneProvider
 from codeintel.reindexer import Reindexer
 
@@ -128,7 +128,8 @@ class Gateway:
             if r is not None:
                 return r
             return safe_null_result(op_str, target_str, engine=engine_str, reason="no-result")
-        except Exception:
+        except Exception as exc:
+            log_swallowed(f"Gateway._dispatch_single[{engine_str}.{op_str}]", exc)
             return safe_null_result(op_str, target_str, engine=engine_str, reason="provider-error")
 
     def query(
@@ -206,14 +207,15 @@ class Gateway:
             provider = self._provider_for(engine_str)
             result = self._dispatch_single(provider, op_str, target_str, budget, project_root, engine_str)
 
-            # overview auto-fallback (F4 Story 2): when auto-routed to graph but graph is
-            # unavailable, try lsp — a file/symbol overview is something lsp can also serve.
+            # overview auto-fallback (F4 Story 2): when auto-routed to graph but graph can't serve
+            # it — the backend is unavailable, OR this repo simply isn't in the graph — try lsp,
+            # which can produce a file/symbol overview without the graph index.
             if (
                 was_auto
                 and op_str == "overview"
                 and engine_str == "graph"
                 and result.get("result") is None
-                and result.get("reason") == "engine-unavailable"
+                and result.get("reason") in ("engine-unavailable", "project-not-indexed")
             ):
                 lsp_result = self._dispatch_single(
                     self.lsp, op_str, target_str, budget, project_root, "lsp"
@@ -224,5 +226,6 @@ class Gateway:
             self._cache.put(op_str, target_str, engine_str, root_str, result, freshness)
             return result
 
-        except Exception:
+        except Exception as exc:
+            log_swallowed("Gateway.query", exc)
             return safe_null_result(op or "", target or "", reason="gateway-error")

codeintel/http_server.py

@@ -1,15 +1,25 @@
 from __future__ import annotations
 
+import hmac
 import json
 import sys
+import threading
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from urllib.parse import parse_qs, urlparse
 
+from codeintel.provider import log_swallowed
 from codeintel.server import code_doctor_handler, code_query_handler, code_status_handler
 
-_MAX_BODY_BYTES = 1_048_576  # 1 MiB
+_MAX_BODY_BYTES = 1_048_576       # 1 MiB
+_REQUEST_TIMEOUT_S = 60           # per-request socket read timeout — drops an idle/half-open client
+_MAX_CONCURRENT_REQUESTS = 64     # cap live worker threads so a burst can't exhaust threads/FDs
 
 
 class _Handler(BaseHTTPRequestHandler):
+    # Applied to the request socket by the base handler's setup(); a slow/half-open client
+    # connection is dropped instead of pinning a worker thread forever (basic slowloris guard).
+    timeout = _REQUEST_TIMEOUT_S
+
     def log_message(self, format: str, *args: object) -> None:
         pass  # suppress default stderr noise
 
@@ -21,7 +31,26 @@ class _Handler(BaseHTTPRequestHandler):
         self.end_headers()
         self.wfile.write(body)
 
+    def _authorized(self) -> bool:
+        """When the server was started with a token, require ``Authorization: Bearer <token>``
+        (constant-time compare). No token configured → auth is disabled (the loopback default)."""
+        token = getattr(self.server, "auth_token", None)
+        if not token:
+            return True
+        header = self.headers.get("Authorization", "")
+        prefix = "Bearer "
+        if not header.startswith(prefix):
+            return False
+        presented = header[len(prefix):].strip()
+        # Compare as UTF-8 bytes: hmac.compare_digest raises TypeError on a non-ASCII str, so a
+        # malformed header (e.g. `Bearer résumé`) would otherwise crash the handler thread. Encoding
+        # is always safe and keeps the comparison constant-time.
+        return hmac.compare_digest(presented.encode("utf-8"), token.encode("utf-8"))
+
     def do_POST(self) -> None:
+        if not self._authorized():
+            self._send_json(401, {"error": "unauthorized"})
+            return
         if self.path not in ("/code/query", "/code/doctor"):
             self._send_json(404, {"error": "not-found"})
             return
@@ -50,19 +79,59 @@ class _Handler(BaseHTTPRequestHandler):
         self._send_json(200, result)
 
     def do_GET(self) -> None:
-        if self.path != "/code/status":
+        if not self._authorized():
+            self._send_json(401, {"error": "unauthorized"})
+            return
+        parsed = urlparse(self.path)
+        if parsed.path != "/code/status":
             self._send_json(404, {"error": "not-found"})
             return
-        result = code_status_handler({})
+        # Optional ?project_root=... scopes the `indexed` flag to that repo.
+        project_root = (parse_qs(parsed.query).get("project_root") or [""])[0]
+        result = code_status_handler({"project_root": project_root})
         self._send_json(200, result)
 
 
 class CodeIntelHTTPServer(ThreadingHTTPServer):
-    # Threaded so one slow request (e.g. an LSP session warming, or a first-time index) can't
-    # block every other agent's query. The gateway is a shared singleton, but its mutable state
-    # is lock-guarded (query cache, reindexer, LSP sessions, graph project cache) and the
-    # semantic engine is thread-confined with WAL, so concurrent requests are safe.
+    # Threaded so one slow request (an LSP session warming, or a first-time index) can't block
+    # every other agent's query. The gateway is a shared singleton, but its mutable state is
+    # lock-guarded (query cache, reindexer, LSP sessions, graph project cache) and the semantic
+    # engine is thread-confined with WAL, so concurrent requests are safe.
     daemon_threads = True
+    auth_token: str | None = None  # set by run() when a token is provided
+
+    def __init__(self, *args, **kwargs) -> None:
+        super().__init__(*args, **kwargs)
+        # Stdlib ThreadingHTTPServer spawns one thread per connection with no ceiling, so many
+        # slow/half-open clients could exhaust threads/FDs. Bound live workers; past the cap we
+        # refuse fast with 503 rather than spawn an unbounded thread. (For a genuinely hostile
+        # network, still front this with a reverse proxy — http.server is not hardened for that.)
+        self._slots = threading.BoundedSemaphore(_MAX_CONCURRENT_REQUESTS)
+
+    def process_request(self, request, client_address) -> None:
+        if not self._slots.acquire(blocking=False):
+            try:
+                request.sendall(
+                    b"HTTP/1.1 503 Service Unavailable\r\n"
+                    b"Content-Type: application/json\r\nContent-Length: 22\r\n"
+                    b'Connection: close\r\n\r\n{"error":"overloaded"}'
+                )
+            except Exception:
+                pass
+            self.shutdown_request(request)
+            return
+        super().process_request(request, client_address)  # spawns the worker thread
+
+    def process_request_thread(self, request, client_address) -> None:
+        try:
+            super().process_request_thread(request, client_address)
+        finally:
+            self._slots.release()
+
+    def handle_error(self, request, client_address) -> None:
+        # A stalled/reset client raises here (socket timeout, broken pipe) — expected noise, not a
+        # server bug (handlers are never-raise). Keep it off stderr; surface with CODEINTEL_DEBUG=1.
+        log_swallowed("http.handle_error", sys.exc_info()[1] or Exception("unknown"))
 
 
 _LOOPBACK_NAMES = {"localhost"}
@@ -83,15 +152,24 @@ def _is_loopback(host: str) -> bool:
         return False  # a non-IP hostname is never treated as loopback
 
 
-def run(host: str = "127.0.0.1", port: int = 8766, *, allow_remote: bool = False) -> None:
+def run(
+    host: str = "127.0.0.1",
+    port: int = 8766,
+    *,
+    allow_remote: bool = False,
+    token: str | None = None,
+) -> None:
     if not _is_loopback(host) and not allow_remote:
         print(f"refusing to bind non-loopback host {host!r} without --allow-remote — this would "
               f"expose an UNAUTHENTICATED code-intel endpoint (your indexed repo) to the network",
               file=sys.stderr)
         raise SystemExit(2)
-    if not _is_loopback(host):
-        print(f"WARNING: serving codeintel on {host}:{port} with NO authentication — anyone who can "
-              f"reach this port can read your indexed repo", file=sys.stderr)
     server = CodeIntelHTTPServer((host, port), _Handler)
-    print(f"Listening on http://{host}:{port}")
+    server.auth_token = (token or "").strip() or None
+    if not _is_loopback(host) and not server.auth_token:
+        print(f"WARNING: serving codeintel on {host}:{port} with NO authentication — anyone who can "
+              f"reach this port can read your indexed repo (set --token to require a bearer token)",
+              file=sys.stderr)
+    auth_note = "  (bearer-token auth required)" if server.auth_token else ""
+    print(f"Listening on http://{host}:{port}{auth_note}")
     server.serve_forever()

codeintel/indexer.py

@@ -37,12 +37,14 @@ class Indexer:
         window: int = 20,
         stride: int = 10,
         max_chunks: int = 500,
+        max_total_chunks: int = 100000,
     ) -> None:
         self.db = db
         self.model_name = model_name
         self.window = window
         self.stride = stride
-        self.max_chunks = max_chunks
+        self.max_chunks = max_chunks               # per file
+        self.max_total_chunks = max_total_chunks   # safety ceiling per index pass (memory backstop)
         self._embedder = None
 
     def _get_embedder(self):
@@ -131,6 +133,13 @@ class Indexer:
         new_chunks: list[tuple[str, str, str, int, str]] = []
 
         for filepath in self._walk_files(root):
+            if len(new_chunks) >= self.max_total_chunks:
+                logger.warning(
+                    "index: reached max_total_chunks=%d this pass — stopping "
+                    "(raise it in .codeintel.toml to embed more of a very large repo)",
+                    self.max_total_chunks,
+                )
+                break
             try:
                 with open(filepath, encoding="utf-8", errors="replace") as f:
                     lines = f.readlines()
@@ -145,6 +154,8 @@ class Indexer:
             chunk_count = 0
 
             for chunk_start in range(0, len(lines), self.stride):
+                if len(new_chunks) >= self.max_total_chunks:
+                    break  # global ceiling reached mid-file; the file-loop guard logs and stops
                 if chunk_count >= self.max_chunks:
                     logger.debug(
                         "chunk cap hit for %s, truncating at %d",

codeintel/onboarding.py

@@ -72,6 +72,7 @@ def _bounded_index(project_root: str, *, timeout_s: float, out) -> dict:
                     db, model_name=str(cfg.get("model") or "BAAI/bge-small-en-v1.5"),
                     window=int(cfg.get("window", 20)), stride=int(cfg.get("stride", 10)),
                     max_chunks=int(cfg.get("max_chunks", 500)),
+                    max_total_chunks=int(cfg.get("max_total_chunks", 100000)),
                 ).index(project_root)
             finally:
                 db.close()

codeintel/provider.py

@@ -1,8 +1,27 @@
 from __future__ import annotations
 
+import logging
+import os
+import traceback
 from typing import Any, Optional, Protocol, runtime_checkable
 from typing_extensions import NotRequired, TypedDict
 
+_logger = logging.getLogger("codeintel")
+_DEBUG = os.environ.get("CODEINTEL_DEBUG", "").strip().lower() in ("1", "true", "on", "yes")
+
+
+def log_swallowed(where: str, exc: BaseException) -> None:
+    """Record an exception the never-raise contract is about to swallow. Quiet by default so the
+    contract stays silent in normal use; set ``CODEINTEL_DEBUG=1`` to surface a full traceback when
+    diagnosing why a query came back as a safe-null. Never raises — logging failures are ignored."""
+    try:
+        if _DEBUG:
+            _logger.warning("codeintel swallowed error in %s: %s\n%s", where, exc, traceback.format_exc())
+        else:
+            _logger.debug("codeintel swallowed error in %s: %s", where, exc)
+    except Exception:
+        pass
+
 
 class Result(TypedDict):
     ok: bool

codeintel/providers/graph.py

@@ -7,7 +7,7 @@ import threading
 import time
 from typing import Any, Optional
 
-from codeintel.provider import Result, safe_null_result
+from codeintel.provider import Result, log_swallowed, safe_null_result
 
 
 def _cypher_literal(s: Any) -> str:
@@ -35,7 +35,8 @@ class GraphProvider:
     """
 
     def __init__(self) -> None:
-        self._project_cache: dict[str, Optional[str]] = {}
+        self._project_cache: dict[str, Optional[str]] = {}          # resolved names (stable, kept)
+        self._negative_until: dict[str, float] = {}                 # failed lookups, short TTL only
         self._project_cache_lock = threading.Lock()  # concurrent HTTP requests share one provider
         self._detect_backend()
 
@@ -129,14 +130,22 @@ class GraphProvider:
     def _resolve_project(self, project_root: str) -> Optional[str]:
         with self._project_cache_lock:
             if project_root in self._project_cache:
-                return self._project_cache[project_root]
+                return self._project_cache[project_root]  # positive: a repo's name is stable
+            neg_until = self._negative_until.get(project_root)
+            if neg_until is not None and time.monotonic() < neg_until:
+                return None  # a recently-failed lookup, still within its short TTL
         # list_projects shells out — resolve it OUTSIDE the lock so a slow backend can't serialize
-        # every concurrent request. A rare duplicate lookup on first contact is harmless (the
-        # result is idempotent); we simply never hold the lock across a subprocess.
+        # every concurrent request. A rare duplicate lookup on first contact is harmless.
         raw = self._run("list_projects", {}, 3000)
         name = self._match_project(raw, project_root)
         with self._project_cache_lock:
-            self._project_cache[project_root] = name
+            if name is not None:
+                self._project_cache[project_root] = name
+                self._negative_until.pop(project_root, None)
+            else:
+                # Cache the MISS only briefly, so a repo indexed into the graph AFTER this failed
+                # lookup is picked up within the TTL rather than staying stuck until a restart.
+                self._negative_until[project_root] = time.monotonic() + 30.0
         return name
 
     def probe(self, project_root: str, timeout_ms: int = 3000) -> dict:
@@ -400,7 +409,8 @@ class GraphProvider:
                 "engine": "graph",
                 "cached": False,
             }
-        except Exception:
+        except Exception as exc:
+            log_swallowed("GraphProvider.build_result", exc)
             return safe_null_result(op, target, engine="graph", reason="error")
 
     def _dispatch(

codeintel/providers/lsp.py

@@ -12,7 +12,7 @@ from typing import Any, Optional
 from mcp import ClientSession
 from mcp.client.stdio import stdio_client
 
-from codeintel.provider import Result, safe_null_result
+from codeintel.provider import Result, log_swallowed, safe_null_result
 
 _COOLDOWN_SECONDS = 60
 _DEFAULT_TIMEOUT_S = 5.0
@@ -231,7 +231,8 @@ class LspProvider:
                 "engine": "lsp",
                 "cached": False,
             }
-        except Exception:
+        except Exception as exc:
+            log_swallowed("LspProvider.build_result", exc)
             return safe_null_result(op, target, engine="lsp", reason="error")
 
     def _dispatch(

codeintel/providers/semantic.py

@@ -1,8 +1,9 @@
 from __future__ import annotations
 
+import os
 import pathlib
 
-from codeintel.provider import Result, safe_null_result
+from codeintel.provider import Result, log_swallowed, safe_null_result
 
 try:
     import fastembed  # noqa: F401
@@ -101,15 +102,26 @@ class SemanticProvider:
             db = SemanticDb(str(_DB_PATH))
             db.init()
 
-            Indexer(
-                db,
-                model_name=model,
-                window=int(cfg.get("window", 20)),
-                stride=int(cfg.get("stride", 10)),
-                max_chunks=int(cfg.get("max_chunks", 500)),
-            ).index(project_root)
-
             searcher = Searcher(db, model_name=model)
+
+            # A full index pass walks and hashes every file — too expensive to run on every query.
+            # The background Reindexer (gated by the CODEINTEL_REINDEX env) already keeps a warm repo
+            # fresh, so we only pay the inline pass on a COLD repo (nothing indexed yet). The one
+            # exception: when that background reindexer is turned off, the inline pass is the only
+            # thing keeping the index current, so we run it every query to preserve freshness.
+            background_reindex_off = (
+                os.environ.get("CODEINTEL_REINDEX", "on").strip().lower() == "off"
+            )
+            if background_reindex_off or not searcher.has_index(project_root):
+                Indexer(
+                    db,
+                    model_name=model,
+                    window=int(cfg.get("window", 20)),
+                    stride=int(cfg.get("stride", 10)),
+                    max_chunks=int(cfg.get("max_chunks", 500)),
+                    max_total_chunks=int(cfg.get("max_total_chunks", 100000)),
+                ).index(project_root)
+
             if not searcher.has_index(project_root):
                 return safe_null_result(
                     op, target, engine="semantic", reason="no-index",
@@ -135,5 +147,6 @@ class SemanticProvider:
                 "cached": False,
             }
             return result
-        except Exception:
+        except Exception as exc:
+            log_swallowed("SemanticProvider.build_result", exc)
             return safe_null_result(op, target, engine="semantic", reason="provider-error")

codeintel/reindexer.py

@@ -71,8 +71,21 @@ class Reindexer:
                 return
             self._last_fired[project_root] = now
 
+        # Passed the debounce gate — honor a per-project `reindex = "never"` opt-out before doing
+        # expensive work, so that config key actually disables background reindexing (not only the
+        # inline path). Checked post-debounce, so config is read at most once per window.
+        if self._reindex_disabled(project_root):
+            return
+
         self._executor.submit(self._do_reindex, project_root)
 
+    def _reindex_disabled(self, project_root: str) -> bool:
+        try:
+            from codeintel.config import load_config
+            return str(load_config(project_root).get("reindex") or "").strip().lower() == "never"
+        except Exception:
+            return False
+
     def _do_reindex(self, project_root: str) -> None:
         try:
             self._semantic_reindex(project_root)
@@ -85,17 +98,27 @@ class Reindexer:
     def _semantic_reindex(self, project_root: str) -> None:
         import pathlib
 
+        from codeintel.config import load_config
         from codeintel.semantic_db import SemanticDb, default_db_path
         from codeintel.indexer import Indexer
 
-        # Same per-machine cache the SemanticProvider reads — index and search must
-        # never diverge onto different files.
+        # Same per-machine cache the SemanticProvider reads — index and search must never diverge
+        # onto different files. Honor the project's config so the background pass indexes exactly
+        # like the inline and CLI paths (same model, window/stride, and chunk ceilings).
+        cfg = load_config(project_root)
         db_path = default_db_path()
         pathlib.Path(db_path).parent.mkdir(parents=True, exist_ok=True)
         db = SemanticDb(db_path)
         try:
             db.init()
-            Indexer(db).index(project_root)
+            Indexer(
+                db,
+                model_name=str(cfg.get("model") or "BAAI/bge-small-en-v1.5"),
+                window=int(cfg.get("window", 20)),
+                stride=int(cfg.get("stride", 10)),
+                max_chunks=int(cfg.get("max_chunks", 500)),
+                max_total_chunks=int(cfg.get("max_total_chunks", 100000)),
+            ).index(project_root)
         finally:
             db.close()
 

codeintel/server.py

@@ -103,15 +103,21 @@ def code_status_handler(args: dict) -> dict:
         if not engines:
             engines.append("none")
 
-        # Report real freshness/model instead of hardcoded nulls (SPEC §7).
+        # Report real freshness/model instead of hardcoded nulls (SPEC §7). When a project_root is
+        # supplied, `indexed` is scoped to THAT repo (does it have indexed chunks?) rather than the
+        # misleading "any semantic db file exists on this machine".
+        project_root = str(args.get("project_root", "") or "")
         indexed = False
         model = None
         try:
-            import os
             from codeintel.semantic_db import DEFAULT_MODEL, default_db_path
             if semantic_available:
                 model = DEFAULT_MODEL
-                indexed = os.path.exists(default_db_path())
+                if project_root:
+                    indexed = bool(SemanticProvider().probe(project_root).get("repo_indexed"))
+                else:
+                    import os
+                    indexed = os.path.exists(default_db_path())
         except Exception:
             pass
 
@@ -199,8 +205,8 @@ def run() -> None:
             {"op": op, "target": target, "project_root": project_root, "engine": engine, "role": role}
         )
 
-    async def _code_status() -> dict:
-        return code_status_handler({})
+    async def _code_status(project_root: str = "") -> dict:
+        return code_status_handler({"project_root": project_root})
 
     async def _code_doctor(project_root: str = "", deep: bool = False) -> dict:
         return code_doctor_handler({"project_root": project_root, "deep": deep})

codecortex-0.2.2.dist-info/RECORD

@@ -1,31 +0,0 @@
-codecortex-0.2.2.dist-info/licenses/LICENSE,sha256=DIRvhlH8EulEUOYQxKDFFiimJrlVT5-kCKxQIgi2m7c,1073
-codeintel/__init__.py,sha256=m6kyaNpwBcP1XYcqrelX2oS3PJuOnElOcRdBa9pEb8c,22
-codeintel/__main__.py,sha256=3MpEhdpAfHiO0rwIB7hxXM_mDN3RovPQBLXNH36aLUg,15189
-codeintel/cache.py,sha256=VhAc_xEA3FtC_dSLXHPe646FKvL-cgPWm27S9i6VuYE,3156
-codeintel/config.py,sha256=Q7Mgq3UleJO6cpBKr1S7I1SeRbyL_rDekmCi8iasy9s,1133
-codeintel/doctor.py,sha256=jiPuZzS801xfWwTyeVO9UHUxQqAJ-YqTlX-_epZfoyI,6659
-codeintel/gateway.py,sha256=1c0rtjGGcgaa63Klk79LtdeOTl2ld8cvcnFlRSpKNsg,8929
-codeintel/http_server.py,sha256=J44uZqpXFJ06X-wDOZZvLEM1wftuokKCvS09Z62eUyU,3942
-codeintel/indexer.py,sha256=QKQtVu79DsEf1qSj9pLSdrspy5ajgzuNmupaZFL128s,9578
-codeintel/injector.py,sha256=IHZTLajQURE2LLxL8dFE9pvJV1RMOAbE7pVRQbFVvgs,2755
-codeintel/installer.py,sha256=PSgK6KA14VEQs2pQKCmjf9JDxJ2mpoEsaWrpFGWSy68,3488
-codeintel/mapper.py,sha256=JN-lMO1CXIyDKSwHfzp0JuK396FR41XhFgtAip77G0E,7493
-codeintel/onboarding.py,sha256=TOpRKkFvN2UesA7lW233s5PUJTFj98i7sby953EGWIQ,8846
-codeintel/policy.py,sha256=mUAlk9oYoszQLEcvoroFLRd9DSTzho7YcbM7xb6RS-k,935
-codeintel/provider.py,sha256=SlIModATCANJA4VyJs_Rz24bpQfzHG8ljHfpS1LWa8c,1214
-codeintel/reindexer.py,sha256=zFNeLgPRZPfeczc5Cx3aPYkJpyrjd6AHyvuqsrLGf8E,4443
-codeintel/reset.py,sha256=8Mpak4Tj5e-yTcM3_EnZJzbJPwPV2o8xpxGjv-wk9EE,3675
-codeintel/searcher.py,sha256=R9PV7jSx-GO--qmh65FXlYxpbKbjCnxV-EF9hGe8hZU,4588
-codeintel/semantic_db.py,sha256=9umQ8UUUtHHy7Ngi0B-e8eGCy62XFIOkcWnd3wc3MMQ,3503
-codeintel/server.py,sha256=9eo531Hj1XZYz3I6Zx2DQZ-fsYPYtmIRJH2TTe5yRjo,7031
-codeintel/term.py,sha256=csbSYD8euC13FnnpThy8kylDgZeejQ6j6Ol6Gu21Clo,6819
-codeintel/providers/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-codeintel/providers/graph.py,sha256=JUAnAo7UioUMZJ3PUGrKT4svWQ3mmaTa14JMKAGVvNk,19401
-codeintel/providers/lsp.py,sha256=3jn2bMfo5jiaL4_X3kIjzzxLsL475xIZsT-okWy-vgg,16759
-codeintel/providers/none.py,sha256=VI84XQkT7opQm6rBAEn0CdM8CNVlKGWYmjAhyeWbHeg,783
-codeintel/providers/semantic.py,sha256=Yka9Puyd8LqY2jfvjoJzAsSMZvb3S-cViQC6fuesdUk,5397
-codecortex-0.2.2.dist-info/METADATA,sha256=kL47WPI6VYU46EBctBloS984tmQ8ZRGNyfKNxRwYzAs,15376
-codecortex-0.2.2.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
-codecortex-0.2.2.dist-info/entry_points.txt,sha256=zrYfo95-8JkK0G2e5f9d94sYqqim7ZrdCW8knIfSy2U,54
-codecortex-0.2.2.dist-info/top_level.txt,sha256=DF3TH1hHLWrrnX-7HhfMX0OnS2EXgqPRhPbiL_oSekw,10
-codecortex-0.2.2.dist-info/RECORD,,