scip-cli 2.0.0__tar.gz → 2.1.0__tar.gz

{scip_cli-2.0.0 → scip_cli-2.1.0}/.cursor/rules/agent.mdc

@@ -7,26 +7,32 @@ alwaysApply: true
 
 **Work from repo root.** CLI usage, flags, and user gotchas: `scip-cli skill` (source: `scip_cli/SKILL.md`). Read once per task — do not duplicate SKILL here.
 
-Use **`.venv/bin/scip-cli`** / **`.venv/bin/pytest`**. Uncommitted code: `PYTHONPATH=$PWD python -m scip_cli …` if the venv binary is stale.
+**Almost always:** bare `scip-cli` on PATH must be **this checkout live** (`pip install -e ".[dev]"`). All examples use `scip-cli` — never `.venv/bin/scip-cli`, `python -m scip_cli`, or other paths. Only swap to PyPI briefly to smoke-test a release.
 
 ## Bootstrap
 
+From repo root — keep this mapping for the whole dev session:
+
 ```bash
-python -m venv .venv && source .venv/bin/activate
 pip install -e ".[dev]"
+scip-cli --version    # must match scip_cli/__init__.py
 pre-commit install
-scip-cli skill
+scip-cli skill ~/.claude/skills/scip-cli/
 ```
 
+After publishing: `pip install scip-cli` → smoke-test → `pip install -e ".[dev]"` to restore local.
+
+Optional venv for isolation only: create/activate, then same `pip install -e ".[dev]"` — `scip-cli` still runs from PATH, not a venv-relative path.
+
 Node.js + `npx` required for integration tests (`scip-typescript`). Optional: `SCIP_CLI_DEBUG=1` (SQL to stderr, truncated).
 
 ## Lint / test
 
 | Task | Command |
 |------|---------|
-| Full suite | `.venv/bin/pytest` |
-| E2e loop | `.venv/bin/pytest tests/test_e2e.py` |
-| Lint / format | `.venv/bin/ruff check .` / `ruff format .` |
+| Full suite | `pytest` |
+| E2e loop | `pytest tests/test_e2e.py` |
+| Lint / format | `ruff check .` / `ruff format .` |
 
 Pre-commit: ruff + ruff-format + full pytest. E2e before commit on command changes.
 
@@ -35,11 +41,11 @@ Pre-commit: ruff + ruff-format + full pytest. E2e before commit on command chang
 **Dogfood on the repo itself** — faster and more honest than spawning reviewers:
 
 ```bash
-.venv/bin/scip-cli reindex
-.venv/bin/scip-cli analyze --limit 25
-.venv/bin/scip-cli analyze --priority high --limit 25   # dead code & cycles only
-.venv/bin/scip-cli analyze scip_cli --limit 15              # directory
-.venv/bin/scip-cli analyze scip_cli/queries.py --limit 20   # file (+ scoped project)
+scip-cli reindex
+scip-cli analyze --limit 25
+scip-cli analyze --priority high --limit 25   # dead code & cycles only
+scip-cli analyze scip_cli --limit 15              # directory
+scip-cli analyze scip_cli/queries.py --limit 20   # file (+ scoped project)
 ```
 
 Interpretation: README § *Finding easy wins with analyze*. Target = omit (project), directory, file, or symbol — not `--path` for scope.
@@ -103,6 +109,32 @@ SQLite has **line ranges** (`defn_enclosing_ranges`) so LOC is cheap. That does
 
 **Rule:** size-only checks stay **low or file-drill-down**; never high/medium unless paired with a **structural smell** (cycle, dead export, zero production consumers). Prefer enriching existing low-tier lines (e.g. show `loc` on bottlenecks) over a new dashboard section.
 
+## SQL / analyze benchmarks
+
+The e2e fixture and this repo's own index are **small** — every query looks fast here. That hides blowups on real projects (40MB+ indexes, dense graphs).
+
+**Before merging new or heavily changed SQL** (`queries.py`, `analyze/*`):
+
+1. Time it against **1–2 existing checks** on a **large cached index** you already have (not the fixture):
+   ```bash
+   python3 -u -c "
+   import sqlite3, time, sys
+   from pathlib import Path
+   sys.path.insert(0, '.')
+   from scip_cli.analyze import project as p
+   from scip_cli.sql import configure_read_connection
+   db_path = Path.home() / '.cache/scip-cli/projects/<project-slug>/index.db'
+   db = sqlite3.connect(f'file:{db_path}?mode=ro', uri=True)
+   configure_read_connection(db)
+   for name, fn in [('hotspots', p.hotspots), ('cycles', p.cycles), ('dead_exports', p.dead_exports)]:
+       t = time.perf_counter(); fn(db, limit=25); print(f'{name}: {(time.perf_counter()-t)*1000:.0f}ms')
+   "
+   ```
+2. Flag anything **notably slower** than peers — orders of magnitude or multi-second, not a few ms drift.
+3. Swap in your new/changed check for one of the rows; keep a fast peer (e.g. `hotspots`) and a heavy peer (e.g. `dead_exports`).
+
+**In CI:** `tests/test_e2e_perf.py` (whole commands) and `tests/test_analyze_perf.py` (per-check on mini DB) catch gross regressions on the tiny fixture only — they **will not** catch large-repo OOMs. Use the manual benchmark above for that.
+
 ## TDD
 
 1. **Red** — `tests/test_e2e.py` + `fixture_catalog` constants.

{scip_cli-2.0.0 → scip_cli-2.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scip-cli
-Version: 2.0.0
+Version: 2.1.0
 Summary: Fast code intelligence via SCIP indexes
 Project-URL: Homepage, https://github.com/flesler/scip-cli
 Author: Ariel Flesler
@@ -62,17 +62,24 @@ scip-cli skill
 pip install scip-cli
 ```
 
-**Local development** (use a project venv — do not rely on global `pip`):
+**Local development** — almost always keep bare `scip-cli` mapped to this checkout via editable install. You edit the repo; the CLI you run is live code, not a frozen PyPI copy.
 
 ```bash
 cd scip-cli
-python -m venv .venv
-source .venv/bin/activate   # Windows: .venv\Scripts\activate
 pip install -e ".[dev]"
 scip-cli --version
 ```
 
-`pip install -e .` keeps `scip-cli` on your PATH inside the venv while you edit the repo. Run tests with `pytest` from the same venv.
+All docs and examples use `scip-cli` (never `.venv/bin/scip-cli` or `python -m scip_cli`). After publishing, briefly `pip install scip-cli` to smoke-test PyPI, then `pip install -e ".[dev]"` again.
+
+Optional venv for isolation — activate it, run the same editable install, still invoke `scip-cli` on PATH:
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+Tests and lint: `pytest`, `ruff check .` from the same environment.
 
 ### 2. Install prerequisites (optional)
 

{scip_cli-2.0.0 → scip_cli-2.1.0}/README.md

@@ -42,17 +42,24 @@ scip-cli skill
 pip install scip-cli
 ```
 
-**Local development** (use a project venv — do not rely on global `pip`):
+**Local development** — almost always keep bare `scip-cli` mapped to this checkout via editable install. You edit the repo; the CLI you run is live code, not a frozen PyPI copy.
 
 ```bash
 cd scip-cli
-python -m venv .venv
-source .venv/bin/activate   # Windows: .venv\Scripts\activate
 pip install -e ".[dev]"
 scip-cli --version
 ```
 
-`pip install -e .` keeps `scip-cli` on your PATH inside the venv while you edit the repo. Run tests with `pytest` from the same venv.
+All docs and examples use `scip-cli` (never `.venv/bin/scip-cli` or `python -m scip_cli`). After publishing, briefly `pip install scip-cli` to smoke-test PyPI, then `pip install -e ".[dev]"` again.
+
+Optional venv for isolation — activate it, run the same editable install, still invoke `scip-cli` on PATH:
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+Tests and lint: `pytest`, `ruff check .` from the same environment.
 
 ### 2. Install prerequisites (optional)
 

{scip_cli-2.0.0 → scip_cli-2.1.0}/scip_cli/SKILL.md

@@ -7,6 +7,8 @@ TypeScript/JavaScript (.ts, .tsx, .js, .jsx) and Python (.py) — not GraphQL, C
 
 All commands are sub-commands of `scip-cli`. Run from the project root.
 
+**Contributors:** keep `pip install -e .` (or `pip install -e ".[dev]"` in this repo) so bare `scip-cli` on PATH runs your live checkout — not a venv-relative path.
+
 ## Quick Decision Guide
 
 | Question                                  | Use                 | What you get                                                                              |

{scip_cli-2.0.0 → scip_cli-2.1.0}/scip_cli/__init__.py

@@ -1,3 +1,3 @@
 """scip-cli: Fast code intelligence via SCIP indexes."""
 
-__version__ = "2.0.0"
+__version__ = "2.1.0"

scip_cli-2.1.0/scip_cli/analyze/graph.py

@@ -0,0 +1,130 @@
+"""File dependency graph helpers for analyze (cycle detection)."""
+
+from __future__ import annotations
+
+from collections import defaultdict
+
+from .common import fetch_all
+
+FILE_EDGES_SQL = """
+    SELECT DISTINCT d1.relative_path AS from_file, d2.relative_path AS to_file
+    FROM mentions m
+    JOIN chunks c ON m.chunk_id = c.id
+    JOIN documents d1 ON c.document_id = d1.id
+    JOIN defn_enclosing_ranges der ON m.symbol_id = der.symbol_id
+    JOIN documents d2 ON der.document_id = d2.id
+    WHERE d1.id != d2.id AND m.role != 1
+"""
+
+
+def fetch_file_edges(db) -> list[tuple[str, str]]:
+    return fetch_all(db, FILE_EDGES_SQL)
+
+
+def _tarjan_sccs(graph: dict[str, list[str]], nodes: set[str]) -> list[list[str]]:
+    index = 0
+    stack: list[str] = []
+    on_stack: set[str] = set()
+    indices: dict[str, int] = {}
+    lowlink: dict[str, int] = {}
+    sccs: list[list[str]] = []
+
+    def strongconnect(vertex: str) -> None:
+        nonlocal index
+        indices[vertex] = index
+        lowlink[vertex] = index
+        index += 1
+        stack.append(vertex)
+        on_stack.add(vertex)
+        for neighbor in graph.get(vertex, ()):
+            if neighbor not in indices:
+                strongconnect(neighbor)
+                lowlink[vertex] = min(lowlink[vertex], lowlink[neighbor])
+            elif neighbor in on_stack:
+                lowlink[vertex] = min(lowlink[vertex], indices[neighbor])
+        if lowlink[vertex] == indices[vertex]:
+            component: list[str] = []
+            while True:
+                w = stack.pop()
+                on_stack.remove(w)
+                component.append(w)
+                if w == vertex:
+                    break
+            sccs.append(component)
+
+    for node in nodes:
+        if node not in indices:
+            strongconnect(node)
+    return sccs
+
+
+def _cycles_in_scc(
+    graph: dict[str, list[str]],
+    scc_nodes: list[str],
+    *,
+    max_depth: int,
+    limit: int,
+) -> list[str]:
+    if len(scc_nodes) <= 2:
+        return []
+
+    scc = set(scc_nodes)
+    subgraph: dict[str, list[str]] = defaultdict(list)
+    for src in scc:
+        for dst in graph.get(src, ()):
+            if dst in scc:
+                subgraph[src].append(dst)
+
+    found: dict[tuple[str, ...], str] = {}
+
+    def record(path: list[str]) -> None:
+        key = min(tuple(path[i:] + path[:i]) for i in range(len(path)))
+        if key not in found:
+            found[key] = " -> ".join([*path, path[0]])
+
+    for origin in sorted(scc):
+        stack: list[tuple[str, list[str]]] = [(origin, [origin])]
+        while stack:
+            node, path = stack.pop()
+            if len(path) > max_depth:
+                continue
+            for nxt in subgraph.get(node, ()):
+                if nxt == origin and len(path) >= 2:
+                    record(path)
+                    if len(found) >= limit:
+                        return sorted(found.values())[:limit]
+                elif nxt not in path:
+                    stack.append((nxt, [*path, nxt]))
+    return sorted(found.values())
+
+
+def find_longer_cycles(
+    edges: list[tuple[str, str]],
+    *,
+    max_depth: int = 8,
+    limit: int,
+) -> list[str]:
+    """Find directed cycles with 3+ distinct files (2-node pairs use the SQL two-way check)."""
+    if limit <= 0 or not edges:
+        return []
+
+    graph: dict[str, list[str]] = defaultdict(list)
+    nodes: set[str] = set()
+    for src, dst in edges:
+        graph[src].append(dst)
+        nodes.add(src)
+        nodes.add(dst)
+
+    found: dict[tuple[str, ...], str] = {}
+    for component in _tarjan_sccs(graph, nodes):
+        if len(component) <= 2:
+            continue
+        for line in _cycles_in_scc(graph, component, max_depth=max_depth, limit=limit):
+            path = line.split(" -> ")
+            body = path[:-1]
+            key = min(tuple(body[i:] + body[:i]) for i in range(len(body)))
+            if key not in found:
+                found[key] = line
+            if len(found) >= limit:
+                return sorted(found.values())[:limit]
+    return sorted(found.values())[:limit]

{scip_cli-2.0.0 → scip_cli-2.1.0}/scip_cli/analyze/project.py

@@ -16,18 +16,9 @@ from .common import (
     short_name,
     stale_type_noise,
 )
+from .graph import FILE_EDGES_SQL, fetch_file_edges, find_longer_cycles
 from .sections import Check, Priority, run_checks
 
-_FILE_EDGES_SQL = """
-    SELECT DISTINCT d1.relative_path AS from_file, d2.relative_path AS to_file
-    FROM mentions m
-    JOIN chunks c ON m.chunk_id = c.id
-    JOIN documents d1 ON c.document_id = d1.id
-    JOIN defn_enclosing_ranges der ON m.symbol_id = der.symbol_id
-    JOIN documents d2 ON der.document_id = d2.id
-    WHERE d1.id != d2.id AND m.role != 1
-"""
-
 
 def _scope_suffix(scope: str | None) -> str:
     return f" [{scope}]" if scope else ""
@@ -134,34 +125,11 @@ def cycles(
     include_tests: bool = False,
     scope: str | None = None,
 ) -> list[str]:
-    rows = fetch_all(
-        db,
-        f"""
-        WITH RECURSIVE
-        edges AS ({_FILE_EDGES_SQL}),
-        walk(origin, current, depth, path) AS (
-            SELECT from_file, from_file, 0, from_file FROM edges
-            UNION ALL
-            SELECT walk.origin, e.to_file, walk.depth + 1,
-                   walk.path || ' -> ' || e.to_file
-            FROM walk
-            JOIN edges e ON walk.current = e.from_file
-            WHERE walk.depth < 8
-              AND instr(walk.path, e.to_file) = 0
-        )
-        SELECT DISTINCT walk.path || ' -> ' || walk.origin AS cycle_path
-        FROM walk
-        JOIN edges e ON walk.current = e.from_file AND e.to_file = walk.origin
-        WHERE walk.depth > 0
-        ORDER BY cycle_path
-        LIMIT ?
-        """,
-        (limit * 5,),
-    )
+    cap = limit * 5
     two_way = fetch_all(
         db,
         f"""
-        WITH edges AS ({_FILE_EDGES_SQL})
+        WITH edges AS ({FILE_EDGES_SQL})
         SELECT e1.from_file || ' <-> ' || e1.to_file
         FROM edges e1
         JOIN edges e2 ON e1.from_file = e2.to_file AND e1.to_file = e2.from_file
@@ -169,11 +137,11 @@ def cycles(
         ORDER BY 1
         LIMIT ?
         """,
-        (limit * 5,),
+        (cap,),
     )
     lines = [row[0] for row in two_way if not cycle_path_noise(row[0], include_tests=include_tests)]
-    for row in rows:
-        path = row[0]
+    longer = find_longer_cycles(fetch_file_edges(db), max_depth=8, limit=cap)
+    for path in longer:
         if path not in lines and not cycle_path_noise(path, include_tests=include_tests):
             lines.append(path)
     if scope:

scip_cli-2.1.0/tests/perf_util.py

@@ -0,0 +1,45 @@
+"""Timing helpers for coarse perf regression guards."""
+
+from __future__ import annotations
+
+import statistics
+import time
+from collections.abc import Callable
+from typing import TypeVar
+
+T = TypeVar("T")
+
+DEFAULT_RUNS = 3
+DEFAULT_WARMUP = 1
+# Ignore sub-millisecond noise; require both ratio and floor before flagging.
+MIN_REPORT_MS = 5.0
+
+
+def median_elapsed(
+    fn: Callable[[], T],
+    *,
+    runs: int = DEFAULT_RUNS,
+    warmup: int = DEFAULT_WARMUP,
+) -> float:
+    for _ in range(warmup):
+        fn()
+    samples: list[float] = []
+    for _ in range(runs):
+        start = time.perf_counter()
+        fn()
+        samples.append(time.perf_counter() - start)
+    return statistics.median(samples)
+
+
+def outliers_vs_median(
+    timings: dict[str, float],
+    *,
+    ratio: float,
+    floor_s: float = MIN_REPORT_MS / 1000.0,
+) -> dict[str, float]:
+    """Names where timing exceeds max(ratio x peer median, floor_s)."""
+    if not timings:
+        return {}
+    peer = statistics.median(timings.values())
+    threshold = max(peer * ratio, floor_s)
+    return {name: elapsed for name, elapsed in timings.items() if elapsed > threshold}

scip_cli-2.1.0/tests/test_analyze_graph.py

@@ -0,0 +1,30 @@
+"""Tests for analyze graph helpers."""
+
+from scip_cli.analyze.graph import find_longer_cycles
+
+from .analyze_db import AnalyzeDbBuilder
+
+
+class TestFindLongerCycles:
+    def test_three_node_cycle(self):
+        edges = [("a.ts", "b.ts"), ("b.ts", "c.ts"), ("c.ts", "a.ts")]
+        lines = find_longer_cycles(edges, max_depth=8, limit=10)
+        assert len(lines) == 1
+        assert "a.ts" in lines[0]
+        assert "b.ts" in lines[0]
+        assert "c.ts" in lines[0]
+
+    def test_two_node_skipped(self):
+        edges = [("a.ts", "b.ts"), ("b.ts", "a.ts")]
+        assert find_longer_cycles(edges, max_depth=8, limit=10) == []
+
+    def test_mini_codebase_no_long_cycles(self):
+        b = AnalyzeDbBuilder()
+        sym_x = b.define("src/cycle/a.ts", "alpha")
+        sym_y = b.define("src/cycle/b.ts", "beta")
+        b.reference("src/cycle/a.ts", sym_y)
+        b.reference("src/cycle/b.ts", sym_x)
+        from scip_cli.analyze.graph import fetch_file_edges
+
+        edges = fetch_file_edges(b.finish())
+        assert find_longer_cycles(edges, max_depth=8, limit=10) == []

scip_cli-2.1.0/tests/test_analyze_perf.py

@@ -0,0 +1,39 @@
+"""Relative timing guard for project analyze checks on the in-memory mini DB."""
+
+from __future__ import annotations
+
+from scip_cli.analyze import project as project_checks
+from tests.analyze_db import mini_codebase_db
+from tests.perf_util import median_elapsed, outliers_vs_median
+
+# Mini DB is tiny — generous ratio; still catches runaway SQL (e.g. recursive blowups).
+CHECK_RATIO = 25.0
+BASELINE_CHECKS = ("hotspots", "coupling", "stale_types", "dead_exports")
+
+
+class TestAnalyzeCheckPerf:
+    def test_project_checks_not_dramatically_slower_than_peers(self):
+        db = mini_codebase_db()
+        checks = {
+            "hotspots": project_checks.hotspots,
+            "coupling": project_checks.top_coupling,
+            "stale_types": project_checks.stale_types,
+            "cycles": project_checks.cycles,
+            "bottlenecks": project_checks.bottlenecks,
+            "dead_exports": project_checks.dead_exports,
+            "unreferenced": project_checks.unreferenced_symbols,
+            "same_file": project_checks.same_file_only,
+            "test_only": project_checks.symbols_test_only_consumers,
+        }
+        timings = {
+            name: median_elapsed(lambda fn=fn: fn(db, limit=20), runs=5, warmup=2) for name, fn in checks.items()
+        }
+
+        baseline = {k: timings[k] for k in BASELINE_CHECKS if k in timings}
+        slow = outliers_vs_median(timings, ratio=CHECK_RATIO)
+        baseline_slow = {k: v for k, v in slow.items() if k not in baseline}
+        assert not baseline_slow, (
+            f"Analyze check(s) >{CHECK_RATIO:.0f}x mini-DB peer median: "
+            + ", ".join(f"{k}={v * 1000:.2f}ms" for k, v in sorted(baseline_slow.items()))
+            + f" (peer median {sorted(baseline.values())[len(baseline) // 2] * 1000:.2f}ms)"
+        )

scip_cli-2.1.0/tests/test_e2e_perf.py

@@ -0,0 +1,53 @@
+"""Coarse e2e timing guard — fixture is tiny; catches gross CLI regressions only."""
+
+from __future__ import annotations
+
+import pytest
+
+from tests.e2e_harness import CliRunner
+from tests.fixture_catalog import CLASS_WIDGET, FN_GREET, HELPER_FILE
+from tests.perf_util import median_elapsed, outliers_vs_median
+
+pytestmark = pytest.mark.integration
+
+# Whole-command smoke: nothing should be orders of magnitude slower than peers.
+FAST_RATIO = 5.0
+ANALYZE_FULL_VS_HIGH_RATIO = 6.0
+
+
+def _time_cli(cli: CliRunner, *argv: str) -> float:
+    return median_elapsed(lambda: cli.run(*argv))
+
+
+class TestE2eCommandPerf:
+    def test_command_timings_no_dramatic_outliers(self, cli):
+        """On the small fixture all commands finish in a few ms — flag huge skew only."""
+        fast = {
+            "search": _time_cli(cli, "search", FN_GREET, "--limit", "3"),
+            "symbols": _time_cli(cli, "symbols", HELPER_FILE, "--limit", "10"),
+            "code": _time_cli(cli, "code", FN_GREET, "--limit", "1"),
+            "refs": _time_cli(cli, "refs", FN_GREET, "--paths-only", "--limit", "10"),
+            "members": _time_cli(cli, "members", CLASS_WIDGET, "--names-only"),
+            "rdeps": _time_cli(cli, "rdeps", HELPER_FILE, "--limit", "10"),
+        }
+        analyze_high = _time_cli(cli, "analyze", "--priority", "high", "--limit", "5")
+        analyze_all = _time_cli(cli, "analyze", "--limit", "5")
+
+        fast_outliers = outliers_vs_median(fast, ratio=FAST_RATIO)
+        assert not fast_outliers, f"Fast commands skewed vs peers on fixture (>{FAST_RATIO:.0f}x median): " + ", ".join(
+            f"{k}={v * 1000:.1f}ms" for k, v in sorted(fast_outliers.items())
+        )
+
+        fast_peer = sorted(fast.values())[len(fast) // 2]
+        analyze_high_limit = max(fast_peer * FAST_RATIO, analyze_high)
+        assert analyze_high <= analyze_high_limit, (
+            f"analyze --priority high too slow on fixture: {analyze_high * 1000:.1f}ms "
+            f"(limit {analyze_high_limit * 1000:.1f}ms)"
+        )
+
+        analyze_all_limit = max(analyze_high * ANALYZE_FULL_VS_HIGH_RATIO, analyze_high)
+        assert analyze_all <= analyze_all_limit, (
+            f"analyze (all priorities) too slow vs high-only on fixture: "
+            f"{analyze_all * 1000:.1f}ms vs high {analyze_high * 1000:.1f}ms "
+            f"(limit {analyze_all_limit * 1000:.1f}ms)"
+        )