scip-cli 1.3.0__tar.gz → 2.0.0__tar.gz

scip_cli-2.0.0/.cursor/rules/agent.mdc

@@ -0,0 +1,133 @@
+---
+description: scip-cli dev workflow and TDD with the shared fixture
+alwaysApply: true
+---
+
+# scip-cli agent
+
+**Work from repo root.** CLI usage, flags, and user gotchas: `scip-cli skill` (source: `scip_cli/SKILL.md`). Read once per task — do not duplicate SKILL here.
+
+Use **`.venv/bin/scip-cli`** / **`.venv/bin/pytest`**. Uncommitted code: `PYTHONPATH=$PWD python -m scip_cli …` if the venv binary is stale.
+
+## Bootstrap
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+pre-commit install
+scip-cli skill
+```
+
+Node.js + `npx` required for integration tests (`scip-typescript`). Optional: `SCIP_CLI_DEBUG=1` (SQL to stderr, truncated).
+
+## Lint / test
+
+| Task | Command |
+|------|---------|
+| Full suite | `.venv/bin/pytest` |
+| E2e loop | `.venv/bin/pytest tests/test_e2e.py` |
+| Lint / format | `.venv/bin/ruff check .` / `ruff format .` |
+
+Pre-commit: ruff + ruff-format + full pytest. E2e before commit on command changes.
+
+## Improving this repo (do this before review subagents)
+
+**Dogfood on the repo itself** — faster and more honest than spawning reviewers:
+
+```bash
+.venv/bin/scip-cli reindex
+.venv/bin/scip-cli analyze --limit 25
+.venv/bin/scip-cli analyze --priority high --limit 25   # dead code & cycles only
+.venv/bin/scip-cli analyze scip_cli --limit 15              # directory
+.venv/bin/scip-cli analyze scip_cli/queries.py --limit 20   # file (+ scoped project)
+```
+
+Interpretation: README § *Finding easy wins with analyze*. Target = omit (project), directory, file, or symbol — not `--path` for scope.
+
+| Priority | Section | Action |
+|----------|---------|--------|
+| 1 | **Cycles** | Break production file dependency cycles (`cache`↔`scope` was one) |
+| 2 | **Unreferenced / dead exports** | No ref usage / no external refs — verify with `rg`, delete or `_` prefix |
+| 3 | **Same-file only** | Module-private by usage — rename to `_` |
+| 4 | **Stale types** | Low-consumer classes — merge or justify |
+
+`analyze` skips test paths by default (`tests/`, `*.test.*`, `*.spec.*`); `--include-tests` to include. Reindex after large edits.
+
+Do **not** spawn broad code-review agents until this pass is done (or explain why skipped).
+
+## Analyze feedback loop (mandatory)
+
+When you or a review subagent finds a repo issue — bug, dead code, cycle, risky hub, stale type, unused import — **stop and ask first:**
+
+> *Could our SQLite index have predicted this, and `analyze` just doesn't surface it yet?*
+
+If **yes** (the signal is in `documents`, `mentions`, `global_symbols`, `defn_enclosing_ranges`):
+
+1. **Extend `analyze` first** — new or tuned SQL check in `scip_cli/analyze/`, priority in `sections.py`, tests in `tests/test_analyze.py` + `analyze_db.py`.
+2. **Dogfood** — `reindex` → confirm the issue appears (or the false positive is filtered).
+3. **Then** fix the production code (or document a known blind spot).
+
+Only skip step 1 when the issue is **outside the index** (runtime, CLI UX, docs drift, git, env, concurrency, test harness).
+
+### Retrospective: caught manually, SQLite already had it
+
+| Issue we hit | Index signal | Analyze today | Gap / next check |
+|--------------|--------------|---------------|------------------|
+| `cache` ↔ `scope` import cycle | cross-file `mentions` edges | `[high] Cycles` | **Caught** — keep dogfooding cycles |
+| Cross-file dead export (`get_refs_for_symbols`) | def with no external `mentions` | `[high] Dead exports` | **Caught** |
+| Same-file-only dead code | def, `role=0` only in def file | `[high] Unreferenced` or `[medium] Same-file only` | **Caught** (since extended) |
+| Same-file helper without `_` (`run_with_fallback`) | same-file `role=0` or none | `[medium] Same-file only` / unreferenced | **Caught** — rename to `_` |
+| Typing-only dataclasses (`ProjectSettings`, `IndexScope`) | type `#` symbol, 0 consumer files | `[high] Stale types` + `stale_type_noise` | **Filtered** — index can't see annotations; document or optional `--types` |
+| `analyze/*` section runners “dead” | no external refs | noise filter in `common.py` | **Filtered** — framework pattern |
+| Unused import | `role=2` mention, no in-file use | `[high] Unused imports` (per file) | **Caught** on file target |
+| Hub before editing (`resolve_symbol`, `queries.py`) | fan-in / consumer count | `[medium] Change surface` / `[low] Hotspots` | **Caught** on file target; project hubs are low tier |
+| Symbol only used from tests | consumers all under `tests/` | `[low] Test-only consumers` | **Partial** — Python index often omits same-file `role=0`; verify with `rg` |
+| Indexer omits same-file Python calls | no `role=0` for in-module use | unreferenced / test-only false positives | **Index gap** — not fixable in SQL alone; demoted test-only to low |
+| Duplicate / parallel helpers | — | — | **Not in index** — `rg`, not analyze |
+
+Add new rows to `dogfood.md` when you find another gap; implement the SQL check before the next manual review pass.
+
+### Size / complexity checks (assessed — don’t promote to high/medium)
+
+SQLite has **line ranges** (`defn_enclosing_ranges`) so LOC is cheap. That does **not** mean “big function” belongs next to dead exports.
+
+| Idea | In index? | Verdict |
+|------|-----------|---------|
+| **LOC ≥ N** alone | Yes | **LOW at best** — noisy (`main`, argparse, legitimate parsers). Not a nuke candidate. |
+| **LOC + fan-in** (large hub) | Yes | **Already covered** by `[low] Bottlenecks` (fan-in × fan-out). Add `loc=` to that output if helpful; don’t duplicate as medium. |
+| **LOC + consumers** on file target | Yes | **Change surface** (medium) already sorts by consumers with line ranges — drill there. |
+| **Symbol pressure** | Yes | **Per-symbol** `analyze Foo` shows `loc × fan_in × fan_out`. |
+| **God class / many members** | Yes | Use `members Class` — not analyze. |
+| **File byte size / cyclomatic complexity** | No | Out of scope for SCIP index. |
+| **Fat file (many symbols)** | Yes | Low value; coupling/hotspots approximate module heat. |
+
+**Rule:** size-only checks stay **low or file-drill-down**; never high/medium unless paired with a **structural smell** (cycle, dead export, zero production consumers). Prefer enriching existing low-tier lines (e.g. show `loc` on bottlenecks) over a new dashboard section.
+
+## TDD
+
+1. **Red** — `tests/test_e2e.py` + `fixture_catalog` constants.
+2. **Green** — `scip_cli/commands/*.py`.
+3. **Refactor** — unit tests for pure helpers only.
+
+- **E2e** (`cli` fixture): in-process `main()`, TS project indexed once per session — default for behavior.
+- **Unit**: `:memory:` SQLite, `tests/analyze_db.py`, merge/index prune tests.
+- **No subprocess** for CLI tests; no `reindex` in e2e (mutates `~/.cache/scip-cli`).
+- New symbols: extend `tests/fixtures/sample-project/` + `fixture_catalog.py`.
+
+## Codebase map (contributors)
+
+| Area | Path | Notes |
+|------|------|-------|
+| Commands | `scip_cli/commands/` | Thin; SQL in `queries.py` / `analyze/` |
+| Index build | `scip_cli/indexing.py` | `index.db.next` → promote; flock; `>10` tsconfigs log progress |
+| Variable prune | postprocess + `merge.py` | Omit const/let/var on **COPY**, not DELETE — no `variable` kind |
+| Session | `scip_cli/session.py` | `setup()`, `resolve_one_symbol` (limit 2 for ambiguity) |
+| Analyze filters | `scip_cli/analyze/common.py` | `is_test_path`, `analyze_noise` (`_` helpers) |
+
+`todo.md` = tracked work; `dogfood.md` = future ideas only. Wishlist/backlog in `docs/` per workspace rules.
+
+## Docs touch points
+
+- User install + analyze guide → `README.md`
+- Agent/user CLI reference → `scip_cli/SKILL.md` (must match `__main__.py` flags)
+- After flag changes: update SKILL + README command list; run `scip-cli skill` to verify output

scip_cli-2.0.0/.gitignore

@@ -0,0 +1,53 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+venv/
+ENV/
+env/
+.venv
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Env/secrets
+.env
+
+# Pytest
+.pytest_cache/
+
+# Project specific
+TODO.md
+todo.md
+dogfood.md
+
+# Local-only dev scripts (repo-specific smoke harnesses)
+scripts/deep-smoke.sh
+tmp/

scip_cli-2.0.0/.pre-commit-config.yaml

@@ -0,0 +1,15 @@
+repos:
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.11.0
+    hooks:
+      - id: ruff
+        args: [--fix]
+      - id: ruff-format
+  - repo: local
+    hooks:
+      - id: pytest
+        name: pytest
+        entry: pytest
+        language: python
+        pass_filenames: false
+        always_run: true

{scip_cli-1.3.0/scip_cli.egg-info → scip_cli-2.0.0}/PKG-INFO

@@ -1,81 +1,78 @@
 Metadata-Version: 2.4
 Name: scip-cli
-Version: 1.3.0
+Version: 2.0.0
 Summary: Fast code intelligence via SCIP indexes
-Home-page: https://github.com/flesler/scip-cli
+Project-URL: Homepage, https://github.com/flesler/scip-cli
 Author: Ariel Flesler
 License: MIT
-Classifier: Programming Language :: Python :: 3
+License-File: LICENSE
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
 Classifier: Topic :: Software Development :: Code Generators
 Requires-Python: >=3.9
+Provides-Extra: dev
+Requires-Dist: build>=1; extra == 'dev'
+Requires-Dist: pre-commit>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: ruff>=0.11; extra == 'dev'
 Description-Content-Type: text/markdown
-License-File: LICENSE
-Dynamic: author
-Dynamic: classifier
-Dynamic: description
-Dynamic: description-content-type
-Dynamic: home-page
-Dynamic: license
-Dynamic: license-file
-Dynamic: requires-python
-Dynamic: summary
 
 # scip-cli
 
 [![PyPI version](https://badge.fury.io/py/scip-cli.svg)](https://badge.fury.io/py/scip-cli)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-Fast code intelligence CLI for TypeScript/JavaScript and Python projects. Query SCIP indexes directly via SQLite for instant results.
+Token-efficient code intelligence for AI agents. Precise refs, definitions, and repo health analysis via SCIP indexes — TypeScript/JavaScript and Python.
+
+## Why
+
+AI agents waste tokens on grep and file scanning. scip-cli gives them precise, type-aware code navigation in milliseconds — and `analyze` surfaces dead code, cycles, and coupling so agents (and humans) can fix real problems fast.
 
 ## Features
 
-- **Fast**: Direct SQLite queries, eliminating skippable overhead
-- **Simple**: Single binary with subcommands
-- **Auto-indexing**: Automatically indexes projects on first query
-- **Token-efficient**: Clean, minimal output optimized for AI consumption
+- **Agent-first**: Install as a skill for Claude Code, Cursor, or any AI agent — precise code navigation without burning context
+- **Token-efficient**: One record per line, stderr for warnings, pipe-friendly output
+- **Fast**: Direct SQLite queries — 10x to 213x faster than alternatives
+- **`analyze`**: Find dead exports, import cycles, stale types, coupling hotspots — actionable health dashboards at project, file, or symbol scope
+- **Auto-indexing**: Indexes on first query, caches in SQLite, zero config
 
 ## For AI Agents
 
-If you're an AI agent, run this to see the quick reference:
+Install as a reusable skill so your agent always knows how to navigate the codebase:
 
 ```bash
-scip-cli skill
+scip-cli skill ~/.claude/skills/scip-cli/   # Claude Code
+scip-cli skill ~/.cursor/skills/scip-cli/   # Cursor
 ```
 
-Or install it to your skills folder:
+Or dump the quick reference for one-off use:
 
 ```bash
-scip-cli skill ~/.claude/skills/scip-cli/SKILL.md
+scip-cli skill
 ```
 
-This enables commands like `def`, `refs`, `search`, `symbols`, `rdeps`, and `members` - just ask "where is X?" or "find references to X".
-
 ## Installation
 
 ### 1. Install scip-cli
 
-**From PyPI:**
+**From PyPI** (end users):
 
 ```bash
 pip install scip-cli
 ```
 
-**From source (local development):**
+**Local development** (use a project venv — do not rely on global `pip`):
 
 ```bash
-git clone https://github.com/flesler/scip-cli.git
 cd scip-cli
-pip install .
+python -m venv .venv
+source .venv/bin/activate   # Windows: .venv\Scripts\activate
+pip install -e ".[dev]"
+scip-cli --version
 ```
 
-For editable development (where `pip install -e .` fails due to permissions):
-
-```bash
-export PYTHONPATH=/path/to/scip-cli:$PYTHONPATH
-python -m scip_cli --help
-```
+`pip install -e .` keeps `scip-cli` on your PATH inside the venv while you edit the repo. Run tests with `pytest` from the same venv.
 
 ### 2. Install prerequisites (optional)
 
@@ -125,11 +122,12 @@ scip-cli <command> [arguments]
 ### Commands
 
 - `refs <symbol>` - Find all references to a symbol (`--path` to scope)
-- `def <symbol>` - Find symbol definition with source code (`--path`, `--max-lines`)
+- `code <symbol>` - Find symbol definition with source code (`--path`, `--max-lines`, `--full`, `--offset`, `--snippet`, `--line-numbers`)
 - `search <pattern>` - Search symbols by name pattern (`--path`)
 - `symbols <file>` - List all symbols in a file (`--path`; bare filename OK)
 - `rdeps <file>` - Find files that depend on a file (`--path`)
 - `members <symbol>` - List members of a class/interface (`--path`)
+- `analyze [target]` - SQL health dashboards (`--limit`, `--priority`, `--include-tests`). No target: project-wide; directory or file path; symbol name. See [Finding easy wins with `analyze`](#finding-easy-wins-with-analyze).
 - `reindex` - Force re-indexing of the current project (`--path` to limit scope; repeatable)
 - `skill [path]` - Install or dump the SKILL.md
 
@@ -140,13 +138,13 @@ scip-cli <command> [arguments]
 scip-cli refs greet
 
 # Get definition of greet
-scip-cli def greet
+scip-cli code greet
 
 # Search for symbols matching "Widget"
 scip-cli search Widget
 
 # Scope to a subdirectory
-scip-cli def greet --path packages/api
+scip-cli code greet --path packages/api
 
 # List symbols by bare filename
 scip-cli symbols helper.ts
@@ -157,13 +155,16 @@ scip-cli rdeps src/helper.ts
 # List members of a class
 scip-cli members Widget
 
+# Project health dashboard (or: scip-cli analyze src/foo.ts / scip-cli analyze greet)
+scip-cli analyze
+
 # Install skill file
 scip-cli skill ~/.claude/skills/scip-cli/SKILL.md
 ```
 
 ### Pipelines
 
-Stdout is one record per line; stderr carries warnings and ambiguity notices. Kinds are lowercase (`function`, `class`, `method`, `property`, `variable`). Pipe-friendly flags: `refs --paths-only`, `search --names-only` / `--paths-only`, `members --names-only`. `rdeps` already prints bare file paths.
+Stdout is one record per line; stderr carries warnings and ambiguity notices. Kinds are lowercase (`function`, `class`, `method`, `property`). Pipe-friendly flags: `refs --paths-only`, `search --names-only` / `--paths-only`, `members --names-only`. `rdeps` already prints bare file paths.
 
 ```bash
 # What do importers of this file export?
@@ -176,7 +177,7 @@ scip-cli refs greet --paths-only
 scip-cli search Handler --kind class --names-only | xargs -I{} scip-cli members {}
 
 # Walk class members to their definitions
-scip-cli members Widget --names-only | xargs -I{} scip-cli def Widget.{}
+scip-cli members Widget --names-only | xargs -I{} scip-cli code Widget.{}
 ```
 
 ## How It Works
@@ -206,10 +207,12 @@ Optional `.scip-cli.json` in the project root:
 
 `SCIP_CLI_INDEX_WORKERS` controls parallel `scip-typescript` runs during first index (default: up to 8). Merge into one database is always serial.
 
+Large monorepos (>10 tsconfig projects) log per-project progress to stderr during indexing; smaller repos stay quiet aside from the final `Indexed … (size)` line.
+
 Scoped indexing without editing `.scip-cli.json`:
 
 ```bash
-scip-cli reindex --path entrypoints/server
+scip-cli reindex --path packages/server
 scip-cli reindex --path packages/api --path packages/worker
 ```
 
@@ -217,20 +220,63 @@ scip-cli reindex --path packages/api --path packages/worker
 
 Run `scip-cli reindex` after changing scope, `.scip-cli.json` index settings, or when you want a fresh index.
 
-This is separate from `.scipquery.json`, which belongs to [scip-query](https://github.com/PlunderStruck/scip-query) and configures its analyzers, watch mode, and diff-gate — not read by scip-cli.
+## Finding easy wins with `analyze`
+
+Use `analyze` on the repo itself before broad refactors or agent review — it surfaces cross-file issues from the SCIP index (not Python `vulture`).
+
+**Quick pass** (after `scip-cli reindex`):
+
+```bash
+scip-cli analyze --limit 25
+scip-cli analyze --priority high --limit 25   # dead exports & cycles only
+```
+
+Sections are tagged `[high]`, `[medium]`, `[low]` and listed in that order.
+
+| Tier | Project sections | Action |
+| ---- | ---------------- | ------ |
+| **high** | Cycles, unreferenced, dead exports, stale types | Nuke or fix cycles; delete unused; `_` prefix |
+| **medium** | Same-file only, change surface (file target) | Module-private by usage |
+| **low** | Test-only consumers, coupling, bottlenecks, hotspots | Noisy on Python (index omits many same-file calls); verify with `rg` |
+
+Use `--priority high` for a quick gate; `--priority high,medium` adds context. File drill-down adds change surface and unused imports.
+
+**What to look at first**
+
+| Section | Easy pickings |
+| ------- | ------------- |
+| **Cycles** | Import/mention cycles between production files — break the edge or extract shared code |
+| **Unreferenced** | No usage in the index at all — delete |
+| **Dead exports** | No external refs — delete or `_` prefix |
+| **Stale types** | Classes/types with ≤1 external consumer — merge, inline, or document why they stay |
+| **Same-file only** | Used only inside defining file — rename to `_` |
+| **Test-only consumers** | Cross-file refs are all from tests — promote to e2e or accept as internal |
+
+**Per-file or package drill-down** on hubs or suspects:
+
+```bash
+scip-cli analyze scip_cli/queries.py --limit 20   # file: scoped project + per-file + top symbols
+scip-cli analyze scip_cli --limit 15              # directory: scoped project + each file under it
+```
+
+`Dead exports in file` lists same-module symbols with no *external* refs — module-private `_helpers` are filtered out. Remaining rows are worth a manual `rg` check.
+
+**Defaults:** project-wide and directory analyze skip `tests/`, `*.test.*`, `*.spec.*`, `conftest.py`, and `__tests__/`. Pass `--include-tests` to include them. File-target analyze always includes that file.
+
+**Limits:** “Dead export” means no cross-file mentions in the index — not unreachable code. Same-file private helpers are expected. Re-run `reindex` after large changes; the index is a snapshot.
 
 ## Performance
 
-Inspired by [scip-query](https://github.com/PlunderStruck/scip-query), scip-cli is a lightweight Python reimplementation optimized for speed. Compared to the original bash wrapper scripts:
+Inspired by [scip-query](https://github.com/PlunderStruck/scip-query), scip-cli is a lightweight Python partial reimplementation optimized for speed. Compared to the original:
 
 - `refs`: 6.4s → 0.03s (213x faster)
-- `def`: 2.8s → 0.05s (56x faster)
+- `code`: 2.8s → 0.05s (56x faster)
 - `search`: 2.6s → 0.03s (87x faster)
 - `symbols`: 0.3s → 0.02s (15x faster)
 - `rdeps`: 0.2s → 0.02s (10x faster)
 - `members`: 3.1s → 0.03s (103x faster)
 
-The speedup comes from direct SQLite queries instead of shell command chains, eliminating subprocess overhead.
+The speedup comes from using optimized direct SQLite queries and cutting some nice but very slow goodies (like ts-morph).
 
 ## Architecture
 
@@ -243,38 +289,41 @@ scip_cli/
 ├── discover.py      # TypeScript project discovery
 ├── merge.py         # SQLite index merging
 ├── scip_tool.py     # scip binary download
-├── constants.py     # Shared constants
 ├── sql.py           # SQLite helpers
 ├── paths.py         # --path scope filtering
 ├── project.py       # Project root + language detection
 ├── cache.py         # Index cache paths
+├── scope.py         # Persisted reindex scope (index-scope.json)
+├── debug.py         # SCIP_CLI_DEBUG stderr helpers
 ├── indexing.py      # SCIP index build + get_db
 ├── symbols.py       # Symbol parsing and kinds
 ├── queries.py       # Symbol/file SQL queries
 ├── source.py        # Filesystem source reads
 ├── output.py        # CLI formatting helpers
 ├── session.py       # setup() and single-match resolution
+├── targets.py       # file-path heuristics (tests; analyze uses analyze/targets.py)
+├── analyze/         # SQL dashboard queries (project/file/symbol)
 └── commands/        # Subcommand implementations
 ```
 
 ## Development
 
 ```bash
-pip install -e .
+pip install -e ".[dev]"
 pytest tests/ -q
 pytest tests/ -m integration -q   # indexes tests/fixtures/sample-project (needs scip-typescript)
 ```
 
 ### Debug Logging
 
-Set `SCIP_CLI_DEBUG=1` to enable SQL query logging to stderr:
+Set `SCIP_CLI_DEBUG=1` to enable SQL query logging to stderr (statements truncated to 200 chars):
 
 ```bash
 SCIP_CLI_DEBUG=1 scip-cli refs MyFunction
 # Shows: SQL: SELECT ... | params: (...)
 ```
 
-This is useful for testing and debugging SQL queries without exposing a `--debug` flag to users.
+This is useful for testing and debugging SQL queries
 
 ## License