scip-cli 1.2.0__tar.gz → 2.0.0__tar.gz

scip_cli-2.0.0/.cursor/rules/agent.mdc

@@ -0,0 +1,133 @@
+---
+description: scip-cli dev workflow and TDD with the shared fixture
+alwaysApply: true
+---
+
+# scip-cli agent
+
+**Work from repo root.** CLI usage, flags, and user gotchas: `scip-cli skill` (source: `scip_cli/SKILL.md`). Read once per task — do not duplicate SKILL here.
+
+Use **`.venv/bin/scip-cli`** / **`.venv/bin/pytest`**. Uncommitted code: `PYTHONPATH=$PWD python -m scip_cli …` if the venv binary is stale.
+
+## Bootstrap
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+pre-commit install
+scip-cli skill
+```
+
+Node.js + `npx` required for integration tests (`scip-typescript`). Optional: `SCIP_CLI_DEBUG=1` (SQL to stderr, truncated).
+
+## Lint / test
+
+| Task | Command |
+|------|---------|
+| Full suite | `.venv/bin/pytest` |
+| E2e loop | `.venv/bin/pytest tests/test_e2e.py` |
+| Lint / format | `.venv/bin/ruff check .` / `ruff format .` |
+
+Pre-commit: ruff + ruff-format + full pytest. E2e before commit on command changes.
+
+## Improving this repo (do this before review subagents)
+
+**Dogfood on the repo itself** — faster and more honest than spawning reviewers:
+
+```bash
+.venv/bin/scip-cli reindex
+.venv/bin/scip-cli analyze --limit 25
+.venv/bin/scip-cli analyze --priority high --limit 25   # dead code & cycles only
+.venv/bin/scip-cli analyze scip_cli --limit 15              # directory
+.venv/bin/scip-cli analyze scip_cli/queries.py --limit 20   # file (+ scoped project)
+```
+
+Interpretation: README § *Finding easy wins with analyze*. Target = omit (project), directory, file, or symbol — not `--path` for scope.
+
+| Priority | Section | Action |
+|----------|---------|--------|
+| 1 | **Cycles** | Break production file dependency cycles (`cache`↔`scope` was one) |
+| 2 | **Unreferenced / dead exports** | No ref usage / no external refs — verify with `rg`, delete or `_` prefix |
+| 3 | **Same-file only** | Module-private by usage — rename to `_` |
+| 4 | **Stale types** | Low-consumer classes — merge or justify |
+
+`analyze` skips test paths by default (`tests/`, `*.test.*`, `*.spec.*`); `--include-tests` to include. Reindex after large edits.
+
+Do **not** spawn broad code-review agents until this pass is done (or explain why skipped).
+
+## Analyze feedback loop (mandatory)
+
+When you or a review subagent finds a repo issue — bug, dead code, cycle, risky hub, stale type, unused import — **stop and ask first:**
+
+> *Could our SQLite index have predicted this, and `analyze` just doesn't surface it yet?*
+
+If **yes** (the signal is in `documents`, `mentions`, `global_symbols`, `defn_enclosing_ranges`):
+
+1. **Extend `analyze` first** — new or tuned SQL check in `scip_cli/analyze/`, priority in `sections.py`, tests in `tests/test_analyze.py` + `analyze_db.py`.
+2. **Dogfood** — `reindex` → confirm the issue appears (or the false positive is filtered).
+3. **Then** fix the production code (or document a known blind spot).
+
+Only skip step 1 when the issue is **outside the index** (runtime, CLI UX, docs drift, git, env, concurrency, test harness).
+
+### Retrospective: caught manually, SQLite already had it
+
+| Issue we hit | Index signal | Analyze today | Gap / next check |
+|--------------|--------------|---------------|------------------|
+| `cache` ↔ `scope` import cycle | cross-file `mentions` edges | `[high] Cycles` | **Caught** — keep dogfooding cycles |
+| Cross-file dead export (`get_refs_for_symbols`) | def with no external `mentions` | `[high] Dead exports` | **Caught** |
+| Same-file-only dead code | def, `role=0` only in def file | `[high] Unreferenced` or `[medium] Same-file only` | **Caught** (since extended) |
+| Same-file helper without `_` (`run_with_fallback`) | same-file `role=0` or none | `[medium] Same-file only` / unreferenced | **Caught** — rename to `_` |
+| Typing-only dataclasses (`ProjectSettings`, `IndexScope`) | type `#` symbol, 0 consumer files | `[high] Stale types` + `stale_type_noise` | **Filtered** — index can't see annotations; document or optional `--types` |
+| `analyze/*` section runners “dead” | no external refs | noise filter in `common.py` | **Filtered** — framework pattern |
+| Unused import | `role=2` mention, no in-file use | `[high] Unused imports` (per file) | **Caught** on file target |
+| Hub before editing (`resolve_symbol`, `queries.py`) | fan-in / consumer count | `[medium] Change surface` / `[low] Hotspots` | **Caught** on file target; project hubs are low tier |
+| Symbol only used from tests | consumers all under `tests/` | `[low] Test-only consumers` | **Partial** — Python index often omits same-file `role=0`; verify with `rg` |
+| Indexer omits same-file Python calls | no `role=0` for in-module use | unreferenced / test-only false positives | **Index gap** — not fixable in SQL alone; demoted test-only to low |
+| Duplicate / parallel helpers | — | — | **Not in index** — `rg`, not analyze |
+
+Add new rows to `dogfood.md` when you find another gap; implement the SQL check before the next manual review pass.
+
+### Size / complexity checks (assessed — don’t promote to high/medium)
+
+SQLite has **line ranges** (`defn_enclosing_ranges`) so LOC is cheap. That does **not** mean “big function” belongs next to dead exports.
+
+| Idea | In index? | Verdict |
+|------|-----------|---------|
+| **LOC ≥ N** alone | Yes | **LOW at best** — noisy (`main`, argparse, legitimate parsers). Not a nuke candidate. |
+| **LOC + fan-in** (large hub) | Yes | **Already covered** by `[low] Bottlenecks` (fan-in × fan-out). Add `loc=` to that output if helpful; don’t duplicate as medium. |
+| **LOC + consumers** on file target | Yes | **Change surface** (medium) already sorts by consumers with line ranges — drill there. |
+| **Symbol pressure** | Yes | **Per-symbol** `analyze Foo` shows `loc × fan_in × fan_out`. |
+| **God class / many members** | Yes | Use `members Class` — not analyze. |
+| **File byte size / cyclomatic complexity** | No | Out of scope for SCIP index. |
+| **Fat file (many symbols)** | Yes | Low value; coupling/hotspots approximate module heat. |
+
+**Rule:** size-only checks stay **low or file-drill-down**; never high/medium unless paired with a **structural smell** (cycle, dead export, zero production consumers). Prefer enriching existing low-tier lines (e.g. show `loc` on bottlenecks) over a new dashboard section.
+
+## TDD
+
+1. **Red** — `tests/test_e2e.py` + `fixture_catalog` constants.
+2. **Green** — `scip_cli/commands/*.py`.
+3. **Refactor** — unit tests for pure helpers only.
+
+- **E2e** (`cli` fixture): in-process `main()`, TS project indexed once per session — default for behavior.
+- **Unit**: `:memory:` SQLite, `tests/analyze_db.py`, merge/index prune tests.
+- **No subprocess** for CLI tests; no `reindex` in e2e (mutates `~/.cache/scip-cli`).
+- New symbols: extend `tests/fixtures/sample-project/` + `fixture_catalog.py`.
+
+## Codebase map (contributors)
+
+| Area | Path | Notes |
+|------|------|-------|
+| Commands | `scip_cli/commands/` | Thin; SQL in `queries.py` / `analyze/` |
+| Index build | `scip_cli/indexing.py` | `index.db.next` → promote; flock; `>10` tsconfigs log progress |
+| Variable prune | postprocess + `merge.py` | Omit const/let/var on **COPY**, not DELETE — no `variable` kind |
+| Session | `scip_cli/session.py` | `setup()`, `resolve_one_symbol` (limit 2 for ambiguity) |
+| Analyze filters | `scip_cli/analyze/common.py` | `is_test_path`, `analyze_noise` (`_` helpers) |
+
+`todo.md` = tracked work; `dogfood.md` = future ideas only. Wishlist/backlog in `docs/` per workspace rules.
+
+## Docs touch points
+
+- User install + analyze guide → `README.md`
+- Agent/user CLI reference → `scip_cli/SKILL.md` (must match `__main__.py` flags)
+- After flag changes: update SKILL + README command list; run `scip-cli skill` to verify output

scip_cli-2.0.0/.gitignore

@@ -0,0 +1,53 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+venv/
+ENV/
+env/
+.venv
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Env/secrets
+.env
+
+# Pytest
+.pytest_cache/
+
+# Project specific
+TODO.md
+todo.md
+dogfood.md
+
+# Local-only dev scripts (repo-specific smoke harnesses)
+scripts/deep-smoke.sh
+tmp/

scip_cli-2.0.0/.pre-commit-config.yaml

@@ -0,0 +1,15 @@
+repos:
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.11.0
+    hooks:
+      - id: ruff
+        args: [--fix]
+      - id: ruff-format
+  - repo: local
+    hooks:
+      - id: pytest
+        name: pytest
+        entry: pytest
+        language: python
+        pass_filenames: false
+        always_run: true

scip_cli-2.0.0/PKG-INFO

@@ -0,0 +1,330 @@
+Metadata-Version: 2.4
+Name: scip-cli
+Version: 2.0.0
+Summary: Fast code intelligence via SCIP indexes
+Project-URL: Homepage, https://github.com/flesler/scip-cli
+Author: Ariel Flesler
+License: MIT
+License-File: LICENSE
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Code Generators
+Requires-Python: >=3.9
+Provides-Extra: dev
+Requires-Dist: build>=1; extra == 'dev'
+Requires-Dist: pre-commit>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: ruff>=0.11; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# scip-cli
+
+[![PyPI version](https://badge.fury.io/py/scip-cli.svg)](https://badge.fury.io/py/scip-cli)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+Token-efficient code intelligence for AI agents. Precise refs, definitions, and repo health analysis via SCIP indexes — TypeScript/JavaScript and Python.
+
+## Why
+
+AI agents waste tokens on grep and file scanning. scip-cli gives them precise, type-aware code navigation in milliseconds — and `analyze` surfaces dead code, cycles, and coupling so agents (and humans) can fix real problems fast.
+
+## Features
+
+- **Agent-first**: Install as a skill for Claude Code, Cursor, or any AI agent — precise code navigation without burning context
+- **Token-efficient**: One record per line, stderr for warnings, pipe-friendly output
+- **Fast**: Direct SQLite queries — 10x to 213x faster than alternatives
+- **`analyze`**: Find dead exports, import cycles, stale types, coupling hotspots — actionable health dashboards at project, file, or symbol scope
+- **Auto-indexing**: Indexes on first query, caches in SQLite, zero config
+
+## For AI Agents
+
+Install as a reusable skill so your agent always knows how to navigate the codebase:
+
+```bash
+scip-cli skill ~/.claude/skills/scip-cli/   # Claude Code
+scip-cli skill ~/.cursor/skills/scip-cli/   # Cursor
+```
+
+Or dump the quick reference for one-off use:
+
+```bash
+scip-cli skill
+```
+
+## Installation
+
+### 1. Install scip-cli
+
+**From PyPI** (end users):
+
+```bash
+pip install scip-cli
+```
+
+**Local development** (use a project venv — do not rely on global `pip`):
+
+```bash
+cd scip-cli
+python -m venv .venv
+source .venv/bin/activate   # Windows: .venv\Scripts\activate
+pip install -e ".[dev]"
+scip-cli --version
+```
+
+`pip install -e .` keeps `scip-cli` on your PATH inside the venv while you edit the repo. Run tests with `pytest` from the same venv.
+
+### 2. Install prerequisites (optional)
+
+On **first index**, scip-cli runs language indexers and builds a SQLite cache. You can let it fetch tools on demand, or install them globally ahead of time so the first run does not download via `npx`:
+
+**Option A: Zero extra setup (recommended)**
+
+Install `scip-cli` and run it. On first index, scip-cli will:
+
+- Download `scip-typescript` / `scip-python` via `npx` when not already on PATH
+- Download the `scip` converter binary from [GitHub releases](https://github.com/scip-code/scip/releases) into `~/.cache/scip-cli/bin/` when not already on PATH
+- Walk the repo for `tsconfig*.json` project roots (TypeScript monorepos), run `scip-typescript` per project (parallel by default), convert each partial index, then merge into one `index.db`
+
+No `.scip-cli.json` required for discovery. Subsequent queries read the cached database only.
+
+**Option B: Install indexers globally ahead of time**
+
+Same indexing steps as Option A; this only avoids `npx` download on the first run:
+
+```bash
+# TypeScript/JavaScript indexer (also handles plain JS via --infer-tsconfig)
+npm install -g @sourcegraph/scip-typescript
+
+# Python indexer
+npm install -g @sourcegraph/scip-python
+
+# SCIP CLI for index conversion (GitHub release — not on npm)
+# https://github.com/scip-code/scip/releases  (v0.8.1+ recommended)
+```
+
+**Verify installation:**
+
+```bash
+scip-cli --help
+scip-typescript --version  # Only if you chose Option B
+scip --version             # Install from GitHub releases; v0.8.1+ recommended
+```
+
+## Usage
+
+All commands are subcommands of `scip-cli`:
+
+```bash
+scip-cli <command> [arguments]
+```
+
+### Commands
+
+- `refs <symbol>` - Find all references to a symbol (`--path` to scope)
+- `code <symbol>` - Find symbol definition with source code (`--path`, `--max-lines`, `--full`, `--offset`, `--snippet`, `--line-numbers`)
+- `search <pattern>` - Search symbols by name pattern (`--path`)
+- `symbols <file>` - List all symbols in a file (`--path`; bare filename OK)
+- `rdeps <file>` - Find files that depend on a file (`--path`)
+- `members <symbol>` - List members of a class/interface (`--path`)
+- `analyze [target]` - SQL health dashboards (`--limit`, `--priority`, `--include-tests`). No target: project-wide; directory or file path; symbol name. See [Finding easy wins with `analyze`](#finding-easy-wins-with-analyze).
+- `reindex` - Force re-indexing of the current project (`--path` to limit scope; repeatable)
+- `skill [path]` - Install or dump the SKILL.md
+
+### Examples
+
+```bash
+# Find where greet is used
+scip-cli refs greet
+
+# Get definition of greet
+scip-cli code greet
+
+# Search for symbols matching "Widget"
+scip-cli search Widget
+
+# Scope to a subdirectory
+scip-cli code greet --path packages/api
+
+# List symbols by bare filename
+scip-cli symbols helper.ts
+
+# Find files that import from a module
+scip-cli rdeps src/helper.ts
+
+# List members of a class
+scip-cli members Widget
+
+# Project health dashboard (or: scip-cli analyze src/foo.ts / scip-cli analyze greet)
+scip-cli analyze
+
+# Install skill file
+scip-cli skill ~/.claude/skills/scip-cli/SKILL.md
+```
+
+### Pipelines
+
+Stdout is one record per line; stderr carries warnings and ambiguity notices. Kinds are lowercase (`function`, `class`, `method`, `property`). Pipe-friendly flags: `refs --paths-only`, `search --names-only` / `--paths-only`, `members --names-only`. `rdeps` already prints bare file paths.
+
+```bash
+# What do importers of this file export?
+scip-cli rdeps src/helper.ts | xargs -I{} scip-cli symbols {}
+
+# Which files reference a symbol?
+scip-cli refs greet --paths-only
+
+# Classes matching a name → list their members
+scip-cli search Handler --kind class --names-only | xargs -I{} scip-cli members {}
+
+# Walk class members to their definitions
+scip-cli members Widget --names-only | xargs -I{} scip-cli code Widget.{}
+```
+
+## How It Works
+
+1. On first query, automatically detects project language from `package.json` (TS/JS) or `pyproject.toml`/`setup.py` (Python)
+2. For TypeScript monorepos, walks the repository for `tsconfig*.json` project roots (nested ancestors deduped; root included only when its `include` is broad)
+3. Runs `scip-typescript` per project (in parallel when there are multiple projects; set `SCIP_CLI_INDEX_WORKERS=1` to force serial), or `scip-python` for Python
+4. Converts each SCIP output to SQLite with `scip expt-convert`, then merges partial databases when needed
+5. Caches the result in `~/.cache/scip-cli/projects/<dirname>-<hash>/index.db` (e.g. `my-monorepo-1a3f7a`)
+6. Subsequent queries are SQLite lookups against that cache (not re-indexing)
+
+## Configuration
+
+Optional `.scip-cli.json` in the project root:
+
+```json
+{
+  "maxHeapMb": 8192,
+  "indexRoots": ["packages/core", "apps/worker"],
+  "onlyIndexRoots": false
+}
+```
+
+- `maxHeapMb` — Node heap for `scip-typescript` / `scip-python` (default **8192 MB** when omitted). Overridden by `SCIP_CLI_MAX_HEAP_MB`. This is the V8 heap cap, not total RAM usage.
+- `indexRoots` — extra TypeScript project directories to include on **first index**, merged with auto-discovered projects.
+- `onlyIndexRoots` — skip auto-discovery and index **only** `indexRoots` (smaller initial index when you only care about part of a monorepo).
+
+`SCIP_CLI_INDEX_WORKERS` controls parallel `scip-typescript` runs during first index (default: up to 8). Merge into one database is always serial.
+
+Large monorepos (>10 tsconfig projects) log per-project progress to stderr during indexing; smaller repos stay quiet aside from the final `Indexed … (size)` line.
+
+Scoped indexing without editing `.scip-cli.json`:
+
+```bash
+scip-cli reindex --path packages/server
+scip-cli reindex --path packages/api --path packages/worker
+```
+
+`--path` limits which discovered tsconfig projects are indexed (prefix match, same idea as query `--path`). The scope is saved as `index-scope.json` next to `index.db` and reused until you run a full `scip-cli reindex` with no `--path`.
+
+Run `scip-cli reindex` after changing scope, `.scip-cli.json` index settings, or when you want a fresh index.
+
+## Finding easy wins with `analyze`
+
+Use `analyze` on the repo itself before broad refactors or agent review — it surfaces cross-file issues from the SCIP index (not Python `vulture`).
+
+**Quick pass** (after `scip-cli reindex`):
+
+```bash
+scip-cli analyze --limit 25
+scip-cli analyze --priority high --limit 25   # dead exports & cycles only
+```
+
+Sections are tagged `[high]`, `[medium]`, `[low]` and listed in that order.
+
+| Tier | Project sections | Action |
+| ---- | ---------------- | ------ |
+| **high** | Cycles, unreferenced, dead exports, stale types | Nuke or fix cycles; delete unused; `_` prefix |
+| **medium** | Same-file only, change surface (file target) | Module-private by usage |
+| **low** | Test-only consumers, coupling, bottlenecks, hotspots | Noisy on Python (index omits many same-file calls); verify with `rg` |
+
+Use `--priority high` for a quick gate; `--priority high,medium` adds context. File drill-down adds change surface and unused imports.
+
+**What to look at first**
+
+| Section | Easy pickings |
+| ------- | ------------- |
+| **Cycles** | Import/mention cycles between production files — break the edge or extract shared code |
+| **Unreferenced** | No usage in the index at all — delete |
+| **Dead exports** | No external refs — delete or `_` prefix |
+| **Stale types** | Classes/types with ≤1 external consumer — merge, inline, or document why they stay |
+| **Same-file only** | Used only inside defining file — rename to `_` |
+| **Test-only consumers** | Cross-file refs are all from tests — promote to e2e or accept as internal |
+
+**Per-file or package drill-down** on hubs or suspects:
+
+```bash
+scip-cli analyze scip_cli/queries.py --limit 20   # file: scoped project + per-file + top symbols
+scip-cli analyze scip_cli --limit 15              # directory: scoped project + each file under it
+```
+
+`Dead exports in file` lists same-module symbols with no *external* refs — module-private `_helpers` are filtered out. Remaining rows are worth a manual `rg` check.
+
+**Defaults:** project-wide and directory analyze skip `tests/`, `*.test.*`, `*.spec.*`, `conftest.py`, and `__tests__/`. Pass `--include-tests` to include them. File-target analyze always includes that file.
+
+**Limits:** “Dead export” means no cross-file mentions in the index — not unreachable code. Same-file private helpers are expected. Re-run `reindex` after large changes; the index is a snapshot.
+
+## Performance
+
+Inspired by [scip-query](https://github.com/PlunderStruck/scip-query), scip-cli is a lightweight Python partial reimplementation optimized for speed. Compared to the original:
+
+- `refs`: 6.4s → 0.03s (213x faster)
+- `code`: 2.8s → 0.05s (56x faster)
+- `search`: 2.6s → 0.03s (87x faster)
+- `symbols`: 0.3s → 0.02s (15x faster)
+- `rdeps`: 0.2s → 0.02s (10x faster)
+- `members`: 3.1s → 0.03s (103x faster)
+
+The speedup comes from using optimized direct SQLite queries and cutting some nice but very slow goodies (like ts-morph).
+
+## Architecture
+
+```
+scip_cli/
+├── __init__.py
+├── __main__.py      # CLI entry point
+├── cli_args.py      # Shared argparse helpers
+├── config.py        # .scip-cli.json loader
+├── discover.py      # TypeScript project discovery
+├── merge.py         # SQLite index merging
+├── scip_tool.py     # scip binary download
+├── sql.py           # SQLite helpers
+├── paths.py         # --path scope filtering
+├── project.py       # Project root + language detection
+├── cache.py         # Index cache paths
+├── scope.py         # Persisted reindex scope (index-scope.json)
+├── debug.py         # SCIP_CLI_DEBUG stderr helpers
+├── indexing.py      # SCIP index build + get_db
+├── symbols.py       # Symbol parsing and kinds
+├── queries.py       # Symbol/file SQL queries
+├── source.py        # Filesystem source reads
+├── output.py        # CLI formatting helpers
+├── session.py       # setup() and single-match resolution
+├── targets.py       # file-path heuristics (tests; analyze uses analyze/targets.py)
+├── analyze/         # SQL dashboard queries (project/file/symbol)
+└── commands/        # Subcommand implementations
+```
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest tests/ -q
+pytest tests/ -m integration -q   # indexes tests/fixtures/sample-project (needs scip-typescript)
+```
+
+### Debug Logging
+
+Set `SCIP_CLI_DEBUG=1` to enable SQL query logging to stderr (statements truncated to 200 chars):
+
+```bash
+SCIP_CLI_DEBUG=1 scip-cli refs MyFunction
+# Shows: SQL: SELECT ... | params: (...)
+```
+
+This is useful for testing and debugging SQL queries
+
+## License
+
+MIT