scip-cli 3.0.0__tar.gz → 3.1.0__tar.gz

{scip_cli-3.0.0 → scip_cli-3.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: scip-cli
-Version: 3.0.0
+Version: 3.1.0
 Summary: Fast code intelligence via SCIP indexes
 Project-URL: Homepage, https://github.com/flesler/scip-cli
 Author: Ariel Flesler
@@ -101,7 +101,8 @@ On **first index**, scip-cli runs language indexers and builds a SQLite cache. Y
 
 Install `scip-cli` and run it. On first index, scip-cli will:
 
-- Download `scip-typescript` / `scip-python` via `npx`, `scip-go` via `go install`, or `rust-analyzer` via `rustup` when not already on PATH
+- Install `scip-typescript` from the [flesler/scip-typescript](https://github.com/flesler/scip-typescript) fork (`feat/partial-files`) when not on PATH — clones, `npm install`, and builds into `~/.cache/scip-cli/tools/github-npm/` (upstream npm does not ship partial `--files` yet)
+- Download `scip-python` via `npx`, `scip-go` via `go install`, or `rust-analyzer` via `rustup` when not already on PATH
 - Download the `scip` converter binary from [GitHub releases](https://github.com/scip-code/scip/releases) into `~/.cache/scip-cli/bin/` when not already on PATH
 - Walk the repo for `tsconfig*.json` project roots (TypeScript monorepos), run `scip-typescript` per project (parallel by default), convert each partial index, then merge into one `index.db`
 
@@ -109,11 +110,12 @@ No `.scip-cli.json` required for discovery. Subsequent queries read the cached d
 
 **Option B: Install indexers globally ahead of time**
 
-Same indexing steps as Option A; this only avoids `npx` download on the first run:
+Same indexing steps as Option A; this only avoids the first-run download/build:
 
 ```bash
-# TypeScript/JavaScript indexer (also handles plain JS via --infer-tsconfig)
-npm install -g @sourcegraph/scip-typescript
+# TypeScript/JavaScript indexer — fork with partial --files (required for incremental reindex)
+git clone --depth 1 --branch feat/partial-files https://github.com/flesler/scip-typescript.git
+cd scip-typescript && npm install --ignore-scripts && npm run build && npm install -g .
 
 # Python indexer
 npm install -g @sourcegraph/scip-python
@@ -154,7 +156,7 @@ scip-cli <command> [arguments]
 - `deps <symbol|file>` - Find outbound dependencies (what a symbol or file calls) (`--path`, `--paths-only`)
 - `members <symbol>` - List members of a class/interface (`--path`)
 - `analyze [target]` - SQL health dashboards (`--limit`, `--per-check-limit`, `--priority`, `--check`, `--include-tests`). No target: project-wide; directory or file path; symbol name. See [Finding easy wins with `analyze`](#finding-easy-wins-with-analyze).
-- `reindex` - Force re-indexing of the current project (`--path` to limit scope; repeatable)
+- `reindex` - Force re-indexing (`--path`, `--tsconfig`, `--exclude`, `--fresh`, `--no-incremental`; incremental by default in git TS repos)
 - `skill [path]` - Install or dump the SKILL.md
 
 ### Examples
@@ -222,7 +224,10 @@ scip-cli deps greet --paths-only | sort -u
 3. Runs `scip-typescript` per project (in parallel when there are multiple projects; set `SCIP_CLI_INDEX_WORKERS=1` to force serial), `scip-python` for Python, `scip-go` for Go, or `rust-analyzer scip` for Rust
 4. Converts each SCIP output to SQLite with `scip expt-convert`, then merges partial databases when needed
 5. Caches the result in `~/.cache/scip-cli/projects/<dirname>-<hash>/index.db` (e.g. `my-monorepo-1a3f7a`)
-6. Subsequent queries are SQLite lookups against that cache (not re-indexing)
+6. Compacts the database after each reindex (`VACUUM` + WAL checkpoint)
+7. Subsequent queries are SQLite lookups against that cache (not re-indexing)
+
+**Incremental reindex (TypeScript, v3.0+):** plain `scip-cli reindex` in a git worktree is incremental by default — reuses unchanged tsconfig shards via `shards/manifest.json` (`git_commit` + per-shard `tsconfig_digest`). Dirty shards reindex only changed files (git delta → importer closure → fork `--files`) and upsert into the live `index.db`. Use `--no-incremental` or `--fresh` for a full rebuild; `--unversioned` and non-git projects always full-reindex. Bench locally with `scripts/bench_incremental_gate.sh --branch fixture`.
 
 ## Configuration
 
@@ -249,27 +254,28 @@ Other environment variables:
 |`SCIP_CLI_MAX_HEAP_MB`|Node heap for `scip-typescript` / `scip-python` (overrides `maxHeapMb` in config)|
 |`SCIP_CLI_TS_INDEX_BATCH_SIZE`|Split large TS repos into multiple `scip-typescript` runs (default: all tsconfigs in one run)|
 |`SCIP_CLI_MERGE_BATCH_SIZE`|SQLite ATTACH batch size when merging part DBs (max 9)|
+|`SCIP_CLI_INDEX_TIMING`|`1` — per-phase `INDEX_TIMING:` lines on stderr during reindex|
 |`SCIP_CLI_MAX_DEF_LINES`|Max definition lines in `code` output|
 |`SCIP_CLI_DEBUG`|Log SQL queries to stderr|
 
-**Version policy:** only the `scip` converter (`expt-convert`) is pinned to the 0.8.x release line because it defines the SQLite schema. `scip-typescript` installs via npx from the GitHub fork in `scip_cli/indexing/constants.py` when not on PATH (until upstream ships incremental flags). `scip-python` via `npx` at latest; `scip-go` via `go install @latest`. `rust-analyzer` installs via `rustup component add`.
+**Version policy:** only the `scip` converter (`expt-convert`) is pinned to the 0.8.x release line because it defines the SQLite schema. **TypeScript** uses [flesler/scip-typescript](https://github.com/flesler/scip-typescript) `feat/partial-files` (see `SCIP_TYPESCRIPT_NPX_PACKAGE` in `scip_cli/indexing/constants.py`) — auto-built when not on PATH; switch back to `@sourcegraph/scip-typescript` on npm once upstream merges `--files`. `scip-python` via `npx` at latest; `scip-go` via `go install @latest`. `rust-analyzer` installs via `rustup component add`.
 
 Large monorepos (>10 tsconfig projects) log per-project progress to stderr during indexing; smaller repos stay quiet aside from the final `Indexed … (size)` line.
 
 Scoped indexing without editing `.scip-cli.json`:
 
 ```bash
-scip-cli reindex --path packages/server
+scip-cli reindex --path packages/core
 scip-cli reindex --path packages/api --path packages/worker
 scip-cli reindex --tsconfig 'apps/api/tsconfig.*.json'
 scip-cli reindex --tsconfig tsconfig.app.json --tsconfig tsconfig.spec.json
 scip-cli reindex --exclude '**/*.test.ts' '**/*.spec.ts'
 scip-cli reindex --exclude          # clear persisted excludes
 scip-cli reindex --fresh            # full index; clear metadata.json
-scip-cli reindex --incremental      # reuse unchanged project shards (see SKILL for language support)
+scip-cli reindex --no-incremental   # full rebuild; clears shard manifest
 ```
 
-`--path` limits which discovered tsconfig **directories** are indexed (prefix match, same idea as query `--path`). `--tsconfig` skips discovery and indexes those `tsconfig*.json` **files** (repeatable; globs are expanded inside the tool). File-based runs default to one `scip-typescript` process per file so each gets its own heap (`SCIP_CLI_TS_INDEX_BATCH_SIZE` still overrides). Cannot combine `--path` and `--tsconfig`. **TypeScript only.** Scope and exclude defaults are saved in `metadata.json` next to `index.db` and reused on later `reindex` runs; use `reindex --fresh` to clear them and restore a full index. `--exclude` updates the persisted exclude list; a lone bare `--exclude` clears it.
+`--path` limits which discovered tsconfig **directories** are indexed (prefix match, same idea as query `--path`). `--tsconfig` skips discovery and indexes those `tsconfig*.json` **files** (repeatable; globs are repo-relative to the detected project root — run from monorepo root or shorten the glob). File-based runs default to one `scip-typescript` process per file so each gets its own heap (`SCIP_CLI_TS_INDEX_BATCH_SIZE` still overrides). Cannot combine `--path` and `--tsconfig`. **TypeScript only** for `--path` and `--tsconfig`. Scope and exclude defaults are saved in `metadata.json` next to `index.db` and reused on later `reindex` runs; use `reindex --fresh` to clear them and restore a full index. `--exclude` updates the persisted exclude list (honored during incremental reindex too); a lone bare `--exclude` clears it. Incremental is the default in git TypeScript repos; `--no-incremental`, `--fresh`, `--unversioned`, or non-git force full reindex.
 
 When a tsconfig (after `extends`) has `allowJs: true`, matching `.js`/`.jsx` files under that config's `include`/`files` are indexed too (same roots as `.ts`/`.tsx`). `allowJs: false` or unset leaves JavaScript out. JS-only repos with no `tsconfig.json` still use `--infer-tsconfig`.
 
@@ -338,35 +344,6 @@ Inspired by [scip-query](https://github.com/PlunderStruck/scip-query), scip-cli
 
 The speedup comes from using optimized direct SQLite queries and cutting some nice but very slow goodies (like ts-morph).
 
-## Architecture
-
-```
-scip_cli/
-├── __init__.py
-├── __main__.py      # CLI entry point
-├── cli_args.py      # Shared argparse helpers
-├── config.py        # .scip-cli.json loader
-├── discover.py      # TypeScript project discovery
-├── merge.py         # SQLite index merging
-├── scip_tool.py     # scip binary download
-├── sql.py           # SQLite helpers
-├── paths.py         # --path scope filtering
-├── project.py       # Project root + language detection
-├── cache.py         # Index cache paths
-├── metadata.py      # Persisted reindex defaults (metadata.json)
-├── scope.py         # Scoped reindex helpers
-├── debug.py         # SCIP_CLI_DEBUG stderr helpers
-├── indexing.py      # SCIP index build + get_db
-├── symbols.py       # Symbol parsing and kinds
-├── queries.py       # Symbol/file SQL queries
-├── source.py        # Filesystem source reads
-├── output.py        # CLI formatting helpers
-├── session.py       # setup() and single-match resolution
-├── targets.py       # file-path heuristics (tests; analyze uses analyze/targets.py)
-├── analyze/         # SQL dashboard queries (project/file/symbol)
-└── commands/        # Subcommand implementations
-```
-
 ## Development
 
 ```bash

{scip_cli-3.0.0 → scip_cli-3.1.0}/README.md

@@ -80,7 +80,8 @@ On **first index**, scip-cli runs language indexers and builds a SQLite cache. Y
 
 Install `scip-cli` and run it. On first index, scip-cli will:
 
-- Download `scip-typescript` / `scip-python` via `npx`, `scip-go` via `go install`, or `rust-analyzer` via `rustup` when not already on PATH
+- Install `scip-typescript` from the [flesler/scip-typescript](https://github.com/flesler/scip-typescript) fork (`feat/partial-files`) when not on PATH — clones, `npm install`, and builds into `~/.cache/scip-cli/tools/github-npm/` (upstream npm does not ship partial `--files` yet)
+- Download `scip-python` via `npx`, `scip-go` via `go install`, or `rust-analyzer` via `rustup` when not already on PATH
 - Download the `scip` converter binary from [GitHub releases](https://github.com/scip-code/scip/releases) into `~/.cache/scip-cli/bin/` when not already on PATH
 - Walk the repo for `tsconfig*.json` project roots (TypeScript monorepos), run `scip-typescript` per project (parallel by default), convert each partial index, then merge into one `index.db`
 
@@ -88,11 +89,12 @@ No `.scip-cli.json` required for discovery. Subsequent queries read the cached d
 
 **Option B: Install indexers globally ahead of time**
 
-Same indexing steps as Option A; this only avoids `npx` download on the first run:
+Same indexing steps as Option A; this only avoids the first-run download/build:
 
 ```bash
-# TypeScript/JavaScript indexer (also handles plain JS via --infer-tsconfig)
-npm install -g @sourcegraph/scip-typescript
+# TypeScript/JavaScript indexer — fork with partial --files (required for incremental reindex)
+git clone --depth 1 --branch feat/partial-files https://github.com/flesler/scip-typescript.git
+cd scip-typescript && npm install --ignore-scripts && npm run build && npm install -g .
 
 # Python indexer
 npm install -g @sourcegraph/scip-python
@@ -133,7 +135,7 @@ scip-cli <command> [arguments]
 - `deps <symbol|file>` - Find outbound dependencies (what a symbol or file calls) (`--path`, `--paths-only`)
 - `members <symbol>` - List members of a class/interface (`--path`)
 - `analyze [target]` - SQL health dashboards (`--limit`, `--per-check-limit`, `--priority`, `--check`, `--include-tests`). No target: project-wide; directory or file path; symbol name. See [Finding easy wins with `analyze`](#finding-easy-wins-with-analyze).
-- `reindex` - Force re-indexing of the current project (`--path` to limit scope; repeatable)
+- `reindex` - Force re-indexing (`--path`, `--tsconfig`, `--exclude`, `--fresh`, `--no-incremental`; incremental by default in git TS repos)
 - `skill [path]` - Install or dump the SKILL.md
 
 ### Examples
@@ -201,7 +203,10 @@ scip-cli deps greet --paths-only | sort -u
 3. Runs `scip-typescript` per project (in parallel when there are multiple projects; set `SCIP_CLI_INDEX_WORKERS=1` to force serial), `scip-python` for Python, `scip-go` for Go, or `rust-analyzer scip` for Rust
 4. Converts each SCIP output to SQLite with `scip expt-convert`, then merges partial databases when needed
 5. Caches the result in `~/.cache/scip-cli/projects/<dirname>-<hash>/index.db` (e.g. `my-monorepo-1a3f7a`)
-6. Subsequent queries are SQLite lookups against that cache (not re-indexing)
+6. Compacts the database after each reindex (`VACUUM` + WAL checkpoint)
+7. Subsequent queries are SQLite lookups against that cache (not re-indexing)
+
+**Incremental reindex (TypeScript, v3.0+):** plain `scip-cli reindex` in a git worktree is incremental by default — reuses unchanged tsconfig shards via `shards/manifest.json` (`git_commit` + per-shard `tsconfig_digest`). Dirty shards reindex only changed files (git delta → importer closure → fork `--files`) and upsert into the live `index.db`. Use `--no-incremental` or `--fresh` for a full rebuild; `--unversioned` and non-git projects always full-reindex. Bench locally with `scripts/bench_incremental_gate.sh --branch fixture`.
 
 ## Configuration
 
@@ -228,27 +233,28 @@ Other environment variables:
 |`SCIP_CLI_MAX_HEAP_MB`|Node heap for `scip-typescript` / `scip-python` (overrides `maxHeapMb` in config)|
 |`SCIP_CLI_TS_INDEX_BATCH_SIZE`|Split large TS repos into multiple `scip-typescript` runs (default: all tsconfigs in one run)|
 |`SCIP_CLI_MERGE_BATCH_SIZE`|SQLite ATTACH batch size when merging part DBs (max 9)|
+|`SCIP_CLI_INDEX_TIMING`|`1` — per-phase `INDEX_TIMING:` lines on stderr during reindex|
 |`SCIP_CLI_MAX_DEF_LINES`|Max definition lines in `code` output|
 |`SCIP_CLI_DEBUG`|Log SQL queries to stderr|
 
-**Version policy:** only the `scip` converter (`expt-convert`) is pinned to the 0.8.x release line because it defines the SQLite schema. `scip-typescript` installs via npx from the GitHub fork in `scip_cli/indexing/constants.py` when not on PATH (until upstream ships incremental flags). `scip-python` via `npx` at latest; `scip-go` via `go install @latest`. `rust-analyzer` installs via `rustup component add`.
+**Version policy:** only the `scip` converter (`expt-convert`) is pinned to the 0.8.x release line because it defines the SQLite schema. **TypeScript** uses [flesler/scip-typescript](https://github.com/flesler/scip-typescript) `feat/partial-files` (see `SCIP_TYPESCRIPT_NPX_PACKAGE` in `scip_cli/indexing/constants.py`) — auto-built when not on PATH; switch back to `@sourcegraph/scip-typescript` on npm once upstream merges `--files`. `scip-python` via `npx` at latest; `scip-go` via `go install @latest`. `rust-analyzer` installs via `rustup component add`.
 
 Large monorepos (>10 tsconfig projects) log per-project progress to stderr during indexing; smaller repos stay quiet aside from the final `Indexed … (size)` line.
 
 Scoped indexing without editing `.scip-cli.json`:
 
 ```bash
-scip-cli reindex --path packages/server
+scip-cli reindex --path packages/core
 scip-cli reindex --path packages/api --path packages/worker
 scip-cli reindex --tsconfig 'apps/api/tsconfig.*.json'
 scip-cli reindex --tsconfig tsconfig.app.json --tsconfig tsconfig.spec.json
 scip-cli reindex --exclude '**/*.test.ts' '**/*.spec.ts'
 scip-cli reindex --exclude          # clear persisted excludes
 scip-cli reindex --fresh            # full index; clear metadata.json
-scip-cli reindex --incremental      # reuse unchanged project shards (see SKILL for language support)
+scip-cli reindex --no-incremental   # full rebuild; clears shard manifest
 ```
 
-`--path` limits which discovered tsconfig **directories** are indexed (prefix match, same idea as query `--path`). `--tsconfig` skips discovery and indexes those `tsconfig*.json` **files** (repeatable; globs are expanded inside the tool). File-based runs default to one `scip-typescript` process per file so each gets its own heap (`SCIP_CLI_TS_INDEX_BATCH_SIZE` still overrides). Cannot combine `--path` and `--tsconfig`. **TypeScript only.** Scope and exclude defaults are saved in `metadata.json` next to `index.db` and reused on later `reindex` runs; use `reindex --fresh` to clear them and restore a full index. `--exclude` updates the persisted exclude list; a lone bare `--exclude` clears it.
+`--path` limits which discovered tsconfig **directories** are indexed (prefix match, same idea as query `--path`). `--tsconfig` skips discovery and indexes those `tsconfig*.json` **files** (repeatable; globs are repo-relative to the detected project root — run from monorepo root or shorten the glob). File-based runs default to one `scip-typescript` process per file so each gets its own heap (`SCIP_CLI_TS_INDEX_BATCH_SIZE` still overrides). Cannot combine `--path` and `--tsconfig`. **TypeScript only** for `--path` and `--tsconfig`. Scope and exclude defaults are saved in `metadata.json` next to `index.db` and reused on later `reindex` runs; use `reindex --fresh` to clear them and restore a full index. `--exclude` updates the persisted exclude list (honored during incremental reindex too); a lone bare `--exclude` clears it. Incremental is the default in git TypeScript repos; `--no-incremental`, `--fresh`, `--unversioned`, or non-git force full reindex.
 
 When a tsconfig (after `extends`) has `allowJs: true`, matching `.js`/`.jsx` files under that config's `include`/`files` are indexed too (same roots as `.ts`/`.tsx`). `allowJs: false` or unset leaves JavaScript out. JS-only repos with no `tsconfig.json` still use `--infer-tsconfig`.
 
@@ -317,35 +323,6 @@ Inspired by [scip-query](https://github.com/PlunderStruck/scip-query), scip-cli
 
 The speedup comes from using optimized direct SQLite queries and cutting some nice but very slow goodies (like ts-morph).
 
-## Architecture
-
-```
-scip_cli/
-├── __init__.py
-├── __main__.py      # CLI entry point
-├── cli_args.py      # Shared argparse helpers
-├── config.py        # .scip-cli.json loader
-├── discover.py      # TypeScript project discovery
-├── merge.py         # SQLite index merging
-├── scip_tool.py     # scip binary download
-├── sql.py           # SQLite helpers
-├── paths.py         # --path scope filtering
-├── project.py       # Project root + language detection
-├── cache.py         # Index cache paths
-├── metadata.py      # Persisted reindex defaults (metadata.json)
-├── scope.py         # Scoped reindex helpers
-├── debug.py         # SCIP_CLI_DEBUG stderr helpers
-├── indexing.py      # SCIP index build + get_db
-├── symbols.py       # Symbol parsing and kinds
-├── queries.py       # Symbol/file SQL queries
-├── source.py        # Filesystem source reads
-├── output.py        # CLI formatting helpers
-├── session.py       # setup() and single-match resolution
-├── targets.py       # file-path heuristics (tests; analyze uses analyze/targets.py)
-├── analyze/         # SQL dashboard queries (project/file/symbol)
-└── commands/        # Subcommand implementations
-```
-
 ## Development
 
 ```bash

{scip_cli-3.0.0 → scip_cli-3.1.0}/scip_cli/SKILL.md

@@ -123,7 +123,7 @@ Priority **high → medium → low**. `--limit` = total finding rows (default 20
 ### reindex
 
 ```bash
-reindex [--path DIR ...] [--tsconfig GLOB ...] [--exclude [GLOB ...]] [--fresh] [--incremental] [--unversioned] [--with-external]
+reindex [--path DIR ...] [--tsconfig GLOB ...] [--exclude [GLOB ...]] [--fresh] [--no-incremental] [--unversioned] [--with-external]
 ```
 
 `--path` and `--tsconfig` are TS-only and mutually exclusive. `--tsconfig` globs expanded in-tool; one heap per file by default (`SCIP_CLI_TS_INDEX_BATCH_SIZE` overrides). Scope/exclude persist in `metadata.json`.
@@ -132,4 +132,4 @@ reindex [--path DIR ...] [--tsconfig GLOB ...] [--exclude [GLOB ...]] [--fresh]
 
 `--fresh` clears persisted metadata (unless combined with scope flags on same command).
 
-`--incremental` — git repo required; errors with `--unversioned` or non-git. Reuses `shards/manifest.json` (`git_commit` + per-shard `tsconfig_digest`); skips clean shards. Dirty: git delta ∪ importer closure → fork `--files` → upsert live `index.db`; deletions remove documents. `SCIP_CLI_FILE_INCREMENTAL=0|1` (default on). `--unversioned` forces glob/full discovery (persisted). Plain `reindex` clears manifest. No `--incremental` with `--fresh`.
+**Incremental by default** in git TypeScript repos: reuses `shards/manifest.json` (`git_commit` + per-shard `tsconfig_digest`); skips clean shards. Dirty: git delta ∪ importer closure → fork `--files` → upsert live `index.db`; deletions remove documents. `--no-incremental` or `--fresh` forces full reindex and clears the manifest. Non-git, `--unversioned`, or non-TypeScript projects always full-reindex.

{scip_cli-3.0.0 → scip_cli-3.1.0}/scip_cli/__init__.py

@@ -1,3 +1,3 @@
 """scip-cli: Fast code intelligence via SCIP indexes."""
 
-__version__ = "3.0.0"
+__version__ = "3.1.0"

{scip_cli-3.0.0 → scip_cli-3.1.0}/scip_cli/__main__.py

@@ -196,10 +196,11 @@ def main() -> None:
         help="Ignore persisted metadata.json; clear it unless scope/exclude flags are set on this run",
     )
     reindex_parser.add_argument(
-        "--incremental",
+        "--no-incremental",
         action="store_true",
         help=(
-            "Reuse cached per-project index parts when inputs are unchanged (full reindex clears the incremental cache)"
+            "Full reindex every shard and clear the shard manifest "
+            "(default: incremental in git TypeScript repos; --fresh also forces full)"
         ),
     )
     reindex_parser.add_argument(

{scip_cli-3.0.0 → scip_cli-3.1.0}/scip_cli/commands/reindex.py

@@ -20,6 +20,24 @@ from ..sql import finalize_index_db
 from ..tsconfig import expand_tsconfig_patterns
 
 
+def _resolve_incremental(
+    root,
+    lang,
+    *,
+    fresh: bool,
+    no_incremental: bool,
+    unversioned: bool,
+) -> bool:
+    """Incremental reindex when eligible; full reindex otherwise."""
+    if fresh or no_incremental:
+        return False
+    if lang != Language.TYPESCRIPT:
+        return False
+    if unversioned or index_unversioned(root):
+        return False
+    return is_versioned_repo(root)
+
+
 def main(args):
     root, lang = find_project_root_and_language()
     if not root:
@@ -30,23 +48,15 @@ def main(args):
     tsconfig_args = getattr(args, "tsconfig", None) or []
     exclude_groups = getattr(args, "exclude", None)
     fresh = getattr(args, "fresh", False)
-    incremental = getattr(args, "incremental", False)
+    no_incremental = getattr(args, "no_incremental", False)
     unversioned = getattr(args, "unversioned", False)
-    if incremental and fresh:
-        print("Error: reindex --incremental and --fresh cannot be combined", file=sys.stderr)
-        sys.exit(1)
-    if incremental and lang is not None and lang != Language.TYPESCRIPT:
-        print("Error: reindex --incremental is only supported for TypeScript projects", file=sys.stderr)
-        sys.exit(1)
-    if incremental and (index_unversioned(root) or not is_versioned_repo(root)):
-        print(
-            (
-                "Error: reindex --incremental requires a git repository "
-                "(use full reindex without --incremental for --unversioned or non-git projects)"
-            ),
-            file=sys.stderr,
-        )
-        sys.exit(1)
+    incremental = _resolve_incremental(
+        root,
+        lang,
+        fresh=fresh,
+        no_incremental=no_incremental,
+        unversioned=unversioned,
+    )
     if path_args and tsconfig_args:
         print("Error: reindex --path and --tsconfig cannot be combined", file=sys.stderr)
         sys.exit(1)

{scip_cli-3.0.0 → scip_cli-3.1.0}/scip_cli/indexing/constants.py

@@ -1,7 +1,5 @@
 """Indexing constants."""
 
-import os
-
 INDEX_TIMEOUT = 300
 DEFAULT_MAX_HEAP_MB = 8192
 PROGRESS_LOG_MIN_PROJECTS = 10
@@ -12,15 +10,9 @@ SCIP_INSTALL_URL = "https://github.com/scip-code/scip/releases"
 #
 # TODO: Remove github_npm.py and runners github: branch when switching back to upstream npm.
 # scip-typescript npx fallback (when the binary is not on PATH). Fork adds --files
-# for partial document indexing (see docs/benchmarks.md). Pin >= ce6e9c9.
+# for partial document indexing. Pin >= ce6e9c9.
 SCIP_TYPESCRIPT_NPX_PACKAGE = "github:flesler/scip-typescript#feat/partial-files"
 
-
-def file_incremental_enabled() -> bool:
-    """Partial reindex via fork --files (default on)."""
-    return os.environ.get("SCIP_CLI_FILE_INCREMENTAL", "1").lower() not in {"0", "false", "no"}
-
-
 # scip-typescript accepts many tsconfig paths per invocation (one .scip, one convert, no merge).
 # Split only when SCIP_CLI_TS_INDEX_BATCH_SIZE is set (OOM/timeout tuning).
 DEFAULT_TS_INDEX_BATCH_SIZE = None

{scip_cli-3.0.0 → scip_cli-3.1.0}/scip_cli/indexing/incremental.py

@@ -10,7 +10,6 @@ from pathlib import Path
 
 from ..cache import index_db_path
 from ..debug import debug_log
-from .constants import file_incremental_enabled
 from .document_upsert import document_paths_in_db, remove_documents_by_paths, upsert_shard_documents
 from .git_delta import (
     GitIndexDelta,
@@ -20,7 +19,7 @@ from .git_delta import (
     shard_dirty_paths,
 )
 from .orchestrate import batch_projects, finalize_part_dbs, index_workers, project_batch_label
-from .performance import flush_summary, metric, note, phase
+from .performance import flush_summary, metric, phase
 from .shard_files import PartialReindexPlan, resolve_partial_reindex_plan, shard_relative_paths
 from .shards import (
     load_manifest_data,
@@ -88,7 +87,7 @@ def _index_dirty_shard(
 ) -> tuple[str, ShardUpdate | None, str | None]:
     project = batch[0]
     partial: PartialReindexPlan | None = None
-    if file_incremental_enabled() and delta is not None:
+    if delta is not None:
         partial = resolve_partial_reindex_plan(
             root,
             project,
@@ -97,8 +96,6 @@ def _index_dirty_shard(
             delta,
             exclude_globs=exclude_globs,
         )
-    elif not file_incremental_enabled():
-        note(f"{shard_key(project)}:partial_off")
 
     if partial is not None and not partial.index_paths and not partial.remove_paths:
         return (

{scip_cli-3.0.0 → scip_cli-3.1.0}/scip_cli/indexing/languages.py

@@ -6,7 +6,7 @@ from pathlib import Path
 
 from .convert import convert_scip_to_db
 from .orchestrate import project_label
-from .runners import run_indexer_with_fallback
+from .runners import indexer_failure_message, run_indexer_with_fallback
 
 
 def _project_cwd(root: Path, project: Path) -> Path:
@@ -29,7 +29,7 @@ def index_python_project(root, project, work_dir, env, *, output_db=None, exclud
         npx_package="@sourcegraph/scip-python",
     )
     if result.returncode != 0:
-        return label, None, result.stderr.strip() or "indexing failed"
+        return label, None, indexer_failure_message(result)
     try:
         convert_scip_to_db(part_scip, db_path, document_path_prefix=project, exclude_globs=exclude_globs)
     finally:
@@ -53,7 +53,7 @@ def index_golang_module(root, module, work_dir, env, *, output_db=None, exclude_
         go_package="github.com/scip-code/scip-go/cmd/scip-go",
     )
     if result.returncode != 0:
-        return label, None, result.stderr.strip() or "indexing failed"
+        return label, None, indexer_failure_message(result)
     try:
         convert_scip_to_db(part_scip, db_path, document_path_prefix=module, exclude_globs=exclude_globs)
     finally:
@@ -77,7 +77,7 @@ def index_rust_crate(root, crate, work_dir, env, *, output_db=None, exclude_glob
         rustup_component="rust-analyzer",
     )
     if result.returncode != 0:
-        return label, None, result.stderr.strip() or "indexing failed"
+        return label, None, indexer_failure_message(result)
     try:
         convert_scip_to_db(part_scip, db_path, document_path_prefix=crate, exclude_globs=exclude_globs)
     finally:

{scip_cli-3.0.0 → scip_cli-3.1.0}/scip_cli/indexing/runners.py

@@ -11,6 +11,17 @@ from ..debug import debug_log
 from .constants import INDEX_TIMEOUT
 
 
+def indexer_failure_message(result: subprocess.CompletedProcess[str]) -> str:
+    """Best-effort detail when an indexer subprocess exits non-zero."""
+    stderr = (result.stderr or "").strip()
+    if stderr:
+        return stderr
+    stdout = (result.stdout or "").strip()
+    if stdout:
+        return stdout
+    return "indexing failed"
+
+
 def run_subprocess(cmd, cwd, env=None):
     """Run subprocess with timeout; raise RuntimeError on timeout."""
     try:

{scip_cli-3.0.0 → scip_cli-3.1.0}/scip_cli/indexing/ts_projects.py

@@ -11,7 +11,7 @@ from .constants import SCIP_TYPESCRIPT_NPX_PACKAGE
 from .convert import convert_scip_to_db
 from .orchestrate import project_batch_label
 from .performance import phase
-from .runners import run_indexer_with_fallback
+from .runners import indexer_failure_message, run_indexer_with_fallback
 
 
 def typescript_index_args(
@@ -96,7 +96,7 @@ def index_ts_projects(
             npx_package=SCIP_TYPESCRIPT_NPX_PACKAGE,
         )
     if result.returncode != 0:
-        return label, None, result.stderr.strip() or "indexing failed"
+        return label, None, indexer_failure_message(result)
     try:
         with phase("scip_convert"):
             convert_scip_to_db(

{scip_cli-3.0.0 → scip_cli-3.1.0}/scripts/bench_fingerprint_smoke.py

@@ -51,7 +51,7 @@ def _restore_snapshot(cache_dir: Path) -> None:
 def _run_reindex(smoke_root: Path, env: dict[str, str]) -> tuple[float, str, str]:
     t0 = time.perf_counter()
     proc = subprocess.run(
-        ["scip-cli", "reindex", "--incremental"],
+        ["scip-cli", "reindex"],
         cwd=smoke_root,
         capture_output=True,
         text=True,
@@ -86,8 +86,6 @@ def _git_delta_scan(smoke_root: Path) -> float:
 def bench_incremental(smoke_root: Path) -> None:
     env = os.environ.copy()
     env["SCIP_CLI_INDEX_TIMING"] = "1"
-    env["SCIP_CLI_FILE_INCREMENTAL"] = "1"
-
     _restore_snapshot(get_cache_dir_from_root(smoke_root))
     warm_wall, warm_line, warm_summary = _run_reindex(smoke_root, env)
     delta_s = _git_delta_scan(smoke_root)

{scip_cli-3.0.0 → scip_cli-3.1.0}/scripts/bench_incremental.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-"""Benchmark reindex --incremental on the versioned multi-shard fixture.
+"""Benchmark incremental reindex on the versioned multi-shard fixture.
 
 Manual only — not part of pre-commit or scripts/test.sh. Requires Node.js + npx.
 
@@ -7,9 +7,7 @@ Manual only — not part of pre-commit or scripts/test.sh. Requires Node.js + np
   scripts/bench_incremental.sh --baseline   # save baseline
   scripts/bench_incremental.sh --compare    # diff against baseline
 
-Default env is Phase 1+3 (shard skip + --files). Phase 2 (TSC --incremental) removed.
-
-  SCIP_CLI_FILE_INCREMENTAL=0  scripts/bench_incremental.sh   # Phase 1 only
+Incremental reindex is the default in git repos (shard skip + partial --files).
 
 Dirty-file cases modify content (append), not mtime-only touch.
 
@@ -168,7 +166,7 @@ def run_benchmarks() -> list[BenchRow]:
     _backup_touch_files(work_root)
     rows: list[BenchRow] = []
 
-    seed = _bench_row(work_root, "seed_warm", ["--incremental"], note="build warm shard cache")
+    seed = _bench_row(work_root, "seed_warm", [], note="build warm shard cache")
     rows.append(seed)
     _save_snapshot(cache)
 
@@ -182,22 +180,22 @@ def run_benchmarks() -> list[BenchRow]:
         _reset_to_warm(work_root, cache)
         rows.append(_bench_row(work_root, scenario, args, note=note, touch=touch))
 
-    bench("warm_all_cached", ["--incremental"], note="no file changes")
+    bench("warm_all_cached", [], note="no file changes")
     bench(
         "touch_small_shard",
-        ["--incremental"],
+        [],
         note="packages/alpha (~4 files)",
         touch=TOUCH_SMALL,
     )
     bench(
         "touch_large_shard",
-        ["--incremental"],
+        [],
         note=f"packages/bulk (~{len(list((FIXTURE_SOURCE / 'packages/bulk/src').rglob('*.ts')))} files)",
         touch=TOUCH_LARGE,
     )
     bench(
         "touch_small_and_large",
-        ["--incremental"],
+        [],
         note="alpha + bulk (2 shards)",
         touch=TOUCH_BOTH,
     )
@@ -208,7 +206,7 @@ def run_benchmarks() -> list[BenchRow]:
         _bench_row(
             work_root,
             "cold_incremental",
-            ["--incremental"],
+            [],
             note="empty shard cache",
         )
     )
@@ -218,7 +216,7 @@ def run_benchmarks() -> list[BenchRow]:
         _bench_row(
             work_root,
             "full_reindex",
-            [],
+            ["--no-incremental"],
             note="non-incremental baseline",
         )
     )

{scip_cli-3.0.0 → scip_cli-3.1.0}/scripts/bench_incremental_gate.py

@@ -33,9 +33,7 @@ BENCH_DIR = ROOT / "tmp" / "benchmarks"
 DECISIONS_PATH = BENCH_DIR / ".incremental-gate-decisions.json"
 FIXTURE_SOURCE = ROOT / "tests" / "fixtures" / "incremental-bench"
 
-BASE_ENV: dict[str, str] = {
-    "SCIP_CLI_FILE_INCREMENTAL": "1",
-}
+BASE_ENV: dict[str, str] = {}
 
 FIXTURE_TOUCH_LARGE = ["packages/bulk/src/modules/module_25.ts"]
 SMOKE_BASELINE_DIR = BENCH_DIR / "gate-smoke-baseline"
@@ -160,7 +158,7 @@ def _run_reindex(work_root: Path, env_overrides: dict[str, str]) -> tuple[float,
     env.update(env_overrides)
     t0 = time.perf_counter()
     proc = subprocess.run(
-        ["scip-cli", "reindex", "--incremental"],
+        ["scip-cli", "reindex"],
         cwd=work_root,
         capture_output=True,
         text=True,

{scip_cli-3.0.0 → scip_cli-3.1.0}/scripts/smoke.local.example.json

@@ -1,6 +1,6 @@
 {
   "root": "/path/to/your-monorepo-checkout",
   "touch_large": [
-    "apps/api/src/services/dataStore.ts"
+    "packages/bulk/src/modules/module_25.ts"
   ]
 }

{scip_cli-3.0.0 → scip_cli-3.1.0}/tests/test_incremental_equivalence.py

@@ -15,9 +15,7 @@ from scip_cli.indexing.typescript import index_typescript, typescript_projects
 FIXTURE_SOURCE = Path(__file__).parent / "fixtures" / "incremental-bench"
 FIXTURE_TOUCH_LARGE = "packages/bulk/src/modules/module_25.ts"
 
-INCREMENTAL_ENV = {
-    "SCIP_CLI_FILE_INCREMENTAL": "1",
-}
+INCREMENTAL_ENV: dict[str, str] = {}
 
 
 def _init_git(root: Path) -> None:

scip_cli-3.1.0/tests/test_indexer_failure_message.py

@@ -0,0 +1,29 @@
+"""Tests for indexer subprocess failure messaging."""
+
+import subprocess
+
+from scip_cli.indexing.runners import indexer_failure_message
+
+
+class TestIndexerFailureMessage:
+    def test_prefers_stderr(self):
+        result = subprocess.CompletedProcess(
+            args=[],
+            returncode=1,
+            stdout="stdout detail",
+            stderr="stderr detail",
+        )
+        assert indexer_failure_message(result) == "stderr detail"
+
+    def test_falls_back_to_stdout_when_stderr_empty(self):
+        result = subprocess.CompletedProcess(
+            args=[],
+            returncode=1,
+            stdout="error: no files got indexed",
+            stderr="",
+        )
+        assert indexer_failure_message(result) == "error: no files got indexed"
+
+    def test_default_when_both_streams_empty(self):
+        result = subprocess.CompletedProcess(args=[], returncode=1, stdout="", stderr="")
+        assert indexer_failure_message(result) == "indexing failed"