scip-cli 2.11.0__tar.gz → 2.12.0__tar.gz

{scip_cli-2.11.0 → scip_cli-2.12.0}/.github/workflows/ci.yml

@@ -29,7 +29,7 @@ jobs:
           python -m venv .venv
           source .venv/bin/activate
           pip install -e ".[dev]"
-          npm install -g @sourcegraph/scip-typescript
+          npm install -g github:flesler/scip-typescript#feat/incremental-tsc
           mkdir -p ~/.local/bin
           curl -sL https://github.com/scip-code/scip/releases/download/v0.9.0/scip-linux-amd64.tar.gz | tar xz -C ~/.local/bin
           echo "$VIRTUAL_ENV/bin" >> $GITHUB_PATH

{scip_cli-2.11.0 → scip_cli-2.12.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: scip-cli
-Version: 2.11.0
+Version: 2.12.0
 Summary: Fast code intelligence via SCIP indexes
 Project-URL: Homepage, https://github.com/flesler/scip-cli
 Author: Ariel Flesler
@@ -252,7 +252,7 @@ Other environment variables:
 |`SCIP_CLI_MAX_DEF_LINES`|Max definition lines in `code` output|
 |`SCIP_CLI_DEBUG`|Log SQL queries to stderr|
 
-**Version policy:** only the `scip` converter (`expt-convert`) is pinned to the 0.8.x release line because it defines the SQLite schema. Language indexers (`scip-typescript`, `scip-python` via `npx`; `scip-go` via `go install @latest`) install at latest on first use. `rust-analyzer` installs via `rustup component add`.
+**Version policy:** only the `scip` converter (`expt-convert`) is pinned to the 0.8.x release line because it defines the SQLite schema. `scip-typescript` installs via npx from the GitHub fork in `scip_cli/indexing/constants.py` when not on PATH (until upstream ships incremental flags). `scip-python` via `npx` at latest; `scip-go` via `go install @latest`. `rust-analyzer` installs via `rustup component add`.
 
 Large monorepos (>10 tsconfig projects) log per-project progress to stderr during indexing; smaller repos stay quiet aside from the final `Indexed … (size)` line.
 

{scip_cli-2.11.0 → scip_cli-2.12.0}/README.md

@@ -231,7 +231,7 @@ Other environment variables:
 |`SCIP_CLI_MAX_DEF_LINES`|Max definition lines in `code` output|
 |`SCIP_CLI_DEBUG`|Log SQL queries to stderr|
 
-**Version policy:** only the `scip` converter (`expt-convert`) is pinned to the 0.8.x release line because it defines the SQLite schema. Language indexers (`scip-typescript`, `scip-python` via `npx`; `scip-go` via `go install @latest`) install at latest on first use. `rust-analyzer` installs via `rustup component add`.
+**Version policy:** only the `scip` converter (`expt-convert`) is pinned to the 0.8.x release line because it defines the SQLite schema. `scip-typescript` installs via npx from the GitHub fork in `scip_cli/indexing/constants.py` when not on PATH (until upstream ships incremental flags). `scip-python` via `npx` at latest; `scip-go` via `go install @latest`. `rust-analyzer` installs via `rustup component add`.
 
 Large monorepos (>10 tsconfig projects) log per-project progress to stderr during indexing; smaller repos stay quiet aside from the final `Indexed … (size)` line.
 

{scip_cli-2.11.0 → scip_cli-2.12.0}/docs/incremental-reindex-roadmap.md

@@ -18,13 +18,16 @@ Per-tsconfig shard cache with content fingerprints. **Shippable:** unit + integr
 - [x] Stderr stats: `Incremental: N shard(s) reused, M reindexed`
 - [x] Update `scip_cli/SKILL.md` and README flag list
 
-## Phase 2 — scip-typescript fork: TSC incremental program
+## Phase 2 — scip-typescript fork: TSC incremental program (in progress)
 
-- [ ] Fork `sourcegraph/scip-typescript`
-- [ ] `--incremental` / `--ts-build-info-file` CLI flags
-- [ ] `createIncrementalProgram` + `createIncrementalCompilerHost` in `ProjectIndexer`
-- [ ] Default `tsBuildInfoFile` under scip-cli cache dir (not repo root)
-- [ ] scip-cli passes `--ts-build-info-file` when shard is dirty
+Fork work lives outside this repo (local clone + PR to `sourcegraph/scip-typescript`).
+
+- [x] Fork `sourcegraph/scip-typescript` (local: implement + tests passing)
+- [x] `--incremental` / `--ts-build-info-dir` CLI flags
+- [x] `createIncrementalProgram` + `createIncrementalCompilerHost` in `ProjectIndexer`
+- [x] scip-cli passes `--incremental --ts-build-info-dir <cache>/tsbuildinfo` on `reindex --incremental` dirty shards
+- [x] npx fallback via `github:flesler/scip-typescript#feat/incremental-tsc` (`SCIP_TYPESCRIPT_NPX_PACKAGE`; switch back when upstream merges)
+- [ ] Upstream PR merged or fork published to npm long-term
 - [ ] Benchmark dirty shard: program build time before/after
 
 ## Phase 3 — File-level incremental inside a shard

{scip_cli-2.11.0 → scip_cli-2.12.0}/scip_cli/SKILL.md

@@ -169,4 +169,4 @@ reindex [--path DIR ...] [--tsconfig FILE_OR_GLOB ...] [--exclude [GLOB ...]] [-
 
 `--fresh` ignores persisted `metadata.json` and clears it before indexing unless `--path`, `--tsconfig`, or `--exclude` are set on the same command (use `reindex --fresh` to restore a full index).
 
-`--incremental` reuses cached per-project part DBs under `shards/` in the project cache dir (`~/.cache/scip-cli/projects/<slug>/`, alongside `index.db` and `metadata.json`) when inputs are unchanged. **Currently TypeScript only** (Python reindex rejects the flag). Forces one indexer run per discovered project. A plain `reindex` clears the incremental cache; run `reindex --incremental` after an initial full index to benefit. Cannot combine with `--fresh`.
+`--incremental` reuses cached per-project part DBs under `shards/` in the project cache dir (`~/.cache/scip-cli/projects/<slug>/`, alongside `index.db` and `metadata.json`) when inputs are unchanged. **Currently TypeScript only** (Python reindex rejects the flag). Forces one indexer run per discovered project. On dirty shards passes TSC incremental flags; `.tsbuildinfo` under `tsbuildinfo/` in the same cache dir. When `scip-typescript` is not on PATH, npx installs from the GitHub fork (`SCIP_TYPESCRIPT_NPX_PACKAGE` in `indexing/constants.py`) until upstream ships those flags. A plain `reindex` clears the incremental cache; run `reindex --incremental` after an initial full index to benefit. Cannot combine with `--fresh`.

{scip_cli-2.11.0 → scip_cli-2.12.0}/scip_cli/__init__.py

@@ -1,3 +1,3 @@
 """scip-cli: Fast code intelligence via SCIP indexes."""
 
-__version__ = "2.11.0"
+__version__ = "2.12.0"

{scip_cli-2.11.0 → scip_cli-2.12.0}/scip_cli/indexing/constants.py

@@ -7,6 +7,12 @@ SCIP_INSTALL_URL = "https://github.com/scip-code/scip/releases"
 
 # Language indexers (npx / go install) use latest on first use.
 # The scip expt-convert binary is pinned separately in scip_tool.py (DB schema).
+#
+# scip-typescript npx fallback (when the binary is not on PATH). Fork adds
+# --incremental / --ts-build-info-dir for reindex --incremental. Switch back to
+# @sourcegraph/scip-typescript once upstream merges and releases; otherwise
+# publish the fork to npm long-term.
+SCIP_TYPESCRIPT_NPX_PACKAGE = "github:flesler/scip-typescript#feat/incremental-tsc"
 # scip-typescript accepts many tsconfig paths per invocation (one .scip, one convert, no merge).
 # Split only when SCIP_CLI_TS_INDEX_BATCH_SIZE is set (OOM/timeout tuning).
 DEFAULT_TS_INDEX_BATCH_SIZE = None

{scip_cli-2.11.0 → scip_cli-2.12.0}/scip_cli/indexing/shards.py

@@ -13,6 +13,7 @@ from ..tsconfig import resolved_exclude, resolved_include_or_files, tsconfig_for
 MANIFEST_VERSION = 1
 MANIFEST_FILENAME = "manifest.json"
 SHARDS_DIRNAME = "shards"
+TSBUILDINFO_DIRNAME = "tsbuildinfo"
 
 _SOURCE_SUFFIXES = frozenset({".ts", ".tsx", ".js", ".jsx", ".mts", ".cts", ".mjs", ".cjs"})
 
@@ -21,6 +22,11 @@ def shards_dir(cache_dir: Path) -> Path:
     return Path(cache_dir) / SHARDS_DIRNAME
 
 
+def ts_build_info_dir(cache_dir: Path) -> Path:
+    """Directory for per-tsconfig .tsbuildinfo files passed to scip-typescript."""
+    return Path(cache_dir) / TSBUILDINFO_DIRNAME
+
+
 def manifest_path(cache_dir: Path) -> Path:
     return shards_dir(cache_dir) / MANIFEST_FILENAME
 

{scip_cli-2.11.0 → scip_cli-2.12.0}/scip_cli/indexing/typescript.py

@@ -14,7 +14,7 @@ from ..config import CONFIG_FILENAME, load_project_config, resolve_index_roots
 from ..discover import discover_typescript_projects
 from ..scope import load_index_scope, projects_matching_scope
 from ..tsconfig import allow_js_overlay, scope_tsconfig_paths, tsconfig_for_project
-from .constants import PROGRESS_LOG_MIN_PROJECTS
+from .constants import PROGRESS_LOG_MIN_PROJECTS, SCIP_TYPESCRIPT_NPX_PACKAGE
 from .convert import convert_scip_to_db
 from .orchestrate import (
     batch_projects,
@@ -34,6 +34,7 @@ from .shards import (
     shard_db_filename,
     shard_db_path,
     shard_key,
+    ts_build_info_dir,
 )
 
 
@@ -67,11 +68,22 @@ def typescript_projects(root: Path) -> list[Path]:
     return sorted(merged.values(), key=str)
 
 
-def _typescript_index_args(root, output_scip, projects):
+def _typescript_index_args(
+    root,
+    output_scip,
+    projects,
+    *,
+    cache_dir: Path | None = None,
+    tsc_incremental: bool = False,
+):
     args = ["index", "--output", str(output_scip)]
     root = Path(root)
     if not (root / "tsconfig.json").exists():
         args.insert(1, "--infer-tsconfig")
+    if tsc_incremental and cache_dir is not None:
+        build_info_dir = ts_build_info_dir(cache_dir)
+        build_info_dir.mkdir(parents=True, exist_ok=True)
+        args.extend(["--incremental", "--ts-build-info-dir", str(build_info_dir)])
     args.extend(str(project) for project in projects)
     return args
 
@@ -110,6 +122,8 @@ def index_ts_projects(
     *,
     output_db: Path | None = None,
     exclude_globs: tuple[str, ...] = (),
+    cache_dir: Path | None = None,
+    tsc_incremental: bool = False,
 ):
     """Index one or more TypeScript projects into work_dir/index.db (or output_db when set)."""
     root = Path(root)
@@ -119,13 +133,19 @@ def index_ts_projects(
     part_scip = work_dir / "index.scip"
     db_path = Path(output_db) if output_db is not None else work_dir / "index.db"
     index_projects = materialize_allow_js_projects(root, projects, work_dir)
-    index_args = _typescript_index_args(root, part_scip, index_projects)
+    index_args = _typescript_index_args(
+        root,
+        part_scip,
+        index_projects,
+        cache_dir=cache_dir,
+        tsc_incremental=tsc_incremental,
+    )
     result = run_indexer_with_fallback(
         "scip-typescript",
         index_args,
         str(root),
         env=env,
-        npx_package="@sourcegraph/scip-typescript",
+        npx_package=SCIP_TYPESCRIPT_NPX_PACKAGE,
     )
     if result.returncode != 0:
         return label, None, result.stderr.strip() or "indexing failed"
@@ -161,6 +181,8 @@ def _index_shard_batch(
         env,
         output_db=output_db,
         exclude_globs=exclude_globs,
+        cache_dir=cache_dir,
+        tsc_incremental=True,
     )
     if db_path is None:
         return label, None, error

{scip_cli-2.11.0 → scip_cli-2.12.0}/tests/test_incremental_shards.py

@@ -13,6 +13,7 @@ from scip_cli.indexing.shards import (
     save_shard_manifest,
     shard_db_path,
     shard_key,
+    ts_build_info_dir,
 )
 
 
@@ -29,6 +30,41 @@ def _write_ts_project(root: Path, rel_dir: str, *, source: str = "export const x
     return Path(rel_dir)
 
 
+class TestTsBuildInfoDir:
+    def test_lives_under_cache_dir(self, tmp_path):
+        cache_dir = tmp_path / "cache"
+        assert ts_build_info_dir(cache_dir) == cache_dir / "tsbuildinfo"
+
+
+class TestTypescriptIndexArgs:
+    def test_incremental_passes_tsc_flags(self, tmp_path):
+        from scip_cli.indexing.typescript import _typescript_index_args
+
+        cache_dir = tmp_path / "cache"
+        args = _typescript_index_args(
+            tmp_path,
+            tmp_path / "out.scip",
+            [Path("pkg/tsconfig.json")],
+            cache_dir=cache_dir,
+            tsc_incremental=True,
+        )
+        assert "--incremental" in args
+        assert "--ts-build-info-dir" in args
+        assert str(ts_build_info_dir(cache_dir)) in args
+
+    def test_full_reindex_omits_tsc_flags(self, tmp_path):
+        from scip_cli.indexing.typescript import _typescript_index_args
+
+        args = _typescript_index_args(
+            tmp_path,
+            tmp_path / "out.scip",
+            [Path("pkg/tsconfig.json")],
+            tsc_incremental=False,
+        )
+        assert "--incremental" not in args
+        assert "--ts-build-info-dir" not in args
+
+
 class TestShardFingerprint:
     def test_fingerprint_stable_for_same_tree(self, tmp_path):
         root = tmp_path / "repo"
@@ -205,7 +241,17 @@ class TestIndexTypescriptIncremental:
         cache_dir = tmp_path / "cache"
         calls: list[list[Path]] = []
 
-        def fake_index_ts_projects(_root, batch, work_dir, _env, *, output_db=None, exclude_globs=()):
+        def fake_index_ts_projects(
+            _root,
+            batch,
+            work_dir,
+            _env,
+            *,
+            output_db=None,
+            exclude_globs=(),
+            cache_dir=None,
+            tsc_incremental=False,
+        ):
             calls.append(list(batch))
             db = Path(output_db) if output_db is not None else Path(work_dir) / "index.db"
             label = batch[0].as_posix()

{scip_cli-2.11.0 → scip_cli-2.12.0}/tests/test_indexer_fallback.py

@@ -247,8 +247,7 @@ class TestRunIndexerWithFallback:
                 ["index"],
                 "/tmp",
                 env={},
-                npx_package="@sourcegraph/scip-typescript",
-                npx_version="0.4.0",
+                npx_package="github:flesler/scip-typescript#feat/incremental-tsc",
             )
             assert result is mock_result
             mock_npx.assert_called_once()