attocode-code-intel 0.1.21__tar.gz → 0.1.22__tar.gz

attocode_code_intel-0.1.22/.gitignore

@@ -0,0 +1,59 @@
+# Python
+dist/
+__pycache__/
+.venv/
+*.egg-info/
+*.db
+*.db-shm
+*.db-wal
+*.jsonl
+.mypy_cache/
+.pytest_cache/
+.ruff_cache/
+
+# Node
+node_modules/
+*.tsbuildinfo
+
+# Environment
+.env
+
+# Claude/Agent
+.claude/
+.agent/
+.traces/
+.production-agent-sessions/
+.test-sandbox/
+
+# Attocode index, caches and private quality runs
+/.attocode/
+
+docs/superpowers/
+eval/charts/
+eval/*.md
+eval/*.log
+eval/*.json
+
+# Build outputs
+*.vsix
+site/
+snapshot_report.html
+.internal
+
+# scripts
+scripts/*
+!scripts/check_pack_licenses.py
+.DS_Store
+
+#playwright
+.playwright-mcp
+
+#gsd
+.gsd/
+.bg-shell/
+
+#cursor
+.cursor/
+
+# search-quality sweep scratch output
+eval/sweep_out/

{attocode_code_intel-0.1.21 → attocode_code_intel-0.1.22}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: attocode-code-intel
-Version: 0.1.21
+Version: 0.1.22
 Summary: Local code search and navigation for coding agents and people
 Project-URL: Homepage, https://github.com/eren23/attocode
 Project-URL: Repository, https://github.com/eren23/attocode
@@ -139,6 +139,11 @@ To use the latest source, install from a clone of this repository: `uv tool inst
 
 ## What your agent gets
 
+Version 0.1.22 adds an opt-in discovery budget of up to 100,000 files
+(`ATTOCODE_INTEL_DISCOVERY_MAX_FILES`; the default stays 50,000). It also packages
+the LLM client, so `ATTOCODE_FLAG_CONFIDENCE=llm` and LLM rule synthesis work from
+a PyPI install when an OpenRouter or Anthropic key is set.
+
 Version 0.1.21 includes configured search-discovery coverage, OCaml interface
 parsing and more conservative Go/Rust fallback evidence. The existing 100 ms native
 budget and 50,000-file discovery safety ceiling remain; bounded fallback is heuristic.
@@ -233,7 +238,7 @@ failed the frozen protocol. It provides no quality, cost or time benefit evidenc
 
 - **First search.** The first tool call starts the index build. A search waits up to 15 seconds for the index. On a large repository, a search can report that the index is warming. Then search again. An empty result during warm-up does not prove that the code is absent.
 - **Index location.** The index is in `.attocode/` in your project. When the server makes that folder, it adds a `.gitignore`, so the index stays out of commits.
-- **What the index reads.** The server reads at most 50,000 files and skips files over 1 MB. It skips files that `.gitignore` ignores, and folders that start with a dot. It also skips folders named `dist`, `build`, `out`, `output`, `target`, `env`, `tmp`, `vendor` or `coverage`, and binary, media and lock files. If a build-named folder contains real source, explicitly re-include it with a root `.gitignore` entry such as `!dist/`; discovery and edit refresh honor that override.
+- **What the index reads.** By default the server discovers at most 50,000 eligible files and skips files over 1 MB. From 0.1.22, the [discovery budget setting](https://github.com/eren23/attocode/blob/main/docs/intelligence-discovery.md) allows an explicit limit up to 100,000; the selected index can be smaller under its independent file limit. It skips files that `.gitignore` ignores, and folders that start with a dot. It also skips folders named `dist`, `build`, `out`, `output`, `target`, `env`, `tmp`, `vendor` or `coverage`, and binary, media and lock files. If a build-named folder contains real source, explicitly re-include it with a root `.gitignore` entry such as `!dist/`; discovery and edit refresh honor that override.
 - **Languages.** The base install parses Bash, C, C#, C++, CSS, Elixir, Go, HCL, HTML, Java, JavaScript, JSON, Kotlin, Lua, PHP, Python, Ruby, Rust, Scala, Swift, TOML, TypeScript, YAML and Zig. Optional grammars are loaded only when already installed or cached; installing the `tree-sitter-all` extra does not trigger grammar downloads during navigation. A slow grammar falls back after a 100 ms parse budget. Static imports and references are partial evidence, not compiler/type resolution.
 - **Degraded parsing (development builds).** Selecting a missing definition in
   a file with a recorded native parse failure permits one bounded retry for

{attocode_code_intel-0.1.21 → attocode_code_intel-0.1.22}/README.md

@@ -26,6 +26,11 @@ To use the latest source, install from a clone of this repository: `uv tool inst
 
 ## What your agent gets
 
+Version 0.1.22 adds an opt-in discovery budget of up to 100,000 files
+(`ATTOCODE_INTEL_DISCOVERY_MAX_FILES`; the default stays 50,000). It also packages
+the LLM client, so `ATTOCODE_FLAG_CONFIDENCE=llm` and LLM rule synthesis work from
+a PyPI install when an OpenRouter or Anthropic key is set.
+
 Version 0.1.21 includes configured search-discovery coverage, OCaml interface
 parsing and more conservative Go/Rust fallback evidence. The existing 100 ms native
 budget and 50,000-file discovery safety ceiling remain; bounded fallback is heuristic.
@@ -120,7 +125,7 @@ failed the frozen protocol. It provides no quality, cost or time benefit evidenc
 
 - **First search.** The first tool call starts the index build. A search waits up to 15 seconds for the index. On a large repository, a search can report that the index is warming. Then search again. An empty result during warm-up does not prove that the code is absent.
 - **Index location.** The index is in `.attocode/` in your project. When the server makes that folder, it adds a `.gitignore`, so the index stays out of commits.
-- **What the index reads.** The server reads at most 50,000 files and skips files over 1 MB. It skips files that `.gitignore` ignores, and folders that start with a dot. It also skips folders named `dist`, `build`, `out`, `output`, `target`, `env`, `tmp`, `vendor` or `coverage`, and binary, media and lock files. If a build-named folder contains real source, explicitly re-include it with a root `.gitignore` entry such as `!dist/`; discovery and edit refresh honor that override.
+- **What the index reads.** By default the server discovers at most 50,000 eligible files and skips files over 1 MB. From 0.1.22, the [discovery budget setting](https://github.com/eren23/attocode/blob/main/docs/intelligence-discovery.md) allows an explicit limit up to 100,000; the selected index can be smaller under its independent file limit. It skips files that `.gitignore` ignores, and folders that start with a dot. It also skips folders named `dist`, `build`, `out`, `output`, `target`, `env`, `tmp`, `vendor` or `coverage`, and binary, media and lock files. If a build-named folder contains real source, explicitly re-include it with a root `.gitignore` entry such as `!dist/`; discovery and edit refresh honor that override.
 - **Languages.** The base install parses Bash, C, C#, C++, CSS, Elixir, Go, HCL, HTML, Java, JavaScript, JSON, Kotlin, Lua, PHP, Python, Ruby, Rust, Scala, Swift, TOML, TypeScript, YAML and Zig. Optional grammars are loaded only when already installed or cached; installing the `tree-sitter-all` extra does not trigger grammar downloads during navigation. A slow grammar falls back after a 100 ms parse budget. Static imports and references are partial evidence, not compiler/type resolution.
 - **Degraded parsing (development builds).** Selecting a missing definition in
   a file with a recorded native parse failure permits one bounded retry for

{attocode_code_intel-0.1.21 → attocode_code_intel-0.1.22}/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "attocode-code-intel"
-version = "0.1.21"
+version = "0.1.22"
 description = "Local code search and navigation for coding agents and people"
 readme = "README.md"
 requires-python = ">=3.12"
@@ -142,8 +142,8 @@ packages = [
 ]
 
 [tool.hatch.build.targets.sdist]
-# Frozen experiment results remain available in the tagged Git repository.
-exclude = ["/evals/results/"]
+# Frozen study designs and results remain available in the tagged Git repository.
+exclude = ["/evals/"]
 
 [tool.pytest.ini_options]
 asyncio_mode = "auto"

{attocode_code_intel-0.1.21 → attocode_code_intel-0.1.22}/src/attocode_intel/__init__.py

@@ -49,4 +49,4 @@ def __getattr__(name: str) -> Any:
     raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
 
 # The other copy is packages/code-intel/pyproject.toml. A test checks that both match.
-__version__ = "0.1.21"
+__version__ = "0.1.22"

{attocode_code_intel-0.1.21 → attocode_code_intel-0.1.22}/src/attocode_intel/_internal/integrations/context/codebase_context.py

@@ -821,6 +821,27 @@ def build_dependency_graph(
     return graph
 
 
+_DISCOVERY_ENV = "ATTOCODE_INTEL_DISCOVERY_MAX_FILES"
+_DEFAULT_DISCOVERY_MAX_FILES = 50_000
+_HARD_DISCOVERY_MAX_FILES = 100_000
+
+
+def _validate_discovery_max_files(value: int) -> int:
+    if type(value) is not int or not 1 <= value <= _HARD_DISCOVERY_MAX_FILES:
+        raise ValueError(f"discovery_max_files must be an integer from 1 to {_HARD_DISCOVERY_MAX_FILES}")
+    return value
+
+
+def _discovery_max_files_from_env() -> int:
+    raw = os.environ.get(_DISCOVERY_ENV)
+    if raw is None:
+        return _DEFAULT_DISCOVERY_MAX_FILES
+    try:
+        return _validate_discovery_max_files(int(raw))
+    except ValueError as exc:
+        raise ValueError(f"{_DISCOVERY_ENV} must be an integer from 1 to {_HARD_DISCOVERY_MAX_FILES}") from exc
+
+
 @dataclass
 class CodebaseContextManager:
     """Manages codebase context for intelligent code understanding.
@@ -853,6 +874,12 @@ class CodebaseContextManager:
     _ignored_checkout_dirs_capped: bool = field(default=False, repr=False)
     cancelled: Callable[[], bool] | None = field(default=None, repr=False, compare=False)
     _discovery_snapshot: Any = field(default=None, repr=False, compare=False)
+    # Separate from max_files, which caps selection after importance scoring.
+    # Capture the environment per instance; preserve existing positional fields.
+    discovery_max_files: int = field(default_factory=_discovery_max_files_from_env, kw_only=True)
+
+    def __post_init__(self) -> None:
+        _validate_discovery_max_files(self.discovery_max_files)
 
     def _check_cancelled(self) -> None:
         if self.cancelled is not None and self.cancelled():
@@ -906,7 +933,8 @@ class CodebaseContextManager:
         directory/file and the ignore-file identity must still match. A failed
         validation leaves this context empty so its normal fresh discovery runs.
         """
-        if snapshot is None or not snapshot.valid_for(self):
+        scan_limit = _validate_discovery_max_files(self.discovery_max_files)
+        if snapshot is None or len(snapshot.files) > scan_limit or not snapshot.valid_for(self):
             return False
         files = [FileInfo(*row) for row in snapshot.files]
         cap = _compute_dynamic_cap(files, self.max_files)
@@ -927,6 +955,7 @@ class CodebaseContextManager:
         Returns:
             List of FileInfo objects for discovered files.
         """
+        scan_limit = _validate_discovery_max_files(self.discovery_max_files)
         root = Path(self.root_dir)
         resolved_root = root.resolve()
         from attocode_intel._internal.integrations.context.discovery_snapshot import (
@@ -962,7 +991,7 @@ class CodebaseContextManager:
         guard(root / ".gitignore", missing_ok=True)
         ignore = IgnoreManager(root, default_dirs=self.ignore_patterns)
         files: list[FileInfo] = []
-        _SAFETY_CEILING = 50_000  # noqa: N806  # OOM guard for massive repos
+        scan_truncated = False
         self._ignored_checkout_dirs = []
         self._ignored_checkout_dirs_capped = False
 
@@ -1028,6 +1057,12 @@ class CodebaseContextManager:
                 if size > 1_000_000:  # 1MB
                     continue
 
+                # One further eligible file proves overflow. Excluded files do
+                # not, and no descriptor/content is allocated for this lookahead.
+                if len(files) >= scan_limit:
+                    scan_truncated = True
+                    break
+
                 is_test = any(p in rel_path.lower() for p in TEST_PATTERNS)
                 is_config = any(p in rel_path.lower() for p in CONFIG_PATTERNS)
 
@@ -1050,10 +1085,7 @@ class CodebaseContextManager:
                     line_count=line_count,
                 ))
 
-                if len(files) >= _SAFETY_CEILING:
-                    break
-
-            if len(files) >= _SAFETY_CEILING:
+            if scan_truncated:
                 break
 
         # Score importance
@@ -1084,14 +1116,14 @@ class CodebaseContextManager:
         # Truncate to dynamic cap AFTER scoring (smart truncation)
         cap = _compute_dynamic_cap(files, self.max_files)
         self.discovered_file_count = len(files)
-        self.discovery_truncated = len(files) > cap
+        self.discovery_truncated = scan_truncated or len(files) > cap
         if len(files) > cap:
             files = files[:cap]
 
         self._files = files
         self.refresh_excluded_checkout_roots()
         self._last_refresh_time = time.monotonic()
-        if snapshot_safe and not self.discovery_truncated and self.discovered_file_count < _SAFETY_CEILING:
+        if snapshot_safe and not self.discovery_truncated and self.discovered_file_count < scan_limit:
             self._discovery_snapshot = DiscoverySnapshot(
                 os.path.abspath(self.root_dir), frozenset(self.ignore_patterns),
                 self._last_refresh_time,
@@ -1375,55 +1407,6 @@ class CodebaseContextManager:
 
         return result
 
-    # --- Recursive context retrieval ---
-
-    def get_recursive_context(
-        self,
-        seed_files: list[str],
-        token_budget: int = 20_000,
-        max_depth: int = 3,
-        max_files: int = 15,
-    ) -> str:
-        """Gather context by recursively following imports from seed files.
-
-        Uses the RecursiveContextRetriever to mechanically follow Python/JS
-        imports starting from seed files, gathering content up to a depth
-        and token budget. Zero LLM cost.
-
-        Args:
-            seed_files: Files to start traversal from.
-            token_budget: Maximum tokens to gather.
-            max_depth: Maximum import-following depth.
-            max_files: Maximum files to visit.
-
-        Returns:
-            Combined content from all gathered files.
-        """
-        from attocode_intel._internal.tricks.recursive_context import RecursiveContextRetriever
-
-        class _FileProvider:
-            def __init__(self, root: str) -> None:
-                self._root = os.path.realpath(root)
-
-            def get_content(self, path: str) -> str | None:
-                candidate = os.path.normpath(os.path.join(self._root, path))
-                if not candidate.startswith(self._root):
-                    return None  # Path traversal rejected
-                try:
-                    with open(candidate, encoding="utf-8", errors="replace") as f:
-                        return f.read()
-                except (OSError, UnicodeDecodeError):
-                    return None
-
-        retriever = RecursiveContextRetriever(
-            _FileProvider(self.root_dir),
-            max_depth=max_depth,
-            token_budget=token_budget,
-            max_files=max_files,
-        )
-        result = retriever.retrieve(seed_files)
-        return result.content
-
     # --- Incremental update API ---
 
     def mark_file_dirty(self, file_path: str) -> None:

{attocode_code_intel-0.1.21 → attocode_code_intel-0.1.22}/src/attocode_intel/_internal/integrations/context/semantic_search.py

@@ -2598,9 +2598,7 @@ class SemanticSearchManager:
             "discovery_max_files": discovery_limit,
             "discovery_discovered_files": discovered,
             "discovery_selected_files": selected,
-            # Discovery stops enumerating at50k. Counts at that boundary are
-            # lower bounds, even when the later composition cap keeps them all.
-            "discovery_truncated": bool(getattr(ctx, "discovery_truncated", False)) or discovered >= 50_000,
+            "discovery_truncated": bool(getattr(ctx, "discovery_truncated", False)),
         })
         self._kw_index_built = True
 

{attocode_code_intel-0.1.21 → attocode_code_intel-0.1.22}/src/attocode_intel/confidence/llm.py

@@ -36,16 +36,50 @@ _ANTHROPIC_MODEL = "claude-haiku-4-5-20251001"
 _WORKERS = 32
 
 
-def _default_llm_caller() -> LLMCaller:
-    """The project's shared client. Only importable inside this repo."""
-    try:
-        from eval.meta_harness._llm_client import call_llm
-    except ModuleNotFoundError as exc:  # pragma: no cover - depends on checkout
-        raise RuntimeError(
-            "No LLM caller available. Pass llm_caller=..., or run inside the "
-            "attocode repo where eval.meta_harness is importable."
-        ) from exc
+def call_llm(prompt: str, model: str = "", max_tokens: int = 4096) -> str:
+    """Send one prompt through OpenRouter or Anthropic and return the text.
+
+    OpenRouter is used when ``OPENROUTER_API_KEY`` is set, otherwise
+    ``ANTHROPIC_API_KEY``. The SDKs are optional and imported on demand.
+    """
+    import os
+
+    or_key = os.environ.get("OPENROUTER_API_KEY", "")
+    anthropic_key = os.environ.get("ANTHROPIC_API_KEY", "")
+
+    if or_key:
+        import openai
+
+        client = openai.OpenAI(base_url="https://openrouter.ai/api/v1", api_key=or_key)
+        response = client.chat.completions.create(
+            model=model or "anthropic/claude-sonnet-5",
+            max_tokens=max_tokens,
+            messages=[{"role": "user", "content": prompt}],
+        )
+        return response.choices[0].message.content or ""
+
+    if anthropic_key:
+        import anthropic
 
+        model = model or "claude-sonnet-5"
+        # Sonnet 5 and Opus 5 think when `thinking` is omitted; keep thinking off.
+        extra = {"thinking": {"type": "disabled"}} if model in ("claude-sonnet-5", "claude-opus-5") else {}
+        response = anthropic.Anthropic(api_key=anthropic_key).messages.create(
+            model=model,
+            max_tokens=max_tokens,
+            messages=[{"role": "user", "content": prompt}],
+            **extra,
+        )
+        # Thinking blocks can come first, so read the first text block.
+        return next((b.text for b in response.content if b.type == "text"), "")
+
+    raise RuntimeError(
+        "No API key found. Set OPENROUTER_API_KEY or ANTHROPIC_API_KEY "
+        "in environment or .env file."
+    )
+
+
+def _default_llm_caller() -> LLMCaller:
     def call(prompt: str, model: str, max_tokens: int) -> str:
         return call_llm(prompt, model=model, max_tokens=max_tokens)
 
@@ -53,12 +87,10 @@ def _default_llm_caller() -> LLMCaller:
 
 
 def _load_env() -> None:
-    """Populate keys from the project .env when running inside the repo."""
-    try:
-        from eval.meta_harness._llm_client import load_env
-    except ModuleNotFoundError:  # pragma: no cover - depends on checkout
-        return
-    load_env()
+    """Populate keys from .env and ~/.jev/env (same loader as the jev scorer)."""
+    from attocode_intel.confidence.jev import _load_env as load
+
+    load()
 
 
 def _cheap_model() -> str:

{attocode_code_intel-0.1.21 → attocode_code_intel-0.1.22}/src/attocode_intel/rules/synthesis.py

@@ -307,27 +307,11 @@ def _validate_llm_yaml(
 
 
 def _default_llm_caller(prompt: str) -> str:
-    """Resolve the project's shared LLM client lazily — keeps the
-    optional ``openai``/``anthropic`` import out of the import path for
-    callers that only want the deterministic synthesizer.
-
-    The default client lives under the dev-only ``eval/meta_harness/``
-    tree, which isn't bundled with the installed wheel. When that path
-    is unavailable we surface a clear error so the synthesizer's
-    ``except`` arm can attach it as a diagnostic, instead of letting an
-    opaque ``ModuleNotFoundError`` bubble out of the MCP tool.
-    """
-    try:
-        from eval.meta_harness._llm_client import call_llm, load_env
-    except ModuleNotFoundError as exc:
-        raise RuntimeError(
-            "LLM synthesis requested but no llm_caller was configured "
-            "and the dev meta-harness client is not importable from this "
-            "process. Pass llm_caller=... explicitly when calling "
-            "synthesize_rule(mode='llm' or 'auto') from production."
-        ) from exc
-
-    load_env()
+    """Call the packaged LLM client. A missing key or SDK raises, and the
+    synthesizer's ``except`` arm reports it as a diagnostic."""
+    from attocode_intel.confidence.llm import _load_env, call_llm
+
+    _load_env()
     return call_llm(prompt, max_tokens=2048)
 
 

attocode_code_intel-0.1.22/tests/fixtures/sample_project/.attocode/.gitignore

@@ -0,0 +1,2 @@
+# Local index data of attocode-code-intel
+*

attocode_code_intel-0.1.22/tests/fixtures/sample_project/main.py

@@ -0,0 +1,34 @@
+"""Sample application entry point for integration tests."""
+
+from utils import parse_config, validate_input
+from models import User, Config
+
+
+class App:
+    """Main application class."""
+
+    def __init__(self, config: Config) -> None:
+        self.config = config
+        self.users: list[User] = []
+
+    def add_user(self, name: str, email: str) -> User:
+        """Create and register a new user."""
+        validate_input(name)
+        user = User(name=name, email=email)
+        self.users.append(user)
+        return user
+
+    def run(self) -> None:
+        """Start the application."""
+        settings = parse_config(self.config.path)
+        print(f"Running with {len(settings)} settings")
+
+
+def main() -> None:
+    config = Config(path="config.yaml", debug=True)
+    app = App(config)
+    app.run()
+
+
+if __name__ == "__main__":
+    main()

attocode_code_intel-0.1.22/tests/fixtures/sample_project/models.py

@@ -0,0 +1,25 @@
+"""Data models for the sample project."""
+
+from dataclasses import dataclass, field
+
+
+@dataclass
+class User:
+    """Represents an application user."""
+
+    name: str
+    email: str
+    active: bool = True
+
+    def deactivate(self) -> None:
+        self.active = False
+
+
+@dataclass
+class Config:
+    """Application configuration."""
+
+    path: str
+    debug: bool = False
+    max_retries: int = 3
+    tags: list[str] = field(default_factory=list)

attocode_code_intel-0.1.22/tests/fixtures/sample_project/utils.py

@@ -0,0 +1,26 @@
+"""Utility functions for the sample project."""
+
+import os
+import json
+
+
+def parse_config(path: str) -> dict:
+    """Parse a configuration file and return settings."""
+    if not os.path.exists(path):
+        return {"default": True}
+    with open(path) as f:
+        return json.load(f)
+
+
+def validate_input(value: str) -> bool:
+    """Validate that input is non-empty and reasonable length."""
+    if not value or not value.strip():
+        raise ValueError("Input cannot be empty")
+    if len(value) > 255:
+        raise ValueError("Input too long")
+    return True
+
+
+def format_output(data: dict) -> str:
+    """Format data dictionary as a readable string."""
+    return "\n".join(f"{k}: {v}" for k, v in sorted(data.items()))

{attocode_code_intel-0.1.21 → attocode_code_intel-0.1.22}/tests/test_capacity_experiment.py

@@ -41,3 +41,32 @@ async def test_capacity_phase_validates_actual_delivered_search_sources(tmp_path
         source_rows = [row for value in validations for row in value if row.get('kind') == 'source']
         assert len(source_rows) >= 3
         assert any(row.get('policy', {}).get('producer') for row in source_rows)
+
+
+async def test_capacity_phase_expands_same_persisted_index_under_new_scan_policy(tmp_path, monkeypatch):
+    monkeypatch.setenv('ATTOCODE_INTEL_MAX_FILES', '100000')
+    monkeypatch.setenv('ATTOCODE_INTEL_RANKING_PROVIDER', 'off')
+    tree = tmp_path / 'project'
+    generate(tree, 3)
+    previous = None
+    for name, budget, selected, truncated in [('default', '2', 2, True),
+                                               ('expanded', '3', 3, False),
+                                               ('restart', '3', 3, False)]:
+        monkeypatch.setenv('ATTOCODE_INTEL_DISCOVERY_MAX_FILES', budget)
+        output = tmp_path / name
+        spec = {'tree': str(tree), 'size': 3, 'phase': name, 'seconds': 40,
+                'discovery_env': budget, 'expected_selected': selected,
+                'expected_truncated': truncated,
+                'expected_outcome': 'limited' if truncated else 'passed',
+                'persisted_before': previous is not None}
+        result = await phase(spec, output)
+        assert result['complete'] and result['closed']
+        assert result['expected_selected_files'] == selected
+        assert result['discovery_truncated'] is truncated
+        assert result['discovery_env'] == budget and result['outcome'] == spec['expected_outcome']
+        if previous is not None:
+            assert result['databases_before'] == previous['databases_after']
+        identities = json.loads((output / 'candidate-identities.json').read_bytes())
+        expected = [f'module_{n:06d}.py' for n in range(selected)]
+        assert identities['keyword_paths'] == identities['body_paths'] == expected
+        previous = result