ai-codeindex 0.26.1__tar.gz → 0.26.2__tar.gz

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/CHANGELOG.md

@@ -7,6 +7,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.26.2] - 2026-06-22
+
+**Theme**: enrichment-quality patch. `scan-all --ai` was silently writing punt
+text ("I need the file names") as the directory description for leaf dirs and
+caching it as `enrichment: ok`. A/B on a real TS monorepo: 23/50 punts → 0/50.
+No contract change.
+
+> **Upgrade note**: existing `README_AI.md` files enriched by ≤0.26.1 may carry
+> poisoned descriptions stamped `enrichment: ok`, which the cache will not
+> refresh on its own. After upgrading, run **`codeindex scan-all --ai --retry-all`**
+> once to flush and regenerate them.
+
+### Fixed
+
+- **`scan-all --ai` enrichment now feeds each directory its OWN files/symbols, fixing leaf-dir punts** (GH #94): `_enrich_directories_with_ai` built the prompt from `build_safe_subdir_context(child_dirs)` — child *directory* names only — while `build_enrich_prompt` instructed the model "based ONLY on the file names and symbol names above." A leaf dir's content is *files*, not subdirs, so the prompt carried nothing to describe; worse, a single uninformative child like `__tests__` made the subdir context non-empty and short-circuited the README fallback. On a TypeScript monorepo this hit 21/50 enriched dirs (`controllers`, `services`, `routes`, `screens`, `pages` — all the leaf code dirs), and the model correctly punted ("I need the file names and symbol names"). New `cli_scan._build_enrich_summary` re-parses the directory's own files **non-recursively** (`scan_directory(recursive=False)` — no subtree leak) and runs them through the existing `extract_symbol_summary`, then `enricher.merge_enrich_context` combines that (primary) with the subdir names (supplementary). Both sources are AST/tree-derived, never README markdown, so the anti-injection-chain property `build_safe_subdir_context` was built for is preserved. `build_enrich_prompt`'s instruction was de-lied ("based ONLY on the information above (file names, symbol names, and subdirectories)") and a dir with genuinely no indexable content is now marked `failed (reason: no indexable content)` — retryable — instead of being sent a self-contradictory prompt. *(The original issue blamed `parallel_workers=16` concurrency; that was a confounded experiment — enrichment is a serial loop and the punts occurred serially. Diagnosis corrected by source read + clean repro before this fix.)*
+- **Punt detection (`looks_like_refusal`) now covers the leaf-dir phrasings it was missing** (GH #94, extends GH #85): the GH #85 refusal table had `i need more` but not `i need specific`, and **zero Chinese prefixes** — so `I need specific file names…` / `我需要…文件名和符号名` / `请提供…` slipped through and were stamped `<!-- enrichment: ok -->`, poisoning the cache (skipped on re-run without `--retry-all`). Added `i need specific` plus a deliberately specific Chinese set (`我需要`, `请提供`, `需要具体`, `信息不足`, `我看到的信息`) — kept narrow so legit one-liners that merely contain `需要` (e.g. "需要登录的接口") are not misclassified, locked by `test_chinese_descriptions_with_xuyao_are_not_refusals`.
+
 ## [0.26.1] - 2026-06-05
 
 **Theme**: GitFlow + non-Python UX patch. Four bug fixes from continued fabricOS dogfood — all paths where `codeindex` silently no-op'd or warned about state that wasn't real. No contract changes.

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ai-codeindex
-Version: 0.26.1
+Version: 0.26.2
 Summary: AI-native code indexing tool for large codebases
 Project-URL: Homepage, https://github.com/dreamlx/codeindex
 Project-URL: Documentation, https://github.com/dreamlx/codeindex
@@ -578,7 +578,7 @@ See [Release Automation Guide](docs/development/QUICK_START_RELEASE.md) for deta
 
 ## Roadmap
 
-**Current version**: v0.25.0
+**Current version**: v0.26.2
 
 **Recent milestones**:
 - v0.23.0 — **AI-Enhanced Module Descriptions**: two-phase pipeline, auto-AI enrichment, post-commit thin wrapper

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/README.md

@@ -521,7 +521,7 @@ See [Release Automation Guide](docs/development/QUICK_START_RELEASE.md) for deta
 
 ## Roadmap
 
-**Current version**: v0.25.0
+**Current version**: v0.26.2
 
 **Recent milestones**:
 - v0.23.0 — **AI-Enhanced Module Descriptions**: two-phase pipeline, auto-AI enrichment, post-commit thin wrapper

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/docs/planning/ROADMAP.md

@@ -1,7 +1,7 @@
 # codeindex Strategic Roadmap
 
 **Last Updated**: 2026-03-06
-**Current Version**: v0.25.0
+**Current Version**: v0.26.2
 **Vision**: Universal Code Parser - Best-in-class multi-language AST parser for AI-assisted development
 **Positioning**: Focused on code parsing and structured data extraction, not AI analysis
 
@@ -534,4 +534,4 @@
 **Next Review**: 2026-03-31
 **Maintained By**: @dreamlx
 **Last Updated**: 2026-03-06
-**Current Version**: v0.25.0
+**Current Version**: v0.26.2

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ai-codeindex"
-version = "0.26.1"
+version = "0.26.2"
 description = "AI-native code indexing tool for large codebases"
 readme = "README.md"
 requires-python = ">=3.10"

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/src/codeindex/README_AI.md

@@ -1,12 +1,12 @@
 <!-- codeindex navigation index — agent: drill into source via Read/Grep for precise mechanism; do not treat this as final word. -->
-<!-- Generated by codeindex (detailed) at 2026-06-05T21:54:59.634399 -->
+<!-- Generated by codeindex (detailed) at 2026-06-21T23:56:14.087118 -->
 
 # codeindex
 
 ## Overview
 
 - **Files**: 89
-- **Symbols**: 572
+- **Symbols**: 574
 
 ## Files
 
@@ -227,6 +227,11 @@ s_
     timeout: int = 120,
     retry_all: bool = False,
 ) -> None`
+- `def _build_enrich_summary(
+    dir_path: Path,
+    child_dirs: list[Path],
+    config: Config,
+) -> str`
 - `def _enrich_directories_with_ai(
     dirs: list[Path],
     tree: DirectoryTree,
@@ -237,14 +242,8 @@ s_
     enrichment_cache: dict[Path, str] | None = None,
 ) -> None`
 - `def _print_enrichment_failure_hint(failed_dirs: list[str]) -> None`
-- `def _process_directory_with_smartwriter(
-    dir_path: Path,
-    tree: DirectoryTree,
-    config: Config,
-    docstring_processor=None,
-) -> tuple[Path, bool, str, int]`
 
-_... and 2 more symbols_
+_... and 3 more symbols_
 
 ### cli_symbols.py
 _CLI commands for symbol indexing and dependency analysis.
@@ -345,7 +344,8 @@ co_
 - `def show_full_config_help() -> None`
 - `def _show_param_section(param_name: str) -> None`
 - `def explain_parameter(
-    param_name: str, curr
+    param_name: str, current_value: Optional[any] = None, cpu_count: Optional[int] = None
+
 
 ---
 _Content truncated due to size limit. See individual module README files for details._

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/src/codeindex/cli_scan.py

@@ -634,6 +634,47 @@ def _process_directories_parallel(
         )
 
 
+def _build_enrich_summary(
+    dir_path: Path,
+    child_dirs: list[Path],
+    config: Config,
+) -> str:
+    """Build the enrichment prompt context for one directory (GH #94).
+
+    Combines the directory's OWN file/symbol summary — re-parsed from source
+    (AST), non-recursively, so subtree content does not leak in — with its
+    subdirectory names. The dir's own content is the primary signal; subdir
+    names are supplementary. This fixes leaf dirs (e.g. ``controllers`` whose
+    only child is ``__tests__``) that previously got only a useless
+    ``Subdirectories: __tests__`` and made the model punt.
+
+    Args:
+        dir_path: Directory being enriched.
+        child_dirs: Indexed child directories (from ``tree.get_children``).
+        config: Active configuration (for language/exclude filtering).
+
+    Returns:
+        Combined context string, or empty if the dir has no indexable content.
+    """
+    from .enricher import (
+        build_safe_subdir_context,
+        extract_symbol_summary,
+        merge_enrich_context,
+    )
+    from .parallel import parse_files_parallel
+    from .scanner import scan_directory
+
+    subdir_context = build_safe_subdir_context(child_dirs)
+
+    own_summary = ""
+    scan_result = scan_directory(dir_path, config, recursive=False)
+    if scan_result.files:
+        parse_results = parse_files_parallel(scan_result.files, config, quiet=True)
+        own_summary = extract_symbol_summary(parse_results)
+
+    return merge_enrich_context(own_summary, subdir_context)
+
+
 def _enrich_directories_with_ai(
     dirs: list[Path],
     tree: DirectoryTree,
@@ -665,8 +706,6 @@ def _enrich_directories_with_ai(
     """
     from .enricher import (
         build_enrich_prompt,
-        build_safe_subdir_context,
-        extract_summary_from_readme,
         inject_blockquote,
         looks_like_refusal,
         mark_enrichment_status,
@@ -715,17 +754,21 @@ def _enrich_directories_with_ai(
     for dir_path in enrich_dirs:
         readme_path = dir_path / config.output_file
 
-        # Prefer tree-derived context (subdir names only) to avoid the
-        # injection chain through markdown-embedded AI descriptions.
-        # Fall back to README parsing only for dirs without indexed children
-        # (rare for enrichable navigation/overview dirs).
+        # Build context from the dir's OWN files/symbols (primary) plus its
+        # subdirectory names (supplementary). Both come from AST/tree sources,
+        # never README markdown, so the anti-injection-chain property holds
+        # (GH #94).
         child_dirs = tree.get_children(dir_path)
-        summary = build_safe_subdir_context(child_dirs)
-
-        if not summary:
-            summary = extract_summary_from_readme(readme_path)
+        summary = _build_enrich_summary(dir_path, child_dirs, config)
 
         if not summary:
+            # Nothing indexable to describe — mark failed (retryable) rather
+            # than send a prompt that asks the model to describe nothing.
+            if readme_path.exists():
+                mark_enrichment_status(
+                    readme_path, "failed", reason="no indexable content"
+                )
+            failed_dirs.append(dir_path.name)
             continue
 
         # Include parent directory name for context

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/src/codeindex/enricher.py

@@ -36,11 +36,19 @@ _REFUSAL_PREFIXES = (
     "i'm unable",
     "i am unable",
     "i need more",
+    "i need specific",  # GH #94: leaf-dir punt asking for omitted file/symbol names
     "no file",
     "insufficient context",
     "without more",
     "sorry, i",
     "as an ai",
+    # Chinese punts (GH #94). Kept specific so legit one-liners that merely
+    # contain 需要 (e.g. "需要登录的接口") are not misclassified.
+    "我需要",
+    "请提供",
+    "需要具体",
+    "信息不足",
+    "我看到的信息",
 )
 
 
@@ -165,6 +173,35 @@ def build_safe_subdir_context(child_dirs: list[Path]) -> str:
     return "Subdirectories: " + ", ".join(names) + suffix
 
 
+def merge_enrich_context(own_summary: str, subdir_context: str) -> str:
+    """Combine a directory's own file/symbol summary with its subdir names.
+
+    GH #94: a directory's OWN files and symbols are the primary signal for a
+    one-line description; subdirectory names are only supplementary. The
+    previous enrichment path used subdir context with a README fallback and a
+    single uninformative child (e.g. ``__tests__``) made the subdir context
+    non-empty, short-circuiting the dir's own content and leaving leaf dirs
+    with an empty prompt — so the model punted ("I need the file names").
+
+    Both inputs come from AST/tree sources (``extract_symbol_summary`` and
+    ``build_safe_subdir_context``), never from README markdown, preserving the
+    anti-injection-chain property ``build_safe_subdir_context`` was built for.
+
+    Args:
+        own_summary: ``extract_symbol_summary`` output (file: symbols; ...).
+        subdir_context: ``build_safe_subdir_context`` output (Subdirectories: ...).
+
+    Returns:
+        Combined context string, or empty string if both inputs are empty.
+    """
+    parts = []
+    if own_summary and own_summary.strip():
+        parts.append(f"Files: {own_summary.strip()}")
+    if subdir_context and subdir_context.strip():
+        parts.append(subdir_context.strip())
+    return "\n".join(parts)
+
+
 def build_enrich_prompt(
     dir_name: str,
     symbol_summary: str,
@@ -188,13 +225,14 @@ def build_enrich_prompt(
     return (
         context
         + "\n"
-        "Based ONLY on the file names and symbol names above, write a concise "
-        "functional description of this module (30 chars or less). "
+        "Based ONLY on the information above (file names, symbol names, and "
+        "subdirectories), write a concise functional description of this "
+        "module (30 chars or less). "
         "Describe WHAT it does, not HOW. "
         "Examples: '会员等级、积分、权益卡管理', 'Payment gateway (Alipay/WeChat)', "
         "'物流配送与运费计算'. "
         "Output ONLY the description text. No quotes, no markdown, no explanation. "
-        "Do NOT invent features not evidenced by the symbol names."
+        "Do NOT invent features not evidenced by the names above."
     )
 
 

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/tests/README_AI.md

@@ -1,12 +1,12 @@
 <!-- codeindex navigation index — agent: drill into source via Read/Grep for precise mechanism; do not treat this as final word. -->
-<!-- Generated by codeindex (detailed) at 2026-06-05T21:54:59.931951 -->
+<!-- Generated by codeindex (detailed) at 2026-06-21T23:56:14.360118 -->
 
 # tests
 
 ## Overview
 
 - **Files**: 125
-- **Symbols**: 2164
+- **Symbols**: 2172
 
 ## Files
 

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/tests/test_enricher.py

@@ -420,6 +420,40 @@ class TestMarkEnrichmentStatus:
         assert "line2" not in enrichment_line or "\n" not in enrichment_line
 
 
+class TestMergeEnrichContext:
+    """GH #94: dir's own files/symbols are primary; subdir names supplementary."""
+
+    def test_own_symbols_primary_subdirs_supplementary(self):
+        from codeindex.enricher import merge_enrich_context
+
+        merged = merge_enrich_context(
+            "user.controller.ts: login, logout; auth.ts: verify",
+            "Subdirectories: __tests__",
+        )
+        assert "login" in merged
+        assert "auth.ts" in merged
+        assert "__tests__" in merged  # supplementary, still present
+
+    def test_uninformative_subdir_does_not_hide_own_symbols(self):
+        """The exact bug: 'Subdirectories: __tests__' used to short-circuit."""
+        from codeindex.enricher import merge_enrich_context
+
+        merged = merge_enrich_context("svc.ts: doWork", "Subdirectories: __tests__")
+        assert "doWork" in merged
+
+    def test_empty_own_summary_falls_back_to_subdirs(self):
+        from codeindex.enricher import merge_enrich_context
+
+        merged = merge_enrich_context("", "Subdirectories: components, pages")
+        assert "components" in merged
+
+    def test_both_empty_returns_empty(self):
+        from codeindex.enricher import merge_enrich_context
+
+        assert merge_enrich_context("", "") == ""
+        assert merge_enrich_context("  ", "  ") == ""
+
+
 class TestLooksLikeRefusal:
     """Tests for the GH #85 refusal detector."""
 
@@ -443,6 +477,26 @@ class TestLooksLikeRefusal:
         assert looks_like_refusal("\n\ni cannot help\n")
         assert looks_like_refusal("I Don't Have Enough Info")
 
+    def test_leaf_dir_punt_phrasings(self):
+        """GH #94: leaf dirs punt with phrasings the GH #85 table missed.
+
+        The empty-symbol enrichment prompt makes the model ask for the
+        very file/symbol names that were omitted. These were stamped
+        `enrichment: ok` because the prefix table had `i need more` but
+        not `i need specific`, and zero Chinese prefixes.
+        """
+        assert looks_like_refusal("I need specific file names and symbol names")
+        assert looks_like_refusal("我需要具体的文件名和符号名才能描述")
+        assert looks_like_refusal("请提供文件名和符号名")
+        assert looks_like_refusal("需要具体的文件列表")
+        assert looks_like_refusal("信息不足，无法生成描述")
+        assert looks_like_refusal("我看到的信息不足以描述该模块")
+
+    def test_chinese_descriptions_with_xuyao_are_not_refusals(self):
+        """Legit Chinese one-liners that merely contain 需要 must pass."""
+        assert not looks_like_refusal("需要登录的接口与权限校验")
+        assert not looks_like_refusal("处理需要异步执行的后台任务")
+
     def test_normal_descriptions_are_not_refusals(self):
         """Real one-line descriptions must pass through unmolested."""
         assert not looks_like_refusal("会员等级管理、积分兑换、权益卡券")

{ai_codeindex-0.26.1 → ai_codeindex-0.26.2}/tests/test_enricher_integration.py

@@ -83,6 +83,36 @@ class TestEnrichmentIntegration:
         # Prompt should ask for concise description
         assert "30" in prompt or "concise" in prompt.lower()
 
+    def test_build_enrich_summary_includes_own_symbols_for_leaf_dir(self, tmp_path):
+        """GH #94: a leaf dir whose only child is __tests__ must still get its
+        OWN file/symbol names into the enrichment summary — not just the
+        useless 'Subdirectories: __tests__' that previously short-circuited it.
+        """
+        from codeindex.cli_scan import _build_enrich_summary
+        from codeindex.config import Config
+
+        controllers = tmp_path / "controllers"
+        controllers.mkdir()
+        (controllers / "user.py").write_text(
+            "class UserController:\n"
+            "    def login(self):\n"
+            "        pass\n"
+            "    def logout(self):\n"
+            "        pass\n"
+        )
+        tests_dir = controllers / "__tests__"
+        tests_dir.mkdir()
+        (tests_dir / "user_test.py").write_text("def test_login():\n    pass\n")
+
+        config = Config(languages=["python"], include=["**/*"], exclude=[])
+        summary = _build_enrich_summary(controllers, [tests_dir], config)
+
+        # Own files/symbols present (the bug: these were entirely absent)
+        assert "user.py" in summary
+        assert "login" in summary
+        # Must NOT recurse into __tests__ (scan_directory recursive=False)
+        assert "test_login" not in summary
+
     def test_re_enrich_updates_description(self, tmp_path):
         """Running enrich twice should update, not duplicate."""
         readme = tmp_path / "README_AI.md"