wikifier 4.6.4__tar.gz → 4.6.5__tar.gz

{wikifier-4.6.4/wikifier.egg-info → wikifier-4.6.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: wikifier
-Version: 4.6.4
+Version: 4.6.5
 Summary: Zero-dependency agent-to-agent codebase wiki for LLMs and AI agents. Autonomously maintained via record-change and mark-green for token-efficient lookup of files, dependencies, health, and summaries — across tiny scripts to 50k+ monorepos. Optional MCP server with rich tools for agents.
 Author-email: Aron Amos <aron@example.com>
 Maintainer: Aron Amos
@@ -109,8 +109,9 @@ Advanced intel as needed: `get_dependencies`, `get_dependents`, `get_cycles`, ba
 
 - **Import analysis** — Python, JS/TS (ESM/CJS, barrels), Rust (`use`/`mod` + best-effort `crate::` paths), Go, C/C++ includes, C# usings; per-edge confidence; barrel expansion for TS/JS
 - **Incremental pipeline** — pure-Python `update-maps`: dirty parse → import cache → reverse deps → cycles → `library.md`
-- **Warm agent maps (4.6.3+ / 4.6.4+)** — zero-dirty fast path skips full graph/`library.md` rewrite; **stdlib SQLite** import cache (dual-read legacy JSON) so warm dirty scans use mtime/hash index not multi‑MB pair JSON; content-hash dirty; `--directory=` scopes walks
-- **Partial-map honesty** — `map_coverage` on `update_maps` / bootstrap (`complete`, `files_remaining_dirty`, `acs_version`, `cache_backend`); budgeted runs are not “done”
+- **Warm agent maps (4.6.3–4.6.5)** — zero-dirty fast path; **stdlib SQLite** cache; **candidate list reuse** + scoped/monitored-first walks (no whole-tree thrash); content-hash dirty
+- **Partial-map honesty** — `map_coverage` on `update_maps` / bootstrap / **`suggest_next`**; incomplete maps yield `update_maps_until_complete` actions
+- **Cache ops** — `wikifier cache-status` (backend, bytes, ACS, coverage, dual-write policy)
 - **Selective agent work** — health + suggest bias to 🔴/actionable 🟡 only; **ACS v1.3** `reason_code` / `agent_signal`; prefer `actionable_low_conf_edges` + reason codes — **never** raw `low_conf_edges` alone
 - **Scale** — reverse index + barrel invalidation so one edit doesn’t re-scan the monorepo
 - **MCP tools** — optional server for Claude, Cursor, Cline, and other MCP clients

{wikifier-4.6.4 → wikifier-4.6.5}/README.md

@@ -77,8 +77,9 @@ Advanced intel as needed: `get_dependencies`, `get_dependents`, `get_cycles`, ba
 
 - **Import analysis** — Python, JS/TS (ESM/CJS, barrels), Rust (`use`/`mod` + best-effort `crate::` paths), Go, C/C++ includes, C# usings; per-edge confidence; barrel expansion for TS/JS
 - **Incremental pipeline** — pure-Python `update-maps`: dirty parse → import cache → reverse deps → cycles → `library.md`
-- **Warm agent maps (4.6.3+ / 4.6.4+)** — zero-dirty fast path skips full graph/`library.md` rewrite; **stdlib SQLite** import cache (dual-read legacy JSON) so warm dirty scans use mtime/hash index not multi‑MB pair JSON; content-hash dirty; `--directory=` scopes walks
-- **Partial-map honesty** — `map_coverage` on `update_maps` / bootstrap (`complete`, `files_remaining_dirty`, `acs_version`, `cache_backend`); budgeted runs are not “done”
+- **Warm agent maps (4.6.3–4.6.5)** — zero-dirty fast path; **stdlib SQLite** cache; **candidate list reuse** + scoped/monitored-first walks (no whole-tree thrash); content-hash dirty
+- **Partial-map honesty** — `map_coverage` on `update_maps` / bootstrap / **`suggest_next`**; incomplete maps yield `update_maps_until_complete` actions
+- **Cache ops** — `wikifier cache-status` (backend, bytes, ACS, coverage, dual-write policy)
 - **Selective agent work** — health + suggest bias to 🔴/actionable 🟡 only; **ACS v1.3** `reason_code` / `agent_signal`; prefer `actionable_low_conf_edges` + reason codes — **never** raw `low_conf_edges` alone
 - **Scale** — reverse index + barrel invalidation so one edit doesn’t re-scan the monorepo
 - **MCP tools** — optional server for Claude, Cursor, Cline, and other MCP clients

{wikifier-4.6.4 → wikifier-4.6.5}/skills/run.md

@@ -12,7 +12,8 @@
 - **First-run:** `init` → `update-maps` → `health --summary` → `suggest_next_actions`. Map is automatic; wiki *prose* is agent-filled over time.
 - **Map-first ≠ wiki-done:** `health_score` **Map Ready** + many 🟡 *Initial stubs* means structural coverage only. Do **not** bulk-wiki stubs. Wiki a file when you edit it, then `mark-green`. Prefer `actionable_yellow` / 🔴 over raw yellow counts.
 - **Steady-state selective work:** only 🔴 and *actionable* 🟡 (never re-wiki 🟢; never re-wiki the whole stub set). **ACS v1.3:** use `actionable_low_conf_edges` + `reason_code_counts` / `agent_signal` (`skip`|`investigate`). Do **not** thrash on raw `low_conf_edges` (includes external/bare scores ~0.48). Prefer `map_coverage.complete` / `files_remaining_dirty` after budgeted `update-maps` — `success: true` ≠ map complete.
-- **Warm maps:** import cache prefers stdlib SQLite (`.wikifier_staging/import_cache.sqlite`); zero-dirty path uses mtime index + meta, not full pair payload.
+- **Warm maps:** import cache prefers stdlib SQLite (`.wikifier_staging/import_cache.sqlite`); zero-dirty path uses mtime index + meta, not full pair payload. Candidate lists reuse when scope fingerprint is unchanged; prefer `--directory=` / lean `monitored_paths` (not bare `.`).
+- **Cache ops:** `wikifier cache-status` / `from wikifier import cache_status` — backend, sizes, `acs_version`, `map_coverage`. Dual-write JSON only ≤400 files or `WIKIFIER_CACHE_JSON=1` (set `0` to never dual-write).
 - **Dual scope:** `monitored_paths` = check-changes/health scope (keep lean). `update-maps --directory=` / `--max-files=` = map walk. Never set `project_root` to a multi-repo parent (e.g. `cloned_sample_projects`).
 - **CLI target:** `wikifier --target /abs/project health --summary` (flags stripped before the command). Or `WIKIFIER_PROJECT_ROOT=…`.
 - **Long-horizon:** `wikifier autonomous-status` (or `readiness`) before unattended runs; daemon writes `.wikifier_staging/daemon_heartbeat.json`. Soak ≥72h is still evidence work, not automatic.

wikifier-4.6.5/tests/test_walk_coverage_resolvers.py

@@ -0,0 +1,203 @@
+"""Walk cost, Core map_coverage, thin C#/C++ resolve, cache-status (shipped APIs)."""
+
+from __future__ import annotations
+
+import os
+import time
+import unittest
+from pathlib import Path
+
+from tests._base import TempProjectTestCase
+
+from wikifier import cli
+from wikifier import cache_store as cs
+from wikifier.candidates import (
+    collect_candidate_source_files,
+    try_cached_candidate_rels,
+    candidate_list_meta,
+)
+from wikifier.agent_loop import build_structured_actions
+
+
+class TestScopedCollect(TempProjectTestCase):
+    def test_directory_scope_limits_candidates(self):
+        self.write("pkg/a.py", "x=1\n")
+        self.write("other/b.py", "y=1\n")
+        cands = collect_candidate_source_files(self.root, directory="pkg")
+        rels = []
+        for p in cands:
+            try:
+                rels.append(str(Path(p).resolve().relative_to(self.root.resolve())))
+            except Exception:
+                rels.append(os.path.realpath(str(p)))
+        self.assertTrue(any("pkg" in r and r.endswith("a.py") for r in rels), rels)
+        self.assertFalse(any("other" in r for r in rels), rels)
+
+    def test_wiki_monitored_paths_do_not_collapse_map(self):
+        """monitored_paths with only wiki files must not yield a 1-file map."""
+        self.write("pkg/a.py", "x=1\n")
+        self.write("pkg/b.py", "y=1\n")
+        self.write("skills/run.md", "# wiki\n")
+        self.write(
+            "monitored_paths.txt",
+            "# wiki only\nskills/run.md\nREADME.md\npkg/a.py\n",
+        )
+        self.write("README.md", "# r\n")
+        cands = collect_candidate_source_files(self.root, directory=None, use_monitored=True)
+        # Must walk full tree, not just a.py from monitored
+        py = [p for p in cands if p.suffix == ".py"]
+        self.assertGreaterEqual(len(py), 2, [str(p) for p in py])
+
+    def test_candidate_reuse_true_when_git_clean(self):
+        """Second warm reuses candidate list when scope clean — honest assert."""
+        self.write("m.py", "print(1)\n")
+        r1 = cli.update_maps(project_root=self.root, full=True)
+        self.assertTrue(r1.get("success"), r1)
+        n1 = int(r1.get("parseable_files") or 0)
+        self.assertGreaterEqual(n1, 1)
+        r2 = cli.update_maps(project_root=self.root, full=False)
+        self.assertTrue(r2.get("success"), r2)
+        r3 = cli.update_maps(project_root=self.root, full=False)
+        self.assertTrue(r3.get("success"), r3)
+        self.assertTrue(
+            r3.get("candidates_reused") is True,
+            f"expected reuse on third warm; r2={r2.get('candidates_reused')} r3={r3}",
+        )
+        self.assertEqual(int(r3.get("parseable_files") or 0), n1)
+
+    def test_nested_new_file_invalidates_reuse(self):
+        """Adding a nested source file must expand candidates (not stale reuse)."""
+        self.write("pkg/a.py", "x=1\n")
+        r1 = cli.update_maps(project_root=self.root, full=True)
+        self.assertTrue(r1.get("success"), r1)
+        n1 = int(r1.get("parseable_files") or 0)
+        # Nested new file under existing package
+        self.write("pkg/nested/b.py", "y=2\n")
+        time.sleep(0.05)
+        r2 = cli.update_maps(project_root=self.root, full=False)
+        self.assertTrue(r2.get("success"), r2)
+        n2 = int(r2.get("parseable_files") or 0)
+        self.assertGreater(
+            n2, n1,
+            f"nested add must expand parseable_files: before={n1} after={n2} reused={r2.get('candidates_reused')}",
+        )
+        # Must not have reused stale list of size n1
+        if r2.get("candidates_reused"):
+            self.fail("must not reuse candidate list after nested source add")
+
+    def test_poisoned_one_file_blob_not_reused(self):
+        """Plant 1-file candidate blob with matching fp; live tree has 2 sources → no reuse."""
+        self.write("pkg/a.py", "x=1\n")
+        self.write("pkg/b.py", "y=2\n")
+        r1 = cli.update_maps(project_root=self.root, full=True)
+        self.assertTrue(r1.get("success"), r1)
+        live_n = int(r1.get("parseable_files") or 0)
+        self.assertGreaterEqual(live_n, 2)
+        # Poison meta: same directory scope, current fp, but only one rel
+        from wikifier.candidates import scope_fingerprint, candidate_list_meta
+        from wikifier import cache_store as cs
+        fp = scope_fingerprint(self.root, None)
+        poison = {
+            "fp": fp,
+            "directory": None,
+            "count": 1,
+            "rels": ["pkg/a.py"],
+        }
+        cs.save_meta_key(self.root, "_candidate_list", poison)
+        # Direct API: try_cached must return None
+        meta = cs.load_meta(self.root, keys=("_candidate_list",))
+        from wikifier.candidates import try_cached_candidate_rels
+        cached = try_cached_candidate_rels(meta, self.root, None)
+        self.assertIsNone(
+            cached,
+            f"poisoned 1-file blob must not reuse when live has {live_n} sources",
+        )
+        # Shipped update_maps must not report parseable_files=1 with reused=true
+        r2 = cli.update_maps(project_root=self.root, full=False)
+        self.assertTrue(r2.get("success"), r2)
+        self.assertGreaterEqual(int(r2.get("parseable_files") or 0), 2, r2)
+        self.assertFalse(
+            r2.get("candidates_reused") is True and int(r2.get("parseable_files") or 0) == 1,
+            r2,
+        )
+
+
+class TestCoverageInCore(TempProjectTestCase):
+    def test_suggest_surfaces_incomplete_map_action(self):
+        for i in range(5):
+            self.write(f"f{i}.py", f"v={i}\n")
+        r = cli.update_maps(project_root=self.root, full=True, max_files=2)
+        self.assertTrue(r.get("success"), r)
+        cov = r.get("map_coverage") or {}
+        self.assertFalse(cov.get("complete"), cov)
+        sug = cli.suggest_next_actions(project_root=self.root, format="json")
+        self.assertTrue(sug.get("success"), sug)
+        self.assertIn("map_coverage", sug)
+        self.assertFalse((sug.get("map_coverage") or {}).get("complete"))
+        actions = sug.get("actions") or []
+        kinds = [a.get("action") for a in actions]
+        self.assertIn("update_maps_until_complete", kinds, actions)
+        text = " ".join(sug.get("suggestions") or [])
+        self.assertIn("INCOMPLETE", text.upper())
+
+    def test_build_structured_actions_incomplete(self):
+        acts = build_structured_actions(
+            clean=True,
+            map_coverage={"complete": False, "files_remaining_dirty": 9},
+        )
+        self.assertTrue(any(a["action"] == "update_maps_until_complete" for a in acts))
+
+
+class TestCacheStatus(TempProjectTestCase):
+    def test_cache_status_shape(self):
+        self.write("z.py", "z=1\n")
+        cli.update_maps(project_root=self.root, full=True)
+        st = cli.cache_status(project_root=self.root)
+        self.assertTrue(st.get("success"), st)
+        self.assertIn(st.get("cache_backend"), ("sqlite", "sqlite+json", "json"))
+        self.assertIn("sqlite_bytes", st)
+        self.assertIn("dual_write_policy", st)
+        self.assertIn("migrate_note", st)
+        st2 = cs.cache_status(self.root)
+        self.assertEqual(st2.get("cache_backend"), st.get("cache_backend"))
+
+
+class TestCsharpProjectResolve(TempProjectTestCase):
+    def test_project_namespace_resolves(self):
+        from wikifier.parsers import csharp as cs_parser
+
+        self.write(
+            "Demo.csproj",
+            "<Project><PropertyGroup><RootNamespace>Demo</RootNamespace></PropertyGroup></Project>\n",
+        )
+        self.write("Util/Helper.cs", "namespace Demo.Util { class Helper {} }\n")
+        main = self.write(
+            "Program.cs",
+            "using Demo.Util;\nnamespace Demo { class Program {} }\n",
+        )
+        edges = cs_parser.parse_csharp_imports(str(main))
+        self.assertTrue(edges)
+        resolved = [e for e in edges if e.get("resolved_path")]
+        self.assertTrue(resolved, msg=f"edges={edges}")
+        self.assertTrue(
+            any("Util" in (e.get("resolved_path") or "") for e in resolved),
+            resolved,
+        )
+
+
+class TestCCppLocalInclude(TempProjectTestCase):
+    def test_include_dir_resolve(self):
+        from wikifier.parsers import c_cpp as cc
+
+        self.write("include/foo.h", "#pragma once\n")
+        src = self.write("src/main.c", '#include "foo.h"\nint main(){return 0;}\n')
+        edges = cc.parse_c_cpp_imports(str(src))
+        self.assertTrue(edges)
+        self.assertTrue(
+            any(e.get("resolved_path") and "foo.h" in e["resolved_path"] for e in edges),
+            edges,
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

{wikifier-4.6.4 → wikifier-4.6.5}/wikifier/__init__.py

@@ -48,6 +48,7 @@ from .cli import (
     why_file,
     seed_source_content_hashes,
     list_core_tools,
+    cache_status,
 )
 from . import agent_loop
 
@@ -78,4 +79,4 @@ from .contracts import (
     compute_acs_confidence,
 )
 
-__version__ = "4.6.4"
+__version__ = "4.6.5"

{wikifier-4.6.4 → wikifier-4.6.5}/wikifier/agent_loop.py

@@ -176,6 +176,7 @@ def build_structured_actions(
     scope_warnings: Optional[List[str]] = None,
     clean: bool = False,
     max_file_actions: int = 12,
+    map_coverage: Optional[Dict[str, Any]] = None,
 ) -> List[Dict[str, Any]]:
     """Build dispatchable action objects for agents (not prose-only).
 
@@ -183,6 +184,28 @@ def build_structured_actions(
     """
     actions: List[Dict[str, Any]] = []
     prio = 1
+    cov = map_coverage if isinstance(map_coverage, dict) else {}
+    remaining = int(cov.get("files_remaining_dirty") or 0)
+    complete = cov.get("complete")
+    # Incomplete map under budget — higher priority than ACS thrash
+    if complete is False or remaining > 0:
+        actions.append({
+            "action": "update_maps_until_complete",
+            "file": None,
+            "priority": 3,
+            "reason": (
+                f"Map incomplete: files_remaining_dirty={remaining}, complete={complete}. "
+                "Re-run update_maps (same directory/max_files) until map_coverage.complete=true; "
+                "success alone does not mean the map is done."
+            ),
+            "preflight": ["update_maps"],
+            "map_coverage": {
+                "complete": complete,
+                "files_remaining_dirty": remaining,
+                "files_skipped": cov.get("files_skipped"),
+                "budget_max_files": cov.get("budget_max_files"),
+            },
+        })
     for f in (red_files or [])[:max_file_actions]:
         actions.append({
             "action": "investigate_red",
@@ -201,7 +224,7 @@ def build_structured_actions(
             "preflight": ["prepare_edit", "why_file"],
         })
         prio = min(prio + 1, 9)
-    if red == 0 and actionable_yellow == 0 and clean:
+    if red == 0 and actionable_yellow == 0 and clean and remaining == 0:
         actions.append({
             "action": "lookup_only",
             "file": None,
@@ -222,7 +245,10 @@ def build_structured_actions(
             "action": "review_acs",
             "file": None,
             "priority": 4,
-            "reason": f"{acs_actionable} actionable low-confidence project edges (not external noise)",
+            "reason": (
+                f"{acs_actionable} actionable low-confidence project edges "
+                "(prefer actionable_low_conf_edges + reason_code_counts; not raw low_conf_edges)"
+            ),
             "preflight": ["get_dependencies"],
         })
     for w in (scope_warnings or [])[:2]:
@@ -233,7 +259,10 @@ def build_structured_actions(
             "reason": str(w),
             "preflight": [],
         })
-    if not any(a["action"] == "update_maps" for a in actions):
+    if not any(
+        a["action"] in ("update_maps", "update_maps_until_complete", "update_maps_if_structure")
+        for a in actions
+    ):
         actions.append({
             "action": "update_maps_if_structure",
             "file": None,
@@ -609,6 +638,7 @@ def session_bootstrap(
     if isinstance(out.get("scope"), dict):
         scope_warnings = list(out["scope"].get("warnings") or [])
 
+    map_cov = out.get("map_coverage") if isinstance(out.get("map_coverage"), dict) else {}
     out["actions"] = build_structured_actions(
         red_files=red_files,
         actionable_yellow_files=actionable_yellow_files,
@@ -618,12 +648,19 @@ def session_bootstrap(
         acs_actionable=acs_actionable,
         scope_warnings=scope_warnings,
         clean=clean,
+        map_coverage=map_cov,
     )
     out["work_items"] = out["actions"]  # alias
     out["selective_work"] = True
     out["map_first"] = True
+    rem = int(map_cov.get("files_remaining_dirty") or 0) if map_cov else 0
     out["message"] = (
         "session_bootstrap ready — use actions[] / attention; "
         "prepare_edit(file) before large edits; record_change after edits."
+        + (
+            f" MAP INCOMPLETE: files_remaining_dirty={rem} — re-run update_maps until complete."
+            if rem > 0 or map_cov.get("complete") is False
+            else ""
+        )
     )
     return out

{wikifier-4.6.4 → wikifier-4.6.5}/wikifier/cache_store.py

@@ -346,3 +346,40 @@ def get_map_coverage_from_meta(root: Path) -> Dict[str, Any]:
         "actionable_low_conf_edges": acs.get("actionable_low_conf_edges"),
         "reason_code_counts": acs.get("reason_code_counts"),
     }
+
+
+def cache_status(root: Path) -> Dict[str, Any]:
+    """Machine-readable dual-cache status for agents/ops (zero full pair load)."""
+    root = Path(root)
+    sp = sqlite_path(root)
+    jp = json_path(root)
+    backend = backend_name(root)
+    sqlite_bytes = int(sp.stat().st_size) if sp.is_file() else 0
+    json_bytes = int(jp.stat().st_size) if jp.is_file() else 0
+    snap = get_map_coverage_from_meta(root)
+    meta = load_meta(root, keys=("_acs_summary", "_map_coverage", "_candidate_list"))
+    acs = meta.get("_acs_summary") if isinstance(meta.get("_acs_summary"), dict) else {}
+    cand = meta.get("_candidate_list") if isinstance(meta.get("_candidate_list"), dict) else {}
+    return {
+        "success": True,
+        "project_root": str(root.resolve()) if root.exists() else str(root),
+        "cache_backend": backend,
+        "sqlite_path": str(sp) if sp.is_file() else None,
+        "json_path": str(jp) if jp.is_file() else None,
+        "sqlite_bytes": sqlite_bytes,
+        "json_bytes": json_bytes,
+        "acs_version": acs.get("acs_version") or snap.get("acs_version"),
+        "actionable_low_conf_edges": acs.get("actionable_low_conf_edges"),
+        "map_coverage": snap.get("map_coverage") or meta.get("_map_coverage"),
+        "candidate_list_count": cand.get("count"),
+        "candidate_list_directory": cand.get("directory"),
+        "dual_write_policy": (
+            "SQLite is primary. Legacy JSON dual-read remains. "
+            "Dual-write JSON only when file count ≤400 or WIKIFIER_CACHE_JSON=1; "
+            "set WIKIFIER_CACHE_JSON=0 to never dual-write. Prefer sqlite for warm agents."
+        ),
+        "migrate_note": (
+            "update_maps migrates legacy import_cache.json → import_cache.sqlite once "
+            "when sqlite is missing."
+        ),
+    }