wikifier 4.5.6__tar.gz → 4.5.8__tar.gz

{wikifier-4.5.6/wikifier.egg-info → wikifier-4.5.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: wikifier
-Version: 4.5.6
+Version: 4.5.8
 Summary: Zero-dependency agent-to-agent codebase wiki for LLMs and AI agents. Autonomously maintained via record-change and mark-green for token-efficient lookup of files, dependencies, health, and summaries — across tiny scripts to 50k+ monorepos. Optional MCP server with rich tools for agents.
 Author-email: Aron Amos <aron@example.com>
 Maintainer: Aron Amos

{wikifier-4.5.6 → wikifier-4.5.8}/skills/run.md

@@ -10,12 +10,15 @@
 **Package 4.5.x + gap-closure notes (additive; protocol still v0.6):**
 - File Tree + `wikifier serve`; MCP status/attention use library + emoji (not `[GREEN]` tags).
 - **First-run:** `init` → `update-maps` → `health --summary` → `suggest_next_actions`. Map is automatic; wiki *prose* is agent-filled over time.
-- **Steady-state selective work:** only 🔴/🟡 (never re-wiki 🟢 unless required). ACS suggestions use `actionable_low_conf_edges` (stdlib/external bare demoted).
-- **Deletions:** `record-deletion` + check-changes ghost detection (missing paths → Red DELETED).
-- **Languages:** deep import maps = Python, JS/TS, Rust, Go, C/C++, C#, Java (regex; not full toolchain resolution). journal/pending = audit queue, not Jira.
-- **Huge monorepos:** lean `monitored_paths.txt` (package roots, not bare `.`). Optional `WIKIFIER_CHECK_CHANGES_MAX` (default 2000). `update-maps` seeds Yellow health stubs for newly parsed files (map-first).
-- **Monitor:** optional background `monitor`/`daemon` for mtime heartbeat; agents still own wiki/mark-green.
-- Health module: use `importlib.import_module("wikifier.health")` or `wikifier.health_module` — `from wikifier import health` is the *function*.
+- **Map-first ≠ wiki-done:** `health_score` **Map Ready** + many 🟡 *Initial stubs* means structural coverage only. Do **not** bulk-wiki stubs. Wiki a file when you edit it, then `mark-green`. Prefer `actionable_yellow` / 🔴 over raw yellow counts.
+- **Steady-state selective work:** only 🔴 and *actionable* 🟡 (never re-wiki 🟢; never re-wiki the whole stub set). ACS uses `actionable_low_conf_edges`.
+- **Dual scope:** `monitored_paths` = check-changes/health scope (keep lean). `update-maps --directory=` / `--max-files=` = map walk. Never set `project_root` to a multi-repo parent (e.g. `cloned_sample_projects`).
+- **CLI target:** `wikifier --target /abs/project health --summary` (flags stripped before the command). Or `WIKIFIER_PROJECT_ROOT=…`.
+- **Long-horizon:** `wikifier autonomous-status` (or `readiness`) before unattended runs; daemon writes `.wikifier_staging/daemon_heartbeat.json`. Soak ≥72h is still evidence work, not automatic.
+- **Deletions:** `record-deletion` + ghost detection. **Languages:** py/js/ts/rust/go/c/c#/java (regex). journal/pending = audit queue, not Jira.
+- **Huge monorepos:** lean `monitored_paths.txt` (not bare `.`). `WIKIFIER_CHECK_CHANGES_MAX` default 2000. Hygiene: `seed-health`, `prune-pending`, `prune-health`.
+- **Monitor/daemon:** mtime heartbeat; agents still own wiki/mark-green. `WIKIFIER_DAEMON_MAPS_INTERVAL` default 600s; `WIKIFIER_DAEMON_MAPS=0` for check-only.
+- Health module: `importlib.import_module("wikifier.health")` or `wikifier.health_module` — `from wikifier import health` is the *function*.
 
 **v0.6 migration notes (packages v4.2.0–v4.3.0)** — changes are additive or strictly-better defaults; v0.5 agent behavior keeps working:
 - `update-maps` (CLI) now runs the **pure-Python full pipeline**: every dirty file is parsed in-process, the canonical per-file cache is persisted, reverse deps + cycles + ACS are computed, and `library.md` is regenerated atomically. `--python-primary` is accepted but redundant; the in-shell first-pass was retired entirely (wikifier.sh's update-maps delegates to this pipeline; `--sh`/`--legacy-sh` are deprecated no-ops). Scoping is explicit via `--directory=`/`--max-files=` and reported in the result (`files_skipped`) — there are no silent caps.
@@ -39,11 +42,15 @@ Do **not** open megamodules (`javascript.py`, `import_cache.py`, `bree.py`) to d
 | Need | Use |
 |------|-----|
 | First-run map | `init` → `update-maps` → `health --summary` → `suggest-next` |
-| Steady-state | `check-changes` → edit 🔴/🟡 only → `record-change` → wiki → `mark-green` |
+| Steady-state | `check-changes` → edit 🔴/actionable 🟡 only → `record-change` → wiki → `mark-green` |
 | Lookup | MCP Core 6: status, check_changes, needing_attention, get_file_wiki, suggest, record/mark |
 | Deps / cycles | `get_dependencies`, `get_dependents`, `get_cycles` (json) |
 | Deletion | `record-deletion` |
-| Code entry | `wikifier/cli.py` (dispatch) + `wikifier/health.py` + parsers — self-tests live in `tests/` + `tests/selftest/` |
+| Long-horizon prep | `autonomous-status` / `readiness`; then `daemon start` with lean monitor |
+| Hygiene | `seed-health`, `prune-pending`, `prune-health`, `validate` |
+| Code entry | `cli.py` dispatch + `health.py` + parsers — self-tests in `tests/` + `tests/selftest/` |
+
+**Do not open megamodules** (`javascript.py` ~2.6k, `import_cache.py` ~2.3k, `bree.py` ~2k) for workflow decisions — use tools + this table.
 
 **CLI pure-Python vs shell:** mutators and maps prefer `python -m wikifier …`. Shell still owns `init`, `monitor`, `daemon`, `serve`, `journal`.  
 **Scope:** `monitored_paths` = change detection; `exclude_patterns` + optional `--directory` = map walk.  

{wikifier-4.5.6 → wikifier-4.5.8}/tests/test_gap_closure.py

@@ -190,6 +190,84 @@ class TestPendingAndPollutionHygiene(TempProjectTestCase):
         self.assertIn("gone_real.py", entries2)
 
 
+class TestMapFirstTaxonomyAndReadiness(TempProjectTestCase):
+    def test_stub_yellow_is_map_ready_not_needs_attention(self):
+        health_mod = importlib.import_module("wikifier.health")
+        health_mod.upsert_entry(
+            self.root, "a.py", "🟡 Yellow",
+            "Initial stub — present in dependency map; agent should wiki + mark-green when editing",
+        )
+        health_mod.upsert_entry(
+            self.root, "b.py", "🟡 Yellow",
+            "Initial stub — parseable source under monitored_paths; agent should wiki + mark-green when editing",
+        )
+        s = health_mod.get_summary(self.root)
+        self.assertEqual(s.get("stub_yellow"), 2)
+        self.assertEqual(s.get("actionable_yellow"), 0)
+        self.assertEqual(s.get("health_score"), "Map Ready")
+
+    def test_actionable_yellow_needs_attention(self):
+        health_mod = importlib.import_module("wikifier.health")
+        health_mod.upsert_entry(
+            self.root, "a.py", "🟡 Yellow",
+            "mtime changed since last check-changes (Python primary auto-detected)",
+        )
+        s = health_mod.get_summary(self.root)
+        self.assertEqual(s.get("actionable_yellow"), 1)
+        self.assertEqual(s.get("health_score"), "Needs Attention")
+
+    def test_suggest_does_not_bulk_wiki_stubs(self):
+        health_mod = importlib.import_module("wikifier.health")
+        health_mod.upsert_entry(self.root, "a.py", "🟡 Yellow", "Initial stub — present in dependency map")
+        text = cli.suggest_next_actions(project_root=self.root, format="text")
+        self.assertIn("Map-first", text)
+        self.assertNotIn("Review the 1 🟡 Yellow file(s) only", text)
+
+    def test_detect_multi_project_container(self):
+        health_mod = importlib.import_module("wikifier.health")
+        for name in ("proj_a", "proj_b", "proj_c"):
+            d = self.root / name
+            d.mkdir()
+            (d / ".git").mkdir()
+        scope = health_mod.detect_scope_risks(self.root)
+        self.assertGreaterEqual(scope.get("child_project_count"), 3)
+        self.assertFalse(scope.get("ok"))
+        self.assertTrue(any("multi-project" in w.lower() for w in scope.get("warnings") or []))
+
+    def test_autonomous_readiness_shape(self):
+        health_mod = importlib.import_module("wikifier.health")
+        (self.root / "monitored_paths.txt").write_text("src\n", encoding="utf-8")
+        self.write("src/app.py", "x=1\n")
+        health_mod.upsert_entry(self.root, "src/app.py", "🟡 Yellow", "Initial stub — present in dependency map")
+        r = health_mod.assess_autonomous_readiness(self.root)
+        self.assertTrue(r.get("success"))
+        self.assertIn(r.get("readiness"), (
+            "blocked", "map_ok_scope_risk", "ready_for_daemon",
+            "ready_with_agent_wiki_work", "not_ready",
+        ))
+        self.assertIn("long_horizon_note", r)
+        self.assertIn("metrics", r)
+
+    def test_write_metrics_snapshot_history(self):
+        health_mod = importlib.import_module("wikifier.health")
+        (self.root / "monitored_paths.txt").write_text("src\n", encoding="utf-8")
+        self.write("src/app.py", "x=1\n")
+        health_mod.upsert_entry(self.root, "src/app.py", "🟢 Green", "ok")
+        s1 = health_mod.write_metrics_snapshot(self.root, source="test1")
+        self.assertTrue(s1.get("success"))
+        latest = self.root / ".wikifier_staging" / "metrics_latest.json"
+        hist = self.root / ".wikifier_staging" / "metrics_history.jsonl"
+        self.assertTrue(latest.exists())
+        self.assertTrue(hist.exists())
+        s2 = health_mod.write_metrics_snapshot(self.root, source="test2")
+        self.assertTrue(s2.get("success"))
+        rows = health_mod.read_metrics_history(self.root, limit=10)
+        self.assertGreaterEqual(len(rows), 2)
+        self.assertEqual(rows[-1].get("source"), "test2")
+        self.assertIn("staging_bytes", rows[-1])
+        self.assertIn("health_score", rows[-1])
+
+
 class TestMapFirstHealthAndValidate(TempProjectTestCase):
     def test_deep_relative_keys_are_under_root(self):
         """Regression: deep monorepo rel paths must not be treated as abs pollution."""

{wikifier-4.5.6 → wikifier-4.5.8}/wikifier/__init__.py

@@ -71,4 +71,4 @@ from .contracts import (
     compute_acs_confidence,
 )
 
-__version__ = "4.5.6"
+__version__ = "4.5.8"

{wikifier-4.5.6 → wikifier-4.5.8}/wikifier/cli.py

@@ -728,18 +728,20 @@ def main():
     i = 0
     while i < len(argv):
         arg = argv[i]
+        # Consume --target / --project-root so the *command* remains filtered_argv[0]
+        # (previously left --target as argv[0] → "Unknown command: --target").
         if arg in ("--target", "--project-root") and i + 1 < len(argv):
             project_root = argv[i + 1]
-            filtered_argv.append(arg)
-            filtered_argv.append(argv[i + 1])
             i += 2
             continue
         elif arg.startswith("--target="):
             project_root = arg.split("=", 1)[1]
-            filtered_argv.append(arg)
+            i += 1
+            continue
         elif arg.startswith("--project-root="):
             project_root = arg.split("=", 1)[1]
-            filtered_argv.append(arg)
+            i += 1
+            continue
         elif arg in ("--use-canonical", "--use_canonical"):
             use_canonical = True
             filtered_argv.append(arg)
@@ -895,6 +897,30 @@ def main():
                 res = _health_mod.prune_health_outside_monitored(root)
                 print(_json.dumps(res, indent=2, default=str))
                 return 0 if res.get("success") else 1
+            if _cmd0 in (
+                "autonomous-status",
+                "autonomous_status",
+                "readiness",
+                "long-horizon",
+            ):
+                import json as _json
+                if _health_mod is None or not hasattr(_health_mod, "assess_autonomous_readiness"):
+                    print("[wikifier] health.assess_autonomous_readiness unavailable", file=sys.stderr)
+                    return 1
+                root = _get_effective_root(project_root)
+                res = _health_mod.assess_autonomous_readiness(root)
+                print(_json.dumps(res, indent=2, default=str))
+                # exit 0 only when not blocked
+                return 0 if res.get("readiness") != "blocked" else 2
+            if _cmd0 in ("metrics-snapshot", "metrics_snapshot", "metrics"):
+                import json as _json
+                if _health_mod is None or not hasattr(_health_mod, "write_metrics_snapshot"):
+                    print("[wikifier] write_metrics_snapshot unavailable", file=sys.stderr)
+                    return 1
+                root = _get_effective_root(project_root)
+                res = _health_mod.write_metrics_snapshot(root, source="cli")
+                print(_json.dumps(res, indent=2, default=str))
+                return 0 if res.get("success") else 1
         except Exception as e:
             print(f"[wikifier] Python-primary {_cmd0} failed: {e}", file=sys.stderr)
             return 1
@@ -1419,12 +1445,14 @@ def suggest_next_actions(
     """
     root = _get_effective_root(project_root)
     try:
-        red = yellow = 0
+        red = yellow = stub_y = action_y = 0
         health_sum: Dict[str, Any] = {}
         if _health_mod is not None:
             health_sum = _health_mod.get_summary(root, directory) or {}
-            red = health_sum.get("red", 0)
-            yellow = health_sum.get("yellow", 0)
+            red = int(health_sum.get("red", 0) or 0)
+            yellow = int(health_sum.get("yellow", 0) or 0)
+            stub_y = int(health_sum.get("stub_yellow", 0) or 0)
+            action_y = int(health_sum.get("actionable_yellow", yellow - stub_y) or 0)
 
         suggestions: List[str] = []
         n = 1
@@ -1434,10 +1462,23 @@ def suggest_next_actions(
                 "Do not re-wiki 🟢 Green files."
             )
             n += 1
-        if yellow > 0:
+        if action_y > 0:
+            suggestions.append(
+                f"{n}. Review {action_y} *actionable* 🟡 Yellow file(s) "
+                "(mtime/record-change/barrel — not Initial stubs). "
+                "record-change → wiki that file → mark-green. Skip green."
+            )
+            n += 1
+        if stub_y > 0 and action_y == 0 and red == 0:
+            suggestions.append(
+                f"{n}. Map-first OK: {stub_y} 🟡 Initial stubs mean \"on the map\", "
+                "NOT \"wiki this tree now\". Lookup via get_file_wiki/deps; "
+                "write prose only when you edit a file, then mark-green."
+            )
+            n += 1
+        elif stub_y > 0 and action_y > 0:
             suggestions.append(
-                f"{n}. Review the {yellow} 🟡 Yellow file(s) only — record-change intent, refresh wiki, mark-green. "
-                "Skip green unless a dependent of a yellow/red requires it."
+                f"{n}. Ignore {stub_y} map-first stubs for bulk work; only actionable yellows need wiki."
             )
             n += 1
         if red == 0 and yellow == 0:
@@ -1445,6 +1486,15 @@ def suggest_next_actions(
                 f"{n}. Health is clean (no red/yellow). Do not re-summarize the tree; use the map for lookup only."
             )
             n += 1
+        # Scope hygiene
+        if _health_mod is not None and hasattr(_health_mod, "detect_scope_risks"):
+            try:
+                scope = _health_mod.detect_scope_risks(root) or {}
+                for w in (scope.get("warnings") or [])[:2]:
+                    suggestions.append(f"{n}. SCOPE: {w}")
+                    n += 1
+            except Exception:
+                pass
         suggestions.append(
             f"{n}. Run `update_maps(directory=...)` only if imports/structure changed (not for wiki-only edits)."
         )
@@ -1453,7 +1503,10 @@ def suggest_next_actions(
             f"{n}. On yellow/red hotspots, query dependents (get_dependents) before editing callers."
         )
         n += 1
-        suggestions.append(f"{n}. Review journal/ for recent record-change intent (self-audit).")
+        suggestions.append(
+            f"{n}. Long-horizon: `wikifier autonomous-status` before unattended daemon; "
+            "lean monitored_paths; never parent multi-repo folders as project_root."
+        )
 
         acs_note = ""
         if _ic_mod is not None:
@@ -1488,10 +1541,14 @@ def suggest_next_actions(
                 "project_root": str(root),
                 "red": red,
                 "yellow": yellow,
+                "stub_yellow": stub_y,
+                "actionable_yellow": action_y,
+                "health_score": health_sum.get("health_score"),
                 "suggestions": suggestions,
                 "health_summary": health_sum,
                 "acs_note": acs_note,
                 "selective_work": True,
+                "map_first": True,
             }
         return "\n".join(suggestions) + (acs_note or "")
     except Exception as e:

{wikifier-4.5.6 → wikifier-4.5.8}/wikifier/daemon.py

@@ -166,6 +166,42 @@ def log(msg: str, also_to_file: bool = True) -> None:
 # Core check runner (used by both initial, periodic, and post-sleep paths)
 # =============================================================================
 
+def _write_heartbeat(event: str, ok: bool, detail: str = "") -> None:
+    """Persist last daemon success/failure for autonomous-status / long-horizon ops."""
+    try:
+        import json as _json
+        path = ensure_state_dir() / "daemon_heartbeat.json"
+        prev = {}
+        if path.exists():
+            try:
+                prev = _json.loads(path.read_text(encoding="utf-8")) or {}
+            except Exception:
+                prev = {}
+        fail_streak = int(prev.get("consecutive_failures", 0) or 0)
+        if ok:
+            fail_streak = 0
+        else:
+            fail_streak += 1
+        payload = {
+            "ts": datetime.now().isoformat(timespec="seconds"),
+            "event": event,
+            "ok": ok,
+            "detail": (detail or "")[:400],
+            "consecutive_failures": fail_streak,
+            "project_root": str(
+                (discover_project_root() if discover_project_root else Path.cwd()).resolve()
+            ),
+        }
+        path.write_text(_json.dumps(payload, indent=2) + "\n", encoding="utf-8")
+        if fail_streak >= 5:
+            log(
+                f"LONG-HORIZON WARN: {fail_streak} consecutive daemon step failures — "
+                "inspect daemon.log; consider lean monitored_paths / WIKIFIER_DAEMON_MAPS=0"
+            )
+    except Exception:
+        pass
+
+
 def _run_check_changes(reason: str) -> None:
     """
     Run the check-changes command in a robust way.
@@ -185,16 +221,18 @@ def _run_check_changes(reason: str) -> None:
             env=os.environ.copy(),
         )
         if result.returncode != 0:
-            log(
-                f"check-changes failed (reason={reason}): "
-                f"{(result.stderr or '')[:500]}"
-            )
+            msg = (result.stderr or "")[:500]
+            log(f"check-changes failed (reason={reason}): {msg}")
+            _write_heartbeat(f"check-changes:{reason}", False, msg)
             return
         log(f"check-changes completed (reason={reason})")
+        _write_heartbeat(f"check-changes:{reason}", True)
     except subprocess.TimeoutExpired:
         log(f"check-changes timed out (reason={reason})")
+        _write_heartbeat(f"check-changes:{reason}", False, "timeout")
     except Exception as e:
         log(f"check-changes error (reason={reason}): {e}")
+        _write_heartbeat(f"check-changes:{reason}", False, str(e))
 
 
 def _run_python_primary_update(reason: str, force_full: bool = False) -> None:
@@ -215,13 +253,20 @@ def _run_python_primary_update(reason: str, force_full: bool = False) -> None:
         files = res.get("files_to_reparse", 0)
         persisted = res.get("sample_persisted_pairs", 0)
         tied = res.get("barrel_creative_tied_in_pure_path", False)
+        ok = bool(res.get("success", True))
         log(
             f"python-primary update completed (reason={reason}): "
             f"files_to_reparse={files} persisted_pairs={persisted} barrel_creative_tied={tied} "
-            f"root={res.get('root')}"
+            f"root={res.get('root')} success={ok}"
+        )
+        _write_heartbeat(
+            f"update-maps:{reason}",
+            ok,
+            f"parsed={res.get('files_parsed')} edges={res.get('edges_persisted')}",
         )
     except Exception as e:
         log(f"python-primary update error (reason={reason}): {e}")
+        _write_heartbeat(f"update-maps:{reason}", False, str(e))
 
 
 # =============================================================================
@@ -246,29 +291,76 @@ def daemon_loop() -> None:
 
     last_check = time.time()
     last_maps = 0.0
+    last_metrics = 0.0
+    # Metrics cadence (seconds); default 1h for soak growth without spam
+    try:
+        metrics_iv = max(0, int(os.environ.get("WIKIFIER_DAEMON_METRICS_INTERVAL", "3600") or "3600"))
+    except ValueError:
+        metrics_iv = 3600
+
+    def _maybe_metrics(reason: str, force: bool = False) -> None:
+        nonlocal last_metrics
+        if metrics_iv <= 0 and not force:
+            return
+        now_m = time.time()
+        if not force and metrics_iv > 0 and (now_m - last_metrics) < metrics_iv:
+            return
+        try:
+            from . import health as health_mod
+            if hasattr(health_mod, "write_metrics_snapshot"):
+                root = None
+                if discover_project_root is not None:
+                    try:
+                        root = discover_project_root()
+                    except Exception:
+                        root = None
+                snap = health_mod.write_metrics_snapshot(
+                    root or Path.cwd(),
+                    source=f"daemon:{reason}",
+                )
+                log(
+                    f"metrics-snapshot (reason={reason}): "
+                    f"staging_bytes={snap.get('staging_bytes')} "
+                    f"score={snap.get('health_score')} ok={snap.get('success')}"
+                )
+                last_metrics = now_m
+        except Exception as me:
+            log(f"metrics-snapshot error (reason={reason}): {me}")
+
     _run_check_changes("initial start / resume")
     if maps_on:
         _run_python_primary_update("initial start / resume", force_full=False)
         last_maps = time.time()
+    _maybe_metrics("initial", force=True)
 
     while True:
-        time.sleep(POLL_INTERVAL_DEFAULT)
-
-        now = time.time()
-        gap = now - last_check
-
-        if gap > SLEEP_THRESHOLD_SECONDS:
-            log(f"Wake detected (gap of {int(gap)}s). Forced check-changes + optional maps.")
-            _run_check_changes("post-sleep resume")
-            if maps_on:
-                _run_python_primary_update("post-sleep resume", force_full=False)
+        try:
+            time.sleep(POLL_INTERVAL_DEFAULT)
+
+            now = time.time()
+            gap = now - last_check
+
+            if gap > SLEEP_THRESHOLD_SECONDS:
+                log(f"Wake detected (gap of {int(gap)}s). Forced check-changes + optional maps.")
+                _run_check_changes("post-sleep resume")
+                if maps_on:
+                    _run_python_primary_update("post-sleep resume", force_full=False)
+                    last_maps = time.time()
+                _maybe_metrics("post-sleep", force=True)
+
+            _run_check_changes("periodic")
+            if maps_on and maps_iv > 0 and (time.time() - last_maps) >= maps_iv:
+                _run_python_primary_update("periodic maps interval", force_full=False)
                 last_maps = time.time()
-
-        _run_check_changes("periodic")
-        if maps_on and maps_iv > 0 and (time.time() - last_maps) >= maps_iv:
-            _run_python_primary_update("periodic maps interval", force_full=False)
-            last_maps = time.time()
-        last_check = time.time()
+            _maybe_metrics("periodic")
+            last_check = time.time()
+        except KeyboardInterrupt:
+            raise
+        except Exception as loop_err:
+            # Never die on a single cycle — long-horizon autonomous ops requirement
+            log(f"daemon loop error (continuing): {loop_err}")
+            _write_heartbeat("loop", False, str(loop_err))
+            time.sleep(POLL_INTERVAL_DEFAULT)
 
 
 # =============================================================================

{wikifier-4.5.6 → wikifier-4.5.8}/wikifier/health.py

@@ -3,13 +3,15 @@ Documentation health matrix (agent-first).
 
 AGENT MAP:
   load_health / save_health / upsert_entry — file_health.json SSOT (+ regenerate .md)
-  get_summary / get_files_needing_attention — 🟢🟡🔴 counts and lists
+  get_summary — 🟢🟡🔴 + stub_yellow/actionable_yellow + health_score (Map Ready|…)
+  assess_autonomous_readiness / detect_scope_risks — long-horizon + dual-scope
   find_ghost_entries / validate_health — map-first gaps + ghosts (parseable sources only)
   seed_health_from_map — backfill 🟡 stubs from import_cache (warm-cache safe)
   prune_pending_to_monitored / prune_health_outside_monitored — lean-monitor hygiene
   mark_green / heal_stubs / apply_barrel_invalidation_reports — status mutations
-  CLI: seed-health | prune-pending | prune-health | validate | health --summary
+  CLI: autonomous-status | seed-health | prune-pending | prune-health | validate
 Agents: prefer MCP health / check_changes / mark_green; only open this for matrix policy.
+  Map Ready ≠ wiki-done — never bulk-wiki Initial stubs.
 
 JSON Schema (v2 - additive from v1):
 {
@@ -1149,6 +1151,359 @@ def validate_health(root: Path) -> Dict[str, Any]:
     }
 
 
+def detect_scope_risks(root: Path) -> Dict[str, Any]:
+    """Dual-scope + multi-project root misuse detector (agent hygiene)."""
+    root = _coerce_root(root)
+    warnings: List[str] = []
+    monitored = _read_monitored_rel_roots(root)
+    bare_dot = monitored == ["."] or monitored == []
+    if bare_dot:
+        warnings.append(
+            "monitored_paths is bare '.' — check-changes will thrash on large trees. "
+            "Prefer lean package roots (e.g. src/, library/std/src)."
+        )
+
+    # Parent multi-project container heuristic
+    child_projects = 0
+    sample_children: List[str] = []
+    try:
+        for p in root.iterdir():
+            if not p.is_dir() or p.name.startswith("."):
+                continue
+            if (p / ".git").exists() or (p / "file_health.json").exists() or (
+                p / ".wikifier_staging" / "import_cache.json"
+            ).exists():
+                child_projects += 1
+                if len(sample_children) < 8:
+                    sample_children.append(p.name)
+    except Exception:
+        pass
+    if child_projects >= 3:
+        warnings.append(
+            f"project_root looks like a multi-project container ({child_projects} child "
+            f"projects e.g. {sample_children}). Always target a *child* project path, "
+            "never the parent folder (e.g. cloned_sample_projects)."
+        )
+    name = root.name.lower()
+    if name in ("cloned_sample_projects", "coding_projects", "repos", "samples"):
+        warnings.append(
+            f"Directory name '{root.name}' often holds many repos — confirm project_root "
+            "is a single project, not the container."
+        )
+
+    return {
+        "monitored_roots": monitored,
+        "bare_dot_monitor": bare_dot,
+        "child_project_count": child_projects,
+        "child_project_sample": sample_children,
+        "warnings": warnings,
+        "ok": len(warnings) == 0,
+    }
+
+
+def _staging_byte_total(staging: Path, cache_path: Path) -> int:
+    if not staging.exists():
+        return 0
+    total = 0
+    try:
+        for p in staging.rglob("*"):
+            if p.is_file():
+                try:
+                    total += p.stat().st_size
+                except Exception:
+                    pass
+    except Exception:
+        try:
+            return cache_path.stat().st_size if cache_path.exists() else 0
+        except Exception:
+            return 0
+    return total
+
+
+def write_metrics_snapshot(
+    root: Path,
+    source: str = "manual",
+    extra: Optional[Dict[str, Any]] = None,
+) -> Dict[str, Any]:
+    """Append a long-horizon metrics sample under .wikifier_staging/.
+
+    Files:
+      - metrics_latest.json  (last sample, overwrite)
+      - metrics_history.jsonl (append-only; pruned to last 500 lines)
+
+    Call from CLI, readiness assessment, or daemon periodic ticks.
+    Does not claim multi-day soak; enables measurable growth over calendar time.
+    """
+    root = _coerce_root(root)
+    staging = root / ".wikifier_staging"
+    staging.mkdir(parents=True, exist_ok=True)
+    cache_path = staging / "import_cache.json"
+    summary = get_summary(root)
+    scope = detect_scope_risks(root)
+    ghosts = find_ghost_entries(root)
+    cache_bytes = cache_path.stat().st_size if cache_path.exists() else 0
+    staging_bytes = _staging_byte_total(staging, cache_path)
+    journal_files = 0
+    journal_root = root / "journal"
+    if journal_root.exists():
+        journal_files = sum(1 for _ in journal_root.rglob("*.md"))
+
+    heartbeat = None
+    hb_path = staging / "daemon_heartbeat.json"
+    if hb_path.exists():
+        try:
+            heartbeat = json.loads(hb_path.read_text(encoding="utf-8"))
+        except Exception:
+            heartbeat = {"error": "unreadable"}
+
+    snap: Dict[str, Any] = {
+        "ts": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
+        "source": source,
+        "root": str(root),
+        "health_score": summary.get("health_score"),
+        "total": summary.get("total"),
+        "green": summary.get("green"),
+        "yellow": summary.get("yellow"),
+        "red": summary.get("red"),
+        "stub_yellow": summary.get("stub_yellow"),
+        "actionable_yellow": summary.get("actionable_yellow"),
+        "pending_updates": summary.get("pending_updates"),
+        "ghost_count": len(ghosts),
+        "cache_bytes": cache_bytes,
+        "staging_bytes": staging_bytes,
+        "journal_files": journal_files,
+        "bare_dot_monitor": scope.get("bare_dot_monitor"),
+        "scope_ok": scope.get("ok"),
+        "daemon_heartbeat_ok": (heartbeat or {}).get("ok") if isinstance(heartbeat, dict) else None,
+        "daemon_fail_streak": (heartbeat or {}).get("consecutive_failures")
+        if isinstance(heartbeat, dict)
+        else None,
+    }
+    if extra:
+        snap["extra"] = extra
+
+    latest_path = staging / "metrics_latest.json"
+    hist_path = staging / "metrics_history.jsonl"
+    try:
+        latest_path.write_text(json.dumps(snap, indent=2) + "\n", encoding="utf-8")
+        with open(hist_path, "a", encoding="utf-8") as f:
+            f.write(json.dumps(snap, ensure_ascii=False) + "\n")
+        # Bound history (last 500 samples)
+        try:
+            lines = hist_path.read_text(encoding="utf-8").splitlines()
+            if len(lines) > 500:
+                hist_path.write_text("\n".join(lines[-500:]) + "\n", encoding="utf-8")
+        except Exception:
+            pass
+        snap["success"] = True
+        snap["metrics_latest_path"] = str(latest_path)
+        snap["metrics_history_path"] = str(hist_path)
+    except Exception as e:
+        snap["success"] = False
+        snap["error"] = str(e)
+    return snap
+
+
+def read_metrics_history(root: Path, limit: int = 20) -> List[Dict[str, Any]]:
+    """Return up to `limit` most recent metrics samples (newest last)."""
+    root = _coerce_root(root)
+    hist_path = root / ".wikifier_staging" / "metrics_history.jsonl"
+    if not hist_path.exists():
+        return []
+    out: List[Dict[str, Any]] = []
+    try:
+        lines = hist_path.read_text(encoding="utf-8").splitlines()
+        for ln in lines[-max(1, min(limit, 500)) :]:
+            ln = ln.strip()
+            if not ln:
+                continue
+            try:
+                out.append(json.loads(ln))
+            except Exception:
+                continue
+    except Exception:
+        return []
+    return out
+
+
+def assess_autonomous_readiness(
+    root: Path,
+    write_metrics: bool = True,
+) -> Dict[str, Any]:
+    """Long-horizon autonomous ops checklist (map + daemon + scope + state).
+
+    Does not replace multi-day soak evidence; surfaces *whether* a target is
+    configured safely enough for unattended check-changes/daemon loops.
+    Optionally appends a metrics snapshot for growth tracking over calendar time.
+    """
+    root = _coerce_root(root)
+    summary = get_summary(root)
+    scope = detect_scope_risks(root)
+    ghosts = find_ghost_entries(root)
+    pending_n = count_pending(root)
+
+    staging = root / ".wikifier_staging"
+    cache_path = staging / "import_cache.json"
+    lock_path = staging / ".lock"
+    pid_path = staging / "wikifier.pid"
+    heartbeat_path = staging / "daemon_heartbeat.json"
+    log_path = staging / "daemon.log"
+    metrics_latest_path = staging / "metrics_latest.json"
+
+    cache_bytes = cache_path.stat().st_size if cache_path.exists() else 0
+    staging_bytes = _staging_byte_total(staging, cache_path)
+
+    journal_root = root / "journal"
+    journal_days = 0
+    journal_files = 0
+    if journal_root.exists():
+        for p in journal_root.rglob("*.md"):
+            journal_files += 1
+        try:
+            journal_days = len({p.parent for p in journal_root.rglob("*.md")})
+        except Exception:
+            pass
+
+    daemon_running = False
+    daemon_pid = None
+    if pid_path.exists():
+        try:
+            daemon_pid = int(pid_path.read_text(encoding="utf-8").strip())
+            os.kill(daemon_pid, 0)
+            daemon_running = True
+        except Exception:
+            daemon_running = False
+
+    heartbeat = None
+    if heartbeat_path.exists():
+        try:
+            with open(heartbeat_path, "r", encoding="utf-8") as f:
+                heartbeat = json.load(f)
+        except Exception:
+            heartbeat = {"error": "unreadable"}
+
+    has_health = (root / "file_health.json").exists() or (root / "file_health.md").exists()
+    has_library = (root / "library.md").exists()
+    has_cache = cache_path.exists()
+
+    metrics_snap = None
+    if write_metrics:
+        try:
+            metrics_snap = write_metrics_snapshot(root, source="autonomous-status")
+        except Exception as e:
+            metrics_snap = {"success": False, "error": str(e)}
+
+    history = read_metrics_history(root, limit=5)
+    growth_note = None
+    if len(history) >= 2:
+        try:
+            a, b = history[0], history[-1]
+            da = int(b.get("staging_bytes") or 0) - int(a.get("staging_bytes") or 0)
+            growth_note = {
+                "samples": len(history),
+                "staging_bytes_delta": da,
+                "first_ts": a.get("ts"),
+                "last_ts": b.get("ts"),
+            }
+        except Exception:
+            growth_note = None
+
+    blockers: List[str] = []
+    recs: List[str] = []
+    if not has_cache and not has_library:
+        blockers.append("No import map (run update-maps first).")
+    if not has_health:
+        blockers.append("No file_health — run seed-health or update-maps.")
+    if scope["warnings"]:
+        recs.extend(scope["warnings"])
+    if len(ghosts) > 50:
+        blockers.append(f"Many ghost health keys ({len(ghosts)}); run record-deletion / prune.")
+    elif ghosts:
+        recs.append(f"{len(ghosts)} ghost health entries — clean with record-deletion.")
+    if pending_n > 500:
+        recs.append(
+            f"pending_updates={pending_n} is large; run prune-pending (auto-detected noise)."
+        )
+    if cache_bytes > 200 * 1024 * 1024:
+        recs.append(
+            f"import_cache is large ({cache_bytes // (1024*1024)}MB); "
+            "prefer scoped update-maps and lean monitors for long runs."
+        )
+    if staging_bytes > 500 * 1024 * 1024:
+        recs.append(f".wikifier_staging ~{staging_bytes // (1024*1024)}MB — watch disk growth.")
+    if not daemon_running:
+        recs.append(
+            "Daemon not running. For long-horizon: "
+            "`WIKIFIER_PROJECT_ROOT=… wikifier daemon start` "
+            "(maps interval default 600s; WIKIFIER_DAEMON_MAPS=0 for check-only)."
+        )
+    else:
+        recs.append(f"Daemon running (PID {daemon_pid}).")
+    recs.append(
+        "Metrics: write_metrics_snapshot / `wikifier metrics-snapshot`; "
+        "history at .wikifier_staging/metrics_history.jsonl for soak growth."
+    )
+
+    score = summary.get("health_score") or "Unknown"
+    # Readiness tier for unattended ops (not M5 95% soak claim)
+    if blockers:
+        readiness = "blocked"
+    elif scope["warnings"] and summary.get("actionable_yellow", 0) == 0 and summary.get("red", 0) == 0:
+        readiness = "map_ok_scope_risk"
+    elif score in ("Map Ready", "Good") and not scope["bare_dot_monitor"]:
+        readiness = "ready_for_daemon"
+    elif score == "Needs Attention" and summary.get("red", 0) == 0:
+        readiness = "ready_with_agent_wiki_work"
+    else:
+        readiness = "not_ready"
+
+    return {
+        "success": True,
+        "root": str(root),
+        "readiness": readiness,
+        "health_score": score,
+        "health_summary": summary,
+        "scope": scope,
+        "ghost_count": len(ghosts),
+        "pending_updates": pending_n,
+        "artifacts": {
+            "has_health": has_health,
+            "has_library": has_library,
+            "has_cache": has_cache,
+            "cache_bytes": cache_bytes,
+            "staging_bytes": staging_bytes,
+            "journal_files": journal_files,
+            "journal_day_dirs": journal_days,
+            "metrics_latest": str(metrics_latest_path) if metrics_latest_path.exists() else None,
+        },
+        "daemon": {
+            "running": daemon_running,
+            "pid": daemon_pid,
+            "heartbeat": heartbeat,
+            "log": str(log_path) if log_path.exists() else None,
+            "maps_env": {
+                "WIKIFIER_DAEMON_MAPS": os.environ.get("WIKIFIER_DAEMON_MAPS", "1"),
+                "WIKIFIER_DAEMON_MAPS_INTERVAL": os.environ.get(
+                    "WIKIFIER_DAEMON_MAPS_INTERVAL", "600"
+                ),
+            },
+        },
+        "metrics": metrics_snap,
+        "metrics_growth": growth_note,
+        "lock_path": str(lock_path),
+        "blockers": blockers,
+        "recommendations": recs,
+        "long_horizon_note": (
+            "ready_for_daemon = safe config for unattended check-changes/maps heartbeat. "
+            "It is NOT a multi-day 0-corruption soak proof (M5.3). "
+            "For soak: run daemon on 1–3 lean-monitored targets, append metrics-snapshot "
+            "periodically, watch staging_bytes delta in metrics_history.jsonl, "
+            "daemon_heartbeat consecutive_failures, and journal continuity over >=72h."
+        ),
+        "map_first_note": summary.get("map_first_note"),
+    }
+
+
 # ----------------------------- apply_barrel... (continues) -----------------------------
 
 def apply_barrel_invalidation_reports(
@@ -1203,33 +1558,84 @@ def apply_barrel_invalidation_reports(
     return updated
 
 
+def _is_map_first_stub_entry(entry: Dict[str, Any]) -> bool:
+    """True for map-first Initial stubs (not agent-action work items)."""
+    if not isinstance(entry, dict):
+        return False
+    reason = str(entry.get("reason") or "")
+    prov = str(entry.get("freshness_provenance") or "")
+    blob = (reason + " " + prov).lower()
+    if "initial stub" in blob:
+        return True
+    if "seed_health_from_map" in blob or "seed_health_for_monitored" in blob:
+        return True
+    if "present in dependency map" in blob:
+        return True
+    if "parseable source under monitored" in blob:
+        return True
+    return False
+
+
+def _compute_health_score(
+    red: int,
+    yellow: int,
+    stub_yellow: int,
+    actionable_yellow: int,
+) -> str:
+    """Agent-facing readiness label (map-first aware).
+
+    Map Ready  — no reds, only map-first stubs (or empty); do NOT treat as unfinished wiki.
+    Good       — clean green-heavy tree, few actionable yellows
+    Needs Attention — actionable yellows / pending work (not mere stubs)
+    Critical   — reds
+    """
+    if red >= 3:
+        return "Critical"
+    if red > 0:
+        return "Needs Attention"
+    if actionable_yellow > 0:
+        return "Needs Attention"
+    if yellow > 0 and stub_yellow >= yellow and actionable_yellow == 0:
+        return "Map Ready"
+    if yellow == 0:
+        return "Good"
+    if yellow < 5:
+        return "Good"
+    return "Needs Attention"
+
+
 def get_summary(root: Path, directory: Optional[str] = None, include_stale: bool = False) -> Dict[str, Any]:
     """Return a summary of the health matrix (fast even for large repos; sharded by dir).
 
-    If directory is provided, only counts files under that subdirectory.
-    This enables scalable views in massive monorepos (e.g. health per package).
-    M2 B: include_stale=True adds bounded stale_wiki count (via B3 detector) for
-    practical summary-only views without full materialization.
+    Map-first taxonomy (additive fields):
+      stub_yellow — Initial stub / map-seed rows (lookup OK; wiki on edit)
+      actionable_yellow — real work (mtime, record-change, barrel stale, …)
+      health_score — \"Map Ready\" | \"Good\" | \"Needs Attention\" | \"Critical\"
+
+    Agents must not treat stub_yellow as \"I understand this file.\"
     """
     health = load_health(root)
-    # Phase 5e (66): health.get_summary (format=summary path) promoted as first-class default/recommended for 20k+ creative monorepos (O(k) sharded, complements compute_acs_summary + ACS/CIABRE/BRC bounded via deque; per crit2/5 + 48/58).
     entries = health.get("entries", {})
 
     green = yellow = red = 0
+    stub_yellow = actionable_yellow = 0
     total = 0
 
     for file_path, entry in entries.items():
         if directory:
-            # Normalize paths for comparison
             if not file_path.startswith(directory.rstrip('/') + '/'):
                 continue
 
         total += 1
-        status = entry.get("status", "")
+        status = entry.get("status", "") if isinstance(entry, dict) else ""
         if "🟢" in status:
             green += 1
         elif "🟡" in status:
             yellow += 1
+            if _is_map_first_stub_entry(entry if isinstance(entry, dict) else {}):
+                stub_yellow += 1
+            else:
+                actionable_yellow += 1
         elif "🔴" in status:
             red += 1
 
@@ -1237,12 +1643,20 @@ def get_summary(root: Path, directory: Optional[str] = None, include_stale: bool
         pending_n = count_pending(root)
     except Exception:
         pending_n = 0
+    score = _compute_health_score(red, yellow, stub_yellow, actionable_yellow)
     out = {
         "total": total,
         "green": green,
         "yellow": yellow,
         "red": red,
+        "stub_yellow": stub_yellow,
+        "actionable_yellow": actionable_yellow,
         "pending_updates": pending_n,
+        "health_score": score,
+        "map_first_note": (
+            "Yellow Initial stubs = map coverage only; wiki prose is optional until you edit. "
+            "Prefer actionable_yellow / red for work. health_score Map Ready ≠ wiki-done."
+        ),
         "directory": directory or ".",
         "version": health.get("version", 2),
         "last_updated": health.get("last_updated")

{wikifier-4.5.6 → wikifier-4.5.8}/wikifier/mcp/server.py

@@ -1709,14 +1709,34 @@ def get_project_status(
                 yellow=summary["yellow"],
                 red=summary["red"],
                 pending_updates=pending_count,
-                health_score="Good" if summary["red"] == 0 and summary["yellow"] < 5 else "Needs Attention" if summary["red"] < 3 else "Critical"
+                health_score=str(
+                    summary.get("health_score")
+                    or (
+                        "Good"
+                        if summary["red"] == 0 and summary["yellow"] < 5
+                        else "Needs Attention"
+                        if summary["red"] < 3
+                        else "Critical"
+                    )
+                ),
             )
-            # attach dep intel (additive, agents can now use for ACS filtering without separate calls)
+            # attach dep intel + map-first taxonomy (additive)
             if isinstance(base, dict):
                 base["dependency_intel"] = dep_intel
+                for k in (
+                    "stub_yellow",
+                    "actionable_yellow",
+                    "map_first_note",
+                    "health_score",
+                ):
+                    if k in summary:
+                        base[k] = summary[k]
             else:
                 try:
                     base.dependency_intel = dep_intel  # type: ignore[attr-defined]
+                    for k in ("stub_yellow", "actionable_yellow", "map_first_note"):
+                        if k in summary:
+                            setattr(base, k, summary[k])
                 except Exception:
                     pass
             return base
@@ -1784,14 +1804,15 @@ Use get_files_needing_attention() for the actual list. Use get_cycles(analysis=T
                 if l.strip().startswith("- ") and not l.strip().startswith("- (")
             ])
         green = yellow = red = total = 0
+        summary_fb: dict = {}
         try:
             import importlib
             health_module = importlib.import_module("wikifier.health")
-            summary = health_module.get_summary(root, directory)
-            total = int(summary.get("total", 0) or 0)
-            green = int(summary.get("green", 0) or 0)
-            yellow = int(summary.get("yellow", 0) or 0)
-            red = int(summary.get("red", 0) or 0)
+            summary_fb = health_module.get_summary(root, directory) or {}
+            total = int(summary_fb.get("total", 0) or 0)
+            green = int(summary_fb.get("green", 0) or 0)
+            yellow = int(summary_fb.get("yellow", 0) or 0)
+            red = int(summary_fb.get("red", 0) or 0)
         except Exception:
             # Last resort: emoji-aware (+ legacy tag) counts from health text.
             try:
@@ -1804,13 +1825,22 @@ Use get_files_needing_attention() for the actual list. Use get_cycles(analysis=T
                 pass
 
         if format == "json":
+            _hs = summary_fb.get("health_score")
+            if not _hs:
+                _hs = (
+                    "Good"
+                    if red == 0 and yellow < 5
+                    else "Needs Attention"
+                    if red < 3
+                    else "Critical"
+                )
             return ProjectHealthSummary(
                 total_files=total or (green + yellow + red),
                 green=green,
                 yellow=yellow,
                 red=red,
                 pending_updates=pending_count,
-                health_score="Good" if red == 0 and yellow < 5 else "Needs Attention" if red < 3 else "Critical"
+                health_score=str(_hs)
             )
 
         dir_str = f" (in {directory})" if directory else ""

{wikifier-4.5.6 → wikifier-4.5.8/wikifier.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: wikifier
-Version: 4.5.6
+Version: 4.5.8
 Summary: Zero-dependency agent-to-agent codebase wiki for LLMs and AI agents. Autonomously maintained via record-change and mark-green for token-efficient lookup of files, dependencies, health, and summaries — across tiny scripts to 50k+ monorepos. Optional MCP server with rich tools for agents.
 Author-email: Aron Amos <aron@example.com>
 Maintainer: Aron Amos