codmap 0.0.21__tar.gz → 0.0.22__tar.gz

{codmap-0.0.21/codmap.egg-info → codmap-0.0.22}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codmap
-Version: 0.0.21
+Version: 0.0.22
 Summary: Static analyzer that turns a package's source into a queryable code graph.
 Author-email: kogriv <kogriv@gmail.com>
 License-Expression: MIT
@@ -44,7 +44,7 @@ and a **SCIP index** for interop with Sourcegraph / Glean and other precise-code
 
 [![CI](https://github.com/kogriv/codemap/actions/workflows/ci.yml/badge.svg)](https://github.com/kogriv/codemap/actions/workflows/ci.yml)
 
-**Status:** 🟢 M0–M20 implemented + research track (R1/R2) — schema 0.13, **990 tests with no failures on
+**Status:** 🟢 M0–M20 implemented + research track (R1/R2) — schema 0.13, **1054 tests with no failures on
 Python 3.11–3.14** ([in CI](docs/ci.md): the full suite including the dogfood pass, a determinism check, a
 wheel smoke test, and ctags/SCIP interop against the real CLIs), warm serve surface with 31 ops (28 exposed
 as MCP tools), and SCIP export. See **[DESIGN.md](DESIGN.md)** (product design &
@@ -221,8 +221,14 @@ codemap serve --graph graph.json --watch
 - **Dataflow** — producers/consumers of a string-keyed DataFrame column.
 - **Semantic search** (opt-in) — a concept query routed to an external adapter, with each fuzzy hit
   resolved to the exact codemap symbol at its location (`codemap semantic`). See [docs/integrations.md](docs/integrations.md).
-- **Context pack** — the most relevant slice of the graph under a token budget, ranked by importance or
-  by relevance to seed symbols (`codemap pack --budget N [--seed X]`). See [docs/pack.md](docs/pack.md).
+- **Context pack** — the highest-ranked modules, classes and functions under a token budget, by
+  importance or by relevance to seed symbols (`codemap pack --budget N [--seed X]`). With a seed it
+  walks both ways — what the seed uses and what uses it (`--direction uses` for one way) — documents are
+  never in it, the answer says which walk it took, and a seed that resolves to nothing is refused rather
+  than silently answered globally. Deterministic and budget-tested. Against git history, seeded with a
+  changed file, it puts 42 % and 60 % of the co-changed files in its first ten on two repositories —
+  ahead of `impact` and of grep by names, though on one of the two not beyond the noise; grep still
+  finds tests best. **Whether it helps an agent is not measured.** See [docs/pack.md](docs/pack.md).
 - **Interop** — export the graph as a [SCIP](https://scip-code.org/) index (definitions + symbol
   info + inherits/implements relationships) so Sourcegraph, Glean and other SCIP consumers can drive
   go-to-definition, symbol search and type hierarchy over it. See [docs/export.md](docs/export.md).
@@ -266,6 +272,13 @@ build says so in its own diagnostics, and `codemap diff` says so when both sides
 delta of a few call edges as possible tool noise before reading it as a change in the code. Anything
 that must be reproducible byte-for-byte — a gate, a two-release comparison — belongs on the fast tier.
 
+**A deep build does not check jedi.** With jedi answering nothing, a deep build still exits 0 and
+declares `tier: deep` — and its call edges equal the fast tier's. `codemap doctor` asks griffe and jedi
+one question each with a known answer and exits 1 when one is absent or answers wrongly
+([accuracy.md](docs/accuracy.md#what-this-means-for-trust)); rebuilding over the previous graph of the
+same tree (`build -o` on the existing file) names the classes of edges that shrank
+([provenance.md](docs/provenance.md#fewer-edges-on-the-same-input-is-named)).
+
 **How many times to build.** A consumer measured 175 full deep builds of one tree and found a real
 edge present in **75 %** of them; eight builds here agreed (5 of 8, the same edge, and nothing else
 unstable among 13 675). So one build misses such an edge about 1 time in 4, two builds 1 in 16, three

{codmap-0.0.21 → codmap-0.0.22}/README.md

@@ -8,7 +8,7 @@ and a **SCIP index** for interop with Sourcegraph / Glean and other precise-code
 
 [![CI](https://github.com/kogriv/codemap/actions/workflows/ci.yml/badge.svg)](https://github.com/kogriv/codemap/actions/workflows/ci.yml)
 
-**Status:** 🟢 M0–M20 implemented + research track (R1/R2) — schema 0.13, **990 tests with no failures on
+**Status:** 🟢 M0–M20 implemented + research track (R1/R2) — schema 0.13, **1054 tests with no failures on
 Python 3.11–3.14** ([in CI](docs/ci.md): the full suite including the dogfood pass, a determinism check, a
 wheel smoke test, and ctags/SCIP interop against the real CLIs), warm serve surface with 31 ops (28 exposed
 as MCP tools), and SCIP export. See **[DESIGN.md](DESIGN.md)** (product design &
@@ -185,8 +185,14 @@ codemap serve --graph graph.json --watch
 - **Dataflow** — producers/consumers of a string-keyed DataFrame column.
 - **Semantic search** (opt-in) — a concept query routed to an external adapter, with each fuzzy hit
   resolved to the exact codemap symbol at its location (`codemap semantic`). See [docs/integrations.md](docs/integrations.md).
-- **Context pack** — the most relevant slice of the graph under a token budget, ranked by importance or
-  by relevance to seed symbols (`codemap pack --budget N [--seed X]`). See [docs/pack.md](docs/pack.md).
+- **Context pack** — the highest-ranked modules, classes and functions under a token budget, by
+  importance or by relevance to seed symbols (`codemap pack --budget N [--seed X]`). With a seed it
+  walks both ways — what the seed uses and what uses it (`--direction uses` for one way) — documents are
+  never in it, the answer says which walk it took, and a seed that resolves to nothing is refused rather
+  than silently answered globally. Deterministic and budget-tested. Against git history, seeded with a
+  changed file, it puts 42 % and 60 % of the co-changed files in its first ten on two repositories —
+  ahead of `impact` and of grep by names, though on one of the two not beyond the noise; grep still
+  finds tests best. **Whether it helps an agent is not measured.** See [docs/pack.md](docs/pack.md).
 - **Interop** — export the graph as a [SCIP](https://scip-code.org/) index (definitions + symbol
   info + inherits/implements relationships) so Sourcegraph, Glean and other SCIP consumers can drive
   go-to-definition, symbol search and type hierarchy over it. See [docs/export.md](docs/export.md).
@@ -230,6 +236,13 @@ build says so in its own diagnostics, and `codemap diff` says so when both sides
 delta of a few call edges as possible tool noise before reading it as a change in the code. Anything
 that must be reproducible byte-for-byte — a gate, a two-release comparison — belongs on the fast tier.
 
+**A deep build does not check jedi.** With jedi answering nothing, a deep build still exits 0 and
+declares `tier: deep` — and its call edges equal the fast tier's. `codemap doctor` asks griffe and jedi
+one question each with a known answer and exits 1 when one is absent or answers wrongly
+([accuracy.md](docs/accuracy.md#what-this-means-for-trust)); rebuilding over the previous graph of the
+same tree (`build -o` on the existing file) names the classes of edges that shrank
+([provenance.md](docs/provenance.md#fewer-edges-on-the-same-input-is-named)).
+
 **How many times to build.** A consumer measured 175 full deep builds of one tree and found a real
 edge present in **75 %** of them; eight builds here agreed (5 of 8, the same edge, and nothing else
 unstable among 13 675). So one build misses such an edge about 1 time in 4, two builds 1 in 16, three

{codmap-0.0.21 → codmap-0.0.22}/codemap/cli.py

@@ -2,6 +2,8 @@
 
     codemap build  <path> [-o graph.json] [--deep]
                    [--consumer PATH ...] [--docs PATH ...] [--mode thin|full]
+    codemap doctor [--json]
+        probe griffe and jedi with a question whose answer is known (R1-C64)
     codemap scope  <path> [--consumer PATH ...] [--docs PATH ...] [--no-git] [--json]
                    | --diff A.meta.json B.meta.json   → input scope manifest (M19.A)
     codemap query  <name> (--graph g.json | --build <path>) [--format json|text]
@@ -30,7 +32,7 @@
         codemap.toml [integrations].enabled + the tool installed)
     codemap semantic <query> (--graph g.json | --build <path>) [--root DIR] [--limit N]
         semantic search via an opt-in adapter, enriched to codemap symbols (R1-C16)
-    codemap pack (--graph g.json | --build <path>) [--budget N] [--seed X …]
+    codemap pack (--graph g.json | --build <path>) [--budget N] [--seed X …] [--direction both|uses]
         token-budgeted context pack: most relevant graph slice under N tokens (R1-C6)
 """
 
@@ -155,6 +157,7 @@ def _cmd_build(args) -> int:
         text = " ".join(p for p in (d["message"], d.get("consequence")) if p)
         print(f"[{d.get('severity', 'warning')}] {text}", file=sys.stderr)
     if args.out:
+        _report_completeness_drop(args.out, graph)
         store.save(graph, args.out)
         # M18: record the build recipe beside the graph so `codemap refresh` can
         # rebuild it, and so the graph's age is meaningful to `serve`/stats.
@@ -175,6 +178,36 @@ def _cmd_build(args) -> int:
     return 0
 
 
+def _report_completeness_drop(out: str, graph) -> None:
+    """Compare a fresh build with the graph it is about to overwrite (R1-C59).
+
+    The one moment both builds of an input are in hand without anyone asking: the previous
+    graph is still on disk at ``-o``. Read it before the write and say when an edge class
+    shrank on the **same** input — on one input that is the builder (tool, dependency,
+    sampling), never the code. A warning, not a refusal: rebuilding with a new tool is
+    legitimate, and the build still succeeds.
+
+    A file that exists and cannot be read is not a file that is absent (R1-C27), so it is
+    named; a missing file is the first build, and there is nothing to say.
+    """
+    from codemap.provenance import completeness_drops, describe_drops
+    if not Path(out).exists():
+        return
+    try:
+        previous = store.load(out)
+    except Exception as exc:  # any unreadable previous graph — never fatal to a build
+        print(f"[note] the graph already at {out} could not be read "
+              f"({type(exc).__name__}); completeness against it not compared",
+              file=sys.stderr)
+        return
+    drops = completeness_drops(previous, graph)
+    if drops:
+        print(f"[warning] {describe_drops(drops)} — against the graph this build replaces. "
+              "Same input, fewer edges: the difference is the builder (the tool, a "
+              "dependency, or the deep tier's sampling), not the code; whatever reads "
+              "those edges now answers from less than before.", file=sys.stderr)
+
+
 def _inputs_with_membership(graph, scope, args):
     """The extractor's ``inputs`` block plus the manifest membership result (R1-C41).
 
@@ -333,6 +366,22 @@ def _cmd_watch(args) -> int:
     return 0
 
 
+def _cmd_doctor(args) -> int:
+    """Probe each dependency with a question whose answer is known (R1-C64).
+
+    Exit 0 when every probe answers as expected, 1 otherwise. A dependency that imports
+    and answers wrongly is the case this exists for: it never breaks a build, it makes the
+    graph poorer while the graph still declares its tier.
+    """
+    from codemap.extract.probe import render, run_probes
+    report = run_probes()
+    if args.json:
+        print(json.dumps(report, indent=2, sort_keys=True, ensure_ascii=False))
+    else:
+        print(render(report), end="")
+    return 0 if report["ok"] else 1
+
+
 def _cmd_scope(args) -> int:
     """Resolve/print the input scope manifest, or diff two (M19.A)."""
     from codemap.scope import resolve_scope, diff_scopes
@@ -741,15 +790,21 @@ def _cmd_semantic(args) -> int:
 
 def _cmd_pack(args) -> int:
     """Token-budgeted context pack — most relevant graph slice under N tokens (R1-C6)."""
-    from codemap.serve.pack import build_pack, render_pack
-    q = Query(_graph_from(args))
+    from codemap.serve.pack import build_pack, render_payload
     seeds = tuple(args.seed or ())
+    if args.direction and not seeds:  # before any build: a usage error costs nothing
+        print("codemap pack: --direction needs --seed — the global rank walks from no "
+              "seed", file=sys.stderr)
+        return 2
+    q = Query(_graph_from(args))
+    p = build_pack(q, budget=args.budget, seeds=seeds, direction=args.direction)
     if args.format == "json":
-        print(json.dumps(build_pack(q, budget=args.budget, seeds=seeds),
-                         ensure_ascii=False, indent=2, sort_keys=True))
+        print(json.dumps(p, ensure_ascii=False, indent=2, sort_keys=True))
     else:
-        print(render_pack(q, budget=args.budget, seeds=seeds), end="")
-    return 0
+        print(render_payload(p, target=q.graph.target, seeds=seeds), end="")
+    # No seed resolved → a refusal, not an empty pack: exit 1, as `query` does on an
+    # unknown name (pack honesty).
+    return 1 if "error" in p else 0
 
 
 def _cmd_check(args) -> int:
@@ -795,7 +850,7 @@ def _cmd_diff(args) -> int:
     # R1-C25/D4: two graphs built by different tools are a before/after of the *tool*,
     # not of the code. Never a refusal — comparing across an upgrade is legitimate; what
     # was missing is being told, since a clean "no breaking changes" reads as proof.
-    cmp = comparability(old.provenance, new.provenance)
+    cmp = comparability(old.provenance, new.provenance, old_graph=old, new_graph=new)
     if not cmp["comparable"]:
         for line in cmp["differences"]:
             print(f"[warning] {line}", file=sys.stderr)
@@ -947,6 +1002,11 @@ def build_parser() -> argparse.ArgumentParser:
                         "of the chain: a different N restarts it from a full build.")
     b.set_defaults(func=_cmd_build)
 
+    dr = sub.add_parser("doctor", help="Probe griffe and jedi with a question whose answer is "
+                                       "known; exit 1 if one is absent or answers wrongly.")
+    dr.add_argument("--json", action="store_true", help="Structured report instead of text.")
+    dr.set_defaults(func=_cmd_doctor)
+
     sc = sub.add_parser("scope", help="Resolve the input scope manifest (scope_id + profile), or --diff two.")
     sc.add_argument("path", nargs="?", help="Package/dir to scope (like build's path).")
     sc.add_argument("--consumer", action="append", metavar="PATH",
@@ -1094,6 +1154,9 @@ def build_parser() -> argparse.ArgumentParser:
     pk.add_argument("--seed", action="append", metavar="X",
                     help="Bias relevance to this symbol / file (repeatable). "
                          "Omit for global importance.")
+    pk.add_argument("--direction", choices=["both", "uses"],
+                    help="Which way a seeded walk runs: both (default) — what the seeds "
+                         "use and what uses them; uses — only what they use.")
     pk.add_argument("--format", choices=["markdown", "json"], default="markdown")
     pk.set_defaults(func=_cmd_pack)
 

codmap-0.0.22/codemap/extract/probe.py

@@ -0,0 +1,152 @@
+"""Does each dependency still answer a known question correctly? (R1-C64)
+
+griffe and jedi do work codemap never re-checks: griffe resolves re-export aliases into the
+structure, jedi types a local variable into a call target. Both are hard dependencies, so the
+only "check" they ever got was that ``import`` did not fail. That is the wrong check for the
+failure that matters: a jedi that stopped resolving after an upgrade does not break a build —
+``_resolve_jedi`` reads any exception as ``unresolved`` — it turns every deep call into a
+fast-tier one, and the graph is simply poorer. Measured: with ``jedi.Script`` answering
+nothing, a deep build of a real 89-module tree exited 0 with ``calls`` equal to the fast tier's
+to the edge, while still declaring ``tier: deep``.
+
+The probe is the shape `jg doctor` uses (research/tools/jevgrep.md, S2): a synthetic input
+whose correct answer is known, and a backend that answers it wrongly is refused rather than
+counted as present. One fact per dependency, through the real pipeline — not a benchmark,
+which would be noisy and get switched off.
+
+Design: ``docs/design/dependency_probe.md``.
+"""
+
+from __future__ import annotations
+
+import importlib
+import tempfile
+from pathlib import Path
+
+PROBE_PACKAGE = "codemap_probe"
+
+#: The probe tree. ``engine.run()`` needs a local variable's type (jedi only — the fast tier
+#: does not build that edge); ``ph()`` goes through a re-export alias whose target griffe
+#: resolves (``Alias.target_path``), which is where the ``export`` edge comes from.
+PROBE_FILES = {
+    "__init__.py": f"from {PROBE_PACKAGE}.core import helper as public_helper\n",
+    "core.py": ("class Engine:\n"
+                "    def run(self):\n"
+                "        return 1\n\n\n"
+                "def helper():\n"
+                "    return 2\n"),
+    "app.py": (f"from {PROBE_PACKAGE} import public_helper as ph\n"
+               f"from {PROBE_PACKAGE}.core import Engine\n\n\n"
+               "def main():\n"
+               "    engine = Engine()\n"
+               "    engine.run()\n"
+               "    return ph()\n"),
+}
+
+#: Outcomes. ``wrong`` is the case this module exists for: present, answering, mistaken.
+OK, WRONG, ABSENT, ERROR, NOT_RUN = "ok", "wrong", "absent", "error", "not-run"
+
+#: dependency → (edge type, source, target, resolution or None, what the fact means)
+EXPECTED = {
+    "griffe": ("export", PROBE_PACKAGE, f"{PROBE_PACKAGE}.core.helper", None,
+               f"the re-export `{PROBE_PACKAGE}.public_helper` resolves to "
+               f"`{PROBE_PACKAGE}.core.helper`"),
+    "jedi": ("calls", f"{PROBE_PACKAGE}.app.main", f"{PROBE_PACKAGE}.core.Engine.run", "deep",
+             f"`engine.run()` on a local `Engine()` resolves to "
+             f"`{PROBE_PACKAGE}.core.Engine.run`"),
+}
+
+
+def write_probe(directory: str | Path) -> Path:
+    """Write the probe package under ``directory`` and return its path."""
+    pkg = Path(directory) / PROBE_PACKAGE
+    pkg.mkdir(parents=True, exist_ok=True)
+    for name, body in PROBE_FILES.items():
+        (pkg / name).write_text(body, encoding="utf-8")
+    return pkg
+
+
+def _version(name: str) -> str | None:
+    from importlib.metadata import PackageNotFoundError, version
+    try:
+        return version(name)
+    except PackageNotFoundError:
+        return None
+
+
+def holds(graph, dependency: str) -> bool:
+    """Is the dependency's fact an edge of ``graph``?"""
+    etype, source, target, resolution, _ = EXPECTED[dependency]
+    return any(e.type == etype and e.source == source and e.target == target
+               and (resolution is None or e.extras.get("resolution") == resolution)
+               for e in graph.edges)
+
+
+def _expected_str(dependency: str) -> str:
+    etype, source, target, resolution, _ = EXPECTED[dependency]
+    return f"{etype} {source} → {target}" + (f" ({resolution})" if resolution else "")
+
+
+def _row(dependency: str, status: str, detail: str) -> dict:
+    return {"dependency": dependency, "version": _version(dependency), "status": status,
+            "fact": EXPECTED[dependency][4], "expected": _expected_str(dependency),
+            "detail": detail}
+
+
+def _importable(name: str) -> str | None:
+    """``None`` when ``name`` imports; otherwise why it does not."""
+    try:
+        importlib.import_module(name)
+    except Exception as exc:  # ImportError, or a broken install raising anything else
+        return f"not importable ({type(exc).__name__}: {exc})"
+    return None
+
+
+def _check(dependency: str, build) -> tuple[dict, bool]:
+    """Run one probe. Returns the row and whether its build produced a graph at all."""
+    why = _importable(dependency)
+    if why:
+        return _row(dependency, ABSENT, why), False
+    try:
+        graph = build()
+    except Exception as exc:
+        return _row(dependency, ERROR, f"the probe build raised {type(exc).__name__}: {exc}"), False
+    if holds(graph, dependency):
+        return _row(dependency, OK, "answered as expected"), True
+    return _row(dependency, WRONG, f"expected edge `{_expected_str(dependency)}` is not in "
+                                   "the graph — the dependency imports and answers, and the "
+                                   "answer is wrong"), True
+
+
+def run_probes() -> dict:
+    """Build the probe twice (fast for griffe, deep for jedi) and grade each dependency.
+
+    jedi's probe stands on griffe's structure: when the fast build already failed, a deep
+    result would mean nothing, so jedi is ``not-run`` — unknown, not wrong.
+    """
+    from codemap.extract import extract
+    with tempfile.TemporaryDirectory(prefix="codemap-probe-") as tmp:
+        pkg = write_probe(tmp)
+        griffe_row, built = _check("griffe", lambda: extract(str(pkg)))
+        if not built:
+            jedi_row = _row("jedi", NOT_RUN, "not run: the structural pass it stands on "
+                                             f"did not produce a graph ({griffe_row['status']})")
+        else:
+            jedi_row, _ = _check("jedi", lambda: extract(str(pkg), deep=True))
+    rows = [griffe_row, jedi_row]
+    return {"ok": all(r["status"] == OK for r in rows), "probes": rows}
+
+
+def render(report: dict) -> str:
+    """Plain text, one line per dependency, then a verdict line."""
+    lines = ["codemap doctor — does each dependency still answer a known question correctly?", ""]
+    for r in report["probes"]:
+        name = f"{r['dependency']} {r['version'] or '(version unknown)'}"
+        lines.append(f"{name:<22} {r['status'].upper():<8} {r['fact']}")
+        if r["status"] != OK:
+            lines.append(f"{'':<22} {'':<8} {r['detail']}")
+    lines.append("")
+    lines.append("✅ every probe answered as expected." if report["ok"] else
+                 "❌ at least one dependency is absent or answers wrongly — a graph built "
+                 "here may be poorer than its tier declares.")
+    return "\n".join(lines) + "\n"

{codmap-0.0.21 → codmap-0.0.22}/codemap/model.py

@@ -93,7 +93,17 @@ SCHEMA_VERSION = "0.13"
 #
 # Starts at 1 with 0.0.21 and makes no claim about earlier releases: the answer format changed
 # many times before it, and numbering that retroactively would be inventing a history.
-ANSWER_FORMAT = 1
+#
+# History:
+#   1 — 0.0.21: first numbered format.
+#   2 — pack honesty: `pack` carries `seeds`, `unreachable_from_seeds` and `coverage`
+#       blocks, and a seed set that resolves to nothing is a refusal instead of the
+#       global pack (docs/design/pack_honesty.md). Before 2 was released, a seeded
+#       pack also began walking both ways by default — what the seeds use and what
+#       uses them — with `coverage.walk` naming the walk (R1-C66,
+#       docs/design/pack_direction.md). Not a separate number: no release ever
+#       answered 2 without it.
+ANSWER_FORMAT = 2
 
 # Closed vocabulary of edge types (R1-C7). Node ``kind`` is deliberately an OPEN set
 # (DESIGN §2 — new entity kinds may appear), but edges are TYPED: every relationship

{codmap-0.0.21 → codmap-0.0.22}/codemap/provenance.py

@@ -269,13 +269,79 @@ def _samples_str(prov: dict) -> str:
     return f"{runs} sample{'s' if runs != 1 else ''}"
 
 
-def comparability(old: dict | None, new: dict | None) -> dict:
+# -- completeness of two builds of one input (R1-C59) -------------------------
+
+#: A class of edges has *dropped* when it lost at least this many edges **and** at least
+#: this share of its previous count. Above the measured noise, not tuned to a case: the
+#: fast tier is byte-stable (any drop there is another builder), and the deep tier's
+#: sampling noise was measured at single edges — ten builds of one tree differed by no
+#: edge at all, a larger tree lost one real edge in 9524 once in seven builds (R1-C42).
+#: The share keeps a large graph from flagging a handful of edges out of tens of thousands.
+COMPLETENESS_MIN_EDGES = 3
+COMPLETENESS_MIN_SHARE = 0.01
+
+
+def same_input(old: dict | None, new: dict | None) -> bool:
+    """Do two provenance blocks describe one input, read on one tier? (R1-C59 / D2)
+
+    Needs a recorded ``scope_id`` on both sides: a graph without one (pre-0.12, or built
+    by ``extract()`` without a manifest) has an *unknown* input, which is not an equal one.
+    """
+    if not old or not new or not old.get("scope_id"):
+        return False
+    return (old.get("scope_id") == new.get("scope_id")
+            and old.get("tier") == new.get("tier")
+            and old.get("roots") == new.get("roots"))
+
+
+def edge_counts(graph) -> dict[str, int]:
+    """Edges per type — derived from the graph, never stored beside it (D1)."""
+    counts: dict[str, int] = {}
+    for e in graph.edges:
+        counts[e.type] = counts.get(e.type, 0) + 1
+    return counts
+
+
+def completeness_drops(old_graph, new_graph) -> list[dict] | None:
+    """Edge classes the newer build of the **same input** holds fewer of (R1-C59).
+
+    ``None`` when the pair is not comparable this way — another input, another tier, or an
+    input nobody recorded; that is "not compared", and it must stay distinguishable from
+    ``[]``, "compared, nothing fell" (R1-C44). Measured on one frozen tree: codmap 0.0.3
+    and the current code read the same ``scope_id`` and differed by −25 ``imports``,
+    −28 ``references`` and −26 ``calls``, and nothing said so — ``diff`` named the other
+    tool, never what it had lost.
+    """
+    if not same_input(old_graph.provenance, new_graph.provenance):
+        return None
+    before, after = edge_counts(old_graph), edge_counts(new_graph)
+    drops = []
+    for etype in sorted(before):
+        lost = before[etype] - after.get(etype, 0)
+        if lost >= COMPLETENESS_MIN_EDGES and lost >= COMPLETENESS_MIN_SHARE * before[etype]:
+            drops.append({"type": etype, "old": before[etype], "new": after.get(etype, 0)})
+    return drops
+
+
+def describe_drops(drops: list[dict]) -> str:
+    """``completeness fell on the same input: calls 899 → 873 (−2.9 %), …``"""
+    parts = [f"{d['type']} {d['old']} → {d['new']} "
+             f"(−{(d['old'] - d['new']) * 100 / d['old']:.1f} %)" for d in drops]
+    return "completeness fell on the same input: " + ", ".join(parts)
+
+
+def comparability(old: dict | None, new: dict | None, *,
+                  old_graph=None, new_graph=None) -> dict:
     """Are two graphs a before/after of the *code*, or of the *tool*? (D4)
 
     Never a refusal — comparing across an upgrade is a legitimate thing to want. What
     was missing is being told: on the R1-C25 evidence pair, ``diff`` said "no breaking
     changes" (true, and useless) about two graphs that disagreed on which functions
     were dead.
+
+    With both graphs passed, a completeness drop on the same input is one more
+    difference (R1-C59): on one input, everything the two graphs disagree on came from
+    the builder, and a class that shrank is the part a reader most needs named.
     """
     differences: list[str] = []
     caveats: list[str] = []
@@ -304,6 +370,10 @@ def comparability(old: dict | None, new: dict | None) -> dict:
                            "builds of an unchanged tree can differ by a few jedi-resolved "
                            "edges, so read a small delta as possible tool noise "
                            f"(old: {_samples_str(old)}, new: {_samples_str(new)})")
+    if old_graph is not None and new_graph is not None:
+        drops = completeness_drops(old_graph, new_graph)
+        if drops:
+            differences.append(describe_drops(drops))
     return {
         "comparable": not differences,
         "differences": differences,

{codmap-0.0.21 → codmap-0.0.22}/codemap/query.py

@@ -74,6 +74,23 @@ _IMPACT_EDGES = ("calls", "references", "inherits", "imports", "decorated_by",
 # is excluded on purpose (structural, would just inflate big modules).
 _RANK_EDGE_TYPES = ("calls", "imports", "references", "inherits", "implements")
 
+# Node kinds `rank` walks (R1-C6). Everything else — documents, attributes, columns — can
+# never be ranked, so never enters a pack; `pack` declares that rather than implying a
+# "most relevant slice" of the whole graph (pack honesty).
+RANKED_KINDS = ("module", "class", "function")
+
+#: Which way a *seeded* rank walks (R1-C66). ``uses``: from the seeds to what they use —
+#: their callers are never reached. ``both``: that, and everything that uses the seeds.
+#: The default for a seeded `pack` is ``both``, decided by a pre-registered rule
+#: (research/pack_direction.md §5); ``uses`` stays available for "what does this need".
+SEED_DIRECTIONS = ("uses", "both")
+
+
+def _check_direction(direction: str) -> None:
+    if direction not in SEED_DIRECTIONS:
+        raise ValueError(f"unknown direction {direction!r}; expected one of "
+                         f"{', '.join(SEED_DIRECTIONS)}")
+
 # Test-mapping walk (R1-C24). Execution and naming relations only: `imports` is
 # module→module and would spread the answer across the package without saying anything
 # about what a *test* runs; `inherits` is kept because exercising a subclass does exercise
@@ -1063,21 +1080,87 @@ class Query:
 
     # -- relevance ranking (R1-C6) -------------------------------------------
 
+    def _expand_seed(self, seed: str) -> tuple[str, set[str]]:
+        """One seed → ``(via, node ids)``: how it was read and what it matched.
+
+        ``via`` is ``id`` | ``file`` | ``name`` | ``none``. The ids are *all* matches,
+        rankable or not — whether they can bias a rank is the caller's question.
+        """
+        if seed in self.graph.nodes:
+            return "id", {seed}
+        in_file = {n.id for n in self.graph.nodes.values() if n.file == seed}
+        if in_file:
+            return "file", in_file
+        by_name = {n.id for n in self.find(seed)}  # short name → matches
+        return ("name", by_name) if by_name else ("none", set())
+
     def _expand_seeds(self, seeds) -> set[str]:
         """Map each seed (node id / short name / file path) to concrete node ids."""
         out: set[str] = set()
-        files = {n.file for n in self.graph.nodes.values() if n.file}
         for s in seeds:
-            if s in self.graph.nodes:
-                out.add(s)
-            elif s in files:
-                out.update(n.id for n in self.graph.nodes.values() if n.file == s)
+            out |= self._expand_seed(s)[1]
+        return out
+
+    def _rank_graph(self, edge_types=_RANK_EDGE_TYPES,
+                    root: str | None = None) -> "nx.DiGraph":
+        """The graph `rank` walks: rankable kinds only, usage edges user→used."""
+        g = nx.DiGraph()
+        for nid, n in self.graph.nodes.items():
+            if n.kind in RANKED_KINDS and (root is None or self.root_of(nid) == root):
+                g.add_node(nid)
+        ets = set(edge_types)
+        for e in self.graph.edges:
+            if e.type in ets and e.source in g and e.target in g:
+                g.add_edge(e.source, e.target)
+        return g
+
+    def seed_resolution(self, seeds, *, root: str | None = None) -> dict:
+        """What each seed became — the answer to "did my seed count?" (pack honesty).
+
+        A seed that matched only nodes `rank` does not walk (a document, an attribute,
+        a column) biases nothing, so it is **unresolved**, and ``unrankable`` says what
+        it did match. Before this, all three shapes fell through to the global rank
+        under a header naming the seed.
+        """
+        from collections import Counter
+        g = self._rank_graph(root=root)
+        per, unresolved, ids = [], [], set()
+        for s in seeds:
+            via, matched = self._expand_seed(s)
+            rankable = sorted(m for m in matched if m in g)
+            other = Counter(self.graph.nodes[m].kind for m in matched if m not in g)
+            per.append({"input": s, "via": via, "matched": rankable,
+                        "ambiguous": via == "name" and len(rankable) > 1,
+                        "unrankable": dict(sorted(other.items()))})
+            if rankable:
+                ids.update(rankable)
             else:
-                out.update(n.id for n in self.find(s))  # short name → matches
+                unresolved.append(s)
+        return {"seeds": per, "unresolved": unresolved, "ids": sorted(ids)}
+
+    def reachable_from(self, ids, *, root: str | None = None,
+                       direction: str = "uses") -> set[str]:
+        """Rankable nodes a seeded walk reaches from ``ids``.
+
+        ``direction`` (R1-C66): ``uses`` — along usage edges, what the seeds use;
+        ``both`` — that, plus everything that uses the seeds, directly or transitively
+        (ancestors ∪ descendants; a *sibling* that only shares a dependency is not
+        reached). Exactly the set a seeded rank of that direction may surface. Decided by
+        walking, never by ``rank == 0`` — ranks are rounded, and a far reachable node can
+        round to zero.
+        """
+        _check_direction(direction)
+        g = self._rank_graph(root=root)
+        seeds = [i for i in ids if i in g]
+        out = set(seeds)
+        for i in seeds:
+            out |= nx.descendants(g, i)
+            if direction == "both":
+                out |= nx.ancestors(g, i)
         return out
 
     def rank(self, *, seeds=(), edge_types=_RANK_EDGE_TYPES,
-             root: str | None = None) -> dict[str, float]:
+             root: str | None = None, direction: str = "uses") -> dict[str, float]:
         """Importance rank over usage edges via PageRank (R1-C6).
 
         Without ``seeds``: global importance — heavily-depended-upon symbols (imports
@@ -1086,16 +1169,15 @@ class Query:
         context (aider's repo-map trick). ``root`` restricts to one provenance root.
         Deterministic: PageRank is a fixed power-iteration; scores rounded, ties broken
         by id at the call sites that order them.
+
+        ``direction`` applies only with seeds. ``both`` (R1-C66) walks the usage edges in
+        both directions — the restart is still the seeds, so the rank is closeness to them
+        whichever way the edge points. Measured against git history on two repositories
+        (``research/pack_direction.md``): recall@10 of the co-changed files 0.19 → 0.42
+        and 0.20 → 0.60.
         """
-        g = nx.DiGraph()
-        for nid, n in self.graph.nodes.items():
-            if n.kind in ("module", "class", "function") and (
-                    root is None or self.root_of(nid) == root):
-                g.add_node(nid)
-        ets = set(edge_types)
-        for e in self.graph.edges:
-            if e.type in ets and e.source in g and e.target in g:
-                g.add_edge(e.source, e.target)
+        _check_direction(direction)
+        g = self._rank_graph(edge_types, root)
         if g.number_of_nodes() == 0:
             return {}
         personalization = None
@@ -1103,6 +1185,9 @@ class Query:
             seed_ids = {s for s in self._expand_seeds(seeds) if s in g}
             if seed_ids:  # restart biased to seeds (normalized inside _pagerank)
                 personalization = {n: (1.0 if n in seed_ids else 0.0) for n in g}
+                if direction == "both":
+                    g = nx.DiGraph(g)
+                    g.add_edges_from([(b, a) for a, b in g.edges])
         pr = _pagerank(g, personalization)
         return {n: round(v, 8) for n, v in pr.items()}