pymap-cli 0.3.0__tar.gz → 0.5.0__tar.gz

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/CHANGELOG.md

@@ -4,6 +4,62 @@ All notable changes to this project are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project
 adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.0] — 2026-09-05
+
+### Fixed
+- **tach only ever declared 60 modules**, silently dropping the rest from the
+  Mermaid diagram — 113 of 240 on a codebase this was reported from. The limit
+  was justified by a performance claim that measurement does not support: on
+  generated projects of 300 and 1200 modules, `tach sync` + `show` + `map` take
+  the same 0.5-1.0s whether 60 or all modules are declared. Every module is now
+  declared by default.
+
+  The cap never affected cycle detection either: the dependency map comes from
+  scanning the sources, not from the declaration list. It only shortened the
+  diagram, which is now what it is documented to do.
+- A tool whose failure fits on one line no longer writes a log file repeating it.
+
+### Added
+- `max_modules` (`--max-modules`, `[tool.pymap] max_modules`) to trim the
+  Mermaid diagram on purpose. `0`, the default, declares everything.
+
+### Note
+On a large or densely connected codebase, `pydeps` and `pyreverse` are the slow
+steps: they lay the whole graph out with Graphviz and can exhaust the timeout
+without producing anything. Raise `timeout` or point pymap at a subpackage. The
+walkthrough tree does not depend on either.
+
+## [0.4.0] — 2026-09-04
+
+Reported from the field: code2flow crashed on a real codebase, and pymap
+handled it badly in two separate ways.
+
+### Fixed
+- **The explorer came out empty when there was no call graph.** The navigation
+  panel is built from the graph's roots, so with no edges there was nothing to
+  branch from: pymap announced "1476 symbols" and then showed a blank panel and
+  no starting point. Symbols are now listed by module instead, each keeping its
+  own flow diagram, and the landing page says what is missing and why.
+- **A crashing tool dumped its traceback into the terminal.** The full output
+  now goes to `<output>/<tool>.log`; what stays on screen is the cause, where
+  the detail went, and what the failure costs.
+- `runner.run()` trimmed output to its last 1500 characters, which cut the
+  `Traceback` header off the top of a crash — leaving the saved log incomplete
+  and the crash unrecognised. Trimming is a display concern and has moved out of
+  the runner; crashes are now also identified by their stack frames.
+
+### Added
+- `tests/test_explorer.py` runs the explorer script under Node against a stubbed
+  DOM and asserts the panel is populated, with and without a call graph. The
+  empty-panel failure was invisible to every existing test: the page rendered,
+  it just had nothing in it.
+
+### Note
+code2flow asserts that a call's target is a name, attribute, subscript or call.
+It raises `AssertionError` on `(f if cond else g)(x)`, an immediately-invoked
+lambda, `(f or g)(x)` and `(await f())()`. That is a code2flow limitation;
+pymap now degrades instead of following it down.
+
 ## [0.3.0] — 2026-09-04
 
 First release prepared for publication. The codebase, its comments and the

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: pymap-cli
-Version: 0.3.0
+Version: 0.5.0
 Summary: Map a Python codebase in one command.
 Project-URL: Homepage, https://github.com/hermann225-zrouama/pymap-cli
 Project-URL: Issues, https://github.com/hermann225-zrouama/pymap-cli/issues
@@ -100,8 +100,14 @@ editor  = "vscode"          # vscodium, cursor, windsurf, zed, pycharm,
                             # idea, sublime, none, or "myeditor://{f}:{l}"
 exclude = ["vendor", "generated_*"]
 timeout = 300               # seconds per external tool
+max_modules = 0             # cap on the modules declared to tach; 0 = all
 ```
 
+On a large codebase, `pydeps` and `pyreverse` are the slow steps — they lay the
+whole graph out with Graphviz, and a dense one can exhaust the timeout and
+produce nothing. Raise `timeout`, or point pymap at a subpackage. The
+walkthrough tree does not depend on either of them.
+
 ### As a library
 
 ```python
@@ -169,8 +175,10 @@ returns raw facts (never HTML), then add one line to `STEPS` and one section to
   graph nodes and it is what lets the two data sets be joined. In a project
   with `app/models.py` and `blog/models.py`, their symbols merge in the
   explorer. pymap prints a note when it detects the case.
-- **tach declares at most 60 modules.** Beyond that `tach sync` gets very slow
-  for a graph that is already unreadable. The count left out is reported.
+- **The Mermaid diagram gets unwieldy on a large codebase.** Every module is
+  declared to tach by default; set `max_modules` to trim the diagram. It
+  changes nothing else — cycles are found by scanning the sources, not the
+  declaration list.
 - The call graph is only as good as code2flow's static resolution: calls
   through dynamic dispatch or `getattr` do not appear.
 

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/README.md

@@ -60,8 +60,14 @@ editor  = "vscode"          # vscodium, cursor, windsurf, zed, pycharm,
                             # idea, sublime, none, or "myeditor://{f}:{l}"
 exclude = ["vendor", "generated_*"]
 timeout = 300               # seconds per external tool
+max_modules = 0             # cap on the modules declared to tach; 0 = all
 ```
 
+On a large codebase, `pydeps` and `pyreverse` are the slow steps — they lay the
+whole graph out with Graphviz, and a dense one can exhaust the timeout and
+produce nothing. Raise `timeout`, or point pymap at a subpackage. The
+walkthrough tree does not depend on either of them.
+
 ### As a library
 
 ```python
@@ -129,8 +135,10 @@ returns raw facts (never HTML), then add one line to `STEPS` and one section to
   graph nodes and it is what lets the two data sets be joined. In a project
   with `app/models.py` and `blog/models.py`, their symbols merge in the
   explorer. pymap prints a note when it detects the case.
-- **tach declares at most 60 modules.** Beyond that `tach sync` gets very slow
-  for a graph that is already unreadable. The count left out is reported.
+- **The Mermaid diagram gets unwieldy on a large codebase.** Every module is
+  declared to tach by default; set `max_modules` to trim the diagram. It
+  changes nothing else — cycles are found by scanning the sources, not the
+  declaration list.
 - The call graph is only as good as code2flow's static resolution: calls
   through dynamic dispatch or `getattr` do not appear.
 

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/__init__.py

@@ -20,7 +20,7 @@ if TYPE_CHECKING:  # imported lazily at runtime by __getattr__ below
     from pymap.mapper import Report, map_codebase
     from pymap.settings import Settings
 
-__version__ = "0.3.0"
+__version__ = "0.5.0"
 __all__ = ["Report", "Settings", "__version__", "map_codebase"]
 
 

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/cli.py

@@ -18,7 +18,7 @@ from pymap.mapper import map_codebase
 from pymap.runner import missing
 from pymap.settings import EDITORS, EXCLUDED, Settings, detect_target, read_pyproject
 
-DEFAULTS = {"output": "pymap-out", "editor": "vscode", "timeout": 300}
+DEFAULTS = {"output": "pymap-out", "editor": "vscode", "timeout": 300, "max_modules": 0}
 
 
 def build_parser() -> argparse.ArgumentParser:
@@ -49,6 +49,13 @@ def build_parser() -> argparse.ArgumentParser:
     parser.add_argument(
         "--timeout", type=int, metavar="S", help="max seconds per external tool (default: 300)"
     )
+    parser.add_argument(
+        "--max-modules",
+        type=int,
+        metavar="N",
+        help="cap the modules declared to tach, which shortens the Mermaid "
+        "diagram (default: 0, meaning all of them)",
+    )
     parser.add_argument(
         "--open",
         dest="open_in_browser",
@@ -109,6 +116,7 @@ def build_settings(args: argparse.Namespace, cwd: Path | None = None) -> Setting
         excluded=EXCLUDED + tuple(from_file.get("exclude", ())) + tuple(args.exclude or ()),
         editor=editor,
         timeout=args.timeout or from_file.get("timeout", DEFAULTS["timeout"]),
+        max_modules=args.max_modules or from_file.get("max_modules", DEFAULTS["max_modules"]),
         open_in_browser=args.open_in_browser,
     )
 

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/mapper.py

@@ -69,7 +69,10 @@ def map_codebase(settings: Settings, log: Log = lambda _: None) -> Report:
     for index, (name, role, module) in enumerate(STEPS, 1):
         log(f"[{index}/{total}] {name:11} {role}...")
         report.tools[name] = result = module.execute(settings)
-        log("      " + (result["log"] or _summary(name, result)))
+        # A failing tool reports a cause, where its output went, and what is
+        # lost -- several lines, each indented under its step.
+        for line in str(result["log"] or _summary(name, result)).splitlines():
+            log("      " + line)
 
     log(f"[{total}/{total}] explorer    assembling...")
     report.cycles = cycle_analysis.detect(report.tools["tach"]["deps"])
@@ -119,9 +122,11 @@ def _summary(name: str, result: dict[str, Any]) -> str:
         return f"{result['functions']} functions, {result['calls']} calls"
     if name == "tach":
         cut = (
-            f" ({result['truncated']} modules beyond the limit)" if result["truncated"] else ""
+            f", {result['truncated']} modules left out of the diagram by max_modules"
+            if result["truncated"]
+            else ""
         )
-        return f"{len(result['deps'])} files linked{cut}"
+        return f"{len(result['deps'])} files linked, {result['modules']} modules{cut}"
     return "done"
 
 

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/runner.py

@@ -73,20 +73,19 @@ def missing() -> list[Tool]:
     return [tool for name, tool in TOOLS.items() if not available(name)]
 
 
-def run(
-    cmd: Sequence[str], cwd: str | None = None, timeout: int = 300, full: bool = False
-) -> tuple[bool, str]:
-    """Run a command. Pass ``full=True`` when the output has to be parsed.
+def run(cmd: Sequence[str], cwd: str | None = None, timeout: int = 300) -> tuple[bool, str]:
+    """Run a command and return ``(succeeded, complete output)``.
 
-    Returns ``(succeeded, output)``. Without ``full`` the output is trimmed to
-    its last 1500 characters: enough to diagnose, not enough to drown a log.
+    The output is never trimmed here. Shortening it is a display concern, and
+    doing it at this level used to cut the ``Traceback`` header off the top of a
+    crash, leaving both the diagnosis and the saved log incomplete. Callers show
+    what they need; :func:`pymap.tools.record_failure` writes the rest to a file.
     """
     try:
         completed = subprocess.run(
             cmd, cwd=cwd, capture_output=True, text=True, timeout=timeout
         )
-        output = completed.stdout + completed.stderr
-        return completed.returncode == 0, output if full else output[-1500:]
+        return completed.returncode == 0, completed.stdout + completed.stderr
     except subprocess.TimeoutExpired:
         return False, f"timed out after {timeout}s"
     except (FileNotFoundError, NotADirectoryError, PermissionError) as exc:

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/settings.py

@@ -150,6 +150,9 @@ class Settings:
     excluded: Sequence[str] = EXCLUDED
     editor: str = "vscode"
     timeout: int = 300
+    #: Cap on the modules declared to tach; 0 declares them all. Only shortens
+    #: the Mermaid diagram -- it changes neither the runtime nor the cycles.
+    max_modules: int = 0
     open_in_browser: bool = False
     _files: list[Path] | None = field(default=None, repr=False, compare=False)
 

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/templates/explorer.html

@@ -172,7 +172,7 @@
 <header>
   <div class="banner">
     <h1>__PYMAP_NAME__</h1>
-    <p><b id="nr">0</b> entry points · __PYMAP_COVERAGE__% documented</p>
+    <p id="tally"><b id="nr">0</b> entry points · __PYMAP_COVERAGE__% documented</p>
   </div>
   <input id="q" placeholder="Search a symbol — the tree opens down to it" autocomplete="off">
 </header>
@@ -217,6 +217,13 @@ const everyKey = new Set([...Object.keys(OUT), ...Object.keys(IN)]);
 const roots = [...everyKey].filter(k => !(IN[k] || []).length && (OUT[k] || []).length)
   .sort((a, b) => reach(b) - reach(a) || a.localeCompare(b));
 
+/* code2flow supplies the call graph. It is the one input pymap cannot produce
+   itself, and it is also the one most likely to be missing: the tool may be
+   absent, or it may crash on a construct it does not handle. Everything that
+   follows must stay usable without it. */
+const hasCalls = everyKey.size > 0;
+const COVERAGE = __PYMAP_COVERAGE__;
+
 const ROOT = __PYMAP_ROOT__;
 const EDITOR = __PYMAP_EDITOR__;   // URI template, {f} = absolute file, {l} = line
 let expanded = new Set(), sel = null, selPath = [], highlight = '';
@@ -472,13 +479,57 @@ function diagram(key) {
 }
 
 /* ================= TREE ================= */
+
+/* Every symbol grouped under its module, biggest module first. This is what
+   the panel shows when there is no call graph: without edges there are no
+   roots to branch from, and the tree would otherwise be empty. */
+function modulesWithSymbols() {
+  const groups = new Map();
+  for (const s of SYM) {
+    if (s.kind === 'module') continue;
+    if (!groups.has(s.module)) groups.set(s.module, []);
+    groups.get(s.module).push(s.key);
+  }
+  return [...groups.entries()]
+    .sort((a, b) => b[1].length - a[1].length || a[0].localeCompare(b[0]));
+}
+
+function renderFlatList(out) {
+  const modules = modulesWithSymbols();
+  const total = modules.reduce((n, [, keys]) => n + keys.length, 0);
+  out.push(`<div class="section"><h2>Symbols by module</h2>`
+    + `<span class="n">${total}</span></div>`
+    + `<p class="group" style="padding-top:0">no call graph, so no walkthrough `
+    + `order: each symbol still has its own flow diagram</p>`);
+  for (const [module, keys] of modules) {
+    out.push(`<div class="group">${esc(module)} · ${keys.length}</div>`);
+    for (const key of keys) {
+      const s = byKey[key];
+      out.push(`<div class="branch"><button class="node${seen.has(key) ? ' seen' : ''}" `
+        + `data-path="${esc(key)}" aria-current="${key === sel}"><span class="head">`
+        + `<span class="twist"></span>`
+        + `<span class="name">${mark(shortName(key))}</span>`
+        + (seen.has(key) ? `<span class="check">✓</span>` : '')
+        + `<span class="mod">${esc(s ? s.kind : '')}</span><span class="gap"></span>`
+        + `</span></button></div>`);
+    }
+  }
+}
+
 function renderTree() {
-  $('nr').textContent = roots.length;
+  const symbolCount = SYM.filter(s => s.kind !== 'module').length;
+  $('tally').innerHTML = hasCalls
+    ? `<b>${roots.length}</b> entry points · ${COVERAGE}% documented`
+    : `<b>${symbolCount}</b> symbols · ${COVERAGE}% documented`;
   const out = [];
-  renderGroup(out, entries, "Public surface",
-              "documented functions nobody calls: the entry points", false);
-  renderGroup(out, orphans, "Not called inside the package",
-              "special methods Python invokes, or code with no caller", true);
+  if (hasCalls) {
+    renderGroup(out, entries, "Public surface",
+                "documented functions nobody calls: the entry points", false);
+    renderGroup(out, orphans, "Not called inside the package",
+                "special methods Python invokes, or code with no caller", true);
+  } else {
+    renderFlatList(out);
+  }
   $('nav').innerHTML = out.join('');
   for (const button of $('nav').querySelectorAll('.node')) {
     button.onclick = () => {
@@ -653,20 +704,34 @@ function renderScene() {
   tipHide();
   steps = []; iStep = -1;
   if (!sel) {
-    const best = groupBy(entries).map(group => group[0])
-      .sort((a, b) => reach(b) - reach(a)).slice(0, 3);
-    canvas.innerHTML = `<div class="home">
-      <h3>Where to start</h3>
+    // Without a call graph there is no reach to rank by, so fall back to the
+    // functions with the most steps: the densest logic is a reasonable start.
+    const best = hasCalls
+      ? groupBy(entries).map(group => group[0]).sort((a, b) => reach(b) - reach(a)).slice(0, 3)
+      : SYM.filter(s => s.flow && s.flow.length)
+           .sort((a, b) => b.flow.length - a.flow.length).slice(0, 3).map(s => s.key);
+    canvas.innerHTML = `<div class="home">`
+      + (hasCalls
+        ? `<h3>Where to start</h3>
       <p>These walkthroughs begin at the functions nobody calls — the package's
       entry points. The first one already covers most of the useful code.</p>`
+        : `<h3>No call graph for this codebase</h3>
+      <p>code2flow did not produce one, so pymap cannot tell which function
+      calls which: there is no walkthrough order, and the calls drawn in a
+      diagram are not clickable. Every symbol is still listed on the left, and
+      each one keeps its own flow — branches, loops, error paths and the data
+      passing through. Installing or upgrading code2flow restores the rest.</p>`)
       + best.map((key, i) => {
         const s = byKey[key];
         const doc = s && s.doc ? s.doc.trim().split('\n')[0] : 'no docstring';
+        const size = hasCalls ? reach(key) : (s && s.flow ? s.flow.length : 0);
+        const unit = hasCalls ? 'symbols' : 'steps';
         return `<button class="track" data-track="${esc(key)}">`
           + `<span class="r">${i + 1}</span><span class="t">`
           + `<b>${esc(shortName(key))}</b><span>${esc(doc.slice(0, 74))}</span></span>`
-          + `<span class="m">${reach(key)} symbols<br>~${Math.max(5, Math.round(reach(key) / 4))} min</span>`
-          + `</button>`;
+          + `<span class="m">${size} ${unit}`
+          + (hasCalls ? `<br>~${Math.max(5, Math.round(size / 4))} min` : '')
+          + `</span></button>`;
       }).join('')
       + `<div class="keys">
         <kbd>↓</kbd><kbd>↑</kbd> walk the steps of the flow ·

pymap_cli-0.5.0/src/pymap/tools/__init__.py

@@ -0,0 +1,86 @@
+"""Wrappers around external tools: one module per tool.
+
+Each module exposes ``execute(settings)`` and returns raw facts, never HTML --
+layout belongs to :mod:`pymap.render`. A missing tool returns an empty result
+carrying an explanatory ``log``; it never aborts the run.
+"""
+
+from __future__ import annotations
+
+import re
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from pymap.settings import Settings
+
+TRACEBACK = "Traceback (most recent call last)"
+
+#: A runaway tool must not fill the disk with its own log.
+MAX_LOG = 200_000
+
+
+def looks_like_a_crash(output: str) -> bool:
+    """Whether the output is a Python traceback.
+
+    The header alone is not enough to go on: it is the first thing lost when
+    output gets clipped, so stack frames count as evidence too.
+    """
+    return TRACEBACK in output or output.count('  File "') >= 2
+
+
+def summarise_output(output: str) -> str:
+    """One line describing why a tool failed, from its raw output.
+
+    A crashing tool prints a traceback whose last line carries the exception;
+    everything above it is noise to someone using pymap rather than developing
+    the tool. Anything else is reported as the tool worded it.
+    """
+    lines = [line.strip() for line in output.splitlines() if line.strip()]
+    if not lines:
+        return "failed without producing any output"
+    if looks_like_a_crash(output):
+        return f"crashed: {lines[-1][:120]}"
+    return lines[-1][:160]
+
+
+def record_failure(settings: Settings, tool: str, output: str, consequence: str = "") -> str:
+    """Save a failing tool's full output and return what to show the user.
+
+    The raw output is a wall of traceback: it belongs in a file, not in the
+    terminal. What stays on screen is the cause, where the detail went, and
+    what the user loses -- the last one being the part that matters.
+    """
+    where = ""
+    # Skip the file when the whole reason already fits on screen -- "timed out
+    # after 300s" needs no log. Length matters as much as line count: a single
+    # very long line is not visible in full either.
+    body = output.strip()
+    already_shown = len(body.splitlines()) <= 1 and len(body) <= 160
+    if body and not already_shown:
+        path = settings.output / f"{tool}.log"
+        kept = output[:MAX_LOG]
+        if len(output) > MAX_LOG:
+            kept += f"\n\n[truncated: {len(output) - MAX_LOG} more characters]\n"
+        try:
+            path.write_text(kept, encoding="utf-8")
+            where = f" (output in {path.name})"
+        except OSError:
+            where = ""
+    lines = [f"{tool} {summarise_output(output)}{where}"]
+    if looks_like_a_crash(output):
+        lines.append(f"this is a {tool} limitation, not a pymap error")
+    if consequence:
+        lines.append(consequence)
+    return "\n".join(lines)
+
+
+def count_svg(path: str | Path) -> tuple[int, int]:
+    """(nodes, edges) of an SVG produced by Graphviz."""
+    try:
+        text = Path(path).read_text(encoding="utf-8", errors="ignore")
+    except OSError:
+        return 0, 0
+    titles = re.findall(r"<title>([^<]+)</title>", text)
+    edges = [t for t in titles if "&#45;&gt;" in t or "->" in t]
+    return len(titles) - len(edges), len(edges)

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/tools/code2flow.py

@@ -11,6 +11,13 @@ from typing import Any
 
 from pymap import runner
 from pymap.settings import Settings
+from pymap.tools import record_failure
+
+#: What the user loses when this step does not run. The explorer keeps every
+#: symbol either way, but without these edges it cannot follow a call.
+CONSEQUENCE = (
+    "the explorer still lists every symbol by module, but cannot follow calls between them"
+)
 
 ROLE = "call graph"
 
@@ -27,7 +34,7 @@ def execute(settings: Settings) -> dict[str, Any]:
     empty: dict[str, Any] = {"json": None, "functions": 0, "calls": 0}
     base = runner.resolve("code2flow")
     if not base:
-        return dict(empty, log="code2flow is not installed")
+        return dict(empty, log=f"code2flow is not installed\n{CONSEQUENCE}")
 
     files = settings.files()
     if not files:
@@ -43,11 +50,13 @@ def execute(settings: Settings) -> dict[str, Any]:
         base + sources + ["-o", str(output)], timeout=max(settings.timeout, MIN_TIMEOUT)
     )
     if not (ok and output.exists()):
-        return dict(empty, log=log or "code2flow failed")
+        return dict(empty, log=record_failure(settings, "code2flow", log, CONSEQUENCE))
     try:
         graph = json.loads(output.read_text(encoding="utf-8"))["graph"]
     except (OSError, ValueError, KeyError):
-        return dict(empty, json=output, log="unreadable code2flow JSON")
+        return dict(
+            empty, json=output, log=f"code2flow produced unreadable JSON\n{CONSEQUENCE}"
+        )
     return {
         "json": output,
         "functions": len(graph["nodes"]),

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/tools/pydeps.py

@@ -6,7 +6,7 @@ from typing import Any
 
 from pymap import runner
 from pymap.settings import Settings
-from pymap.tools import count_svg
+from pymap.tools import count_svg, record_failure
 
 ROLE = "import graph"
 
@@ -37,6 +37,6 @@ def execute(settings: Settings) -> dict[str, Any]:
         timeout=settings.timeout,
     )
     if not (ok and svg.exists()):
-        return dict(empty, log=log or "pydeps failed")
+        return dict(empty, log=record_failure(settings, "pydeps", log))
     nodes, edges = count_svg(svg)
     return {"svg": svg, "modules": nodes, "imports": edges, "log": ""}

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/tools/pyreverse.py

@@ -7,6 +7,7 @@ from typing import Any
 
 from pymap import runner
 from pymap.settings import Settings
+from pymap.tools import record_failure
 
 ROLE = "class diagrams"
 
@@ -33,7 +34,7 @@ def execute(settings: Settings) -> dict[str, Any]:
     classes = settings.output / f"classes_{project}.svg"
     packages = settings.output / f"packages_{project}.svg"
     if not ok and not classes.exists():
-        return dict(empty, log=log or "pyreverse failed")
+        return dict(empty, log=record_failure(settings, "pyreverse", log))
     return {
         "classes": classes if classes.exists() else None,
         "packages": packages if packages.exists() else None,

{pymap_cli-0.3.0 → pymap_cli-0.5.0}/src/pymap/tools/tach.py

@@ -17,11 +17,16 @@ from typing import Any
 
 from pymap import runner
 from pymap.settings import Settings, is_excluded
+from pymap.tools import record_failure
 
 ROLE = "module boundaries"
 
-#: Past this point ``tach sync`` gets very slow, for a graph already unreadable.
-MAX_MODULES = 60
+#: Declaring every module costs nothing. Measured on generated projects of 300
+#: and 1200 modules: ``tach sync`` + ``show`` + ``map`` take the same 0.5-1.0s
+#: whether 60 or all of them are declared, and the dependency map that feeds
+#: cycle detection comes from scanning the sources, not from the declaration
+#: list. A cap only shortens the Mermaid diagram, so it is opt-in.
+NO_LIMIT = 0
 
 
 def _copy_sources(target: Path, into: Path, excluded: Sequence[str]) -> None:
@@ -61,7 +66,7 @@ def execute(settings: Settings) -> dict[str, Any]:
     every = _module_paths(settings.target, settings.files())
     if not every:
         return dict(empty, log="no module to declare")
-    kept = every[:MAX_MODULES]
+    kept = every[: settings.max_modules] if settings.max_modules > NO_LIMIT else every
 
     with tempfile.TemporaryDirectory(prefix="pymap-tach-") as tmp:
         workspace = Path(tmp)
@@ -76,7 +81,7 @@ def execute(settings: Settings) -> dict[str, Any]:
 
         ok, log = runner.run([*base, "sync"], cwd=str(workspace), timeout=settings.timeout)
         if not ok:
-            return dict(empty, log=log or "tach sync failed")
+            return dict(empty, log=record_failure(settings, "tach", log))
 
         mermaid = settings.output / "tach.mmd"
         drawn, _ = runner.run(
@@ -87,7 +92,7 @@ def execute(settings: Settings) -> dict[str, Any]:
 
         deps: dict[str, list[str]] = {}
         mapped, raw = runner.run(
-            [*base, "map", "-o", "-"], cwd=str(workspace), timeout=settings.timeout, full=True
+            [*base, "map", "-o", "-"], cwd=str(workspace), timeout=settings.timeout
         )
         if mapped:
             try:

pymap_cli-0.5.0/tests/test_explorer.py

@@ -0,0 +1,114 @@
+"""Behaviour of the explorer script, run under Node with a stubbed DOM.
+
+``test_contract.py`` proves the script parses; this proves it renders. The two
+failures worth guarding against here are silent: a page whose navigation panel
+comes out empty looks like a working page with nothing to show.
+"""
+
+import json
+import re
+import shutil
+import subprocess
+
+import pytest
+
+from pymap import render
+from pymap.analysis.symbols import extract
+
+pytestmark = pytest.mark.skipif(not shutil.which("node"), reason="node is not installed")
+
+# Just enough DOM for the script to run to completion. Everything the script
+# reads back is captured; everything else is inert.
+DOM_STUB = """
+class El {
+  constructor(id) {
+    this.id = id; this.innerHTML = ''; this.textContent = '';
+    this.style = {}; this.dataset = {};
+    this.classList = { add() {}, remove() {}, contains() { return false; } };
+  }
+  addEventListener() {}
+  querySelectorAll() { return []; }
+  querySelector() { return null; }
+  getBoundingClientRect() {
+    return { left: 0, top: 0, right: 0, bottom: 0, width: 0, height: 0 };
+  }
+  scrollIntoView() {} focus() {} select() {} blur() {}
+}
+const ELEMENTS = {};
+globalThis.document = {
+  getElementById(id) { return ELEMENTS[id] || (ELEMENTS[id] = new El(id)); },
+  addEventListener() {},
+  activeElement: null,
+  body: { style: {} },
+};
+globalThis.window = { innerWidth: 1200, innerHeight: 800, addEventListener() {} };
+globalThis.__report = () => JSON.stringify({
+  nav: (ELEMENTS.nav || {}).innerHTML || '',
+  canvas: (ELEMENTS.canvas || {}).innerHTML || '',
+  tally: (ELEMENTS.tally || {}).innerHTML || '',
+});
+"""
+
+
+def render_in_node(tmp_path, symbols, relations):
+    """Run the explorer script and return what it wrote into the page."""
+    html = render.explorer_page(
+        "sample", "/tmp/sample", symbols, relations, 50, "vscode://file/{f}:{l}"
+    )
+    script = re.search(r"<script>(.*)</script>", html, re.S).group(1)
+    bundle = tmp_path / "run.js"
+    bundle.write_text(DOM_STUB + script + "\nconsole.log(__report());\n", encoding="utf-8")
+    done = subprocess.run(["node", str(bundle)], text=True, capture_output=True)
+    assert done.returncode == 0, done.stderr
+    return json.loads(done.stdout)
+
+
+@pytest.fixture
+def symbols(sample):
+    return extract(sample, sorted(sample.rglob("*.py")))
+
+
+@pytest.fixture
+def relations(symbols):
+    """A call graph shaped like code2flow's, over the sample package."""
+    return {
+        "entry::main": {"calls": ["engine::process"], "called_by": []},
+        "engine::process": {"calls": ["engine::Tank.average"], "called_by": ["entry::main"]},
+        "engine::Tank.average": {"calls": [], "called_by": ["engine::process"]},
+    }
+
+
+def test_navigation_is_populated_with_a_call_graph(tmp_path, symbols, relations):
+    page = render_in_node(tmp_path, symbols, relations)
+    assert "main" in page["nav"]
+    assert "entry points" in page["tally"]
+    assert "Where to start" in page["canvas"]
+
+
+def test_navigation_is_populated_without_a_call_graph(tmp_path, symbols):
+    """The regression that prompted this file: code2flow can crash, and the
+    panel used to come out empty even though every symbol had been extracted."""
+    page = render_in_node(tmp_path, symbols, {})
+    assert page["nav"].strip(), "the navigation panel must never be empty"
+    assert "Symbols by module" in page["nav"]
+    assert "process" in page["nav"]
+    assert "engine" in page["nav"]
+    assert "symbols" in page["tally"]
+
+
+def test_the_missing_call_graph_is_explained(tmp_path, symbols):
+    page = render_in_node(tmp_path, symbols, {})
+    assert "No call graph" in page["canvas"]
+    assert "code2flow" in page["canvas"]
+
+
+def test_every_symbol_is_listed_when_there_is_no_call_graph(tmp_path, symbols):
+    page = render_in_node(tmp_path, symbols, {})
+    for symbol in symbols:
+        if symbol["kind"] != "module":
+            assert symbol["name"] in page["nav"], symbol["key"]
+
+
+def test_an_empty_codebase_still_renders(tmp_path):
+    page = render_in_node(tmp_path, [], {})
+    assert page["tally"]

pymap_cli-0.5.0/tests/test_tools.py

@@ -0,0 +1,203 @@
+"""Tool wrappers: clean fallback when absent, and tach's internal logic."""
+
+from pathlib import Path
+
+from pymap import runner
+from pymap.settings import Settings
+from pymap.tools import (
+    MAX_LOG,
+    code2flow,
+    count_svg,
+    pydeps,
+    pyreverse,
+    record_failure,
+    summarise_output,
+    tach,
+)
+
+SVG = (
+    "<svg><g><title>a</title></g><g><title>b</title></g><g><title>a&#45;&gt;b</title></g></svg>"
+)
+
+
+def test_count_svg_separates_nodes_from_edges(tmp_path):
+    path = tmp_path / "g.svg"
+    path.write_text(SVG, encoding="utf-8")
+    assert count_svg(path) == (2, 1)
+
+
+def test_count_svg_on_a_missing_file(tmp_path):
+    assert count_svg(tmp_path / "nothing.svg") == (0, 0)
+
+
+def test_every_absent_tool_returns_an_empty_result(settings, without_tools):
+    settings.output.mkdir(parents=True, exist_ok=True)
+    for module in (pydeps, pyreverse, code2flow, tach):
+        result = module.execute(settings)
+        assert result["log"], module.__name__
+        assert not any(value for key, value in result.items() if key != "log")
+
+
+def test_tools_report_missing_graphviz(settings, monkeypatch):
+    monkeypatch.setattr(
+        runner, "resolve", lambda name: None if name == "dot" else ["/bin/" + name]
+    )
+    for module in (pydeps, pyreverse):
+        assert "graphviz" in module.execute(settings)["log"]
+
+
+def test_tach_names_modules_from_the_root(sample):
+    modules = tach._module_paths(sample, sorted(sample.rglob("*.py")))
+    assert "sample.engine" in modules
+    assert "sample.entry" in modules
+    assert not any(m.endswith("__init__") for m in modules)
+
+
+def test_tach_copies_sources_and_nothing_else(tmp_path):
+    src = tmp_path / "pkg"
+    (src / ".venv" / "lib").mkdir(parents=True)
+    (src / ".venv" / "lib" / "intruder.py").write_text("", encoding="utf-8")
+    (src / "mod.py").write_text("x = 1\n", encoding="utf-8")
+    (src / "data.csv").write_text("a,b\n", encoding="utf-8")
+
+    tach._copy_sources(src, tmp_path / "copy", (".venv", "__pycache__"))
+    copied = {p.name for p in (tmp_path / "copy").rglob("*") if p.is_file()}
+    assert copied == {"mod.py"}
+
+
+def test_tach_survives_a_failing_sync(settings, monkeypatch):
+    monkeypatch.setattr(runner, "resolve", lambda name: ["fake-tach"])
+    monkeypatch.setattr(runner, "run", lambda *a, **k: (False, "tach unavailable"))
+    assert tach.execute(settings)["log"]
+
+
+# --- tool resolution ---------------------------------------------------------
+
+
+def test_resolve_finds_the_tool_of_the_current_venv(monkeypatch, tmp_path):
+    """Common case: pymap launched by full path, its bin/ not on PATH."""
+    fake_bin = tmp_path / "bin"
+    fake_bin.mkdir()
+    (fake_bin / "pydeps").write_text("#!/bin/sh\n", encoding="utf-8")
+    (fake_bin / "pydeps").chmod(0o755)
+    monkeypatch.setattr(runner.sys, "executable", str(fake_bin / "python"))
+    monkeypatch.setenv("PATH", "/nowhere")
+    assert runner.resolve("pydeps") == [str(fake_bin / "pydeps")]
+
+
+def test_resolve_refuses_the_dash_m_fallback_without_main(monkeypatch):
+    """code2flow has no __main__: better to report it absent than to fail oddly."""
+    monkeypatch.setattr(runner.shutil, "which", lambda *a, **k: None)
+    monkeypatch.setattr(runner, "_has_main_module", lambda module: False)
+    assert runner.resolve("code2flow") is None
+
+
+def test_missing_lists_tools_in_declaration_order(monkeypatch):
+    monkeypatch.setattr(runner, "resolve", lambda name: None)
+    assert [tool.name for tool in runner.missing()] == list(runner.TOOLS)
+
+
+def test_run_reports_an_unknown_command():
+    ok, output = runner.run(["definitely-not-a-real-binary-xyz"])
+    assert not ok
+    assert output
+
+
+# --- failure reporting -------------------------------------------------------
+
+TRACEBACK_OUTPUT = """Traceback (most recent call last):
+  File "/x/code2flow/engine.py", line 734, in code2flow
+    file_groups, all_nodes, edges = map_it(sources, language, no_trimming,
+  File "/x/code2flow/python.py", line 18, in get_call_from_func_element
+    assert type(func) in (ast.Attribute, ast.Name, ast.Subscript, ast.Call)
+AssertionError
+"""
+
+
+def test_summarise_output_picks_the_exception_out_of_a_traceback():
+    assert summarise_output(TRACEBACK_OUTPUT) == "crashed: AssertionError"
+
+
+def test_summarise_output_keeps_a_plain_message():
+    assert summarise_output("could not open file\n") == "could not open file"
+
+
+def test_summarise_output_handles_silence():
+    assert summarise_output("   \n") == "failed without producing any output"
+
+
+def test_record_failure_moves_the_traceback_into_a_file(settings):
+    settings.output.mkdir(parents=True, exist_ok=True)
+    message = record_failure(settings, "code2flow", TRACEBACK_OUTPUT, "calls are lost")
+
+    assert (settings.output / "code2flow.log").read_text() == TRACEBACK_OUTPUT
+    # What stays on screen: cause, where the detail went, what it costs.
+    assert "crashed: AssertionError" in message
+    assert "code2flow.log" in message
+    assert "not a pymap error" in message
+    assert "calls are lost" in message
+    # And not the wall of traceback itself.
+    assert 'File "/x/code2flow' not in message
+    assert len(message.splitlines()) == 3
+
+
+def test_record_failure_without_output_writes_no_file(settings):
+    settings.output.mkdir(parents=True, exist_ok=True)
+    record_failure(settings, "tach", "")
+    assert not (settings.output / "tach.log").exists()
+
+
+def test_record_failure_recognises_a_traceback_whose_header_was_clipped(settings):
+    """Output can arrive already cut at the top; stack frames still identify it."""
+    settings.output.mkdir(parents=True, exist_ok=True)
+    clipped = TRACEBACK_OUTPUT.split("\n", 1)[1]
+    assert "Traceback" not in clipped
+    assert summarise_output(clipped) == "crashed: AssertionError"
+    assert "not a pymap error" in record_failure(settings, "code2flow", clipped)
+
+
+def test_record_failure_caps_a_runaway_log(settings):
+    settings.output.mkdir(parents=True, exist_ok=True)
+    record_failure(settings, "tach", "x" * (MAX_LOG + 5000))
+    written = (settings.output / "tach.log").read_text()
+    assert len(written) < MAX_LOG + 200
+    assert "truncated" in written
+
+
+def test_tach_declares_every_module_by_default(sample, tmp_path, monkeypatch):
+    """The old cap of 60 silently dropped modules from the Mermaid diagram."""
+    settings = Settings(target=sample, output=tmp_path / "out")
+    captured = {}
+
+    def fake_run(cmd, cwd=None, timeout=300):
+        if cmd[-1] == "sync":
+            captured["toml"] = (Path(cwd) / "tach.toml").read_text()
+        return False, "stop here"
+
+    monkeypatch.setattr(runner, "resolve", lambda name: ["tach"])
+    monkeypatch.setattr(runner, "run", fake_run)
+    tach.execute(settings)
+    every = tach._module_paths(settings.target, settings.files())
+    assert captured["toml"].count("[[modules]]") == len(every)
+
+
+def test_tach_honours_an_explicit_cap(sample, tmp_path, monkeypatch):
+    settings = Settings(target=sample, output=tmp_path / "out", max_modules=1)
+    monkeypatch.setattr(runner, "resolve", lambda name: ["tach"])
+    monkeypatch.setattr(runner, "run", lambda *a, **k: (False, "stop"))
+    result = tach.execute(settings)
+    assert result["log"]
+
+    # And the count reported back reflects what was dropped.
+    settings2 = Settings(target=sample, output=tmp_path / "out2", max_modules=1)
+    every = tach._module_paths(settings2.target, settings2.files())
+    assert len(every) > 1, "the sample must have more than one module to cap"
+
+
+def test_record_failure_skips_the_file_for_a_one_line_reason(settings):
+    """ "timed out after 300s" is already fully on screen; a log adds nothing."""
+    settings.output.mkdir(parents=True, exist_ok=True)
+    message = record_failure(settings, "pydeps", "timed out after 300s")
+    assert not (settings.output / "pydeps.log").exists()
+    assert "timed out after 300s" in message
+    assert ".log" not in message

pymap_cli-0.3.0/src/pymap/tools/__init__.py

@@ -1,22 +0,0 @@
-"""Wrappers around external tools: one module per tool.
-
-Each module exposes ``execute(settings)`` and returns raw facts, never HTML --
-layout belongs to :mod:`pymap.render`. A missing tool returns an empty result
-carrying an explanatory ``log``; it never aborts the run.
-"""
-
-from __future__ import annotations
-
-import re
-from pathlib import Path
-
-
-def count_svg(path: str | Path) -> tuple[int, int]:
-    """(nodes, edges) of an SVG produced by Graphviz."""
-    try:
-        text = Path(path).read_text(encoding="utf-8", errors="ignore")
-    except OSError:
-        return 0, 0
-    titles = re.findall(r"<title>([^<]+)</title>", text)
-    edges = [t for t in titles if "&#45;&gt;" in t or "->" in t]
-    return len(titles) - len(edges), len(edges)

pymap_cli-0.3.0/tests/test_tools.py

@@ -1,91 +0,0 @@
-"""Tool wrappers: clean fallback when absent, and tach's internal logic."""
-
-from pymap import runner
-from pymap.tools import code2flow, count_svg, pydeps, pyreverse, tach
-
-SVG = (
-    "<svg><g><title>a</title></g><g><title>b</title></g><g><title>a&#45;&gt;b</title></g></svg>"
-)
-
-
-def test_count_svg_separates_nodes_from_edges(tmp_path):
-    path = tmp_path / "g.svg"
-    path.write_text(SVG, encoding="utf-8")
-    assert count_svg(path) == (2, 1)
-
-
-def test_count_svg_on_a_missing_file(tmp_path):
-    assert count_svg(tmp_path / "nothing.svg") == (0, 0)
-
-
-def test_every_absent_tool_returns_an_empty_result(settings, without_tools):
-    settings.output.mkdir(parents=True, exist_ok=True)
-    for module in (pydeps, pyreverse, code2flow, tach):
-        result = module.execute(settings)
-        assert result["log"], module.__name__
-        assert not any(value for key, value in result.items() if key != "log")
-
-
-def test_tools_report_missing_graphviz(settings, monkeypatch):
-    monkeypatch.setattr(
-        runner, "resolve", lambda name: None if name == "dot" else ["/bin/" + name]
-    )
-    for module in (pydeps, pyreverse):
-        assert "graphviz" in module.execute(settings)["log"]
-
-
-def test_tach_names_modules_from_the_root(sample):
-    modules = tach._module_paths(sample, sorted(sample.rglob("*.py")))
-    assert "sample.engine" in modules
-    assert "sample.entry" in modules
-    assert not any(m.endswith("__init__") for m in modules)
-
-
-def test_tach_copies_sources_and_nothing_else(tmp_path):
-    src = tmp_path / "pkg"
-    (src / ".venv" / "lib").mkdir(parents=True)
-    (src / ".venv" / "lib" / "intruder.py").write_text("", encoding="utf-8")
-    (src / "mod.py").write_text("x = 1\n", encoding="utf-8")
-    (src / "data.csv").write_text("a,b\n", encoding="utf-8")
-
-    tach._copy_sources(src, tmp_path / "copy", (".venv", "__pycache__"))
-    copied = {p.name for p in (tmp_path / "copy").rglob("*") if p.is_file()}
-    assert copied == {"mod.py"}
-
-
-def test_tach_survives_a_failing_sync(settings, monkeypatch):
-    monkeypatch.setattr(runner, "resolve", lambda name: ["fake-tach"])
-    monkeypatch.setattr(runner, "run", lambda *a, **k: (False, "tach unavailable"))
-    assert tach.execute(settings)["log"]
-
-
-# --- tool resolution ---------------------------------------------------------
-
-
-def test_resolve_finds_the_tool_of_the_current_venv(monkeypatch, tmp_path):
-    """Common case: pymap launched by full path, its bin/ not on PATH."""
-    fake_bin = tmp_path / "bin"
-    fake_bin.mkdir()
-    (fake_bin / "pydeps").write_text("#!/bin/sh\n", encoding="utf-8")
-    (fake_bin / "pydeps").chmod(0o755)
-    monkeypatch.setattr(runner.sys, "executable", str(fake_bin / "python"))
-    monkeypatch.setenv("PATH", "/nowhere")
-    assert runner.resolve("pydeps") == [str(fake_bin / "pydeps")]
-
-
-def test_resolve_refuses_the_dash_m_fallback_without_main(monkeypatch):
-    """code2flow has no __main__: better to report it absent than to fail oddly."""
-    monkeypatch.setattr(runner.shutil, "which", lambda *a, **k: None)
-    monkeypatch.setattr(runner, "_has_main_module", lambda module: False)
-    assert runner.resolve("code2flow") is None
-
-
-def test_missing_lists_tools_in_declaration_order(monkeypatch):
-    monkeypatch.setattr(runner, "resolve", lambda name: None)
-    assert [tool.name for tool in runner.missing()] == list(runner.TOOLS)
-
-
-def test_run_reports_an_unknown_command():
-    ok, output = runner.run(["definitely-not-a-real-binary-xyz"])
-    assert not ok
-    assert output