pymodularity 0.2.0__tar.gz → 0.3.1__tar.gz

pymodularity-0.3.1/PKG-INFO

@@ -0,0 +1,112 @@
+Metadata-Version: 2.4
+Name: pymodularity
+Version: 0.3.1
+Summary: See and check a repo's real Python imports: a privacy convention on PEP 8 underscores, an import-cycle ban, and a coupling report, all derived from the code.
+Author: Tilo Himmelsbach
+License-Expression: MIT
+Requires-Python: >=3.11
+Project-URL: Homepage, https://github.com/dertilo/pymodularity
+Project-URL: Repository, https://github.com/dertilo/pymodularity
+Project-URL: Issues, https://github.com/dertilo/pymodularity/issues
+Description-Content-Type: text/markdown
+
+# pymodularity
+
+The underscore's position in a Python import path sets the import's scope.
+model: Rust's `pub(super)` and `pub(crate)`, applied to PEP 8 underscores.
+
+`pymodularity` does 3 jobs on a repo's real Python imports:
+
+- it draws them as mermaid diagrams.
+- it checks them against conventions derived from the code itself, with no allow-list to maintain.
+- it reports which files and directories carry the most structural coupling.
+
+The analysis needs only the standard library, so `pymodularity` has no runtime dependencies.
+exception: `check`'s dependency rule needs `uv` on PATH.
+
+## Install and run
+
+```bash
+uvx pymodularity check .
+uvx pymodularity graph . --out docs/imports
+uvx pymodularity metrics .
+```
+
+Or, inside a project: `uv add --dev pymodularity` and run `pymodularity ...` in that project's venv.
+
+## The rules `check` enforces
+
+`pymodularity check .` exits 1 on any finding.
+Each finding names the importing file, the target and the owner.
+
+- **No import cycles:** between files, and between directories at every depth.
+  - `TYPE_CHECKING` imports and in-function imports count.
+  - no baseline: an existing cycle fails too.
+- **Private items stay inside their owner:** only code inside the owner may import a `_`-prefixed module or name.
+  - private module: the owner is the directory holding it.
+  - private name from a package: the owner is the package directory.
+  - private name from a regular file: the owner is that file alone, so even a sibling file is a finding.
+  - attribute access such as `mod._x` counts like an import.
+- **Declared dependencies match the imports:** [deptry](https://github.com/fpgmaas/deptry) reports unused, missing and transitive dependencies per package root.
+  - requirement: `uv` on PATH. Without it, `check` exits 2.
+  - `--no-deps` skips this rule.
+- **Structure limits:** at most 500 lines per file, 10 entries per directory, 100 files per package root and 10 public names per file.
+  - no config, no baseline.
+  - ceiling: a limit marks the size that is clearly too much. It is no target.
+  - below a limit: a smaller interface is still better. `metrics` reports interface width and exposure share.
+
+Worked cases: the [showcase](tests/showcase/README.md) shows each rule except the dependency rule as allowed and finding cases.
+Definitions: [docs/rules.md](docs/rules.md) holds the counting rules, the deptry run, and the ruff rules for functions and classes.
+
+## The 3 subcommands
+
+All 3 also take a repeatable `--exclude PATTERN`, merged with `[tool.pymodularity] exclude` in `<repo>`'s own pyproject.toml.
+Pattern: `fnmatch` against the POSIX path relative to `<repo>`, matched on the path itself or any parent directory.
+Use case: generated code (an OpenAPI client, protobuf stubs) and vendored code.
+Effect: an excluded tree is invisible to every rule and report, except deptry's scan of a kept package that contains it.
+An import from kept code into it behaves like an import of code `pymodularity` can't resolve.
+Details: [docs/rules.md](docs/rules.md#excluding-code).
+
+- `pymodularity check <repo> [--no-deps]`: runs the 4 rules above and prints the findings.
+  - exit code: 1 on any finding.
+  - `--no-deps`: skips the dependency rule.
+- `pymodularity graph <repo> --out DIR [--html]`: writes one mermaid view per directory under `DIR`.
+  - `index.md`: written for every view.
+  - `--html`: adds a clickable `index.html` per view. The browser needs network access to load mermaid from a CDN.
+- `pymodularity metrics <repo>`: prints a coupling report to stdout.
+  - reach: transitive dependents and dependencies, plus a `score` per file and per directory.
+  - interface width: the names other code imports from a unit.
+  - exposure share: how much of what a file defines other code uses.
+  - entry files: how much of a directory outside code reaches into.
+  - misplaced files: files whose closest partners sit mostly in another directory.
+  - exit code: always 0, because the report is no gate.
+
+## Blind spots
+
+Worked cases: the [showcase](tests/showcase/README.md) gap cases G1 to G8, each tagged with its outcome.
+
+`pymodularity` shows import coupling only.
+Import coupling is a lower bound on logical coupling: "if A changes, must B change?"
+Coupling that bypasses Python imports stays invisible:
+
+- HTTP APIs, shared database schemas and tables, config keys and env vars, message and file formats, string-keyed registries, runtime `sys.path` edits.
+- Two files with no edge can still force each other's edits. **Never read a missing edge as independence.**
+
+Other limits:
+
+- ambiguous import: `check` prints `warning: ambiguous import` to stderr and draws no edge.
+  - cause: several [import roots](docs/internals.md#how-an-absolute-import-resolves-to-a-file) provide the module, and none is an ancestor of the importer.
+- runtime `sys.path` edit: an import that resolves only through such an edit gets an edge only if the inserted directory is an import root.
+- script sibling: `python main.py` importing its sibling `execution.py`, with no package root nearby, gets no edge.
+  - reason: resolution looks only through import roots, never through the importer's own directory.
+- tests: `pymodularity` always skips test directories (`tests/`, `test/`) and test files (`test_*.py`, `*_test.py`, `conftest.py`).
+  - reason: a test importing a sibling would draw a production dependency that production never has.
+- `metrics` as a gate: its numbers move even on a repo that passes `check`.
+  - consequence: a gate on them rewards gaming the number, for example replacing an import with a string-keyed registry.
+- attribute form, missed cases:
+  - an instance attribute such as `obj._y`, because `pymodularity` has no type information.
+  - complement for that case: ruff's `SLF001`. It has no owner scope, so it also flags access from inside the owner.
+  - a base name bound to a third-party module, or to a module outside the repo.
+  - `getattr(mod, "_x")` and other `importlib` access.
+  - a name the file also rebinds anywhere: a parameter, an assignment target, a `for`, `with`, `except` or comprehension target, a nested `def` or `class`.
+  - reason for the last case: a gate must trade false findings for missed ones, so `pymodularity` skips such a name.

pymodularity-0.3.1/README.md

@@ -0,0 +1,100 @@
+# pymodularity
+
+The underscore's position in a Python import path sets the import's scope.
+model: Rust's `pub(super)` and `pub(crate)`, applied to PEP 8 underscores.
+
+`pymodularity` does 3 jobs on a repo's real Python imports:
+
+- it draws them as mermaid diagrams.
+- it checks them against conventions derived from the code itself, with no allow-list to maintain.
+- it reports which files and directories carry the most structural coupling.
+
+The analysis needs only the standard library, so `pymodularity` has no runtime dependencies.
+exception: `check`'s dependency rule needs `uv` on PATH.
+
+## Install and run
+
+```bash
+uvx pymodularity check .
+uvx pymodularity graph . --out docs/imports
+uvx pymodularity metrics .
+```
+
+Or, inside a project: `uv add --dev pymodularity` and run `pymodularity ...` in that project's venv.
+
+## The rules `check` enforces
+
+`pymodularity check .` exits 1 on any finding.
+Each finding names the importing file, the target and the owner.
+
+- **No import cycles:** between files, and between directories at every depth.
+  - `TYPE_CHECKING` imports and in-function imports count.
+  - no baseline: an existing cycle fails too.
+- **Private items stay inside their owner:** only code inside the owner may import a `_`-prefixed module or name.
+  - private module: the owner is the directory holding it.
+  - private name from a package: the owner is the package directory.
+  - private name from a regular file: the owner is that file alone, so even a sibling file is a finding.
+  - attribute access such as `mod._x` counts like an import.
+- **Declared dependencies match the imports:** [deptry](https://github.com/fpgmaas/deptry) reports unused, missing and transitive dependencies per package root.
+  - requirement: `uv` on PATH. Without it, `check` exits 2.
+  - `--no-deps` skips this rule.
+- **Structure limits:** at most 500 lines per file, 10 entries per directory, 100 files per package root and 10 public names per file.
+  - no config, no baseline.
+  - ceiling: a limit marks the size that is clearly too much. It is no target.
+  - below a limit: a smaller interface is still better. `metrics` reports interface width and exposure share.
+
+Worked cases: the [showcase](tests/showcase/README.md) shows each rule except the dependency rule as allowed and finding cases.
+Definitions: [docs/rules.md](docs/rules.md) holds the counting rules, the deptry run, and the ruff rules for functions and classes.
+
+## The 3 subcommands
+
+All 3 also take a repeatable `--exclude PATTERN`, merged with `[tool.pymodularity] exclude` in `<repo>`'s own pyproject.toml.
+Pattern: `fnmatch` against the POSIX path relative to `<repo>`, matched on the path itself or any parent directory.
+Use case: generated code (an OpenAPI client, protobuf stubs) and vendored code.
+Effect: an excluded tree is invisible to every rule and report, except deptry's scan of a kept package that contains it.
+An import from kept code into it behaves like an import of code `pymodularity` can't resolve.
+Details: [docs/rules.md](docs/rules.md#excluding-code).
+
+- `pymodularity check <repo> [--no-deps]`: runs the 4 rules above and prints the findings.
+  - exit code: 1 on any finding.
+  - `--no-deps`: skips the dependency rule.
+- `pymodularity graph <repo> --out DIR [--html]`: writes one mermaid view per directory under `DIR`.
+  - `index.md`: written for every view.
+  - `--html`: adds a clickable `index.html` per view. The browser needs network access to load mermaid from a CDN.
+- `pymodularity metrics <repo>`: prints a coupling report to stdout.
+  - reach: transitive dependents and dependencies, plus a `score` per file and per directory.
+  - interface width: the names other code imports from a unit.
+  - exposure share: how much of what a file defines other code uses.
+  - entry files: how much of a directory outside code reaches into.
+  - misplaced files: files whose closest partners sit mostly in another directory.
+  - exit code: always 0, because the report is no gate.
+
+## Blind spots
+
+Worked cases: the [showcase](tests/showcase/README.md) gap cases G1 to G8, each tagged with its outcome.
+
+`pymodularity` shows import coupling only.
+Import coupling is a lower bound on logical coupling: "if A changes, must B change?"
+Coupling that bypasses Python imports stays invisible:
+
+- HTTP APIs, shared database schemas and tables, config keys and env vars, message and file formats, string-keyed registries, runtime `sys.path` edits.
+- Two files with no edge can still force each other's edits. **Never read a missing edge as independence.**
+
+Other limits:
+
+- ambiguous import: `check` prints `warning: ambiguous import` to stderr and draws no edge.
+  - cause: several [import roots](docs/internals.md#how-an-absolute-import-resolves-to-a-file) provide the module, and none is an ancestor of the importer.
+- runtime `sys.path` edit: an import that resolves only through such an edit gets an edge only if the inserted directory is an import root.
+- script sibling: `python main.py` importing its sibling `execution.py`, with no package root nearby, gets no edge.
+  - reason: resolution looks only through import roots, never through the importer's own directory.
+- tests: `pymodularity` always skips test directories (`tests/`, `test/`) and test files (`test_*.py`, `*_test.py`, `conftest.py`).
+  - reason: a test importing a sibling would draw a production dependency that production never has.
+- `metrics` as a gate: its numbers move even on a repo that passes `check`.
+  - consequence: a gate on them rewards gaming the number, for example replacing an import with a string-keyed registry.
+- attribute form, missed cases:
+  - an instance attribute such as `obj._y`, because `pymodularity` has no type information.
+  - complement for that case: ruff's `SLF001`. It has no owner scope, so it also flags access from inside the owner.
+  - a base name bound to a third-party module, or to a module outside the repo.
+  - `getattr(mod, "_x")` and other `importlib` access.
+  - a name the file also rebinds anywhere: a parameter, an assignment target, a `for`, `with`, `except` or comprehension target, a nested `def` or `class`.
+  - reason for the last case: a gate must trade false findings for missed ones, so `pymodularity` skips such a name.

{pymodularity-0.2.0 → pymodularity-0.3.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "pymodularity"
-version = "0.2.0"
+version = "0.3.1"
 description = "See and check a repo's real Python imports: a privacy convention on PEP 8 underscores, an import-cycle ban, and a coupling report, all derived from the code."
 readme = "README.md"
 license = "MIT"
@@ -22,22 +22,27 @@ pymodularity = "pymodularity.cli:main"
 dev = [
     "pytest",
     "ruff",
-    "basedpyright",
+    "ty==0.0.90",
 ]
 
 [build-system]
 requires = ["uv_build>=0.10.7,<0.11.0"]
 build-backend = "uv_build"
 
+[tool.pytest.ini_options]
+pythonpath = ["tests"]
+
 [tool.ruff]
 line-length = 90
 target-version = "py311"
+exclude = ["tests/showcase"]
+
+[tool.ty.environment]
+python-version = "3.11"
 
-[tool.basedpyright]
+[tool.ty.src]
 include = [
     "src",
     "tests",
 ]
-pythonVersion = "3.11"
-reportUnusedCallResult = false
-reportAny = false
+exclude = ["tests/showcase"]

{pymodularity-0.2.0 → pymodularity-0.3.1}/pyproject.toml.orig

@@ -1,6 +1,6 @@
 [project]
 name = "pymodularity"
-version = "0.2.0"
+version = "0.3.1"
 description = "See and check a repo's real Python imports: a privacy convention on PEP 8 underscores, an import-cycle ban, and a coupling report, all derived from the code."
 readme = "README.md"
 license = "MIT"
@@ -17,18 +17,29 @@ Issues = "https://github.com/dertilo/pymodularity/issues"
 pymodularity = "pymodularity.cli:main"
 
 [dependency-groups]
-dev = ["pytest", "ruff", "basedpyright"]
+dev = [
+    "pytest",
+    "ruff",
+    "ty==0.0.90",
+]
 
 [build-system]
 requires = ["uv_build>=0.10.7,<0.11.0"]
 build-backend = "uv_build"
 
+[tool.pytest.ini_options]
+pythonpath = ["tests"]
+
 [tool.ruff]
 line-length = 90
 target-version = "py311"
+# a static fixture repo, not real project code: committed golden-test input
+exclude = ["tests/showcase"]
+
+[tool.ty.environment]
+python-version = "3.11"
 
-[tool.basedpyright]
+[tool.ty.src]
 include = ["src", "tests"]
-pythonVersion = "3.11"
-reportUnusedCallResult = false
-reportAny = false
+# a static fixture repo, not real project code: committed golden-test input
+exclude = ["tests/showcase"]

{pymodularity-0.2.0 → pymodularity-0.3.1}/src/pymodularity/_check.py

@@ -34,15 +34,22 @@ def _visibility_scope(rel: Path, package_roots: Sequence[Path]) -> Path:
 @dataclass(frozen=True, slots=True)
 class _PrivacyViolation:
     """An import that reaches a module private to a directory it is not
-    under."""
+    under. `via_attribute` marks a module reached through an attribute
+    chain (`import a; a._internal.x`) rather than an import statement
+    itself -- same scope rule, same `find_privacy_violations`, called a
+    second time on the attribute-form edges `_parse.graph` collects."""
 
     importer: Path
     target: Path
     owner: Path
+    via_attribute: bool = False
 
 
 def find_privacy_violations(
-    edge_names: EdgeNames, package_roots: Sequence[Path]
+    edge_names: EdgeNames,
+    package_roots: Sequence[Path],
+    *,
+    via_attribute: bool = False,
 ) -> tuple[_PrivacyViolation, ...]:
     violations: list[_PrivacyViolation] = []
     scope_cache: dict[Path, Path] = {}
@@ -56,7 +63,9 @@ def find_privacy_violations(
         scope = scope_of(target_rel)
         if importer_rel.is_relative_to(scope):
             continue
-        violations.append(_PrivacyViolation(importer_rel, target_rel, scope))
+        violations.append(
+            _PrivacyViolation(importer_rel, target_rel, scope, via_attribute)
+        )
 
     violations.sort(key=lambda v: (str(v.importer), str(v.target)))
     return tuple(violations)
@@ -84,16 +93,22 @@ def _name_owner(target_rel: Path) -> _NameOwner:
 @dataclass(frozen=True, slots=True)
 class _NameViolation:
     """An import of a private name (`from m import _x`) that reaches
-    outside the directory (package) or file that name is private to."""
+    outside the directory (package) or file that name is private to.
+    `via_attribute` marks a name reached as `mod._x` rather than imported --
+    same scope rule, same `find_name_violations`, called a second time on
+    the attribute-form edges `_parse.graph` collects."""
 
     importer: Path
     target: Path
     name: str
     owner: Path
     owner_is_dir: bool
+    via_attribute: bool = False
 
 
-def find_name_violations(edge_names: EdgeNames) -> tuple[_NameViolation, ...]:
+def find_name_violations(
+    edge_names: EdgeNames, *, via_attribute: bool = False
+) -> tuple[_NameViolation, ...]:
     violations: list[_NameViolation] = []
     for (importer_rel, target_rel), names in edge_names.items():
         for name in names:
@@ -109,6 +124,7 @@ def find_name_violations(edge_names: EdgeNames) -> tuple[_NameViolation, ...]:
                     name,
                     owner_info.owner,
                     owner_info.owner_is_dir,
+                    via_attribute,
                 )
             )
     violations.sort(key=lambda v: (str(v.importer), str(v.target), v.name))
@@ -149,12 +165,14 @@ def print_check_findings(
     total_private = len(privacy_violations) + len(name_violations)
     print(f"## Private-item violations: {total_private} finding(s)")
     for v in privacy_violations:
-        print(f"- {v.importer} -> {v.target}: allowed only inside {v.owner}")
+        suffix = " (via attribute access)" if v.via_attribute else ""
+        print(f"- {v.importer} -> {v.target}: allowed only inside {v.owner}{suffix}")
     for nv in name_violations:
         scope = "directory" if nv.owner_is_dir else "file"
+        suffix = " (via attribute access)" if nv.via_attribute else ""
         print(
             f"- {nv.importer} -> {nv.target}: name {nv.name} private to "
-            + f"{scope} {nv.owner}"
+            + f"{scope} {nv.owner}{suffix}"
         )
     print()
 

{pymodularity-0.2.0 → pymodularity-0.3.1}/src/pymodularity/_deps.py

@@ -22,7 +22,7 @@ from dataclasses import dataclass, field
 from pathlib import Path
 from typing import cast
 
-from pymodularity._parse.packages import find_package_roots
+from pymodularity._parse.packages import find_package_roots, should_skip_dir
 
 _DEPTRY_VERSION = "0.25.1"
 
@@ -85,6 +85,59 @@ def _has_runtime_dependencies(pyproject: Path) -> bool:
     return isinstance(deps, list) and len(cast(list[object], deps)) > 0
 
 
+def _top_level_package_names(repo: Path, pkg: Path) -> tuple[str, ...]:
+    """Top-level first-party package/module names deptry should be told
+    about explicitly: deptry does not treat a hatch-autodetected src-layout
+    package (no explicit `[tool.hatch.build.targets.wheel] packages`) as
+    first-party on its own, so a module importing its own package is
+    wrongly reported as DEP003. `pkg/src` is the base if present, the same
+    src-layout check `_resolve.build_index` uses for import roots;
+    otherwise pkg itself."""
+    base = repo / pkg / "src"
+    if not base.is_dir():
+        base = repo / pkg
+    if not base.is_dir():
+        return ()
+    names: set[str] = set()
+    for entry in base.iterdir():
+        if entry.name.startswith(".") or should_skip_dir(entry.name):
+            continue
+        if entry.is_dir() and (entry / "__init__.py").is_file():
+            names.add(entry.name)
+        elif entry.is_file() and entry.suffix == ".py" and entry.stem != "__init__":
+            names.add(entry.stem)
+    return tuple(sorted(names))
+
+
+def _configured_known_first_party(pyproject: Path) -> tuple[str, ...]:
+    """`[tool.deptry] known_first_party` from the package's own
+    pyproject.toml. deptry's CLI `--known-first-party` replaces this
+    config value rather than merging with it (deptry's own
+    `read_configuration_from_pyproject_toml`: pyproject.toml overrides the
+    default, the command line overrides pyproject.toml -- the command line
+    always wins, it never merges), so callers must merge this in
+    themselves to avoid dropping a user's own setting."""
+    with pyproject.open("rb") as f:
+        data = tomllib.load(f)
+    tool = cast(dict[str, object], data.get("tool", {}))
+    deptry_cfg = cast(dict[str, object], tool.get("deptry", {}))
+    configured = deptry_cfg.get("known_first_party", [])
+    if not isinstance(configured, list):
+        return ()
+    return tuple(str(name) for name in cast(list[object], configured))
+
+
+def _known_first_party_names(repo: Path, pkg: Path, pyproject: Path) -> tuple[str, ...]:
+    """Union of the autodetected top-level package names and whatever the
+    package's own pyproject.toml already configures for deptry."""
+    return tuple(
+        sorted(
+            set(_top_level_package_names(repo, pkg))
+            | set(_configured_known_first_party(pyproject))
+        )
+    )
+
+
 def _nested_exclusions(pkg: Path, package_roots: Sequence[Path]) -> tuple[Path, ...]:
     """Subdirectories of pkg that are themselves package roots, relative to
     pkg, so a package's own scan does not re-scan a nested package."""
@@ -155,6 +208,7 @@ def _run_deptry_on(
     cmd = [
         "uv",
         "run",
+        "--isolated",
         "--directory",
         str(pkg_abs),
         "--with",
@@ -167,6 +221,8 @@ def _run_deptry_on(
     for rel in _nested_exclusions(pkg, package_roots):
         pattern = "^" + re.escape(rel.as_posix()) + "(/|$)"
         cmd += ["--extend-exclude", pattern]
+    for name in _known_first_party_names(repo, pkg, pkg_abs / "pyproject.toml"):
+        cmd += ["--known-first-party", name]
     json_out = tmp_dir / f"{pkg.as_posix().replace('/', '__') or 'root'}.json"
     cmd += ["--json-output", str(json_out)]
 
@@ -189,13 +245,17 @@ class UvMissing:
     all, never a silent skip."""
 
 
-def run_dependency_checks(repo: Path) -> _DepsResult | UvMissing:
+def run_dependency_checks(
+    repo: Path, excludes: Sequence[str] = ()
+) -> _DepsResult | UvMissing:
     """Scans every package root under repo with deptry, one uv-managed
-    environment per package."""
+    environment per package. `excludes` prunes matching trees out of
+    package-root discovery first, so a package root entirely under an
+    excluded tree gets no deptry run at all."""
     if shutil.which("uv") is None:
         return UvMissing()
 
-    package_roots = find_package_roots(repo)
+    package_roots = find_package_roots(repo, excludes)
     findings: list[_DepFinding] = []
     skipped: list[_Skipped] = []
     failed: list[_Failed] = []

{pymodularity-0.2.0 → pymodularity-0.3.1}/src/pymodularity/_metrics.py

@@ -254,7 +254,7 @@ def _compute_misplaced(
             # tie-break deterministically: smallest path among the ties
             other_dir = min(d for d, c in other_counts.items() if c == best)
             other = best
-        if other >= 3 and other > own:
+        if other >= 3 and 2 * other > len(partners):
             imported_by = sum(1 for p in importers if p.parent == other_dir)
             imports = sum(1 for p in imports_of if p.parent == other_dir)
             results.append(_Misplaced(f, own, other, other_dir, imported_by, imports))
@@ -434,8 +434,10 @@ def render_metrics(
     lines.append(
         "partners(f) = files f imports directly plus files importing f "
         + "directly; own = partners in f's own parent directory; other = "
-        + "the largest count among all other parent directories, naming "
-        + "that directory; misplaced when other >= 3 and other > own. "
+        + "the largest count among all other parent directories (a "
+        + "subdirectory of f's own directory counts as other), naming "
+        + "that directory; misplaced when other >= 3 and other exceeds "
+        + "half of all partners. "
         + "'imported by' = those other-directory partners that import f; "
         + "'imports' = those other-directory partners f imports; a "
         + "partner doing both counts in both, so the two may sum to more "