gitmole 0.17.0__tar.gz → 0.19.0__tar.gz

{gitmole-0.17.0 → gitmole-0.19.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gitmole
-Version: 0.17.0
+Version: 0.19.0
 Summary: Offline git repository analysis with a terminal report: hotspots, coupling, ownership, code age, secrets, repo health.
 License: MIT
 Project-URL: Homepage, https://github.com/antvinni/gitmole
@@ -44,7 +44,7 @@ Free. Any Stack. Local. Offline. Deterministic. Fast.
 - **Free.** MIT licence, no paid tier, no account, no token. A local clone needs no credentials, and a public `owner/repo` is cloned with plain git. Your `gh` login is only used for private repositories and for `owner/*`, and only when you ask for them. The tools it runs are open source too.
 - **Any stack.** It reads what every repository has: the git log, git blame and the files themselves.
 - **Local & Offline.** Everything runs against a clone on your machine. Nothing is uploaded, nothing phones home; the vulnerability database is a copy you download once.
-- **Deterministic.** No AI at runtime. Every finding is a plain rule over counts you can recompute by hand. The JSON export carries each finding's rule and the numbers it fired on. The same clone gives the same report every time. 
+- **Deterministic.** No AI at runtime. Every finding is a plain rule over counts you can recompute by hand. The JSON export carries each finding's rule, the numbers it fired on and, where a rule rests on a paper, the citation. The same commit gives the same bytes: gitmole's own CI runs it twice on every commit, compares the exports and attests the report.
 
 ## Install
 
@@ -73,12 +73,17 @@ gitmole . --markdown report.md         # the same report as a Markdown document
 gitmole . --json report.json           # every table, the watch list and the findings
 gitmole . --fail-on warning            # exit 3 if any finding is a warning or worse
 gitmole . --risk main --risk-threshold 10  # exit 3 if the files changed since main hold over 10% of the risk
+gitmole . --sarif gitmole.sarif        # the findings for GitHub code scanning or GitLab
+gitmole . --compare last.json          # what changed since an earlier --json export
+gitmole analysis-repo --no-run --hook  # a coding agent's edit hook: history's view of the files it just touched
 gitmole . --since 2y --full            # the current team, every row and column
 gitmole --clean                        # list what gitmole left behind, delete on a yes
 ```
 
 A CI job that runs `gitmole . --fail-on critical --markdown - >> "$GITHUB_STEP_SUMMARY"`
-blocks on secrets in source files and still posts the report. Every option:
+blocks on secrets in source files and still posts the report. The same
+scoring wires into Claude Code, Cursor, Gemini CLI and pre-commit as a hook
+that exits 2 over a threshold. Every option:
 [docs/cli.md](https://github.com/antvinni/gitmole/blob/main/docs/cli.md).
 
 ## What you get
@@ -87,9 +92,9 @@ Reports on repositories you know, each at a pinned commit, published as gitmole
 
 | Repository | Commit | Commits | Lines | gitmole run |
 |---|---|---:|---:|---:|
-| [curl](https://github.com/antvinni/gitmole/blob/main/docs/examples/curl.md) | [`540ee5b5`](https://github.com/curl/curl/commit/540ee5b560cc6e775e11317048a13cc7e355bf91) | 39,758 | 247,179 | 61 s |
-| [django](https://github.com/antvinni/gitmole/blob/main/docs/examples/django.md) | [`8cbdd4a8`](https://github.com/django/django/commit/8cbdd4a814397f81adf0129288f32b615bd1f94f) | 34,933 | 431,749 | 153 s |
-| [react](https://github.com/antvinni/gitmole/blob/main/docs/examples/react.md) | [`2b19aecd`](https://github.com/facebook/react/commit/2b19aecd0e9111b774fad0fad9862e50bcb5bc8a) | 21,703 | 681,078 | 138 s |
+| [curl](https://github.com/antvinni/gitmole/blob/main/docs/examples/curl.md) | [`540ee5b5`](https://github.com/curl/curl/commit/540ee5b560cc6e775e11317048a13cc7e355bf91) | 39,758 | 247,179 | 57 s |
+| [django](https://github.com/antvinni/gitmole/blob/main/docs/examples/django.md) | [`8cbdd4a8`](https://github.com/django/django/commit/8cbdd4a814397f81adf0129288f32b615bd1f94f) | 34,933 | 431,749 | 132 s |
+| [react](https://github.com/antvinni/gitmole/blob/main/docs/examples/react.md) | [`2b19aecd`](https://github.com/facebook/react/commit/2b19aecd0e9111b774fad0fad9862e50bcb5bc8a) | 21,703 | 681,078 | 151 s |
 
 Run times are one `gitmole CLONE` with every default step, on a MacBook Pro (M4, 16 GB).
 
@@ -110,6 +115,7 @@ you about a clone.
 | Which blocks of code appear more than once | [jscpd](https://github.com/kucherenko/jscpd) | brew |
 | Have secrets ever been committed | [betterleaks](https://github.com/betterleaks/betterleaks) | brew |
 | Do the dependencies have known vulnerabilities | [osv-scanner](https://github.com/google/osv-scanner), offline against a local copy of the OSV database | brew, plus a one-time database download |
+| How deeply nested is the code, what did the authors flag, what imports what | [tree-sitter](https://github.com/tree-sitter/py-tree-sitter) grammars for eleven languages | pip, opt-in with `gitmole[structure]` |
 
 Why these and not others: [docs/tools.md](https://github.com/antvinni/gitmole/blob/main/docs/tools.md).
 
@@ -121,6 +127,7 @@ Why these and not others: [docs/tools.md](https://github.com/antvinni/gitmole/bl
 - [Example reports](https://github.com/antvinni/gitmole/tree/main/docs/examples): curl, django and react at pinned commits, regenerated by `bin/render-examples`.
 - [Validation](https://github.com/antvinni/gitmole/blob/main/docs/validation.md): the watch list against other ways of ranking the same files at six cut-offs on three repositories.
 - [Why these tools](https://github.com/antvinni/gitmole/blob/main/docs/tools.md): the rationale, what was left out, licences.
+- [References](https://github.com/antvinni/gitmole/blob/main/docs/references.md): the research and tools gitmole's rules are built on.
 - [Development](https://github.com/antvinni/gitmole/blob/main/docs/development.md): setup, tests, releases, code layout.
 - [Contributing](https://github.com/antvinni/gitmole/blob/main/CONTRIBUTING.md): bugs, ideas, pull requests, security reports.
 

{gitmole-0.17.0 → gitmole-0.19.0}/README.md

@@ -11,7 +11,7 @@ Free. Any Stack. Local. Offline. Deterministic. Fast.
 - **Free.** MIT licence, no paid tier, no account, no token. A local clone needs no credentials, and a public `owner/repo` is cloned with plain git. Your `gh` login is only used for private repositories and for `owner/*`, and only when you ask for them. The tools it runs are open source too.
 - **Any stack.** It reads what every repository has: the git log, git blame and the files themselves.
 - **Local & Offline.** Everything runs against a clone on your machine. Nothing is uploaded, nothing phones home; the vulnerability database is a copy you download once.
-- **Deterministic.** No AI at runtime. Every finding is a plain rule over counts you can recompute by hand. The JSON export carries each finding's rule and the numbers it fired on. The same clone gives the same report every time. 
+- **Deterministic.** No AI at runtime. Every finding is a plain rule over counts you can recompute by hand. The JSON export carries each finding's rule, the numbers it fired on and, where a rule rests on a paper, the citation. The same commit gives the same bytes: gitmole's own CI runs it twice on every commit, compares the exports and attests the report.
 
 ## Install
 
@@ -40,12 +40,17 @@ gitmole . --markdown report.md         # the same report as a Markdown document
 gitmole . --json report.json           # every table, the watch list and the findings
 gitmole . --fail-on warning            # exit 3 if any finding is a warning or worse
 gitmole . --risk main --risk-threshold 10  # exit 3 if the files changed since main hold over 10% of the risk
+gitmole . --sarif gitmole.sarif        # the findings for GitHub code scanning or GitLab
+gitmole . --compare last.json          # what changed since an earlier --json export
+gitmole analysis-repo --no-run --hook  # a coding agent's edit hook: history's view of the files it just touched
 gitmole . --since 2y --full            # the current team, every row and column
 gitmole --clean                        # list what gitmole left behind, delete on a yes
 ```
 
 A CI job that runs `gitmole . --fail-on critical --markdown - >> "$GITHUB_STEP_SUMMARY"`
-blocks on secrets in source files and still posts the report. Every option:
+blocks on secrets in source files and still posts the report. The same
+scoring wires into Claude Code, Cursor, Gemini CLI and pre-commit as a hook
+that exits 2 over a threshold. Every option:
 [docs/cli.md](https://github.com/antvinni/gitmole/blob/main/docs/cli.md).
 
 ## What you get
@@ -54,9 +59,9 @@ Reports on repositories you know, each at a pinned commit, published as gitmole
 
 | Repository | Commit | Commits | Lines | gitmole run |
 |---|---|---:|---:|---:|
-| [curl](https://github.com/antvinni/gitmole/blob/main/docs/examples/curl.md) | [`540ee5b5`](https://github.com/curl/curl/commit/540ee5b560cc6e775e11317048a13cc7e355bf91) | 39,758 | 247,179 | 61 s |
-| [django](https://github.com/antvinni/gitmole/blob/main/docs/examples/django.md) | [`8cbdd4a8`](https://github.com/django/django/commit/8cbdd4a814397f81adf0129288f32b615bd1f94f) | 34,933 | 431,749 | 153 s |
-| [react](https://github.com/antvinni/gitmole/blob/main/docs/examples/react.md) | [`2b19aecd`](https://github.com/facebook/react/commit/2b19aecd0e9111b774fad0fad9862e50bcb5bc8a) | 21,703 | 681,078 | 138 s |
+| [curl](https://github.com/antvinni/gitmole/blob/main/docs/examples/curl.md) | [`540ee5b5`](https://github.com/curl/curl/commit/540ee5b560cc6e775e11317048a13cc7e355bf91) | 39,758 | 247,179 | 57 s |
+| [django](https://github.com/antvinni/gitmole/blob/main/docs/examples/django.md) | [`8cbdd4a8`](https://github.com/django/django/commit/8cbdd4a814397f81adf0129288f32b615bd1f94f) | 34,933 | 431,749 | 132 s |
+| [react](https://github.com/antvinni/gitmole/blob/main/docs/examples/react.md) | [`2b19aecd`](https://github.com/facebook/react/commit/2b19aecd0e9111b774fad0fad9862e50bcb5bc8a) | 21,703 | 681,078 | 151 s |
 
 Run times are one `gitmole CLONE` with every default step, on a MacBook Pro (M4, 16 GB).
 
@@ -77,6 +82,7 @@ you about a clone.
 | Which blocks of code appear more than once | [jscpd](https://github.com/kucherenko/jscpd) | brew |
 | Have secrets ever been committed | [betterleaks](https://github.com/betterleaks/betterleaks) | brew |
 | Do the dependencies have known vulnerabilities | [osv-scanner](https://github.com/google/osv-scanner), offline against a local copy of the OSV database | brew, plus a one-time database download |
+| How deeply nested is the code, what did the authors flag, what imports what | [tree-sitter](https://github.com/tree-sitter/py-tree-sitter) grammars for eleven languages | pip, opt-in with `gitmole[structure]` |
 
 Why these and not others: [docs/tools.md](https://github.com/antvinni/gitmole/blob/main/docs/tools.md).
 
@@ -88,6 +94,7 @@ Why these and not others: [docs/tools.md](https://github.com/antvinni/gitmole/bl
 - [Example reports](https://github.com/antvinni/gitmole/tree/main/docs/examples): curl, django and react at pinned commits, regenerated by `bin/render-examples`.
 - [Validation](https://github.com/antvinni/gitmole/blob/main/docs/validation.md): the watch list against other ways of ranking the same files at six cut-offs on three repositories.
 - [Why these tools](https://github.com/antvinni/gitmole/blob/main/docs/tools.md): the rationale, what was left out, licences.
+- [References](https://github.com/antvinni/gitmole/blob/main/docs/references.md): the research and tools gitmole's rules are built on.
 - [Development](https://github.com/antvinni/gitmole/blob/main/docs/development.md): setup, tests, releases, code layout.
 - [Contributing](https://github.com/antvinni/gitmole/blob/main/CONTRIBUTING.md): bugs, ideas, pull requests, security reports.
 

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/__init__.py

@@ -1,3 +1,3 @@
 """gitmole: offline git repository analysis with a terminal report."""
 
-__version__ = "0.17.0"
+__version__ = "0.19.0"

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/cli.py

@@ -592,7 +592,7 @@ def _render(out_dir: str, console: Console, ui: Console, args, err: Console) ->
             return 2
         comparison = _compare.compare(before, report, found)
     if args.json:
-        _write(json.dumps(render.to_json(report, found, risk=risk, compare=comparison), indent=2) + "\n", args.json, console)
+        _write(render.dumps_json(report, found, risk=risk, compare=comparison), args.json, console)
     if args.markdown:
         _write(render.markdown(report, found, full=args.full, risk=risk, base=args.risk, compare=comparison), args.markdown, console)
     if args.sarif:

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/compare.py

@@ -65,4 +65,13 @@ def compare(before: dict, report: dict, found: list, top: int = watch.WATCH_TOP)
             "watch_entered": [f for f in after_watch if f not in before_watch], "watch_left": [f for f in before_watch if f not in after_watch],
             "tally": {"before": _tally(before.get("findings") or []), "after": _tally(found)},
             "before": {"commit": (meta_b.get("run") or {}).get("commit"), "date": meta_b.get("last_date"),
-                       "options_differ": _options_differ(meta_b, meta_a)}}
+                       "options_differ": _options_differ(meta_b, meta_a), "database": _database_changed(before, report)}}
+
+
+def _database_changed(before: dict, report: dict):
+    """{before, after} dates when the two runs scanned against different snapshots of the OSV database,
+    else None: a new advisory changes the vulnerable-dependency finding without any change to the code."""
+    b, a = before.get("dependencies") or {}, report.get("dependencies") or {}
+    if b.get("database_digest") and a.get("database_digest") and b["database_digest"] != a["database_digest"]:
+        return {"before": b.get("database_date"), "after": a.get("database_date")}
+    return None

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/deps.py

@@ -58,6 +58,28 @@ def database_date(base: str = None) -> str | None:
     return dt.datetime.fromtimestamp(newest).date().isoformat() if newest else None
 
 
+def database_digest(base: str = None) -> str | None:
+    """A digest of the local database snapshot: every file under the cache directories, by path, size
+    and modification time. Two runs against the same snapshot agree; a refresh changes it, which is how
+    --compare can say a dependency finding moved because the database did. None when there is none."""
+    import hashlib
+    base = cache_dir() if base is None else base
+    entries = []
+    for name in CACHE_DIRS:
+        root = os.path.join(base, name)
+        for dirpath, _, files in os.walk(root):
+            for f in files:
+                full = os.path.join(dirpath, f)
+                try:
+                    st = os.stat(full)
+                except OSError:
+                    continue
+                entries.append(f"{os.path.relpath(full, base)}\0{st.st_size}\0{int(st.st_mtime)}")
+    if not entries:
+        return None
+    return hashlib.sha256("\n".join(sorted(entries)).encode("utf-8", "surrogateescape")).hexdigest()[:16]
+
+
 def _key(version: str) -> tuple:
     return tuple(int(n) for n in _NUMBER.findall(version or ""))
 
@@ -94,7 +116,7 @@ def _score(groups: list, vulns: list) -> float | None:
 
 
 _WORDS = {"CRITICAL": 9.5, "HIGH": 8.0, "MODERATE": 5.5, "MEDIUM": 5.5, "LOW": 2.0}
-MALICIOUS_PREFIX = "MAL-"   # OpenSSF malicious-packages records, in the same OSV database; they carry no CVSS
+MALICIOUS_PREFIX = "MAL-"   # OpenSSF malicious-packages records (ossf/malicious-packages), in the same OSV database; they carry no CVSS
 
 
 def is_malicious(vulns: list) -> bool:
@@ -185,6 +207,7 @@ def main(argv=None) -> int:
             return 1
         result = summarise(data, os.getcwd())
         result["database_date"] = database_date()
+        result["database_digest"] = database_digest()
     else:
         print(f"deps.py: osv-scanner exited {proc.returncode}; no report written", file=sys.stderr)
         return proc.returncode

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/findings.py

@@ -1,4 +1,5 @@
-"""Heuristics that turn a loaded report into a short list of flagged findings."""
+"""Heuristics that turn a loaded report into a short list of flagged findings. A rule that rests on a paper
+carries the short citation in its rule dict's `ref` (see REFS); docs/references.md has the full entries."""
 from __future__ import annotations
 
 import re
@@ -11,11 +12,23 @@ PLACEHOLDER_NAMES = {"your name", "unknown", "root", "user"}
 PLACEHOLDER_EMAIL = re.compile(r"(@example\.(com|org|net)$|^you@|^user@|^root@|@localhost$)")
 
 
+# The paper a rule rests on, as the short citation the rule dict carries in `ref`; the full entries are in
+# docs/references.md. A rule that is gitmole's own heuristic has none.
+REFS = {"minor_contributors": "Bird et al., FSE 2011", "tangled_commits": "Herzig and Zeller, MSR 2013",
+        "brain_methods": "Lanza and Marinescu, 2006", "tight_coupling": "Gall, Hajek and Jazayeri, ICSM 1998",
+        "hotspot_dominance": "Tornhill, Your Code as a Crime Scene, 2024", "trojan_source": "Boucher and Anderson, USENIX Security 2023",
+        "debt_in_hotspots": "Maldonado and Shihab, MTD 2015", "hidden_coupling": "Ajienka and Capiluppi, JSS 2017",
+        "unreferenced_files": "Romano et al., TSE 2020", "signoff_by_co_author": "Linux kernel, Documentation/process/coding-assistants.rst",
+        "deep_nesting": "SonarSource cognitive complexity; CodeScene code health"}
+
+
 def _f(severity: str, title: str, statement: str, advice: str, rule: dict, evidence: dict) -> dict:
     """A finding: the facts, then the next step. `detail` is the two joined for anyone reading the
     JSON; `advice` says which part is the step so the report can show it on its own line. `rule` is
     the rule's id and the thresholds it fired on, `evidence` the numbers they were compared with:
     between them a reader of the JSON can check the finding without reading this file."""
+    if rule.get("id") in REFS and "ref" not in rule:
+        rule = {**rule, "ref": REFS[rule["id"]]}
     return {"severity": severity, "title": title, "detail": f"{statement.rstrip()} {advice}", "advice": advice,
             "rule": rule, "evidence": evidence}
 
@@ -1088,10 +1101,118 @@ def signoff_by_co_author(report: dict, min_commits: int = 2) -> list:
                evidence={"identities": rows[:10]})]
 
 
+def _pool_files(report: dict) -> list:
+    """The source files still in the tree that no classifier reason sets aside."""
+    from . import classify
+    cls = classify.Classifier(report)
+    return sorted(f for f in _tree(report) if cls.reason(f) is None)
+
+
+def _authors_of(report: dict, files: list, key: str = "is_author") -> dict:
+    wanted = set(files)
+    out = {f: set() for f in files}
+    for r in report.get("doa") or []:
+        if r["entity"] in wanted and r.get(key):
+            out[r["entity"]].add(r["author"])
+    return out
+
+
+def truck_factor(report: dict, min_files: int = 20, area_files: int = 10) -> list:
+    """Avelino et al.'s truck factor over the degree of authorship: how many people have to leave before
+    more than half the source files have no author. One is a warning, two a note. Changes rather than
+    lines, and a creator's bonus, so it can disagree with the surviving-code share, which the bus-factor
+    finding reads; the finding says so when it does. Also per area, and with knowledge halving every
+    five months."""
+    if not report.get("doa"):
+        return []
+    files = _pool_files(report)
+    authored = {f: a for f, a in _authors_of(report, files).items()}
+    if len(files) < min_files:
+        return []
+    tf, removed, share = knowledge.truck_factor(authored)
+    tf_d, removed_d, _ = knowledge.truck_factor(_authors_of(report, files, "is_author_decayed"))
+    depth = knowledge.depth_for(files)
+    areas = {}
+    for f in files:
+        areas.setdefault(knowledge._area(f, depth), []).append(f)
+    lone = []
+    for area, fs in sorted(areas.items()):
+        if len(fs) >= area_files and area != knowledge.ROOT:
+            n, who, _ = knowledge.truck_factor({f: authored[f] for f in fs})
+            if n == 1:
+                lone.append((area, who[0]))
+    if tf > 2 and not lone:
+        return []
+    orphans = round(share * len(files))
+    statement = (f"Truck factor {tf}: without {textfmt.join_and(removed)}, {orphans} of the {len(files)} source files ({_pct(orphans, len(files))}) "
+                 f"have no author left.")
+    if tf_d != tf:
+        statement += f" With knowledge halving every five months it is {tf_d} ({textfmt.join_and(removed_d)})."
+    if lone:
+        statement += " Areas with a truck factor of one: " + ", ".join(f"{a} ({w})" for a, w in lone[:5]) + (f" and {len(lone) - 5} more" if len(lone) > 5 else "") + "."
+    shares = report.get("theseus_authors") or {}
+    if shares and removed:
+        top, lines = max(shares.items(), key=lambda kv: kv[1])
+        if top != removed[0]:
+            statement += f" The surviving code's largest share is {top}'s ({_pct(lines, sum(shares.values()))}), which the bus-factor finding reads."
+    first_area = next((a for a, w in lone if w == removed[0]), lone[0][0] if lone else None)
+    advice = f"Pair someone with {removed[0]}" + (f" on {first_area}" if first_area else "") + " first; they author most of what would be left without an author."
+    return [_f("warning" if tf == 1 else "info", "Truck factor", statement, advice,
+               rule={"id": "truck_factor", "doa_author_share": 0.75, "doa_floor": 3.293, "orphan_share": 0.5, "decay_months": 5,
+                     "ref": "Avelino et al., ICPC 2016"},
+               evidence={"truck_factor": tf, "removed": removed, "truck_factor_decayed": tf_d, "removed_decayed": removed_d,
+                         "files": len(files), "orphaned": orphans, "areas": [{"area": a, "author": w} for a, w in lone[:10]]})]
+
+
+def authors_gone(report: dict, min_files: int = 5) -> list:
+    """Files whose every author by degree of authorship has stopped committing, while others still
+    change them: "creator left, editors remain", knowledge the blame share cannot show."""
+    if not report.get("doa"):
+        return []
+    months = report["meta"].get("gone_months", loss.DEFAULT_MONTHS)
+    gone = {g["name"] for g in loss.gone(report, months)}
+    fresh = {a["entity"] for a in report.get("age") or [] if a["age-months"] < 12}
+    files = [f for f in _pool_files(report) if f in fresh]
+    authored = _authors_of(report, files)
+    left = [(f, sorted(a)) for f, a in authored.items() if a and a <= gone]
+    if len(left) < min_files:
+        return []
+    listed = "; ".join(f"{f} ({textfmt.join_and(a)})" for f, a in left[:5]) + (f" and {len(left) - 5} more" if len(left) > 5 else "")
+    return [_f("info", "Files whose authors have left", f"{len(left)} source files changed in the last year have no author still committing: {listed}.",
+               f"Make the people who edit {left[0][0]} its authors: review its design with them and write down what only {left[0][1][0]} knew.",
+               rule={"id": "authors_gone", "gone_months": months, "min_files": min_files, "ref": "Avelino et al., ICPC 2016"},
+               evidence={"count": len(left), "files": [{"file": f, "authors": a} for f, a in left[:10]]})]
+
+
+def component_coupling(report: dict, min_degree: int = 30) -> list:
+    """Components (top-level directories, or the level below a lone src/) that change together in a
+    large share of their changes: coupling at the level of the architecture, where two files in one
+    directory is only a layout."""
+    rows = report.get("components") or []
+    if not rows:
+        return []
+    depth = knowledge.depth_for(list(_tree(report)) or [r["entity"] + "x" for r in rows])
+
+    def aside(c):
+        probe = c + "x.py"
+        return filetypes.is_test_path(probe) or filetypes.is_sample_path(probe) or filetypes.is_doc_path(probe) or filetypes.is_vendor_path(probe)
+    pairs = [r for r in rows if r["depth"] == depth and r["degree"] >= min_degree and not aside(r["entity"]) and not aside(r["coupled"])]
+    if not pairs:
+        return []
+    listed = "; ".join(f"{p['entity']} and {p['coupled']} change together in {p['degree']}% of their changes ({p['shared']} shared)" for p in pairs[:3])
+    more = f" ({len(pairs) - 3} more pairs)" if len(pairs) > 3 else ""
+    first = pairs[0]
+    return [_f("info", "Components that change together", f"{listed}{more}.",
+               f"Look at what {first['entity']} and {first['coupled']} share: a change that keeps landing in both is an interface nobody named.",
+               rule={"id": "component_coupling", "min_degree": min_degree, "depth": depth, "ref": "Tornhill, Your Code as a Crime Scene, 2024"},
+               evidence={"pairs": [{"a": p["entity"], "b": p["coupled"], "degree": p["degree"], "shared": p["shared"]} for p in pairs[:10]]})]
+
+
 RULES = [dormant, secrets_found, credential_files, vulnerable_dependencies, placeholder_identity, bus_factor, sizer_concerns, hotspot_dominance, bug_magnets,
          minor_contributors, reverts, brain_methods, complexity_growth, tight_coupling, duplication, stale_files, knowledge_islands, knowledge_loss,
          sweeping_commits, tangled_commits, hygiene_findings, debt_in_hotspots, deep_nesting, hidden_coupling, unreferenced_files,
-         agent_approval_disabled, agent_local_settings, mcp_literal_env, agent_instructions_drift, signoff_by_co_author]
+         agent_approval_disabled, agent_local_settings, mcp_literal_env, agent_instructions_drift, signoff_by_co_author,
+         truck_factor, authors_gone, component_coupling]
 
 
 def evaluate(report: dict) -> list:

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/hook.py

@@ -1,4 +1,5 @@
-"""The agent-hook gate: `gitmole OUT_DIR --no-run --hook [--risk-threshold N] [-- FILE...]`.
+"""The agent-hook gate: `gitmole OUT_DIR --no-run --hook [--risk-threshold N] [-- FILE...]`, after Codacy's
+argument for deterministic findings before inference and Zimmermann et al.'s co-change recommendations.
 
 Deterministic findings should run before inference, so the model reasons over a short list rather
 than rediscovering known issues. A linter says what is wrong in the diff; gitmole says what history

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/hygiene.py

@@ -1,5 +1,6 @@
 """Repository hygiene from the clone alone: the checks OpenSSF Scorecard and the OSPS Baseline make
-through the GitHub API, done with file and git reads.
+through the GitHub API, done with file and git reads; Trojan Source after Boucher and Anderson (USENIX
+Security 2023).
 
 Runs as a pipeline step, `python -m gitmole.hygiene OUT_DIR`, from inside the repository, and writes
 hygiene.json; findings.py turns it into findings. Each check keys on a convention of the ecosystem (a

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/knowledge.py

@@ -78,3 +78,37 @@ def islands(areas_list: list, min_lines: int = 200, min_share: float = 0.9) -> l
         if n / a["lines"] >= min_share:
             out.append({"area": a["area"], "owner": owner, "share": round(100 * n / a["lines"]), "lines": a["lines"]})
     return out
+
+
+def truck_factor(authors_of: dict, orphan_share: float = 0.5) -> tuple:
+    """Avelino et al.'s truck factor: remove the person who authors the most files, again and again, until
+    more than half the files have no author left. (the number removed, their names in order, the share
+    orphaned at the end). authors_of: {file: set of names}."""
+    files = list(authors_of)
+    if not files:
+        return 0, [], 0.0
+    remaining = {f: set(a) for f, a in authors_of.items()}
+    removed = []
+
+    def orphaned():
+        return sum(1 for a in remaining.values() if not a) / len(files)
+    while orphaned() <= orphan_share:
+        counts = {}
+        for a in remaining.values():
+            for who in a:
+                counts[who] = counts.get(who, 0) + 1
+        if not counts:
+            break
+        top = min(counts, key=lambda w: (-counts[w], w))
+        removed.append(top)
+        for a in remaining.values():
+            a.discard(top)
+    return len(removed), removed, orphaned()
+
+
+def depth_for(paths: list, dominant: float = 0.8) -> int:
+    """1 for top-level directories, 2 when one top-level directory holds `dominant` of the files (a lone
+    src/), as the knowledge map chooses."""
+    from collections import Counter
+    tops = Counter(_area(p, 1) for p in paths)
+    return 2 if tops and tops.most_common(1)[0][1] >= dominant * len(paths) and tops.most_common(1)[0][0] != ROOT else 1

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/leaks.py

@@ -228,6 +228,9 @@ def scan_unreachable(repo: str, out_dir: str, found: dict) -> list:
         sha = os.path.basename(r.get("File") or "")
         r["File"] = f"(unreachable blob {sha[:12]})"
         r["Commit"] = ""
+        # betterleaks' own fingerprint names the scratch path the blob was written to; this one names the blob,
+        # so it is the same in every run and can go into .betterleaksignore
+        r["Fingerprint"] = f"unreachable:{sha}:{r.get('RuleID', '')}:{r.get('StartLine', '')}"
     return rows
 
 

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/load.py

@@ -82,7 +82,9 @@ def parse_scc(text: str, types=None) -> dict:
 
 
 NUMERIC_COLUMNS = {"n-revs", "degree", "average-revs", "n-authors", "age-months", "added", "deleted", "n-fixes", "recent-fixes", "tiny-revs",
-                   "minor", "soc", "partners", "n-sets", "with-tests", "periods"}
+                   "minor", "soc", "partners", "n-sets", "with-tests", "periods", "fa", "dl", "ac", "is_author", "is_author_decayed", "late",
+                   "depth", "shared"}
+FLOAT_COLUMNS = {"doa", "doa_decayed", "hcm"}
 
 
 def parse_maat_csv(text: str) -> list:
@@ -91,10 +93,17 @@ def parse_maat_csv(text: str) -> list:
         return []
     out = []
     for row in csv.DictReader(io.StringIO(text)):
-        out.append({k: (_num(v) if k in NUMERIC_COLUMNS else v) for k, v in row.items()})
+        out.append({k: (_num(v) if k in NUMERIC_COLUMNS else _float(v) if k in FLOAT_COLUMNS else v) for k, v in row.items()})
     return out
 
 
+def _float(v):
+    try:
+        return float(v)
+    except (TypeError, ValueError):
+        return 0.0
+
+
 def _num(v):
     """An int for a numeric cell; 0 for a missing, empty or garbage one (a row cut short by a killed step)."""
     try:
@@ -182,6 +191,7 @@ def parse_functions(text: str) -> list:
         anonymous = name in ("", "(anonymous)")
         rows.append({"file": _rel(r[6]), "function": textfmt.cut(label if anonymous and label else name, NAME_CAP) or "(anonymous)", "anonymous": anonymous,
                      "ccn": _num(r[1]), "nloc": _num(r[0]), "params": _num(r[3]), "start": _num(r[9]), "end": _num(r[10]), "suspect": suspect})
+    rows.sort(key=lambda f: (f["file"], f["start"], f["end"], f["function"]))   # the function step works in parallel; the order is this one
     return rows
 
 
@@ -239,6 +249,8 @@ def parse_secrets(text: str) -> list:
             value, placeholder = None, False
         out.append({"rule": r.get("RuleID", ""), "file": r.get("File", ""), "commit": r.get("Commit", "")[:7], "line": r.get("StartLine"),
                     "fingerprint": r.get("Fingerprint", ""), "value": value, "placeholder": placeholder})
+    # betterleaks scans in parallel and does not promise an order; everything downstream reads the rows in this one
+    out.sort(key=lambda r: (r["file"], r["commit"], r["line"] or 0, r["rule"], r["fingerprint"]))
     return out
 
 
@@ -251,6 +263,8 @@ def parse_dependencies(data) -> dict:
     if data["status"] == "scanned":
         out.update({"sources": data.get("sources") or [], "packages": _num(data.get("packages")),
                     "vulnerable": data.get("vulnerable") or [], "database_date": data.get("database_date")})
+        if data.get("database_digest"):
+            out["database_digest"] = data["database_digest"]
     elif data["status"] == "no-database":
         out["download"] = data.get("download") or ""
     return out
@@ -313,6 +327,9 @@ def load_report(out_dir: str, nested: bool = True) -> dict:
         "soc": parse_maat_csv(_read(out_dir, "maat-soc.csv")),   # sum of coupling; empty for an output directory from before 0.11
         "tests": parse_maat_csv(_read(out_dir, "maat-tests.csv")),   # test co-change per production file; empty before 0.12
         "entropy": parse_maat_csv(_read(out_dir, "maat-entropy.csv")),   # Hassan's change entropy per file; empty before 0.13
+        "doa": parse_maat_csv(_read(out_dir, "maat-doa.csv")),   # degree of authorship per file and person; empty before 0.19
+        "latenight": parse_maat_csv(_read(out_dir, "maat-latenight.csv")),
+        "components": parse_maat_csv(_read(out_dir, "maat-components.csv")),
         "authors": parse_maat_csv(_read(out_dir, "maat-authors.csv")),
         "age": parse_maat_csv(_read(out_dir, "maat-age.csv")),
         "ownership": ownership,

{gitmole-0.17.0 → gitmole-0.19.0}/gitmole/maat.py

@@ -1,5 +1,7 @@
 #!/usr/bin/env python3
-"""Change analysis over a git log export, in the layout code-maat produced.
+"""Change analysis over a git log export, in the layout code-maat produced: revisions, coupling and sum of
+coupling after Adam Tornhill's code-maat, minor contributors after Bird et al. (FSE 2011), change entropy
+after Hassan (ICSE 2009), oversized fixes after Herzig and Zeller (MSR 2013); see docs/references.md.
 
 Standalone on purpose: gitmole runs it as a pipeline step with
 `python3 maat.py LOG OUT_DIR [--aliases META_JSON]` and it must not need the
@@ -379,6 +381,102 @@ def entropy(commits: list, now: str = None, decay: float = ENTROPY_DECAY) -> lis
     return rows
 
 
+DOA_DECAY_MONTHS = 5   # JetBrains' Bus Factor Explorer: knowledge halves every five months
+DOA_AUTHOR_SHARE = 0.75
+DOA_FLOOR = 3.293
+
+
+def _doa(fa: int, dl: float, ac: float) -> float:
+    """Avelino et al.'s degree of authorship: a creator's bonus, the author's own changes, and a
+    logarithmic dilution by everyone else's."""
+    return 3.293 + 1.098 * fa + 0.164 * dl - 0.321 * math.log(1 + ac)
+
+
+def doa(commits: list, now: str = None) -> list:
+    """Per file and person: created it (the first commit that added lines to it; a pure move creates
+    nothing), their changes, others' changes, the degree of authorship, and whether they count as an
+    author of it (DOA at least three quarters of the file's highest and at least 3.293), undecayed and
+    with knowledge halving every five months. Changes, not lines, so a reformat transfers nothing."""
+    now = dt.date.fromisoformat(now or dt.date.today().isoformat())
+    changes, decayed, first = defaultdict(Counter), defaultdict(Counter), {}
+    for c in commits:
+        weight = 0.5 ** (max(0, (now - dt.date.fromisoformat(c["date"])).days) / 30.44 / DOA_DECAY_MONTHS)
+        for p, added, _ in c["files"]:
+            for who in people(c):
+                changes[p][who] += 1
+                decayed[p][who] += weight
+            if added > 0 and (p not in first or (c["date"], c.get("time", "")) < first[p][0]):
+                first[p] = ((c["date"], c.get("time", "")), c["author"])
+    rows = []
+    for p, per in changes.items():
+        total, total_d = sum(per.values()), sum(decayed[p].values())
+        creator = first.get(p, (None, None))[1]
+        scores = {}
+        for who, n in per.items():
+            fa = int(who == creator)
+            scores[who] = (fa, n, total - n, _doa(fa, n, total - n), _doa(fa, decayed[p][who], total_d - decayed[p][who]))
+        top, top_d = max(s[3] for s in scores.values()), max(s[4] for s in scores.values())
+        for who, (fa, n, others, value, value_d) in sorted(scores.items()):
+            rows.append({"entity": p, "author": who, "fa": fa, "dl": n, "ac": others, "doa": round(value, 4), "doa_decayed": round(value_d, 4),
+                         "is_author": int(value >= DOA_FLOOR and value >= DOA_AUTHOR_SHARE * top),
+                         "is_author_decayed": int(value_d >= DOA_FLOOR and value_d >= DOA_AUTHOR_SHARE * top_d)})
+    rows.sort(key=lambda r: (r["entity"], r["author"]))
+    return rows
+
+
+def _local_hour(stamp: str):
+    try:
+        return dt.datetime.fromisoformat(stamp[:-1] + "+00:00" if stamp.endswith("Z") else stamp).hour if len(stamp) > 10 else None
+    except ValueError:
+        return None
+
+
+LATE_HOURS = range(0, 4)   # Eyolfson, Tan and Lam: commits between midnight and 4 am, in the author's own time, were buggier
+
+
+def latenight(commits: list) -> list:
+    """Per file: its revisions, and how many were committed between midnight and 4 am in the author's
+    own offset. A reason beside a file, never a rank: the effect is far weaker than churn or ownership."""
+    revs, late = Counter(), Counter()
+    for c in commits:
+        hour = _local_hour(c.get("time") or "")
+        for p, _, _ in c["files"]:
+            revs[p] += 1
+            late[p] += hour is not None and hour in LATE_HOURS
+    rows = [{"entity": p, "n-revs": n, "late": late[p]} for p, n in revs.items()]
+    rows.sort(key=lambda r: (-r["late"], r["entity"]))
+    return rows
+
+
+def component(path: str, depth: int) -> str:
+    dirs = path.split("/")[:-1]
+    return "/".join(dirs[:depth]) + "/" if dirs else "(root files)"
+
+
+def components(commits: list, min_shared: int = 10, min_degree: int = 20, max_components: int = 10) -> list:
+    """Coupling between components, the files truncated to their first one and two directories, over
+    the logical changes: two files in one directory changing together is a layout, `auth/` and
+    `billing/` changing together 40% of the time is architecture. A change that spans more than
+    `max_components` components is a sweep and couples nothing."""
+    out = []
+    for depth in (1, 2):
+        revs, shared = Counter(), Counter()
+        for c in changesets(commits):
+            comps = sorted({component(p, depth) for p, _, _ in c["files"]} - {"(root files)"})
+            if not comps or len(comps) > max_components:
+                continue
+            revs.update(comps)
+            for a, b in itertools.combinations(comps, 2):
+                shared[(a, b)] += 1
+        for (a, b), n in shared.items():
+            avg = (revs[a] + revs[b]) / 2
+            degree = int(math.floor(100 * n / avg + 0.5))
+            if n >= min_shared and degree >= min_degree:
+                out.append({"depth": depth, "entity": a, "coupled": b, "degree": degree, "shared": n, "average-revs": int(math.floor(avg + 0.5))})
+    out.sort(key=lambda r: (r["depth"], -r["degree"], -r["shared"], r["entity"], r["coupled"]))
+    return out
+
+
 RECENT_MONTHS = 6
 OVERSIZED_PERCENTILE = 0.99   # a fix changing more lines than this share of the history's commits credits nothing
 OVERSIZED_FLOOR = 500         # ...and never under this many lines, so a small repository's percentile does not bite
@@ -557,8 +655,11 @@ ANALYSES = {
     "entity-ownership": (entity_ownership, ["entity", "author", "added", "deleted"]),
     "fixes": (fixes, ["entity", "n-fixes", "last-fix", "recent-fixes"]),
     "entropy": (entropy, ["entity", "periods", "hcm"]),
+    "doa": (doa, ["entity", "author", "fa", "dl", "ac", "doa", "doa_decayed", "is_author", "is_author_decayed"]),
+    "latenight": (latenight, ["entity", "n-revs", "late"]),
+    "components": (components, ["depth", "entity", "coupled", "degree", "shared", "average-revs"]),
 }
-NEEDS_NOW = {"age", "fixes", "entropy"}
+NEEDS_NOW = {"age", "fixes", "entropy", "doa"}
 
 
 def aliases_from_meta(path: str) -> dict: