gitmole 0.16.0__tar.gz → 0.18.0__tar.gz

{gitmole-0.16.0 → gitmole-0.18.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gitmole
-Version: 0.16.0
+Version: 0.18.0
 Summary: Offline git repository analysis with a terminal report: hotspots, coupling, ownership, code age, secrets, repo health.
 License: MIT
 Project-URL: Homepage, https://github.com/antvinni/gitmole
@@ -44,7 +44,7 @@ Free. Any Stack. Local. Offline. Deterministic. Fast.
 - **Free.** MIT licence, no paid tier, no account, no token. A local clone needs no credentials, and a public `owner/repo` is cloned with plain git. Your `gh` login is only used for private repositories and for `owner/*`, and only when you ask for them. The tools it runs are open source too.
 - **Any stack.** It reads what every repository has: the git log, git blame and the files themselves.
 - **Local & Offline.** Everything runs against a clone on your machine. Nothing is uploaded, nothing phones home; the vulnerability database is a copy you download once.
-- **Deterministic.** No AI at runtime. Every finding is a plain rule over counts you can recompute by hand. The JSON export carries each finding's rule and the numbers it fired on. The same clone gives the same report every time. 
+- **Deterministic.** No AI at runtime. Every finding is a plain rule over counts you can recompute by hand. The JSON export carries each finding's rule, the numbers it fired on and, where a rule rests on a paper, the citation. The same commit gives the same bytes: gitmole's own CI runs it twice on every commit, compares the exports and attests the report.
 
 ## Install
 
@@ -73,12 +73,17 @@ gitmole . --markdown report.md         # the same report as a Markdown document
 gitmole . --json report.json           # every table, the watch list and the findings
 gitmole . --fail-on warning            # exit 3 if any finding is a warning or worse
 gitmole . --risk main --risk-threshold 10  # exit 3 if the files changed since main hold over 10% of the risk
+gitmole . --sarif gitmole.sarif        # the findings for GitHub code scanning or GitLab
+gitmole . --compare last.json          # what changed since an earlier --json export
+gitmole analysis-repo --no-run --hook  # a coding agent's edit hook: history's view of the files it just touched
 gitmole . --since 2y --full            # the current team, every row and column
 gitmole --clean                        # list what gitmole left behind, delete on a yes
 ```
 
 A CI job that runs `gitmole . --fail-on critical --markdown - >> "$GITHUB_STEP_SUMMARY"`
-blocks on secrets in source files and still posts the report. Every option:
+blocks on secrets in source files and still posts the report. The same
+scoring wires into Claude Code, Cursor, Gemini CLI and pre-commit as a hook
+that exits 2 over a threshold. Every option:
 [docs/cli.md](https://github.com/antvinni/gitmole/blob/main/docs/cli.md).
 
 ## What you get
@@ -87,9 +92,9 @@ Reports on repositories you know, each at a pinned commit, published as gitmole
 
 | Repository | Commit | Commits | Lines | gitmole run |
 |---|---|---:|---:|---:|
-| [curl](https://github.com/antvinni/gitmole/blob/main/docs/examples/curl.md) | [`540ee5b5`](https://github.com/curl/curl/commit/540ee5b560cc6e775e11317048a13cc7e355bf91) | 39,758 | 247,179 | 61 s |
-| [django](https://github.com/antvinni/gitmole/blob/main/docs/examples/django.md) | [`8cbdd4a8`](https://github.com/django/django/commit/8cbdd4a814397f81adf0129288f32b615bd1f94f) | 34,933 | 431,749 | 153 s |
-| [react](https://github.com/antvinni/gitmole/blob/main/docs/examples/react.md) | [`2b19aecd`](https://github.com/facebook/react/commit/2b19aecd0e9111b774fad0fad9862e50bcb5bc8a) | 21,703 | 681,078 | 138 s |
+| [curl](https://github.com/antvinni/gitmole/blob/main/docs/examples/curl.md) | [`540ee5b5`](https://github.com/curl/curl/commit/540ee5b560cc6e775e11317048a13cc7e355bf91) | 39,758 | 247,179 | 57 s |
+| [django](https://github.com/antvinni/gitmole/blob/main/docs/examples/django.md) | [`8cbdd4a8`](https://github.com/django/django/commit/8cbdd4a814397f81adf0129288f32b615bd1f94f) | 34,933 | 431,749 | 132 s |
+| [react](https://github.com/antvinni/gitmole/blob/main/docs/examples/react.md) | [`2b19aecd`](https://github.com/facebook/react/commit/2b19aecd0e9111b774fad0fad9862e50bcb5bc8a) | 21,703 | 681,078 | 151 s |
 
 Run times are one `gitmole CLONE` with every default step, on a MacBook Pro (M4, 16 GB).
 
@@ -110,6 +115,7 @@ you about a clone.
 | Which blocks of code appear more than once | [jscpd](https://github.com/kucherenko/jscpd) | brew |
 | Have secrets ever been committed | [betterleaks](https://github.com/betterleaks/betterleaks) | brew |
 | Do the dependencies have known vulnerabilities | [osv-scanner](https://github.com/google/osv-scanner), offline against a local copy of the OSV database | brew, plus a one-time database download |
+| How deeply nested is the code, what did the authors flag, what imports what | [tree-sitter](https://github.com/tree-sitter/py-tree-sitter) grammars for eleven languages | pip, opt-in with `gitmole[structure]` |
 
 Why these and not others: [docs/tools.md](https://github.com/antvinni/gitmole/blob/main/docs/tools.md).
 
@@ -121,6 +127,7 @@ Why these and not others: [docs/tools.md](https://github.com/antvinni/gitmole/bl
 - [Example reports](https://github.com/antvinni/gitmole/tree/main/docs/examples): curl, django and react at pinned commits, regenerated by `bin/render-examples`.
 - [Validation](https://github.com/antvinni/gitmole/blob/main/docs/validation.md): the watch list against other ways of ranking the same files at six cut-offs on three repositories.
 - [Why these tools](https://github.com/antvinni/gitmole/blob/main/docs/tools.md): the rationale, what was left out, licences.
+- [References](https://github.com/antvinni/gitmole/blob/main/docs/references.md): the research and tools gitmole's rules are built on.
 - [Development](https://github.com/antvinni/gitmole/blob/main/docs/development.md): setup, tests, releases, code layout.
 - [Contributing](https://github.com/antvinni/gitmole/blob/main/CONTRIBUTING.md): bugs, ideas, pull requests, security reports.
 

{gitmole-0.16.0 → gitmole-0.18.0}/README.md

@@ -11,7 +11,7 @@ Free. Any Stack. Local. Offline. Deterministic. Fast.
 - **Free.** MIT licence, no paid tier, no account, no token. A local clone needs no credentials, and a public `owner/repo` is cloned with plain git. Your `gh` login is only used for private repositories and for `owner/*`, and only when you ask for them. The tools it runs are open source too.
 - **Any stack.** It reads what every repository has: the git log, git blame and the files themselves.
 - **Local & Offline.** Everything runs against a clone on your machine. Nothing is uploaded, nothing phones home; the vulnerability database is a copy you download once.
-- **Deterministic.** No AI at runtime. Every finding is a plain rule over counts you can recompute by hand. The JSON export carries each finding's rule and the numbers it fired on. The same clone gives the same report every time. 
+- **Deterministic.** No AI at runtime. Every finding is a plain rule over counts you can recompute by hand. The JSON export carries each finding's rule, the numbers it fired on and, where a rule rests on a paper, the citation. The same commit gives the same bytes: gitmole's own CI runs it twice on every commit, compares the exports and attests the report.
 
 ## Install
 
@@ -40,12 +40,17 @@ gitmole . --markdown report.md         # the same report as a Markdown document
 gitmole . --json report.json           # every table, the watch list and the findings
 gitmole . --fail-on warning            # exit 3 if any finding is a warning or worse
 gitmole . --risk main --risk-threshold 10  # exit 3 if the files changed since main hold over 10% of the risk
+gitmole . --sarif gitmole.sarif        # the findings for GitHub code scanning or GitLab
+gitmole . --compare last.json          # what changed since an earlier --json export
+gitmole analysis-repo --no-run --hook  # a coding agent's edit hook: history's view of the files it just touched
 gitmole . --since 2y --full            # the current team, every row and column
 gitmole --clean                        # list what gitmole left behind, delete on a yes
 ```
 
 A CI job that runs `gitmole . --fail-on critical --markdown - >> "$GITHUB_STEP_SUMMARY"`
-blocks on secrets in source files and still posts the report. Every option:
+blocks on secrets in source files and still posts the report. The same
+scoring wires into Claude Code, Cursor, Gemini CLI and pre-commit as a hook
+that exits 2 over a threshold. Every option:
 [docs/cli.md](https://github.com/antvinni/gitmole/blob/main/docs/cli.md).
 
 ## What you get
@@ -54,9 +59,9 @@ Reports on repositories you know, each at a pinned commit, published as gitmole
 
 | Repository | Commit | Commits | Lines | gitmole run |
 |---|---|---:|---:|---:|
-| [curl](https://github.com/antvinni/gitmole/blob/main/docs/examples/curl.md) | [`540ee5b5`](https://github.com/curl/curl/commit/540ee5b560cc6e775e11317048a13cc7e355bf91) | 39,758 | 247,179 | 61 s |
-| [django](https://github.com/antvinni/gitmole/blob/main/docs/examples/django.md) | [`8cbdd4a8`](https://github.com/django/django/commit/8cbdd4a814397f81adf0129288f32b615bd1f94f) | 34,933 | 431,749 | 153 s |
-| [react](https://github.com/antvinni/gitmole/blob/main/docs/examples/react.md) | [`2b19aecd`](https://github.com/facebook/react/commit/2b19aecd0e9111b774fad0fad9862e50bcb5bc8a) | 21,703 | 681,078 | 138 s |
+| [curl](https://github.com/antvinni/gitmole/blob/main/docs/examples/curl.md) | [`540ee5b5`](https://github.com/curl/curl/commit/540ee5b560cc6e775e11317048a13cc7e355bf91) | 39,758 | 247,179 | 57 s |
+| [django](https://github.com/antvinni/gitmole/blob/main/docs/examples/django.md) | [`8cbdd4a8`](https://github.com/django/django/commit/8cbdd4a814397f81adf0129288f32b615bd1f94f) | 34,933 | 431,749 | 132 s |
+| [react](https://github.com/antvinni/gitmole/blob/main/docs/examples/react.md) | [`2b19aecd`](https://github.com/facebook/react/commit/2b19aecd0e9111b774fad0fad9862e50bcb5bc8a) | 21,703 | 681,078 | 151 s |
 
 Run times are one `gitmole CLONE` with every default step, on a MacBook Pro (M4, 16 GB).
 
@@ -77,6 +82,7 @@ you about a clone.
 | Which blocks of code appear more than once | [jscpd](https://github.com/kucherenko/jscpd) | brew |
 | Have secrets ever been committed | [betterleaks](https://github.com/betterleaks/betterleaks) | brew |
 | Do the dependencies have known vulnerabilities | [osv-scanner](https://github.com/google/osv-scanner), offline against a local copy of the OSV database | brew, plus a one-time database download |
+| How deeply nested is the code, what did the authors flag, what imports what | [tree-sitter](https://github.com/tree-sitter/py-tree-sitter) grammars for eleven languages | pip, opt-in with `gitmole[structure]` |
 
 Why these and not others: [docs/tools.md](https://github.com/antvinni/gitmole/blob/main/docs/tools.md).
 
@@ -88,6 +94,7 @@ Why these and not others: [docs/tools.md](https://github.com/antvinni/gitmole/bl
 - [Example reports](https://github.com/antvinni/gitmole/tree/main/docs/examples): curl, django and react at pinned commits, regenerated by `bin/render-examples`.
 - [Validation](https://github.com/antvinni/gitmole/blob/main/docs/validation.md): the watch list against other ways of ranking the same files at six cut-offs on three repositories.
 - [Why these tools](https://github.com/antvinni/gitmole/blob/main/docs/tools.md): the rationale, what was left out, licences.
+- [References](https://github.com/antvinni/gitmole/blob/main/docs/references.md): the research and tools gitmole's rules are built on.
 - [Development](https://github.com/antvinni/gitmole/blob/main/docs/development.md): setup, tests, releases, code layout.
 - [Contributing](https://github.com/antvinni/gitmole/blob/main/CONTRIBUTING.md): bugs, ideas, pull requests, security reports.
 

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/__init__.py

@@ -1,3 +1,3 @@
 """gitmole: offline git repository analysis with a terminal report."""
 
-__version__ = "0.16.0"
+__version__ = "0.18.0"

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/cli.py

@@ -369,6 +369,9 @@ def _meta_for_run(repo_dir: str, args, estimate, age_ok: bool, plots_ok: bool, p
     from . import maat as _maat
     cut = _maat.months_before(meta["last_date"], 6) if meta["last_date"] else None
     first = meta.get("first_date_all") or meta["first_date"]   # the backtest reads the whole history, window or not
+    year = _maat.months_before(meta["last_date"], 12) if meta["last_date"] else None
+    if year and first and first <= year:
+        meta["duplicates"]["then"] = year   # the rate a year back, when the history reaches it
     if cut and first and first <= _maat.months_before(cut, 6):
         meta["backtest"] = {"status": "planned", "until": cut}
     else:
@@ -427,7 +430,7 @@ def _analyse(repo_dir: str, out_dir: str, args, ui: Console, planner, estimator)
     run.clear_outputs(out_dir)
     steps = planner(repo_dir, out_dir, branch=meta["branch"], age=age_ok, plots=plots_ok, ignore=ignore, types=types_spec, now=args.now,
                     since=args.since_date, lizard=lizard_ok, duplicates=duplicates_ok, backtest=cut, ignore_revs=run.ignore_revs_files(repo_dir),
-                    structure=getattr(args, "structure", False))
+                    structure=getattr(args, "structure", False), duplicates_then=meta["duplicates"].get("then"))
     run.save_meta(meta, out_dir)
     results = _execute(steps, log_path, repo_dir, args.workers, ui, timeout=args.timeout)
     if _control.cancelled.is_set():
@@ -589,7 +592,7 @@ def _render(out_dir: str, console: Console, ui: Console, args, err: Console) ->
             return 2
         comparison = _compare.compare(before, report, found)
     if args.json:
-        _write(json.dumps(render.to_json(report, found, risk=risk, compare=comparison), indent=2) + "\n", args.json, console)
+        _write(render.dumps_json(report, found, risk=risk, compare=comparison), args.json, console)
     if args.markdown:
         _write(render.markdown(report, found, full=args.full, risk=risk, base=args.risk, compare=comparison), args.markdown, console)
     if args.sarif:

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/compare.py

@@ -65,4 +65,13 @@ def compare(before: dict, report: dict, found: list, top: int = watch.WATCH_TOP)
             "watch_entered": [f for f in after_watch if f not in before_watch], "watch_left": [f for f in before_watch if f not in after_watch],
             "tally": {"before": _tally(before.get("findings") or []), "after": _tally(found)},
             "before": {"commit": (meta_b.get("run") or {}).get("commit"), "date": meta_b.get("last_date"),
-                       "options_differ": _options_differ(meta_b, meta_a)}}
+                       "options_differ": _options_differ(meta_b, meta_a), "database": _database_changed(before, report)}}
+
+
+def _database_changed(before: dict, report: dict):
+    """{before, after} dates when the two runs scanned against different snapshots of the OSV database,
+    else None: a new advisory changes the vulnerable-dependency finding without any change to the code."""
+    b, a = before.get("dependencies") or {}, report.get("dependencies") or {}
+    if b.get("database_digest") and a.get("database_digest") and b["database_digest"] != a["database_digest"]:
+        return {"before": b.get("database_date"), "after": a.get("database_date")}
+    return None

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/deps.py

@@ -58,6 +58,28 @@ def database_date(base: str = None) -> str | None:
     return dt.datetime.fromtimestamp(newest).date().isoformat() if newest else None
 
 
+def database_digest(base: str = None) -> str | None:
+    """A digest of the local database snapshot: every file under the cache directories, by path, size
+    and modification time. Two runs against the same snapshot agree; a refresh changes it, which is how
+    --compare can say a dependency finding moved because the database did. None when there is none."""
+    import hashlib
+    base = cache_dir() if base is None else base
+    entries = []
+    for name in CACHE_DIRS:
+        root = os.path.join(base, name)
+        for dirpath, _, files in os.walk(root):
+            for f in files:
+                full = os.path.join(dirpath, f)
+                try:
+                    st = os.stat(full)
+                except OSError:
+                    continue
+                entries.append(f"{os.path.relpath(full, base)}\0{st.st_size}\0{int(st.st_mtime)}")
+    if not entries:
+        return None
+    return hashlib.sha256("\n".join(sorted(entries)).encode("utf-8", "surrogateescape")).hexdigest()[:16]
+
+
 def _key(version: str) -> tuple:
     return tuple(int(n) for n in _NUMBER.findall(version or ""))
 
@@ -94,7 +116,7 @@ def _score(groups: list, vulns: list) -> float | None:
 
 
 _WORDS = {"CRITICAL": 9.5, "HIGH": 8.0, "MODERATE": 5.5, "MEDIUM": 5.5, "LOW": 2.0}
-MALICIOUS_PREFIX = "MAL-"   # OpenSSF malicious-packages records, in the same OSV database; they carry no CVSS
+MALICIOUS_PREFIX = "MAL-"   # OpenSSF malicious-packages records (ossf/malicious-packages), in the same OSV database; they carry no CVSS
 
 
 def is_malicious(vulns: list) -> bool:
@@ -185,6 +207,7 @@ def main(argv=None) -> int:
             return 1
         result = summarise(data, os.getcwd())
         result["database_date"] = database_date()
+        result["database_digest"] = database_digest()
     else:
         print(f"deps.py: osv-scanner exited {proc.returncode}; no report written", file=sys.stderr)
         return proc.returncode

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/duplicates.py

@@ -6,8 +6,10 @@ jscpd walks the working tree and reports every pair of matching fragments. This
 whose two sides are both tracked files inside the analysed types, folds the pairs of one fragment into
 a block with all its places, measures the duplicated share over the kept files, and writes
 duplicates.json. jscpd's own report quotes every fragment, so it is written to a temporary directory
-under OUT and removed before this returns: no source text lands in the output directory. Standalone,
-like maat.py and blame.py.
+under OUT and removed before this returns: no source text lands in the output directory. With --then
+DATE the tree at the last commit before that date is exported under OUT, measured the same way and
+removed, so the rate has a direction: GitClear's longitudinal data shows duplication is where the
+change is, and one number without the year before it says little. Standalone, like maat.py and blame.py.
 """
 from __future__ import annotations
 
@@ -118,6 +120,40 @@ def run_jscpd(repo: str, out: str, procs: int, ignore=()) -> tuple:
         shutil.rmtree(tmp, ignore_errors=True)
 
 
+def rev_before(repo: str, date: str):
+    out = subprocess.run(["git", "rev-list", "-1", f"--before={date}T00:00:00", "HEAD"], cwd=repo, capture_output=True, text=True)
+    return out.stdout.strip() or None
+
+
+def files_at(repo: str, rev: str, ignore=(), types_spec: str = None) -> list:
+    """The tracked text files of the tree at `rev`, in the analysed types, as select_files lists HEAD's."""
+    import fnmatch
+    proc = subprocess.run([*filetypes.GIT, "grep", "-I", "--name-only", "-z", "-e", "", rev], cwd=repo, capture_output=True)
+    types = filetypes.parse(types_spec)
+    paths = sorted(p.decode("utf-8", "surrogateescape").split(":", 1)[1] for p in proc.stdout.split(b"\0") if p)
+    return [f for f in paths if filetypes.matches(f, types) and not any(fnmatch.fnmatch(f, g) for g in ignore)]
+
+
+def rate_at(repo: str, out: str, rev: str, procs: int, ignore=(), types_spec: str = None):
+    """The duplicated share of the tree at `rev`: the tree exported through a temporary index under
+    `out` (as the backtest exports its cut-off), jscpd over it, the same fold and rate as HEAD's."""
+    tmp = tempfile.mkdtemp(prefix=".dup-then-", dir=out)
+    try:
+        tree = os.path.join(tmp, "tree")
+        os.makedirs(tree)
+        env = dict(os.environ, GIT_INDEX_FILE=os.path.join(tmp, "index"))
+        subprocess.run(["git", "read-tree", rev], cwd=repo, env=env, check=True, capture_output=True)
+        subprocess.run(["git", "checkout-index", "-a", f"--prefix={tree}/"], cwd=repo, env=env, check=True, capture_output=True)
+        files = files_at(repo, rev, ignore, types_spec)
+        rc, report = run_jscpd(tree, out, procs, ignore)
+        if rc != 0:
+            return None
+        blocks = fold(report.get("duplicates") or [], set(files))
+        return {"files": len(files), "rate": rate(blocks, tree, files)}
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
 def write(result: dict, target: str) -> None:
     fd, tmp = tempfile.mkstemp(dir=os.path.dirname(os.path.abspath(target)), prefix=".duplicates-", suffix=".json")
     try:
@@ -136,6 +172,7 @@ def main(argv=None) -> int:
     p.add_argument("--procs", type=int, default=1)
     p.add_argument("--ignore", action="append", default=[])
     p.add_argument("--types", default=None, help="file types spec as for gitmole --file-types")
+    p.add_argument("--then", metavar="YYYY-MM-DD", help="also measure the tree at the last commit before this date, for the direction")
     args = p.parse_args(argv)
     repo, out = os.path.abspath(args.repo), os.path.abspath(args.out)
     files = select_files(repo, args.ignore, args.types)
@@ -145,6 +182,11 @@ def main(argv=None) -> int:
     blocks = fold(report.get("duplicates") or [], set(files))
     result = {"tool": "jscpd", "files": len(files), "clones": sum(len(b["places"]) - 1 for b in blocks),
               "rate": rate(blocks, repo, files), "blocks": blocks[:BLOCKS_KEPT]}
+    if args.then:
+        rev = rev_before(repo, args.then)
+        measured = rate_at(repo, out, rev, args.procs, args.ignore, args.types) if rev else None
+        if measured:
+            result["then"] = {"date": args.then, "rev": rev[:12], **measured}
     write(result, os.path.join(out, "duplicates.json"))
     return 0
 

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/findings.py

@@ -1,4 +1,5 @@
-"""Heuristics that turn a loaded report into a short list of flagged findings."""
+"""Heuristics that turn a loaded report into a short list of flagged findings. A rule that rests on a paper
+carries the short citation in its rule dict's `ref` (see REFS); docs/references.md has the full entries."""
 from __future__ import annotations
 
 import re
@@ -11,11 +12,23 @@ PLACEHOLDER_NAMES = {"your name", "unknown", "root", "user"}
 PLACEHOLDER_EMAIL = re.compile(r"(@example\.(com|org|net)$|^you@|^user@|^root@|@localhost$)")
 
 
+# The paper a rule rests on, as the short citation the rule dict carries in `ref`; the full entries are in
+# docs/references.md. A rule that is gitmole's own heuristic has none.
+REFS = {"minor_contributors": "Bird et al., FSE 2011", "tangled_commits": "Herzig and Zeller, MSR 2013",
+        "brain_methods": "Lanza and Marinescu, 2006", "tight_coupling": "Gall, Hajek and Jazayeri, ICSM 1998",
+        "hotspot_dominance": "Tornhill, Your Code as a Crime Scene, 2024", "trojan_source": "Boucher and Anderson, USENIX Security 2023",
+        "debt_in_hotspots": "Maldonado and Shihab, MTD 2015", "hidden_coupling": "Ajienka and Capiluppi, JSS 2017",
+        "unreferenced_files": "Romano et al., TSE 2020", "signoff_by_co_author": "Linux kernel, Documentation/process/coding-assistants.rst",
+        "deep_nesting": "SonarSource cognitive complexity; CodeScene code health"}
+
+
 def _f(severity: str, title: str, statement: str, advice: str, rule: dict, evidence: dict) -> dict:
     """A finding: the facts, then the next step. `detail` is the two joined for anyone reading the
     JSON; `advice` says which part is the step so the report can show it on its own line. `rule` is
     the rule's id and the thresholds it fired on, `evidence` the numbers they were compared with:
     between them a reader of the JSON can check the finding without reading this file."""
+    if rule.get("id") in REFS and "ref" not in rule:
+        rule = {**rule, "ref": REFS[rule["id"]]}
     return {"severity": severity, "title": title, "detail": f"{statement.rstrip()} {advice}", "advice": advice,
             "rule": rule, "evidence": evidence}
 
@@ -1022,9 +1035,76 @@ def unreferenced_files(report: dict) -> list:
                rule={"id": "unreferenced_files", "ref": "Romano et al., TSE 2020"}, evidence={"count": n, "files": paths[:10]})]
 
 
+def _agents(report: dict) -> dict:
+    return ((report.get("provenance") or {}).get("agents")) or {}
+
+
+def agent_approval_disabled(report: dict) -> list:
+    """A committed agent configuration that turns approval prompts off: every clone that picks the
+    settings up runs the agent's tools without asking."""
+    rows = _agents(report).get("approval_disabled") or []
+    if not rows:
+        return []
+    listed = "; ".join(f"{r['file']} sets {r['setting']}" for r in rows)
+    return [_f("warning", "Agent approval prompts turned off in the repository", f"{listed}. Anyone who opens the clone with that agent runs its tools unasked.",
+               "Move the setting to your personal settings file, which is not committed, and keep the shared one to what everyone should get.",
+               rule={"id": "agent_approval_disabled", "by": "tracked agent settings"}, evidence={"settings": rows})]
+
+
+def agent_local_settings(report: dict) -> list:
+    rows = _agents(report).get("local_settings") or []
+    if not rows:
+        return []
+    return [_f("warning", "Personal agent settings tracked", f"{_files_list(rows)} {'is' if len(rows) == 1 else 'are'} tracked; the file is meant for one machine and to stay out of git.",
+               f"git rm --cached {rows[0]} and add it to .gitignore.",
+               rule={"id": "agent_local_settings", "by": "path convention"}, evidence={"files": rows})]
+
+
+def mcp_literal_env(report: dict) -> list:
+    """An MCP server declaration whose environment holds a literal value long enough to be a
+    credential rather than a ${VAR} reference. The key is named, the value never."""
+    rows = [(m["file"], x) for m in _agents(report).get("mcp") or [] for x in m.get("literal_env") or []]
+    if not rows:
+        return []
+    listed = "; ".join(f"{x['server']} sets {x['key']} in {f} to a literal value" for f, x in rows[:5])
+    f0, x0 = rows[0]
+    return [_f("warning", "Literal values in MCP server declarations", f"{listed}. A committed declaration shares whatever it holds.",
+               f"Replace the value of {x0['key']} with ${{{x0['key']}}} and set it in the environment; rotate it if it was a live credential.",
+               rule={"id": "mcp_literal_env", "min_length": 16, "placeholders": "left out"},
+               evidence={"entries": [{"file": f, "server": x["server"], "key": x["key"]} for f, x in rows[:10]]})]
+
+
+def agent_instructions_drift(report: dict, min_months: int = 6, min_commits: int = 100) -> list:
+    """Agent instruction files (AGENTS.md and the like) far behind the code they describe."""
+    last = (report.get("meta") or {}).get("last_date") or ""
+    stale = [r for r in _agents(report).get("instructions") or []
+             if last and _months_apart(r["last"], last) >= min_months and r["commits_behind"] >= min_commits]
+    if not stale:
+        return []
+    listed = "; ".join(f"{r['file']} last changed on {r['last']}, {_months_apart(r['last'], last)} months and {r['commits_behind']:,} commits before the last commit" for r in stale)
+    return [_f("info", "Agent instructions behind the code", f"{listed}.",
+               f"Read {stale[0]['file']} against the tree and update what moved; an agent follows it literally.",
+               rule={"id": "agent_instructions_drift", "min_months": min_months, "min_commits": min_commits},
+               evidence={"files": stale})]
+
+
+def signoff_by_co_author(report: dict, min_commits: int = 2) -> list:
+    """An identity that signs off commits it co-authors but never authors one: the Linux kernel's policy
+    on coding assistants forbids an agent to add Signed-off-by, since the DCO is a human's statement."""
+    rows = [r for r in ((report.get("provenance") or {}).get("trailers") or {}).get("signoff_by_co_author") or [] if r["commits"] >= min_commits]
+    if not rows:
+        return []
+    listed = "; ".join(f"{r['name']} <{r['email']}> signs off {_plural(r['commits'], 'commit')} but never authors one" for r in rows[:3])
+    return [_f("info", "Sign-offs by identities that only co-author", f"{listed}.",
+               "A Signed-off-by line certifies the Developer Certificate of Origin; have a person who authors commits add it.",
+               rule={"id": "signoff_by_co_author", "min_commits": min_commits, "ref": "Linux kernel, Documentation/process/coding-assistants.rst"},
+               evidence={"identities": rows[:10]})]
+
+
 RULES = [dormant, secrets_found, credential_files, vulnerable_dependencies, placeholder_identity, bus_factor, sizer_concerns, hotspot_dominance, bug_magnets,
          minor_contributors, reverts, brain_methods, complexity_growth, tight_coupling, duplication, stale_files, knowledge_islands, knowledge_loss,
-         sweeping_commits, tangled_commits, hygiene_findings, debt_in_hotspots, deep_nesting, hidden_coupling, unreferenced_files]
+         sweeping_commits, tangled_commits, hygiene_findings, debt_in_hotspots, deep_nesting, hidden_coupling, unreferenced_files,
+         agent_approval_disabled, agent_local_settings, mcp_literal_env, agent_instructions_drift, signoff_by_co_author]
 
 
 def evaluate(report: dict) -> list:

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/hook.py

@@ -1,4 +1,5 @@
-"""The agent-hook gate: `gitmole OUT_DIR --no-run --hook [--risk-threshold N] [-- FILE...]`.
+"""The agent-hook gate: `gitmole OUT_DIR --no-run --hook [--risk-threshold N] [-- FILE...]`, after Codacy's
+argument for deterministic findings before inference and Zimmermann et al.'s co-change recommendations.
 
 Deterministic findings should run before inference, so the model reasons over a short list rather
 than rediscovering known issues. A linter says what is wrong in the diff; gitmole says what history

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/hygiene.py

@@ -1,5 +1,6 @@
 """Repository hygiene from the clone alone: the checks OpenSSF Scorecard and the OSPS Baseline make
-through the GitHub API, done with file and git reads.
+through the GitHub API, done with file and git reads; Trojan Source after Boucher and Anderson (USENIX
+Security 2023).
 
 Runs as a pipeline step, `python -m gitmole.hygiene OUT_DIR`, from inside the repository, and writes
 hygiene.json; findings.py turns it into findings. Each check keys on a convention of the ecosystem (a

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/leaks.py

@@ -228,6 +228,9 @@ def scan_unreachable(repo: str, out_dir: str, found: dict) -> list:
         sha = os.path.basename(r.get("File") or "")
         r["File"] = f"(unreachable blob {sha[:12]})"
         r["Commit"] = ""
+        # betterleaks' own fingerprint names the scratch path the blob was written to; this one names the blob,
+        # so it is the same in every run and can go into .betterleaksignore
+        r["Fingerprint"] = f"unreachable:{sha}:{r.get('RuleID', '')}:{r.get('StartLine', '')}"
     return rows
 
 

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/load.py

@@ -182,6 +182,7 @@ def parse_functions(text: str) -> list:
         anonymous = name in ("", "(anonymous)")
         rows.append({"file": _rel(r[6]), "function": textfmt.cut(label if anonymous and label else name, NAME_CAP) or "(anonymous)", "anonymous": anonymous,
                      "ccn": _num(r[1]), "nloc": _num(r[0]), "params": _num(r[3]), "start": _num(r[9]), "end": _num(r[10]), "suspect": suspect})
+    rows.sort(key=lambda f: (f["file"], f["start"], f["end"], f["function"]))   # the function step works in parallel; the order is this one
     return rows
 
 
@@ -197,7 +198,10 @@ def parse_duplicates_json(data) -> dict | None:
     blocks = [{"lines": _num(b.get("lines")), "places": sorted(tuple(p[:3]) for p in b.get("places") or [] if len(p) >= 3)}
               for b in data.get("blocks") or []]
     rate = data.get("rate")
-    return {"rate": float(rate) if rate is not None else None, "blocks": blocks, "files": _num(data.get("files"))}
+    out = {"rate": float(rate) if rate is not None else None, "blocks": blocks, "files": _num(data.get("files"))}
+    if isinstance(data.get("then"), dict):
+        out["then"] = data["then"]   # the rate at the last commit a year before: the direction
+    return out
 
 
 def parse_duplicates(text: str) -> dict:
@@ -236,6 +240,8 @@ def parse_secrets(text: str) -> list:
             value, placeholder = None, False
         out.append({"rule": r.get("RuleID", ""), "file": r.get("File", ""), "commit": r.get("Commit", "")[:7], "line": r.get("StartLine"),
                     "fingerprint": r.get("Fingerprint", ""), "value": value, "placeholder": placeholder})
+    # betterleaks scans in parallel and does not promise an order; everything downstream reads the rows in this one
+    out.sort(key=lambda r: (r["file"], r["commit"], r["line"] or 0, r["rule"], r["fingerprint"]))
     return out
 
 
@@ -248,6 +254,8 @@ def parse_dependencies(data) -> dict:
     if data["status"] == "scanned":
         out.update({"sources": data.get("sources") or [], "packages": _num(data.get("packages")),
                     "vulnerable": data.get("vulnerable") or [], "database_date": data.get("database_date")})
+        if data.get("database_digest"):
+            out["database_digest"] = data["database_digest"]
     elif data["status"] == "no-database":
         out["download"] = data.get("download") or ""
     return out
@@ -331,6 +339,7 @@ def load_report(out_dir: str, nested: bool = True) -> dict:
         "signing": _read_json(out_dir, "signing.json", {}) or {},   # commit signing coverage; {} before the step or after a killed one
         "hygiene": _read_json(out_dir, "hygiene.json", {}) or {},   # the hygiene checks (hygiene.py); {} before 0.15
         "unreachable": _read_json(out_dir, "unreachable.json", {}) or {},
-        "structure": _read_json(out_dir, "structure.json", {}) or {},   # tree-sitter metrics (structure.py); {} without gitmole[structure]   # what the secrets step found outside reachable history
+        "structure": _read_json(out_dir, "structure.json", {}) or {},
+        "provenance": _read_json(out_dir, "provenance.json", {}) or {},   # trailers, cohorts, commit shape, agent files; {} before 0.17   # tree-sitter metrics (structure.py); {} without gitmole[structure]   # what the secrets step found outside reachable history
         "backtest": _nested(out_dir) if nested else None,
     }

{gitmole-0.16.0 → gitmole-0.18.0}/gitmole/maat.py

@@ -1,5 +1,7 @@
 #!/usr/bin/env python3
-"""Change analysis over a git log export, in the layout code-maat produced.
+"""Change analysis over a git log export, in the layout code-maat produced: revisions, coupling and sum of
+coupling after Adam Tornhill's code-maat, minor contributors after Bird et al. (FSE 2011), change entropy
+after Hassan (ICSE 2009), oversized fixes after Herzig and Zeller (MSR 2013); see docs/references.md.
 
 Standalone on purpose: gitmole runs it as a pipeline step with
 `python3 maat.py LOG OUT_DIR [--aliases META_JSON]` and it must not need the