gitlogstats 1.4.0__tar.gz → 1.5.0__tar.gz

{gitlogstats-1.4.0 → gitlogstats-1.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: gitlogstats
-Version: 1.4.0
+Version: 1.5.0
 Summary: Track developers' contributions to one or more git repositories within a particular time range.
 Project-URL: Homepage, https://github.com/bloombar/gitlogstats
 Project-URL: Repository, https://github.com/bloombar/gitlogstats.git
@@ -709,7 +709,7 @@ This script calculates the following, for each developer in each repository:
 - number of **files changed**
 - number of **merges** into the analyzed branch
 
-With `--github`, it also reports each developer's GitHub **pull requests** opened, merged, closed without merging and merged by themselves, their **pushes** to the analyzed branch and their commits made on GitHub's **website** (see [GitHub pull requests](#github-pull-requests)).
+With `--github`, it also reports each developer's GitHub **pull requests** opened, merged, closed without merging and merged by themselves, their **pushes** to the analyzed branch, their commits made on GitHub's **website**, and their **git and GitHub email addresses**. It matches each GitHub account to the developer's git names and combines a developer who committed under several names into one row (see [GitHub statistics](#github-statistics)).
 
 The results can be formatted as `csv`, `json`, or `markdown`.
 
@@ -726,7 +726,7 @@ The command `gitlogstats --help` shows the usage instructions:
 ```
 usage: gitlogstats [-h] (-r REPOSITORY | -rf REPOFILE) [-u USER] [-s START]
                    [-e END] [-x EXCLUSIONS] [-f {csv,json,markdown}]
-                   [-b BRANCH] [-v] [-c] [--github]
+                   [-b BRANCH] [-v] [-c] [--github] [--clobber]
 
 options:
   -h, --help            show this help message and exit
@@ -757,6 +757,9 @@ options:
                         and their commits made on the GitHub website, and
                         match GitHub users to git authors. Needs the GitHub
                         CLI (gh), signed in
+  --clobber             Delete the repository clones this run used from the
+                        repos/ directory after printing the results (default:
+                        keep them for faster later runs)
 ```
 
 In case `gitlogstats` is not found as a command, even after installing, try `python -m gitlogstats` or `python3 -m gitlogstats` instead.
@@ -837,9 +840,19 @@ By default, every contributor in a repository's history is listed, with zeros fo
 gitlogstats -rf repos.txt -c
 ```
 
-### GitHub pull requests
+### Cloned repositories
 
-Add `--github` to also count each contributor's pull requests, pushes and website commits. Nine columns are appended to every output format:
+Each repository is cloned into a `repos/` directory under the directory you run the command from, and later runs update (`git pull`) the existing clone instead of cloning again, which makes them much faster. Add `--clobber` to delete the clones the run used, once the results have been printed; `repos/` itself is removed too if nothing else is left in it. Clones from other runs are kept, and nothing is deleted if the run fails before printing, so you can inspect it. With `-v`, each deleted path is listed on standard error.
+
+```
+gitlogstats -rf repos.txt --clobber
+```
+
+`--clobber` is unrelated to `-c/--clean`, which only hides contributors without activity.
+
+### GitHub statistics
+
+Add `--github` to also report each contributor's pull requests, pushes, website commits and email addresses. Nine columns are appended to every output format:
 
 | Column                | Meaning                                                                                                 |
 | --------------------- | ------------------------------------------------------------------------------------------------------- |
@@ -864,7 +877,7 @@ gitlogstats: --github needs the GitHub CLI (gh), which is not installed. Install
 gitlogstats: --github needs the GitHub CLI (gh) to be signed in. Run 'gh auth login', or run without --github.
 ```
 
-**Empty versus zero.** A zero means GitHub was asked and the person has none. An empty value (`null` in JSON) means GitHub could not be asked for that repository: it is not on github.com, or the request failed (no network, rate limit, a private repository you cannot see). The git statistics are still reported, the reason is printed to standard error, and the other repositories carry on. A contributor with no GitHub account matched has an empty `github_username` and zero pull requests. A GitHub user who opened pull requests but has no commits is listed in a row of their own with an empty `username` and zero git statistics. When GitHub could not be asked, `git_email` is still filled in and `github_email` is empty. The emails come from the whole history of the branch, not only from commits in the date range.
+**Empty versus zero.** A zero means GitHub was asked and the person has none. An empty value (`null` in JSON) means GitHub could not be asked for that repository: it is not on github.com, or the request failed (no network, rate limit, a private repository you cannot see). The git statistics are still reported, the reason is printed to standard error, and the other repositories carry on. A contributor with no GitHub account matched has an empty `github_username` and zero pull requests. A GitHub user who opened pull requests or pushed but has no commits is listed in a row of their own with an empty `username` and zero git statistics. When GitHub could not be asked, `git_email` is still filled in and `github_email` is empty. The emails come from the whole history of the branch, not only from commits in the date range.
 
 **Limits of pushes and website commits.** Pushes come from GitHub's repository activity record, which starts in September 2023: earlier pushes are not counted, however old the date range. A file edited on the website directly on the analyzed branch is recorded by GitHub as a push too, so it counts in both `pushes_to_default` and `web_commits`. Reading it works for any public repository you can see; for a private one you need access to it, and if GitHub refuses, the repository's GitHub columns are empty with the reason on standard error, as above. A push is credited to the GitHub user who pushed it, so a person who only pushes (and has no commits or pull requests in the range) still gets a row of their own. Pushes made while a tag or commit (not a branch) is checked out with `-b` are not counted. Website commits are recognised by GitHub being the committer, so commits made through GitHub's API, for example in the `github.dev` editor, are counted as website commits too, and so is the first commit GitHub creates when a repository is created on the website. Website commits follow `commits`: the same date range, exclusions (`-x`) and branch, credited to the commit's author and added up when names are combined into one row.
 
@@ -878,11 +891,11 @@ If a step finds several users it is skipped as ambiguous and the next is tried.
 
 **How the other options apply.**
 
-- `-s` / `-e`: each pull request count uses its own timestamp (opened, merged or closed) against the same inclusive local-time range as commits.
-- `-b`: only pull requests whose base branch is that branch are counted; without `-b`, all pull requests count.
+- `-s` / `-e`: each pull request count uses its own timestamp (opened, merged or closed), and each push its push time, against the same inclusive local-time range as commits.
+- `-b`: only pull requests whose base branch is that branch are counted (without `-b`, all pull requests count), and pushes and website commits are counted on that branch instead of the default branch.
 - `-u`: selects the combined row whose author names or emails equal the value exactly, or whose GitHub username equals it ignoring case. The whole combined row is reported, so `-u` with one email address also includes the person's commits under their other names and emails, and a value that matches nobody prints no row.
-- `-c`: removes only rows with no git activity and no pull requests.
-- `-x`: has no effect on pull requests.
+- `-c`: removes only rows with no git activity, no pull requests and no pushes. Email addresses do not count as activity.
+- `-x`: has no effect on pull requests or pushes; it does apply to `web_commits`, which only counts commits that `commits` counts.
 
 ### Formatting the results
 
@@ -924,7 +937,7 @@ Some merges are not counted:
 - Merges on other branches, unless that branch is analyzed with `-b`.
 - Local merges that were never pushed.
 
-A merge made on GitHub uses the person's GitHub profile name and email, which may differ from their local git settings. They can then appear under two names (see [Redundant usernames](#redundant-usernames)).
+A merge made on GitHub uses the person's GitHub profile name and email, which may differ from their local git settings. They can then appear under two names; `--github` combines them into one row (see [Redundant usernames](#redundant-usernames)).
 
 Each commit is credited to its author's name, after applying the repository's `.mailmap` if it has one. Merges are credited the same way. Commit messages are never read, so statistics are the same whether messages are one line, many lines, or empty.
 
@@ -949,6 +962,8 @@ Most likely, the contributor has failed to update their version control settings
 
 ### Redundant usernames
 
-The output of the script sometimes lists the same individual contributor under more than one git username... This is most likely due to different username settings for various git and GitHub clients.
+The output of the script sometimes lists the same individual contributor under more than one git username. This is most likely due to different username settings for various git and GitHub clients: for example, a merge made on GitHub's website uses the person's GitHub display name, while their own commits use their local `user.name`.
+
+With `--github`, names that belong to the same GitHub account are combined into one row automatically, with their statistics added up (see [GitHub statistics](#github-statistics)). Check the `github_username` and `git_email` columns to see which names were combined. A name GitHub cannot link to any account, for example one whose email is a laptop's default address, keeps a row of its own; add the two names to a `.mailmap` file to combine them.
 
-If a single developer has multiple usernames that all show the same statistics, then only count those stats once. Otherwise, if a single developer has multiple usernames that show different statistics, then add them together to come up with the total for that developer.
+Without `--github`, or for a repository not on GitHub, add up the rows of a developer who appears under several names yourself, or map the names together in a `.mailmap` file.

{gitlogstats-1.4.0 → gitlogstats-1.5.0}/README.md

@@ -12,7 +12,7 @@ This script calculates the following, for each developer in each repository:
 - number of **files changed**
 - number of **merges** into the analyzed branch
 
-With `--github`, it also reports each developer's GitHub **pull requests** opened, merged, closed without merging and merged by themselves, their **pushes** to the analyzed branch and their commits made on GitHub's **website** (see [GitHub pull requests](#github-pull-requests)).
+With `--github`, it also reports each developer's GitHub **pull requests** opened, merged, closed without merging and merged by themselves, their **pushes** to the analyzed branch, their commits made on GitHub's **website**, and their **git and GitHub email addresses**. It matches each GitHub account to the developer's git names and combines a developer who committed under several names into one row (see [GitHub statistics](#github-statistics)).
 
 The results can be formatted as `csv`, `json`, or `markdown`.
 
@@ -29,7 +29,7 @@ The command `gitlogstats --help` shows the usage instructions:
 ```
 usage: gitlogstats [-h] (-r REPOSITORY | -rf REPOFILE) [-u USER] [-s START]
                    [-e END] [-x EXCLUSIONS] [-f {csv,json,markdown}]
-                   [-b BRANCH] [-v] [-c] [--github]
+                   [-b BRANCH] [-v] [-c] [--github] [--clobber]
 
 options:
   -h, --help            show this help message and exit
@@ -60,6 +60,9 @@ options:
                         and their commits made on the GitHub website, and
                         match GitHub users to git authors. Needs the GitHub
                         CLI (gh), signed in
+  --clobber             Delete the repository clones this run used from the
+                        repos/ directory after printing the results (default:
+                        keep them for faster later runs)
 ```
 
 In case `gitlogstats` is not found as a command, even after installing, try `python -m gitlogstats` or `python3 -m gitlogstats` instead.
@@ -140,9 +143,19 @@ By default, every contributor in a repository's history is listed, with zeros fo
 gitlogstats -rf repos.txt -c
 ```
 
-### GitHub pull requests
+### Cloned repositories
 
-Add `--github` to also count each contributor's pull requests, pushes and website commits. Nine columns are appended to every output format:
+Each repository is cloned into a `repos/` directory under the directory you run the command from, and later runs update (`git pull`) the existing clone instead of cloning again, which makes them much faster. Add `--clobber` to delete the clones the run used, once the results have been printed; `repos/` itself is removed too if nothing else is left in it. Clones from other runs are kept, and nothing is deleted if the run fails before printing, so you can inspect it. With `-v`, each deleted path is listed on standard error.
+
+```
+gitlogstats -rf repos.txt --clobber
+```
+
+`--clobber` is unrelated to `-c/--clean`, which only hides contributors without activity.
+
+### GitHub statistics
+
+Add `--github` to also report each contributor's pull requests, pushes, website commits and email addresses. Nine columns are appended to every output format:
 
 | Column                | Meaning                                                                                                 |
 | --------------------- | ------------------------------------------------------------------------------------------------------- |
@@ -167,7 +180,7 @@ gitlogstats: --github needs the GitHub CLI (gh), which is not installed. Install
 gitlogstats: --github needs the GitHub CLI (gh) to be signed in. Run 'gh auth login', or run without --github.
 ```
 
-**Empty versus zero.** A zero means GitHub was asked and the person has none. An empty value (`null` in JSON) means GitHub could not be asked for that repository: it is not on github.com, or the request failed (no network, rate limit, a private repository you cannot see). The git statistics are still reported, the reason is printed to standard error, and the other repositories carry on. A contributor with no GitHub account matched has an empty `github_username` and zero pull requests. A GitHub user who opened pull requests but has no commits is listed in a row of their own with an empty `username` and zero git statistics. When GitHub could not be asked, `git_email` is still filled in and `github_email` is empty. The emails come from the whole history of the branch, not only from commits in the date range.
+**Empty versus zero.** A zero means GitHub was asked and the person has none. An empty value (`null` in JSON) means GitHub could not be asked for that repository: it is not on github.com, or the request failed (no network, rate limit, a private repository you cannot see). The git statistics are still reported, the reason is printed to standard error, and the other repositories carry on. A contributor with no GitHub account matched has an empty `github_username` and zero pull requests. A GitHub user who opened pull requests or pushed but has no commits is listed in a row of their own with an empty `username` and zero git statistics. When GitHub could not be asked, `git_email` is still filled in and `github_email` is empty. The emails come from the whole history of the branch, not only from commits in the date range.
 
 **Limits of pushes and website commits.** Pushes come from GitHub's repository activity record, which starts in September 2023: earlier pushes are not counted, however old the date range. A file edited on the website directly on the analyzed branch is recorded by GitHub as a push too, so it counts in both `pushes_to_default` and `web_commits`. Reading it works for any public repository you can see; for a private one you need access to it, and if GitHub refuses, the repository's GitHub columns are empty with the reason on standard error, as above. A push is credited to the GitHub user who pushed it, so a person who only pushes (and has no commits or pull requests in the range) still gets a row of their own. Pushes made while a tag or commit (not a branch) is checked out with `-b` are not counted. Website commits are recognised by GitHub being the committer, so commits made through GitHub's API, for example in the `github.dev` editor, are counted as website commits too, and so is the first commit GitHub creates when a repository is created on the website. Website commits follow `commits`: the same date range, exclusions (`-x`) and branch, credited to the commit's author and added up when names are combined into one row.
 
@@ -181,11 +194,11 @@ If a step finds several users it is skipped as ambiguous and the next is tried.
 
 **How the other options apply.**
 
-- `-s` / `-e`: each pull request count uses its own timestamp (opened, merged or closed) against the same inclusive local-time range as commits.
-- `-b`: only pull requests whose base branch is that branch are counted; without `-b`, all pull requests count.
+- `-s` / `-e`: each pull request count uses its own timestamp (opened, merged or closed), and each push its push time, against the same inclusive local-time range as commits.
+- `-b`: only pull requests whose base branch is that branch are counted (without `-b`, all pull requests count), and pushes and website commits are counted on that branch instead of the default branch.
 - `-u`: selects the combined row whose author names or emails equal the value exactly, or whose GitHub username equals it ignoring case. The whole combined row is reported, so `-u` with one email address also includes the person's commits under their other names and emails, and a value that matches nobody prints no row.
-- `-c`: removes only rows with no git activity and no pull requests.
-- `-x`: has no effect on pull requests.
+- `-c`: removes only rows with no git activity, no pull requests and no pushes. Email addresses do not count as activity.
+- `-x`: has no effect on pull requests or pushes; it does apply to `web_commits`, which only counts commits that `commits` counts.
 
 ### Formatting the results
 
@@ -227,7 +240,7 @@ Some merges are not counted:
 - Merges on other branches, unless that branch is analyzed with `-b`.
 - Local merges that were never pushed.
 
-A merge made on GitHub uses the person's GitHub profile name and email, which may differ from their local git settings. They can then appear under two names (see [Redundant usernames](#redundant-usernames)).
+A merge made on GitHub uses the person's GitHub profile name and email, which may differ from their local git settings. They can then appear under two names; `--github` combines them into one row (see [Redundant usernames](#redundant-usernames)).
 
 Each commit is credited to its author's name, after applying the repository's `.mailmap` if it has one. Merges are credited the same way. Commit messages are never read, so statistics are the same whether messages are one line, many lines, or empty.
 
@@ -252,6 +265,8 @@ Most likely, the contributor has failed to update their version control settings
 
 ### Redundant usernames
 
-The output of the script sometimes lists the same individual contributor under more than one git username... This is most likely due to different username settings for various git and GitHub clients.
+The output of the script sometimes lists the same individual contributor under more than one git username. This is most likely due to different username settings for various git and GitHub clients: for example, a merge made on GitHub's website uses the person's GitHub display name, while their own commits use their local `user.name`.
+
+With `--github`, names that belong to the same GitHub account are combined into one row automatically, with their statistics added up (see [GitHub statistics](#github-statistics)). Check the `github_username` and `git_email` columns to see which names were combined. A name GitHub cannot link to any account, for example one whose email is a laptop's default address, keeps a row of its own; add the two names to a `.mailmap` file to combine them.
 
-If a single developer has multiple usernames that all show the same statistics, then only count those stats once. Otherwise, if a single developer has multiple usernames that show different statistics, then add them together to come up with the total for that developer.
+Without `--github`, or for a repository not on GitHub, add up the rows of a developer who appears under several names yourself, or map the names together in a `.mailmap` file.

{gitlogstats-1.4.0 → gitlogstats-1.5.0}/docs/SPEC.md

@@ -29,7 +29,7 @@ This document describes what the tool does today and what is planned. Each requi
 
 #### REPO-3 Local clone cache
 
-Repositories are cloned into a `repos/` directory under the current working directory, in a folder named the way `git clone` names it (the last path segment, minus a trailing `.git`; dots in the name are kept). On later runs the existing clone is updated with `git pull` instead of being cloned again.
+Repositories are cloned into a `repos/` directory under the current working directory, in a folder named the way `git clone` names it (the last path segment, minus a trailing `.git`; dots in the name are kept). On later runs the existing clone is updated with `git pull` instead of being cloned again. `--clobber` removes the clones after the run (REPO-5).
 
 **Status:** Done
 
@@ -39,6 +39,12 @@ Repositories are cloned into a `repos/` directory under the current working dire
 
 **Status:** Done
 
+#### REPO-5 Remove clones after the run
+
+`--clobber` deletes the clones this run used from the `repos/` directory (REPO-3) once the results have been printed, and then removes `repos/` itself if nothing else is left in it. Clones in `repos/` that this run did not use are kept. It is off by default, so clones are kept and reused by later runs. If the run fails before the results are printed, nothing is deleted, so the clones can be inspected. Only directories inside `repos/` are deleted; a symbolic link in place of a clone is removed without following it. If `repos/` is itself a symbolic link to another directory, the clones are deleted from that directory and the link is kept. A clone that cannot be fully deleted (e.g. it contains a folder the user cannot open) is reported on standard error and the remaining clones are still deleted; the exit status stays 0, since the results were printed. Read-only files that git writes inside a clone (as on Windows) are deleted too. With `-v`, each deleted path is printed to standard error. The name follows build tools such as Rake, where `clean` removes intermediate files and `clobber` also removes caches; it is unrelated to `-c/--clean` (FILT-4).
+
+**Status:** Done
+
 ### 2. Contribution Statistics
 
 #### STAT-1 Commit count

{gitlogstats-1.4.0 → gitlogstats-1.5.0}/pyproject.toml

@@ -5,7 +5,7 @@ build-backend = "hatchling.build"
 [project]
 name = "gitlogstats"
 description = "Track developers' contributions to one or more git repositories within a particular time range."
-version = "1.4.0"
+version = "1.5.0"
 authors = [
   { name="Tobody Tothing", email="nobody.nothing@nowhere.com" },
 ]

{gitlogstats-1.4.0 → gitlogstats-1.5.0}/scripts/board/manifest.yaml

@@ -194,6 +194,7 @@
     directory, in a folder named the way git clone names it (the last path
     segment, minus a trailing .git; dots in the name are kept). On later runs
     the existing clone is updated with git pull instead of being cloned again.
+    --clobber removes the clones after the run (REPO-5).
   section: §1 Repository Selection
   family: REPO
   phase: 1
@@ -209,6 +210,22 @@
   family: REPO
   phase: 1
   status: Done
+- id: REPO-5
+  title: Remove clones after the run
+  full: --clobber deletes the clones this run used from the repos/ directory
+    (REPO-3) once the results have been printed, and then removes repos/ itself
+    if nothing else is left in it. Clones in repos/ that this run did not use
+    are kept. It is off by default, so clones are kept and reused by later runs.
+    If the run fails before the results are printed, nothing is deleted, so the
+    clones can be inspected. Only directories inside repos/ are deleted; a
+    symbolic link in place of a clone is removed without following it. If repos/
+    is itself a symbolic link to another directory, the clones are deleted from
+    that directory and the link is kept. A clone that cannot be fully deleted
+    (e.g. it contains a folder…
+  section: §1 Repository Selection
+  family: REPO
+  phase: 1
+  status: Done
 - id: STAT-1
   title: Commit count
   full: "For each contributor, the number of non-merge commits they authored in

{gitlogstats-1.4.0 → gitlogstats-1.5.0}/src/gitlogstats/__main__.py

@@ -5,6 +5,7 @@ import datetime
 import re
 import sys
 from . import GitLogsParser
+from .clones import remove_clones
 from .git_logs_parser import FIELDNAMES
 from .github import (
     ACTIVITY_FIELDNAMES,
@@ -155,6 +156,14 @@ def main():
         default=False,
         action="store_true",
     )
+    parser.add_argument(
+        "--clobber",
+        help="Delete the repository clones this run used from the repos/ "
+        "directory after printing the results (default: keep them for faster "
+        "later runs)",
+        default=False,
+        action="store_true",
+    )
     args = parser.parse_args()
 
     if datetime.datetime.strptime(args.start, "%m/%d/%Y") > datetime.datetime.strptime(
@@ -179,6 +188,7 @@ def main():
         with open(args.repofile, "r", encoding="utf8") as f:
             repository_urls = GitLogsParser.read_repository_urls(f.read())
 
+    start_dir = os.getcwd()  # REPO-5: where the command was started from
     # navigate into the directory where repos will be stored
     repos_dir = os.path.join(
         os.getcwd(), "repos"
@@ -190,6 +200,7 @@ def main():
     # loop through each git repository url, accumulating all results
     all_results = []
     last_parser = None
+    used_clones = []  # REPO-5: the clone directories this run cloned or pulled
     for repo_url in repository_urls:
         os.chdir(repos_dir)  # start from the parent directory of all repos
 
@@ -221,6 +232,8 @@ def main():
                 ["git", "pull"], capture_output=True, check=True
             )  # get the latest code from github
 
+        used_clones.append(repo_dir)
+
         # with --github, -u and -c are applied after GitHub users are combined
         last_parser = GitLogsParser(
             repo=repo_dir,
@@ -250,6 +263,13 @@ def main():
         # end with exactly one newline: CSV and Markdown output already has one
         print(output, end="" if output.endswith("\n") else "\n")
 
+    # REPO-5: only now that the results are out, delete the clones. Leave the
+    # clone we were last inside first, so the deletion never depends on it.
+    if args.clobber:
+        sys.stdout.flush()
+        os.chdir(start_dir)
+        remove_clones(repos_dir, used_clones, args.verbose)
+
 
 # if this script is being run directly...
 if __name__ == "__main__":

gitlogstats-1.5.0/src/gitlogstats/clones.py

@@ -0,0 +1,85 @@
+"""Deleting the repository clones a run used (REPO-5, --clobber)."""
+
+import os
+import shutil
+import stat
+import sys
+
+
+def _make_writable_and_retry(function, path, exc_info):
+    """
+    Error handler for shutil.rmtree: git marks some files read-only (notably
+    pack files on Windows), which stops them being deleted. Clear the read-only
+    bit on the file and its folder, then try again. Any other error is raised,
+    as is a failure to open or list a folder: retrying os.open or os.scandir
+    with only a path would itself fail with a TypeError.
+    """
+    if not isinstance(exc_info[1], PermissionError) or function not in (
+        os.unlink,
+        os.remove,
+        os.rmdir,
+    ):
+        raise exc_info[1]
+    for target in (os.path.dirname(path), path):
+        try:
+            os.chmod(target, os.stat(target).st_mode | stat.S_IRWXU)
+        except OSError:
+            pass
+    function(path)
+
+
+def _delete(path):
+    """
+    Delete one clone. A symbolic link is unlinked, never followed.
+    @param path: the clone's path
+    """
+    if os.path.islink(path) or not os.path.isdir(path):
+        os.remove(path)
+    else:
+        shutil.rmtree(path, onerror=_make_writable_and_retry)
+
+
+def remove_clones(repos_dir, clone_dirs, verbose=False):
+    """
+    Delete the given clones, then remove repos_dir if nothing else is left in it.
+    Only direct children of repos_dir are deleted; any other path is skipped.
+    A failed deletion is reported on standard error and does not stop the rest.
+    @param repos_dir: the directory holding the clones
+    @param clone_dirs: the paths of the clones to delete (duplicates are fine)
+    @param verbose: whether to report each deleted or skipped path on stderr
+    """
+    real_repos = os.path.realpath(repos_dir)
+    seen = set()
+    for clone_dir in clone_dirs:
+        path = os.path.normpath(os.path.abspath(clone_dir))
+        if path in seen:
+            continue
+        seen.add(path)
+        # the folder that holds the clone must be repos_dir itself; checking the
+        # parent (not the path) means a symbolic link is judged by where it sits
+        if os.path.realpath(os.path.dirname(path)) != real_repos:
+            if verbose:
+                print(
+                    f"gitlogstats: not deleting {path}: not inside {repos_dir}",
+                    file=sys.stderr,
+                )
+            continue
+        if not os.path.lexists(path):
+            continue
+        try:
+            _delete(path)
+        except OSError as exc:
+            print(
+                f"gitlogstats: could not delete {path}: {exc.strerror or exc}",
+                file=sys.stderr,
+            )
+            continue
+        if verbose:
+            print(f"gitlogstats: deleted {path}", file=sys.stderr)
+
+    try:
+        os.rmdir(repos_dir)  # only succeeds when empty
+    except OSError:
+        return
+    if verbose:
+        print(f"gitlogstats: deleted {repos_dir}", file=sys.stderr)

gitlogstats-1.5.0/tests/test_clobber.py

@@ -0,0 +1,252 @@
+"""Tests for --clobber (REPO-5): deleting the clones a run used."""
+
+import json
+import os
+import stat
+import sys
+
+import pytest
+
+from gitlogstats import clones
+from gitlogstats.__main__ import main
+from gitlogstats.clones import remove_clones
+from test_integration import (
+    DATES,
+    GITHUB_URL,
+    Repo,
+    github_world,
+    lines,
+    pr,
+    run_github,
+)
+
+
+@pytest.fixture
+def remotes(tmp_path):
+    """Two 'remote' repositories that the command clones by path."""
+    one = Repo(tmp_path / "remotes" / "one")
+    one.write("a.txt", lines(3))
+    one.commit("Alice", "one: add")
+    two = Repo(tmp_path / "remotes" / "two.repo")
+    two.write("c.txt", lines(2))
+    two.commit("Alice", "two: add")
+    return one, two
+
+
+@pytest.fixture
+def work(tmp_path):
+    path = tmp_path / "work"
+    path.mkdir()
+    return path
+
+
+def run(monkeypatch, capsys, work, *argv):
+    """Run the command in *work*; returns (stdout, stderr)."""
+    monkeypatch.chdir(work)
+    monkeypatch.setattr(
+        sys, "argv", ["gitlogstats", *argv, "-s", "01/01/2024"] + ["-e", "12/31/2024"]
+    )
+    main()
+    captured = capsys.readouterr()
+    return captured.out, captured.err
+
+
+class TestCommand:
+    def test_clones_are_kept_without_the_flag(self, remotes, work, monkeypatch, capsys):
+        run(monkeypatch, capsys, work, "-r", remotes[0].path)
+        assert (work / "repos" / "one" / ".git").is_dir()
+
+    def test_clobber_prints_the_same_results_and_removes_everything(
+        self, remotes, work, monkeypatch, capsys
+    ):
+        kept, _ = run(monkeypatch, capsys, work, "-r", remotes[0].path, "-f", "json")
+        assert (work / "repos").exists()
+        # a fresh directory so that the second run clones again
+        other = work.parent / "work2"
+        other.mkdir()
+        out, err = run(
+            monkeypatch, capsys, other, "-r", remotes[0].path, "-f", "json", "--clobber"
+        )
+        assert out == kept and json.loads(out)[0]["commits"] == 1
+        assert err == ""
+        assert os.listdir(other) == []  # repos/ itself is gone
+
+    def test_other_clones_and_repos_dir_are_kept(
+        self, remotes, work, monkeypatch, capsys
+    ):
+        run(monkeypatch, capsys, work, "-r", remotes[1].path)
+        out, _ = run(monkeypatch, capsys, work, "-r", remotes[0].path, "--clobber")
+        assert "Alice,one," in out
+        assert sorted(os.listdir(work / "repos")) == ["two.repo"]
+
+    def test_repofile_with_two_repos_and_a_duplicate(
+        self, remotes, work, monkeypatch, capsys
+    ):
+        repofile = work.parent / "list.txt"
+        repofile.write_text(
+            f"{remotes[0].path}\n{remotes[1].path}\n{remotes[0].path}\n"
+        )
+        out, _ = run(monkeypatch, capsys, work, "-rf", str(repofile), "--clobber")
+        assert "Alice,one," in out and "Alice,two.repo," in out
+        assert os.listdir(work) == []
+
+    def test_failure_before_output_deletes_nothing(
+        self, remotes, work, monkeypatch, capsys
+    ):
+        repofile = work.parent / "list.txt"
+        repofile.write_text(f"{remotes[0].path}\n{work.parent / 'missing'}\n")
+        with pytest.raises(Exception):
+            run(monkeypatch, capsys, work, "-rf", str(repofile), "--clobber")
+        assert (work / "repos" / "one" / ".git").is_dir()
+        assert capsys.readouterr().out == ""
+
+    def test_verbose_lists_deleted_paths_on_stderr_only(
+        self, remotes, work, monkeypatch, capsys
+    ):
+        quiet, _ = run(monkeypatch, capsys, work, "-r", remotes[0].path, "--clobber")
+        out, err = run(
+            monkeypatch, capsys, work, "-r", remotes[0].path, "--clobber", "-v"
+        )
+        assert out == quiet
+        repos = os.path.realpath(work / "repos")
+        assert f"gitlogstats: deleted {os.path.join(repos, 'one')}" in err
+        assert f"gitlogstats: deleted {repos}\n" in err
+        assert "deleted" not in out
+
+    def test_clobber_with_github(self, remotes, tmp_path, monkeypatch, capsys):
+        github_world(tmp_path, monkeypatch, remotes[0], prs=[], contributors=[])
+        out, _ = run_github(
+            monkeypatch,
+            capsys,
+            tmp_path,
+            "-r",
+            GITHUB_URL,
+            "--github",
+            *DATES,
+            "--clobber",
+        )
+        assert "Alice" in out
+        assert not (tmp_path / "work" / "repos").exists()
+
+    def test_nothing_to_delete_when_no_repositories(self, work, monkeypatch, capsys):
+        repofile = work.parent / "empty.txt"
+        repofile.write_text("# nothing\n")
+        out, _ = run(monkeypatch, capsys, work, "-rf", str(repofile), "--clobber")
+        assert out == "" and os.listdir(work) == []
+
+
+class TestRemoveClones:
+    def test_read_only_files_are_deleted(self, tmp_path):
+        repos = tmp_path / "repos"
+        clone = repos / "x"
+        (clone / "objects").mkdir(parents=True)
+        for name in ("a", "objects/pack"):
+            (clone / name).write_text("data")
+            os.chmod(clone / name, stat.S_IREAD)
+        os.chmod(clone / "objects", stat.S_IREAD | stat.S_IEXEC)
+        remove_clones(str(repos), [str(clone)])
+        assert not repos.exists()
+
+    def test_symlink_is_unlinked_and_target_untouched(self, tmp_path):
+        target = tmp_path / "precious"
+        target.mkdir()
+        (target / "keep.txt").write_text("keep")
+        repos = tmp_path / "repos"
+        repos.mkdir()
+        os.symlink(target, repos / "link")
+        remove_clones(str(repos), [str(repos / "link")])
+        assert not repos.exists()
+        assert (target / "keep.txt").read_text() == "keep"
+
+    def test_paths_outside_repos_are_skipped(self, tmp_path, capsys):
+        repos = tmp_path / "repos"
+        repos.mkdir()
+        outside = tmp_path / "outside"
+        outside.mkdir()
+        nested = repos / "a" / "b"
+        nested.mkdir(parents=True)
+        for path in (outside, repos / ".." / "outside", nested, repos):
+            remove_clones(str(repos), [str(path)], verbose=True)
+        assert outside.is_dir() and nested.is_dir() and repos.is_dir()
+        assert capsys.readouterr().err.count("not deleting") == 4
+
+    def test_skipped_paths_are_silent_without_verbose(self, tmp_path, capsys):
+        repos = tmp_path / "repos"
+        repos.mkdir()
+        outside = tmp_path / "outside"
+        outside.mkdir()
+        remove_clones(str(repos), [str(outside)])
+        assert outside.is_dir() and capsys.readouterr().err == ""
+
+    def test_a_file_in_place_of_a_clone_is_removed(self, tmp_path):
+        repos = tmp_path / "repos"
+        repos.mkdir()
+        (repos / "f").write_text("x")
+        remove_clones(str(repos), [str(repos / "f")])
+        assert not repos.exists()
+
+    def test_deletion_error_is_reported_and_the_rest_continues(
+        self, tmp_path, monkeypatch, capsys
+    ):
+        repos = tmp_path / "repos"
+        for name in ("bad", "good"):
+            (repos / name).mkdir(parents=True)
+
+        real = clones._delete
+
+        def flaky(path):
+            if path.endswith("bad"):
+                raise PermissionError(13, "Permission denied")
+            real(path)
+
+        monkeypatch.setattr(clones, "_delete", flaky)
+        remove_clones(str(repos), [str(repos / "bad"), str(repos / "good")])
+        err = capsys.readouterr().err
+        assert (
+            f"gitlogstats: could not delete {repos / 'bad'}: Permission denied" in err
+        )
+        assert os.listdir(repos) == ["bad"]
+
+    @pytest.mark.skipif(
+        sys.platform == "win32" or os.geteuid() == 0,
+        reason="needs POSIX permissions that apply to the current user",
+    )
+    def test_unreadable_folder_is_reported_and_the_rest_continues(
+        self, tmp_path, capsys
+    ):
+        # a folder that cannot be opened makes rmtree call the handler with
+        # os.open; retrying that must not crash after the results are printed
+        repos = tmp_path / "repos"
+        locked = repos / "one" / "notes" / "private"
+        locked.mkdir(parents=True)
+        (locked / "a.txt").write_text("x")
+        (repos / "two").mkdir()
+        locked.chmod(0)
+        try:
+            remove_clones(str(repos), [str(repos / "one"), str(repos / "two")])
+        finally:
+            locked.chmod(stat.S_IRWXU)
+        err = capsys.readouterr().err
+        assert f"gitlogstats: could not delete {repos / 'one'}" in err
+        assert not (repos / "two").exists()
+
+    def test_other_errors_while_deleting_are_not_retried(self, tmp_path):
+        with pytest.raises(FileNotFoundError):
+            clones._make_writable_and_retry(
+                os.remove, str(tmp_path / "gone"), (None, FileNotFoundError(), None)
+            )
+
+    def test_retry_gives_up_when_chmod_is_impossible(self, tmp_path):
+        # the paths do not exist, so chmod fails quietly and the retry raises
+        with pytest.raises(FileNotFoundError):
+            clones._make_writable_and_retry(
+                os.remove,
+                str(tmp_path / "no" / "gone"),
+                (None, PermissionError(), None),
+            )
+
+    def test_a_clone_that_is_already_gone_is_not_an_error(self, tmp_path, capsys):
+        repos = tmp_path / "repos"
+        repos.mkdir()
+        remove_clones(str(repos), [str(repos / "gone")], verbose=True)
+        assert capsys.readouterr().err == f"gitlogstats: deleted {repos}\n"