ci-time-tracker 0.3.0__tar.gz → 0.3.1__tar.gz

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ci-time-tracker
-Version: 0.3.0
+Version: 0.3.1
 Summary: Find where CI time and money go: cost, flaky steps and regressions from GitHub Actions, and a PR comment action
 Author: ci-time-tracker contributors
 License-Expression: MIT
@@ -36,22 +36,44 @@ Dynamic: license-file
 
 Find where your CI time and money go. ci-time-tracker reads your GitHub Actions
 history and reports the steps that cost the most, the ones that are flaky, and
-the ones that got slower, with the commit where it started. Its GitHub Action
-comments on each pull request with how it changes CI time and cost.
+the ones that got slower, with the commit where it started. It also recommends
+changes to your workflows, backed by what your runs actually did. Its GitHub
+Action comments on each pull request with how it changes CI time and cost.
+
+```bash
+pip install ci-time-tracker
+ci-time-tracker --github owner/repo --last 300 --format html --output report.html
+```
+
+[![Report for microsoft/TypeScript](docs/example-report.png)](https://0xwhisp.github.io/ci-time-tracker/example-report.html)
+
+**[See the full example report](https://0xwhisp.github.io/ci-time-tracker/example-report.html)**
+for the last 300 runs of microsoft/TypeScript: three flaky steps, and
+recommendations with up to 135 hours of runner time at stake, such as running
+the Windows legs of three jobs only after merging, since in 87 builds they
+never failed when Linux passed.
+
+Set `GITHUB_TOKEN` first (with the GitHub CLI: `export GITHUB_TOKEN=$(gh auth token)`):
+each run costs a request, and without a token GitHub allows 60 an hour.
 
 ## Features
 
-- **Config Analysis**: Parse GitHub Actions, GitLab CI, and CircleCI configurations to extract pipeline structure
-- **Log Analysis**: Parse build logs to compute actual step durations and statistics
+- **History from the API**: Reads workflow runs, jobs and steps straight from
+  the GitHub Actions API, cached locally so later runs take seconds
 - **Flaky Detection**: Flag steps that both failed and passed on the same commit,
   including runs that only passed on a re-run
-- **Regression Detection**: Flag steps whose duration jumped and stayed up, and
-  name the commit where it started
-- **Cost and Time**: Rank steps by the time they consume, estimate cost per runner,
-  and separate queue time from execution time
+- **Regression Detection**: Flag steps whose duration jumped and stayed up, well
+  beyond their usual variation, and name the commit where it started
+- **Recommendations**: Dependency caching, cancelling superseded pull request
+  runs, and matrix legs that never catch anything, each with the time at stake
+- **Cost and Time**: Rank steps by the time they consume, estimate cost per runner
+  at GitHub's list prices, and separate queue time from execution time
 - **Pull Request Comments**: A GitHub Action that tells each pull request how it
   changes CI time and cost, and whether its failures are in known flaky steps
-- **Multiple Output Formats**: Human-readable text or JSON for dashboard integration
+- **Output Formats**: A self-contained HTML report, Markdown for
+  `$GITHUB_STEP_SUMMARY`, text, or JSON for dashboards; `--fail-on` gates
+- **Config and Log Analysis**: Also parses GitHub Actions, GitLab CI and
+  CircleCI configurations and build logs, for providers without API support yet
 
 ## Installation
 
@@ -376,7 +398,7 @@ jobs:
     if: github.event.workflow_run.event == 'pull_request'
     runs-on: ubuntu-latest
     steps:
-      - uses: 0xWhisp/ci-time-tracker@v0.3.0
+      - uses: 0xWhisp/ci-time-tracker@v0.3.1
 ```
 
 `workflow_run` runs the copy of the workflow on the default branch with the
@@ -558,7 +580,11 @@ A step has **regressed** when its duration jumped and stayed up:
 2. The later segment's median must be at least **25%** higher
    (`--regression-threshold`) and at least **10 seconds** higher, so `1s -> 2s`
    steps are not reported.
-3. The most recent 5 builds must still be that slow, so a spike that already
+3. The jump must be at least **three times** the step's usual variation (the
+   median absolute deviation of the noisier segment). With hundreds of steps
+   checked, the best split of a merely noisy one would otherwise clear the
+   thresholds above by chance.
+4. The most recent 5 builds must still be that slow, so a spike that already
    recovered is not reported.
 
 The report names the commit and build where the regression started. A step
@@ -566,6 +592,10 @@ needs at least **10 builds** to be judged; `regression_checked_steps` says how
 many had enough history. With `--group-matrix`, the legs of a build are
 combined into one sample (their median).
 
+Runs that never executed (awaiting a maintainer's approval, skipped by the
+workflow's conditions, or stale) have no timings and are left out; they do not
+count towards `--last`.
+
 When the analyzed runs span several branches, regressions are looked for on
 the repository's default branch only (`regression_branch`). Pull request
 branches run different code with different caches; mixed into one history
@@ -621,7 +651,7 @@ Then, for each release:
 1. Bump `version` in `pyproject.toml` and `__version__` in
    `ci_time_tracker/__init__.py` (a test checks they agree), and add the
    release to [CHANGELOG.md](CHANGELOG.md).
-2. Tag and push: `git tag v0.3.0 && git push origin v0.3.0`.
+2. Tag and push: `git tag v0.3.1 && git push origin v0.3.1`.
 3. Create the GitHub release from the tag, with the changelog entry as its
    notes, and tick "Publish this Action to the GitHub Marketplace".
 

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/README.md

@@ -5,22 +5,44 @@
 
 Find where your CI time and money go. ci-time-tracker reads your GitHub Actions
 history and reports the steps that cost the most, the ones that are flaky, and
-the ones that got slower, with the commit where it started. Its GitHub Action
-comments on each pull request with how it changes CI time and cost.
+the ones that got slower, with the commit where it started. It also recommends
+changes to your workflows, backed by what your runs actually did. Its GitHub
+Action comments on each pull request with how it changes CI time and cost.
+
+```bash
+pip install ci-time-tracker
+ci-time-tracker --github owner/repo --last 300 --format html --output report.html
+```
+
+[![Report for microsoft/TypeScript](docs/example-report.png)](https://0xwhisp.github.io/ci-time-tracker/example-report.html)
+
+**[See the full example report](https://0xwhisp.github.io/ci-time-tracker/example-report.html)**
+for the last 300 runs of microsoft/TypeScript: three flaky steps, and
+recommendations with up to 135 hours of runner time at stake, such as running
+the Windows legs of three jobs only after merging, since in 87 builds they
+never failed when Linux passed.
+
+Set `GITHUB_TOKEN` first (with the GitHub CLI: `export GITHUB_TOKEN=$(gh auth token)`):
+each run costs a request, and without a token GitHub allows 60 an hour.
 
 ## Features
 
-- **Config Analysis**: Parse GitHub Actions, GitLab CI, and CircleCI configurations to extract pipeline structure
-- **Log Analysis**: Parse build logs to compute actual step durations and statistics
+- **History from the API**: Reads workflow runs, jobs and steps straight from
+  the GitHub Actions API, cached locally so later runs take seconds
 - **Flaky Detection**: Flag steps that both failed and passed on the same commit,
   including runs that only passed on a re-run
-- **Regression Detection**: Flag steps whose duration jumped and stayed up, and
-  name the commit where it started
-- **Cost and Time**: Rank steps by the time they consume, estimate cost per runner,
-  and separate queue time from execution time
+- **Regression Detection**: Flag steps whose duration jumped and stayed up, well
+  beyond their usual variation, and name the commit where it started
+- **Recommendations**: Dependency caching, cancelling superseded pull request
+  runs, and matrix legs that never catch anything, each with the time at stake
+- **Cost and Time**: Rank steps by the time they consume, estimate cost per runner
+  at GitHub's list prices, and separate queue time from execution time
 - **Pull Request Comments**: A GitHub Action that tells each pull request how it
   changes CI time and cost, and whether its failures are in known flaky steps
-- **Multiple Output Formats**: Human-readable text or JSON for dashboard integration
+- **Output Formats**: A self-contained HTML report, Markdown for
+  `$GITHUB_STEP_SUMMARY`, text, or JSON for dashboards; `--fail-on` gates
+- **Config and Log Analysis**: Also parses GitHub Actions, GitLab CI and
+  CircleCI configurations and build logs, for providers without API support yet
 
 ## Installation
 
@@ -345,7 +367,7 @@ jobs:
     if: github.event.workflow_run.event == 'pull_request'
     runs-on: ubuntu-latest
     steps:
-      - uses: 0xWhisp/ci-time-tracker@v0.3.0
+      - uses: 0xWhisp/ci-time-tracker@v0.3.1
 ```
 
 `workflow_run` runs the copy of the workflow on the default branch with the
@@ -527,7 +549,11 @@ A step has **regressed** when its duration jumped and stayed up:
 2. The later segment's median must be at least **25%** higher
    (`--regression-threshold`) and at least **10 seconds** higher, so `1s -> 2s`
    steps are not reported.
-3. The most recent 5 builds must still be that slow, so a spike that already
+3. The jump must be at least **three times** the step's usual variation (the
+   median absolute deviation of the noisier segment). With hundreds of steps
+   checked, the best split of a merely noisy one would otherwise clear the
+   thresholds above by chance.
+4. The most recent 5 builds must still be that slow, so a spike that already
    recovered is not reported.
 
 The report names the commit and build where the regression started. A step
@@ -535,6 +561,10 @@ needs at least **10 builds** to be judged; `regression_checked_steps` says how
 many had enough history. With `--group-matrix`, the legs of a build are
 combined into one sample (their median).
 
+Runs that never executed (awaiting a maintainer's approval, skipped by the
+workflow's conditions, or stale) have no timings and are left out; they do not
+count towards `--last`.
+
 When the analyzed runs span several branches, regressions are looked for on
 the repository's default branch only (`regression_branch`). Pull request
 branches run different code with different caches; mixed into one history
@@ -590,7 +620,7 @@ Then, for each release:
 1. Bump `version` in `pyproject.toml` and `__version__` in
    `ci_time_tracker/__init__.py` (a test checks they agree), and add the
    release to [CHANGELOG.md](CHANGELOG.md).
-2. Tag and push: `git tag v0.3.0 && git push origin v0.3.0`.
+2. Tag and push: `git tag v0.3.1 && git push origin v0.3.1`.
 3. Create the GitHub release from the tag, with the changelog entry as its
    notes, and tick "Publish this Action to the GitHub Marketplace".
 

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/ci_time_tracker/__init__.py

@@ -1,3 +1,3 @@
 """ci-time-tracker: A CLI tool for analyzing CI/CD pipeline configurations and build logs."""
 
-__version__ = "0.3.0"
+__version__ = "0.3.1"

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/ci_time_tracker/cache.py

@@ -2,7 +2,8 @@
 
 Listing workflow runs is cheap, but every run costs an extra request for its
 jobs. Those payloads no longer change once a run has completed, so they are
-stored in a small SQLite database between invocations.
+stored in a small SQLite database between invocations. Workflow files are
+stored too, keyed by the commit they were read at, which pins their content.
 """
 
 import json
@@ -21,6 +22,14 @@ CREATE TABLE IF NOT EXISTS run_jobs (
     payload TEXT NOT NULL,
     PRIMARY KEY (repo, run_id, run_attempt)
 );
+CREATE TABLE IF NOT EXISTS workflow_files (
+    repo TEXT NOT NULL,
+    path TEXT NOT NULL,
+    ref TEXT NOT NULL,
+    fetched_at TEXT NOT NULL,
+    content TEXT NOT NULL,
+    PRIMARY KEY (repo, path, ref)
+);
 """
 
 
@@ -93,6 +102,23 @@ class RunCache:
         )
         self._connection.commit()
 
+    def get_workflow_file(self, repo: str, path: str, ref: str) -> str | None:
+        """Return a workflow file as read at commit `ref`, or None when not cached."""
+        row = self._connection.execute(
+            "SELECT content FROM workflow_files WHERE repo = ? AND path = ? AND ref = ?",
+            (repo, path, ref),
+        ).fetchone()
+        return None if row is None else row[0]
+
+    def store_workflow_file(self, repo: str, path: str, ref: str, content: str) -> None:
+        """Cache a workflow file as read at commit `ref`."""
+        self._connection.execute(
+            "INSERT OR REPLACE INTO workflow_files "
+            "(repo, path, ref, fetched_at, content) VALUES (?, ?, ?, ?, ?)",
+            (repo, path, ref, datetime.now(timezone.utc).isoformat(), content),
+        )
+        self._connection.commit()
+
     def count(self, repo: str | None = None) -> int:
         """Number of cached runs, optionally limited to one repository."""
         if repo is None:

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/ci_time_tracker/cli.py

@@ -7,11 +7,12 @@ parsing, file I/O, and orchestration of parsing, analysis, and reporting.
 import argparse
 import json
 import sys
-from concurrent.futures import ThreadPoolExecutor
+from concurrent.futures import Future, ThreadPoolExecutor
 from dataclasses import replace
 from pathlib import Path
 from typing import Any
 
+from ci_time_tracker import __version__
 from ci_time_tracker.analyzer import analyze_config, analyze_logs
 from ci_time_tracker.cache import RunCache
 from ci_time_tracker.config_parser import parse_config
@@ -86,6 +87,7 @@ def parse_args(args: list[str] | None = None) -> argparse.Namespace:
         description="Analyze CI/CD pipeline configurations and build logs for performance bottlenecks and flaky steps.",
         epilog="Example: ci-time-tracker --config .github/workflows/ci.yml --format text",
     )
+    parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
     
     # Mode selection (mutually exclusive)
     mode_group = parser.add_mutually_exclusive_group(required=True)
@@ -362,19 +364,29 @@ def load_estimates(path: str) -> dict[str, float]:
     return estimates
 
 
-def _default_branch_if_mixed(repo: str, logs: list[Any]) -> str | None:
+def _default_branch(repo: str) -> str | None:
+    """The repository's default branch, or None when it cannot be read."""
+    try:
+        return GitHubClient(repo=repo, token=resolve_token()).fetch_default_branch()
+    except (GitHubAPIError, KeyError, TypeError):
+        return None
+
+
+def _default_branch_if_mixed(
+    repo: str,
+    logs: list[Any],
+    lookup: Future[str | None] | None = None,
+) -> str | None:
     """The repository's default branch, when the runs span several branches.
 
     Regressions are looked for on that branch only; with a single branch
-    there is nothing to choose and no request is made.
+    there is nothing to choose. `lookup` is a request already under way;
+    without one, a request is made only when needed.
     """
     branches = {log.metadata.get("branch") for log in logs if log.metadata.get("branch")}
     if len(branches) < 2:
         return None
-    try:
-        return GitHubClient(repo=repo, token=resolve_token()).fetch_default_branch()
-    except (GitHubAPIError, KeyError, TypeError):
-        return None
+    return lookup.result() if lookup is not None else _default_branch(repo)
 
 
 def _visibility_pricing(logs: list[Any], pricing: Any) -> tuple[PricingTable, str | None]:
@@ -400,38 +412,69 @@ def _visibility_pricing(logs: list[Any], pricing: Any) -> tuple[PricingTable, st
 _MAX_WORKFLOW_FILES = 20
 
 
-def _fetch_workflows(repo: str, logs: list[Any]) -> dict[str, dict[str, Any]]:
+def _workflow_refs(repo: str, logs: list[Any]) -> dict[str, str | None]:
+    """Map each workflow file behind the runs to the commit to read it at.
+
+    That is the newest run's commit from the repository itself (a fork's
+    commit may not be readable through it). None reads the default branch.
+    """
+    def rank(log: Any) -> tuple[bool, float]:
+        own = log.metadata.get("head_repository") in (None, repo)
+        return bool(own and log.head_sha), log.timestamp.timestamp() if log.timestamp else float("-inf")
+
+    newest: dict[str, Any] = {}
+    for log in logs:
+        path = str(log.metadata.get("workflow_path") or "")
+        if path.startswith(".github/workflows/") and (path not in newest or rank(log) > rank(newest[path])):
+            newest[path] = log
+
+    return {
+        path: newest[path].head_sha if rank(newest[path])[0] else None
+        for path in sorted(newest)[:_MAX_WORKFLOW_FILES]
+    }
+
+
+def _fetch_workflows(repo: str, logs: list[Any], cache: Any = None) -> dict[str, dict[str, Any]]:
     """Fetch and parse the workflow files behind the analyzed runs.
 
     Recommendations that need a workflow's configuration skip the ones that
     cannot be read, so a failure here only narrows what is recommended.
+    Files read at a commit are cached, since that pins their content.
     """
-    paths = sorted({
-        log.metadata["workflow_path"]
-        for log in logs
-        if str(log.metadata.get("workflow_path") or "").startswith(".github/workflows/")
-    })[:_MAX_WORKFLOW_FILES]
-
-    if not paths:
-        return {}
-
-    client = GitHubClient(repo=repo, token=resolve_token())
+    refs = _workflow_refs(repo, logs)
+    texts: dict[str, str | None] = {}
+    if cache is not None:
+        for path, ref in refs.items():
+            if ref:
+                cached = cache.get_workflow_file(repo, path, ref)
+                if cached is not None:
+                    texts[path] = cached
+
+    missing = [path for path in refs if path not in texts]
+    if missing:
+        client = GitHubClient(repo=repo, token=resolve_token())
+
+        def fetch(path: str) -> tuple[str, str | None]:
+            try:
+                return path, client.fetch_workflow_file(path, ref=refs[path])
+            except GitHubAPIError as e:
+                print(f"Warning: could not read {path} ({e}); skipping its recommendations", file=sys.stderr)
+                return path, None
 
-    def fetch(path: str) -> tuple[str, str | None]:
-        try:
-            return path, client.fetch_workflow_file(path)
-        except GitHubAPIError as e:
-            print(f"Warning: could not read {path} ({e}); skipping its recommendations", file=sys.stderr)
-            return path, None
+        # Fetched concurrently: one after the other they added over a second
+        # to an analysis whose runs were all cached
+        with ThreadPoolExecutor(max_workers=min(DEFAULT_WORKERS, len(missing))) as pool:
+            for path, text in pool.map(fetch, missing):
+                texts[path] = text
+                if text is not None and cache is not None and refs[path]:
+                    cache.store_workflow_file(repo, path, refs[path], text)
 
-    # Fetched concurrently: these requests are not cached, and one after the
-    # other they added over a second to a cached analysis
     workflows: dict[str, dict[str, Any]] = {}
-    with ThreadPoolExecutor(max_workers=min(DEFAULT_WORKERS, len(paths))) as pool:
-        for path, text in pool.map(fetch, paths):
-            parsed = parse_workflow(text) if text else None
-            if parsed is not None:
-                workflows[path] = parsed
+    for path in refs:
+        text = texts.get(path)
+        parsed = parse_workflow(text) if text else None
+        if parsed is not None:
+            workflows[path] = parsed
 
     return workflows
 
@@ -672,6 +715,14 @@ def main() -> int:
                 except Exception as e:
                     print(f"Warning: run cache unavailable ({e})", file=sys.stderr)
 
+            # Runs of every branch almost always need the default branch to
+            # look for regressions on, so it is asked for while listing
+            lookup = None
+            if args.branch is None:
+                lookups = ThreadPoolExecutor(max_workers=1)
+                lookup = lookups.submit(_default_branch, args.github)
+                lookups.shutdown(wait=False)
+
             skipped: list[str] = []
             try:
                 logs = fetch_build_logs(
@@ -682,6 +733,7 @@ def main() -> int:
                     cache=cache,
                     skipped=skipped,
                 )
+                workflows = _fetch_workflows(args.github, logs, cache) if logs else {}
             except GitHubAPIError as e:
                 print(f"Error: {e}", file=sys.stderr)
                 return 3
@@ -700,7 +752,7 @@ def main() -> int:
                 pricing,
                 args.group_matrix,
                 regression_threshold=args.regression_threshold,
-                regression_branch=_default_branch_if_mixed(args.github, logs),
+                regression_branch=_default_branch_if_mixed(args.github, logs, lookup),
             )
             result.metadata["repository"] = args.github
             if visibility:
@@ -713,7 +765,7 @@ def main() -> int:
                     file=sys.stderr,
                 )
             result.recommendations = [
-                item.to_dict() for item in recommend(logs, _fetch_workflows(args.github, logs), pricing)
+                item.to_dict() for item in recommend(logs, workflows, pricing)
             ]
 
         else:

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/ci_time_tracker/detection.py

@@ -8,8 +8,9 @@ its job.
 
 Regression: a step whose duration jumped and stayed up. The step's history is
 split at the point that best separates "before" from "after", and the jump
-must be large in both relative and absolute terms and still hold for the most
-recent builds, so that a spike that already recovered is not reported.
+must be large in both relative and absolute terms, large against the step's
+own variation, and still hold for the most recent builds, so that a spike that
+already recovered is not reported.
 """
 
 import heapq
@@ -24,6 +25,12 @@ DEFAULT_REGRESSION_THRESHOLD = 0.25
 DEFAULT_MIN_SAMPLES = 5
 DEFAULT_MIN_INCREASE_SECONDS = 10.0
 
+# The jump must be this many times the noisier segment's median absolute
+# deviation. Checking hundreds of steps, the best split of a merely noisy one
+# often clears the relative and absolute thresholds: in field tests such
+# splits stayed under 3 times the noise, a real regression went well above.
+DEFAULT_MIN_SHIFT_SPREADS = 3.0
+
 _EXAMPLE_COMMITS = 3
 _SHORT_SHA = 7
 
@@ -160,19 +167,26 @@ def _prefix_medians_and_costs(values: list[float]) -> list[tuple[float, float]]:
     return results
 
 
+def _median_absolute_deviation(values: list[float]) -> float:
+    center = median(values)
+    return median(abs(value - center) for value in values)
+
+
 def detect_regression(
     samples: list[tuple[float, BuildLog]],
     threshold: float = DEFAULT_REGRESSION_THRESHOLD,
     min_samples: int = DEFAULT_MIN_SAMPLES,
     min_increase_seconds: float = DEFAULT_MIN_INCREASE_SECONDS,
+    min_shift_spreads: float = DEFAULT_MIN_SHIFT_SPREADS,
 ) -> dict[str, Any] | None:
     """Detect a lasting increase in a step's duration.
 
     Finds the single split that best divides the history into two stable
     segments (least total absolute deviation from each segment's median),
     then reports a regression only when the later segment is slower by at
-    least `threshold` and `min_increase_seconds`, and the most recent
-    `min_samples` builds are still that slow.
+    least `threshold`, `min_increase_seconds` and `min_shift_spreads` times
+    the noisier segment's variation, and the most recent `min_samples` builds
+    are still that slow.
 
     Args:
         samples: (duration, build) pairs in chronological order, one per build
@@ -180,6 +194,9 @@ def detect_regression(
         min_samples: Minimum builds on each side of the split
         min_increase_seconds: Minimum absolute increase, so that 1s -> 2s
             steps are not reported
+        min_shift_spreads: Minimum increase in units of the noisier
+            segment's median absolute deviation, so that the best split of a
+            noisy step is not reported
 
     Returns:
         Regression details, or None when there is none or too little history
@@ -217,6 +234,10 @@ def detect_regression(
     if current - baseline < min_increase_seconds or recent - baseline < min_increase_seconds:
         return None
 
+    spread = max(_median_absolute_deviation(durations[:split]), _median_absolute_deviation(durations[split:]))
+    if current - baseline < min_shift_spreads * spread:
+        return None
+
     started = samples[split][1]
 
     return {

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/ci_time_tracker/github_api.py

@@ -7,6 +7,7 @@ so analysis does not depend on parsing runner log text.
 
 import base64
 import json
+import math
 import os
 import time
 import urllib.error
@@ -30,6 +31,19 @@ DEFAULT_WORKERS = 8
 
 _PER_PAGE = 100
 
+# GitHub takes longer to build a bigger page of runs, so a limit is split into
+# smaller pages, one per worker, down to this size
+_MIN_RUNS_PAGE = 25
+
+# Runs that never ran are skipped and made up from later pages, but the
+# listing stops after this many times the pages the limit needed
+_MAX_LISTING_FACTOR = 3
+
+# Run conclusions for runs that never executed: waiting for a maintainer's
+# approval (pull requests from forks), skipped by the workflow's conditions,
+# or superseded while queued. They have no timings to analyze.
+_NOT_RUN_CONCLUSIONS = frozenset({"action_required", "skipped", "stale"})
+
 # A run with many jobs, each with many steps, can make GitHub time out
 # building a 100-job page (502/504); such runs are paged in smaller pieces
 _SMALL_JOBS_PAGE = 30
@@ -320,6 +334,7 @@ class GitHubClient:
         branch: str | None = None,
         limit: int = DEFAULT_RUN_LIMIT,
         status: str | None = "completed",
+        include: Callable[[dict[str, Any]], bool] | None = None,
     ) -> Iterator[dict[str, Any]]:
         """Yield up to `limit` workflow runs, newest first.
 
@@ -330,6 +345,8 @@ class GitHubClient:
             limit: Maximum number of runs to yield
             status: Run status filter; "completed" skips in-flight runs whose
                 timings are not final yet
+            include: Optional filter; runs it rejects are not yielded and do
+                not count towards `limit`, so later pages are read instead
         """
         if limit <= 0:
             return
@@ -343,9 +360,15 @@ class GitHubClient:
         # together: listing is the slowest part of a cached analysis, at a few
         # seconds a page. The pull requests attached to each run are left out
         # of the payload; nothing reads them (a run's pull request is looked up
-        # by its commit).
-        per_page = min(_PER_PAGE, limit)
-        pages = -(-limit // per_page)
+        # by its commit). Eight pages of 25 come back in about half the time
+        # of two pages of 100.
+        # With a filter, a margin is listed up front: a second round of pages
+        # for the runs it dropped would add a few seconds.
+        target = limit + limit // 4 if include is not None else limit
+        per_page = min(_PER_PAGE, max(_MIN_RUNS_PAGE, -(-target // DEFAULT_WORKERS)), target)
+        pages = -(-target // per_page)
+        # Filtered out runs are made up from later pages, within reason
+        last_page = pages * _MAX_LISTING_FACTOR
 
         def fetch(page: int) -> list[dict[str, Any]]:
             payload = self._get(
@@ -360,27 +383,39 @@ class GitHubClient:
             )
             return payload.get("workflow_runs", []) if isinstance(payload, dict) else []
 
-        with ThreadPoolExecutor(max_workers=min(DEFAULT_WORKERS, pages)) as pool:
-            batches = list(pool.map(fetch, range(1, pages + 1)))
-
         seen: set[Any] = set()
-        yielded = 0
-
-        for runs in batches:
-            for run in runs:
-                # A run created while the pages were fetched shifts the others
-                # by one, which can show a run on two pages
-                if run.get("id") in seen:
-                    continue
-                seen.add(run.get("id"))
-                yield run
-                yielded += 1
-                if yielded >= limit:
+        yielded = listed = 0
+        first = 1
+
+        while first <= last_page:
+            with ThreadPoolExecutor(max_workers=min(DEFAULT_WORKERS, pages)) as pool:
+                batches = list(pool.map(fetch, range(first, first + pages)))
+
+            for runs in batches:
+                for run in runs:
+                    # A run created while the pages were fetched shifts the
+                    # others by one, which can show a run on two pages
+                    if run.get("id") in seen:
+                        continue
+                    seen.add(run.get("id"))
+                    listed += 1
+                    if include is not None and not include(run):
+                        continue
+                    yield run
+                    yielded += 1
+                    if yielded >= limit:
+                        return
+
+                # A short page is the last one; later pages can only be empty
+                if len(runs) < per_page:
                     return
 
-            # A short page is the last one; later pages can only be empty
-            if len(runs) < per_page:
-                return
+            # Enough further pages for the runs still missing, at the share
+            # of listed runs the filter has kept so far
+            first += pages
+            kept = max(yielded / listed, 0.1)
+            needed = math.ceil((limit - yielded) / (per_page * kept))
+            pages = max(1, min(DEFAULT_WORKERS, last_page - first + 1, needed))
 
     def fetch_run_jobs(self, run_id: int | str, run_attempt: int | None = None) -> list[dict[str, Any]]:
         """Return every job of a workflow run attempt, following pagination.
@@ -567,6 +602,11 @@ def _earlier_attempt(run: dict[str, Any], attempt: int) -> dict[str, Any]:
     }
 
 
+def _ran(run: dict[str, Any]) -> bool:
+    """Whether a run executed at all, so that it has timings to analyze."""
+    return run.get("conclusion") not in _NOT_RUN_CONCLUSIONS
+
+
 def fetch_build_logs(
     repo: str,
     workflow: str | None = None,
@@ -587,6 +627,9 @@ def fetch_build_logs(
     served from `cache` (see cache.RunCache) and the rest are fetched
     concurrently.
 
+    Runs that never executed (awaiting approval, skipped, stale) are left
+    out and do not count towards `limit`.
+
     The runs listing only shows the latest attempt of a re-run. The earlier
     attempts are fetched too, as BuildLogs of their own: a step that failed
     and then passed on a re-run is the strongest evidence of flakiness.
@@ -623,7 +666,7 @@ def fetch_build_logs(
     # One entry per (run, attempt), newest first: (payload, attempt, superseded)
     attempts: list[tuple[dict[str, Any], int, bool]] = []
 
-    for run in client.iter_workflow_runs(workflow=workflow, branch=branch, limit=limit):
+    for run in client.iter_workflow_runs(workflow=workflow, branch=branch, limit=limit, include=_ran):
         if run.get("id") is None:
             continue
 

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/ci_time_tracker/html_report.py

@@ -8,6 +8,7 @@ pull requests) is escaped.
 
 import html
 import math
+from collections import Counter
 from statistics import median
 from typing import Any
 
@@ -139,8 +140,13 @@ def _summary_cards(report: Report) -> str:
             note = "at GitHub list prices"
         cards.append(_card("Estimated cost", format_cost(summary["estimated_cost_usd"]), note))
 
-    issues = len(report.issues)
-    cards.append(_card("Issues", str(issues), "regressions and flaky steps"))
+    kinds = Counter(issue.get("type") for issue in report.issues)
+    parts = [
+        f"{count} {label if count == 1 else label + 's'}"
+        for label, count in (("regression", kinds["regression"]), ("flaky step", kinds["flaky"]))
+        if count
+    ]
+    cards.append(_card("Issues", str(len(report.issues)), " · ".join(parts) or "no regressions or flaky steps"))
 
     return f'<div class="cards">{"".join(cards)}</div>'
 

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/ci_time_tracker.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ci-time-tracker
-Version: 0.3.0
+Version: 0.3.1
 Summary: Find where CI time and money go: cost, flaky steps and regressions from GitHub Actions, and a PR comment action
 Author: ci-time-tracker contributors
 License-Expression: MIT
@@ -36,22 +36,44 @@ Dynamic: license-file
 
 Find where your CI time and money go. ci-time-tracker reads your GitHub Actions
 history and reports the steps that cost the most, the ones that are flaky, and
-the ones that got slower, with the commit where it started. Its GitHub Action
-comments on each pull request with how it changes CI time and cost.
+the ones that got slower, with the commit where it started. It also recommends
+changes to your workflows, backed by what your runs actually did. Its GitHub
+Action comments on each pull request with how it changes CI time and cost.
+
+```bash
+pip install ci-time-tracker
+ci-time-tracker --github owner/repo --last 300 --format html --output report.html
+```
+
+[![Report for microsoft/TypeScript](docs/example-report.png)](https://0xwhisp.github.io/ci-time-tracker/example-report.html)
+
+**[See the full example report](https://0xwhisp.github.io/ci-time-tracker/example-report.html)**
+for the last 300 runs of microsoft/TypeScript: three flaky steps, and
+recommendations with up to 135 hours of runner time at stake, such as running
+the Windows legs of three jobs only after merging, since in 87 builds they
+never failed when Linux passed.
+
+Set `GITHUB_TOKEN` first (with the GitHub CLI: `export GITHUB_TOKEN=$(gh auth token)`):
+each run costs a request, and without a token GitHub allows 60 an hour.
 
 ## Features
 
-- **Config Analysis**: Parse GitHub Actions, GitLab CI, and CircleCI configurations to extract pipeline structure
-- **Log Analysis**: Parse build logs to compute actual step durations and statistics
+- **History from the API**: Reads workflow runs, jobs and steps straight from
+  the GitHub Actions API, cached locally so later runs take seconds
 - **Flaky Detection**: Flag steps that both failed and passed on the same commit,
   including runs that only passed on a re-run
-- **Regression Detection**: Flag steps whose duration jumped and stayed up, and
-  name the commit where it started
-- **Cost and Time**: Rank steps by the time they consume, estimate cost per runner,
-  and separate queue time from execution time
+- **Regression Detection**: Flag steps whose duration jumped and stayed up, well
+  beyond their usual variation, and name the commit where it started
+- **Recommendations**: Dependency caching, cancelling superseded pull request
+  runs, and matrix legs that never catch anything, each with the time at stake
+- **Cost and Time**: Rank steps by the time they consume, estimate cost per runner
+  at GitHub's list prices, and separate queue time from execution time
 - **Pull Request Comments**: A GitHub Action that tells each pull request how it
   changes CI time and cost, and whether its failures are in known flaky steps
-- **Multiple Output Formats**: Human-readable text or JSON for dashboard integration
+- **Output Formats**: A self-contained HTML report, Markdown for
+  `$GITHUB_STEP_SUMMARY`, text, or JSON for dashboards; `--fail-on` gates
+- **Config and Log Analysis**: Also parses GitHub Actions, GitLab CI and
+  CircleCI configurations and build logs, for providers without API support yet
 
 ## Installation
 
@@ -376,7 +398,7 @@ jobs:
     if: github.event.workflow_run.event == 'pull_request'
     runs-on: ubuntu-latest
     steps:
-      - uses: 0xWhisp/ci-time-tracker@v0.3.0
+      - uses: 0xWhisp/ci-time-tracker@v0.3.1
 ```
 
 `workflow_run` runs the copy of the workflow on the default branch with the
@@ -558,7 +580,11 @@ A step has **regressed** when its duration jumped and stayed up:
 2. The later segment's median must be at least **25%** higher
    (`--regression-threshold`) and at least **10 seconds** higher, so `1s -> 2s`
    steps are not reported.
-3. The most recent 5 builds must still be that slow, so a spike that already
+3. The jump must be at least **three times** the step's usual variation (the
+   median absolute deviation of the noisier segment). With hundreds of steps
+   checked, the best split of a merely noisy one would otherwise clear the
+   thresholds above by chance.
+4. The most recent 5 builds must still be that slow, so a spike that already
    recovered is not reported.
 
 The report names the commit and build where the regression started. A step
@@ -566,6 +592,10 @@ needs at least **10 builds** to be judged; `regression_checked_steps` says how
 many had enough history. With `--group-matrix`, the legs of a build are
 combined into one sample (their median).
 
+Runs that never executed (awaiting a maintainer's approval, skipped by the
+workflow's conditions, or stale) have no timings and are left out; they do not
+count towards `--last`.
+
 When the analyzed runs span several branches, regressions are looked for on
 the repository's default branch only (`regression_branch`). Pull request
 branches run different code with different caches; mixed into one history
@@ -621,7 +651,7 @@ Then, for each release:
 1. Bump `version` in `pyproject.toml` and `__version__` in
    `ci_time_tracker/__init__.py` (a test checks they agree), and add the
    release to [CHANGELOG.md](CHANGELOG.md).
-2. Tag and push: `git tag v0.3.0 && git push origin v0.3.0`.
+2. Tag and push: `git tag v0.3.1 && git push origin v0.3.1`.
 3. Create the GitHub release from the tag, with the changelog entry as its
    notes, and tick "Publish this Action to the GitHub Marketplace".
 

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/ci_time_tracker.egg-info/SOURCES.txt

@@ -36,4 +36,5 @@ tests/test_log_parser.py
 tests/test_metrics.py
 tests/test_packaging.py
 tests/test_recommendations.py
-tests/test_reporter.py
+tests/test_reporter.py
+tests/test_workflow_cache.py

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ci-time-tracker"
-version = "0.3.0"
+version = "0.3.1"
 description = "Find where CI time and money go: cost, flaky steps and regressions from GitHub Actions, and a PR comment action"
 readme = "README.md"
 license = "MIT"

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/tests/test_cli.py

@@ -574,3 +574,14 @@ structure: true
         finally:
             Path(config_path).unlink()
 
+
+
+def test_version(monkeypatch, capsys):
+    from ci_time_tracker import __version__
+
+    monkeypatch.setattr("sys.argv", ["ci-time-tracker", "--version"])
+    with pytest.raises(SystemExit) as exit_info:
+        main()
+
+    assert exit_info.value.code == 0
+    assert capsys.readouterr().out.strip() == f"ci-time-tracker {__version__}"

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/tests/test_detection.py

@@ -300,7 +300,8 @@ class TestFastChangePoint:
         durations = [float(value) for value in values]
 
         expected = reference_regression(durations)
-        result = detect_regression(samples(durations))
+        # The oracle covers the search; the noise rule has its own tests
+        result = detect_regression(samples(durations), min_shift_spreads=0)
 
         if expected is None:
             assert result is None
@@ -381,3 +382,30 @@ class TestRegressionThresholdOption:
     def test_invalid_values_are_rejected(self, value):
         with pytest.raises(SystemExit):
             parse_args(["--logs", "x", "--regression-threshold", value])
+
+
+class TestRegressionAgainstNoise:
+    """Seen on pydantic: 16 regressions in one day, all the best split of a noisy step."""
+
+    def test_the_best_split_of_a_noisy_step_is_not_reported(self):
+        # pydantic's "Test ubuntu-latest / 3.12 / Run pytest": medians 121 -> 166,
+        # yet both halves spread as widely as the jump between them
+        durations = [121.0, 95.0, 160.0, 118.0, 140.0, 166.0, 120.0, 190.0, 171.0, 150.0]
+
+        assert detect_regression(samples(durations), min_shift_spreads=0) is not None
+        assert detect_regression(samples(durations)) is None
+
+    def test_a_clean_jump_is_still_reported(self):
+        durations = [100.0, 104.0, 97.0, 101.0, 99.0, 103.0, 160.0, 158.0, 163.0, 161.0, 157.0, 162.0]
+
+        result = detect_regression(samples(durations))
+
+        assert result is not None
+        assert result["baseline_builds"] == 6
+
+    def test_a_noisy_step_with_a_large_jump_is_still_reported(self):
+        # TypeScript's 623s -> 787s step: noisy, but the jump is four times the noise
+        durations = [600.0, 640.0, 623.0, 610.0, 650.0, 615.0, 630.0,
+                     787.0, 760.0, 800.0, 790.0, 770.0, 810.0, 785.0]
+
+        assert detect_regression(samples(durations)) is not None

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/tests/test_field_test_3.py

@@ -2,6 +2,7 @@
 spring-boot, django, rubocop): regressions on one branch, and matrix-leg
 recommendations grouped per workflow and platform."""
 
+from concurrent.futures import Future
 from datetime import datetime, timedelta, timezone
 
 from ci_time_tracker.analyzer import analyze_logs
@@ -120,6 +121,17 @@ class TestDefaultBranchLookup:
 
         assert _default_branch_if_mixed("acme/web", [build(1, 1.0, "main"), build(2, 1.0, "fix")]) is None
 
+    def test_a_lookup_under_way_is_used(self, monkeypatch):
+        FakeClient.calls = 0
+        monkeypatch.setattr("ci_time_tracker.cli.GitHubClient", FakeClient)
+        lookup = Future()
+        lookup.set_result("trunk")
+
+        mixed = [build(1, 1.0, "main"), build(2, 1.0, "fix")]
+        assert _default_branch_if_mixed("acme/web", mixed, lookup) == "trunk"
+        assert _default_branch_if_mixed("acme/web", mixed[:1], lookup) is None
+        assert FakeClient.calls == 0
+
     def test_client_reads_the_default_branch(self):
         client = GitHubClient(repo="acme/web", transport=lambda url, headers: ({"default_branch": "trunk"}, {}))
 

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/tests/test_github_api.py

@@ -251,34 +251,83 @@ class TestClientRequests:
 
     def test_pagination_walks_pages_until_limit(self):
         """A limit above one page keeps paging."""
-        first_page = [{**RUN, "id": 1000 + index} for index in range(100)]
-        # "&page=1&", not "page=1", which "per_page=100" also contains
+        first_page = [{**RUN, "id": 1000 + index} for index in range(25)]
+        # "&page=1&", not "page=1", which "per_page=..." could also contain
         transport = FakeTransport([
             ("/jobs", JOBS),
             ("&page=1&", {"workflow_runs": first_page}),
             ("&page=2&", {"workflow_runs": [{**RUN, "id": 2000}]}),
         ])
 
-        logs = fetch_build_logs("acme/web", limit=101, transport=transport)
+        logs = fetch_build_logs("acme/web", limit=26, transport=transport)
 
-        assert len(logs) == 101
-        assert len({log.build_id for log in logs}) == 101
+        assert len(logs) == 26
+        assert len({log.build_id for log in logs}) == 26
         assert transport.calls_matching("&page=2&")
 
     def test_pages_are_requested_together_and_deduplicated(self):
         """A run created while listing shifts it onto two pages: it counts once."""
-        first_page = [{**RUN, "id": 1000 + index} for index in range(100)]
-        second_page = [{**RUN, "id": 1099}] + [{**RUN, "id": 2000 + index} for index in range(99)]
+        first_page = [{**RUN, "id": 1000 + index} for index in range(25)]
+        second_page = [{**RUN, "id": 1024}] + [{**RUN, "id": 2000 + index} for index in range(24)]
         transport = FakeTransport([
             ("/jobs", JOBS),
             ("&page=1&", {"workflow_runs": first_page}),
             ("&page=2&", {"workflow_runs": second_page}),
+            ("&page=3&", {"workflow_runs": [{**RUN, "id": 3000}]}),
         ])
 
-        logs = fetch_build_logs("acme/web", limit=200, transport=transport)
+        logs = fetch_build_logs("acme/web", limit=50, transport=transport)
+
+        # The duplicate is made up from the next page
+        assert len(logs) == 50
+        assert len({log.build_id for log in logs}) == 50
+
+    @pytest.mark.parametrize(("limit", "per_page", "pages"), [
+        (10, 10, 1),
+        (50, 25, 2),
+        (200, 25, 8),
+        (400, 50, 8),
+        (1000, 100, 10),
+    ])
+    def test_a_limit_is_split_into_a_page_per_worker(self, limit, per_page, pages):
+        """Smaller pages come back faster, so the workers share the limit."""
+        transport = FakeTransport([("/actions/runs?", {"workflow_runs": []})])
+        list(GitHubClient(repo="acme/web", transport=transport).iter_workflow_runs(limit=limit))
+
+        calls = transport.calls_matching("/actions/runs?")
+        assert len(calls) == pages
+        assert all(f"per_page={per_page}&" in url for url in calls)
+
+    def test_runs_that_never_ran_are_left_out(self):
+        """Awaiting approval, skipped or stale: no timings, and no place in the limit."""
+        never_ran = [{**RUN, "id": 100 + i, "conclusion": c} for i, c in enumerate(["action_required", "skipped", "stale"])]
+        ran = [{**RUN, "id": 200 + i} for i in range(22)]
+        transport = FakeTransport([
+            ("/jobs", JOBS),
+            ("&page=1&", {"workflow_runs": never_ran + ran}),
+            ("&page=2&", {"workflow_runs": [{**RUN, "id": 300 + i} for i in range(25)]}),
+        ])
+
+        logs = fetch_build_logs("acme/web", limit=25, transport=transport)
+
+        assert len(logs) == 25
+        assert not {log.build_id for log in logs} & {"100", "101", "102"}
+        assert not transport.calls_matching("/runs/100/")
+
+    def test_cancelled_runs_are_kept(self):
+        transport = FakeTransport(default_routes(runs=[{**RUN, "conclusion": "cancelled"}]))
+
+        assert len(fetch_build_logs("acme/web", limit=1, transport=transport)) == 1
 
-        assert len(logs) == 199
-        assert len({log.build_id for log in logs}) == 199
+    def test_a_listing_of_runs_that_never_ran_stops(self):
+        """A workflow that is always skipped must not page through its whole history."""
+        transport = FakeTransport([
+            ("/actions/runs?", {"workflow_runs": [{**RUN, "id": 100 + i, "conclusion": "skipped"} for i in range(25)]}),
+        ])
+
+        assert fetch_build_logs("acme/web", limit=50, transport=transport) == []
+        # Three pages hold the limit and its margin; at most three times that are read
+        assert len(transport.calls_matching("/actions/runs?")) == 9
 
     def test_pull_requests_are_left_out_of_the_listing(self):
         transport = FakeTransport(default_routes())
@@ -287,14 +336,17 @@ class TestClientRequests:
         assert "exclude_pull_requests=true" in transport.calls[0]
 
     def test_short_page_is_the_last_one(self):
-        """Fewer runs than asked for means there is no next page to request."""
+        """Fewer runs than a page holds means the later pages are not read."""
         runs = [{**RUN, "id": 100 + index} for index in range(3)]
-        transport = FakeTransport(default_routes(runs=runs))
+        transport = FakeTransport([
+            ("/jobs", JOBS),
+            ("&page=1&", {"workflow_runs": runs}),
+            ("&page=", {"workflow_runs": [{**RUN, "id": 999}]}),
+        ])
 
         logs = fetch_build_logs("acme/web", limit=50, transport=transport)
 
-        assert len(logs) == 3
-        assert len(transport.calls_matching("/actions/runs?")) == 1
+        assert sorted(log.build_id for log in logs) == ["100", "101", "102"]
 
     def test_empty_page_stops_pagination(self):
         transport = FakeTransport([

{ci_time_tracker-0.3.0 → ci_time_tracker-0.3.1}/tests/test_html_report.py

@@ -82,6 +82,20 @@ class TestSummaryCards:
         assert "public repository: standard runners are free" in html
 
 
+    def test_issues_card_counts_each_kind(self):
+        # A regression on main, and a step that failed then passed on one commit
+        logs = [build(i, 100.0 if i < 6 else 200.0) for i in range(12)]
+        flaky = build(12, 200.0, steps=[StepExecution(name="build / Run tests", duration_seconds=200.0, status="failure")])
+        flaky.head_sha = logs[-1].head_sha
+
+        html = page(logs + [flaky])
+
+        assert "1 regression · 1 flaky step" in html
+
+    def test_issues_card_without_issues(self):
+        assert "no regressions or flaky steps" in page([build(i, 600.0) for i in range(6)])
+
+
 class TestBuildDurationCharts:
     def test_one_chart_per_workflow(self):
         """A 10-second workflow next to a 10-minute one would only zigzag."""

ci_time_tracker-0.3.1/tests/test_workflow_cache.py

@@ -0,0 +1,115 @@
+"""Workflow files are cached by the commit they were read at, so an analysis
+whose runs are all cached makes no request for them either."""
+
+import base64
+from datetime import datetime, timedelta, timezone
+
+from ci_time_tracker.cache import RunCache
+from ci_time_tracker.cli import _fetch_workflows, _workflow_refs
+from ci_time_tracker.github_api import GitHubClient
+from ci_time_tracker.models import BuildLog
+
+START = datetime(2026, 10, 1, 10, 0, tzinfo=timezone.utc)
+PATH = ".github/workflows/ci.yml"
+YAML = "jobs:\n  test:\n    steps:\n      - run: pip install .\n"
+
+
+def log(index, sha="", path=PATH, head_repository="acme/web"):
+    return BuildLog(
+        build_id=str(index),
+        head_sha=sha or f"{index:040x}",
+        timestamp=START + timedelta(hours=index),
+        metadata={"workflow_path": path, "head_repository": head_repository},
+    )
+
+
+class Recorder:
+    """Stands in for GitHubClient(repo, token) and records the URLs asked for."""
+
+    urls: list[str] = []
+
+    def __new__(cls, repo, token=None):
+        def transport(url, headers):
+            Recorder.urls.append(url)
+            return {"encoding": "base64", "content": base64.b64encode(YAML.encode()).decode()}, {}
+
+        return GitHubClient(repo=repo, transport=transport)
+
+
+class TestCacheTable:
+    def test_a_file_is_cached_per_commit(self, tmp_path):
+        with RunCache(tmp_path / "runs.db") as cache:
+            cache.store_workflow_file("acme/web", PATH, "abc", YAML)
+
+            assert cache.get_workflow_file("acme/web", PATH, "abc") == YAML
+            assert cache.get_workflow_file("acme/web", PATH, "def") is None
+            assert cache.get_workflow_file("acme/api", PATH, "abc") is None
+
+    def test_an_existing_database_gains_the_table(self, tmp_path):
+        RunCache(tmp_path / "runs.db").close()
+
+        with RunCache(tmp_path / "runs.db") as cache:
+            cache.store_workflow_file("acme/web", PATH, "abc", YAML)
+            assert cache.get_workflow_file("acme/web", PATH, "abc") == YAML
+
+
+class TestWorkflowRefs:
+    def test_the_newest_run_picks_the_commit(self):
+        refs = _workflow_refs("acme/web", [log(1, "a" * 40), log(3, "c" * 40), log(2, "b" * 40)])
+
+        assert refs == {PATH: "c" * 40}
+
+    def test_a_fork_commit_is_not_used(self):
+        refs = _workflow_refs("acme/web", [log(1, "a" * 40), log(2, "f" * 40, head_repository="fork/web")])
+
+        assert refs == {PATH: "a" * 40}
+
+    def test_only_fork_commits_read_the_default_branch(self):
+        assert _workflow_refs("acme/web", [log(1, head_repository="fork/web")]) == {PATH: None}
+
+    def test_runs_without_a_workflow_file_are_ignored(self):
+        assert _workflow_refs("acme/web", [log(1, path="dynamic/dependabot/x")]) == {}
+
+
+class TestFetchWorkflows:
+    def test_a_second_analysis_makes_no_request(self, tmp_path, monkeypatch):
+        monkeypatch.setattr("ci_time_tracker.cli.GitHubClient", Recorder)
+        Recorder.urls = []
+        logs = [log(1, "a" * 40)]
+
+        with RunCache(tmp_path / "runs.db") as cache:
+            first = _fetch_workflows("acme/web", logs, cache)
+            second = _fetch_workflows("acme/web", logs, cache)
+
+        assert first == second
+        assert PATH in first
+        assert len(Recorder.urls) == 1
+        assert "ref=" + "a" * 40 in Recorder.urls[0]
+
+    def test_a_new_commit_is_fetched_again(self, tmp_path, monkeypatch):
+        monkeypatch.setattr("ci_time_tracker.cli.GitHubClient", Recorder)
+        Recorder.urls = []
+
+        with RunCache(tmp_path / "runs.db") as cache:
+            _fetch_workflows("acme/web", [log(1, "a" * 40)], cache)
+            _fetch_workflows("acme/web", [log(1, "a" * 40), log(2, "b" * 40)], cache)
+
+        assert len(Recorder.urls) == 2
+
+    def test_the_default_branch_is_never_cached(self, tmp_path, monkeypatch):
+        monkeypatch.setattr("ci_time_tracker.cli.GitHubClient", Recorder)
+        Recorder.urls = []
+        logs = [log(1, head_repository="fork/web")]
+
+        with RunCache(tmp_path / "runs.db") as cache:
+            _fetch_workflows("acme/web", logs, cache)
+            _fetch_workflows("acme/web", logs, cache)
+
+        assert len(Recorder.urls) == 2
+        assert "ref=" not in Recorder.urls[0]
+
+    def test_without_a_cache_files_are_still_read(self, monkeypatch):
+        monkeypatch.setattr("ci_time_tracker.cli.GitHubClient", Recorder)
+        Recorder.urls = []
+
+        assert PATH in _fetch_workflows("acme/web", [log(1, "a" * 40)])