github-security-report 0.14.0__tar.gz → 0.14.2__tar.gz

{github_security_report-0.14.0 → github_security_report-0.14.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: github-security-report
-Version: 0.14.0
+Version: 0.14.2
 Summary: Security and quality reporting across GitHub organisations
 Project-URL: Homepage, https://github.com/lfreleng-actions/github-security-report-action
 Project-URL: Repository, https://github.com/lfreleng-actions/github-security-report-action
@@ -29,7 +29,7 @@ Requires-Dist: jinja2==3.1.6
 Requires-Dist: jsonschema==4.26.0
 Requires-Dist: pyyaml==6.0.3
 Requires-Dist: rich==15.0.0
-Requires-Dist: typer==0.27.1
+Requires-Dist: typer==0.27.2
 Provides-Extra: dev
 Requires-Dist: mypy==2.3.1; extra == 'dev'
 Requires-Dist: pytest-asyncio==1.4.0; extra == 'dev'
@@ -302,6 +302,7 @@ environment-variable name, never embedded.
     "include_test": false,
     "repo_min_age_days": 28,
     "release_max_age_days": 60,
+    "graph_batch": 10,
     "order": { "style": "auto" }
   },
   "organizations": [
@@ -788,10 +789,15 @@ pull request that cannot progress until somebody responds. The count also feeds
 the row ranking, so a backlog awaiting review outranks an untouched one of the
 same size.
 
-The scan reads up to **20 review threads per pull request**, which is the single
-most expensive part of the prefetch (see the cost figures above). Where a pull
-request carries more threads than that and none of the collected ones qualifies,
-the pull request is treated as **indeterminate** rather than clear — see below.
+The scan reads the **newest 20 review threads per pull request**, which is the
+single most expensive part of the prefetch (see the cost figures above). The
+newest end is deliberate: GitHub returns review threads oldest-first and offers
+no way to order them, so reading from the front would window the threads a
+review has already worked through — the resolved ones — and report a pull
+request still waiting on its latest round as clear. Unanswered feedback is the
+recent kind. Where a pull request carries more threads than the window and none
+of the collected ones qualifies, it is treated as **indeterminate** rather than
+clear — see below.
 
 #### Automation backlog thresholds
 
@@ -1127,6 +1133,55 @@ Gating decides **collection**; the per-category render toggles above decide
 **presentation**. A skipped section still renders (as the one-line notice)
 unless its category is also disabled.
 
+### GraphQL prefetch batch size
+
+The releases/tags, Dependabot-enablement, open-issues and pull-request data
+for every in-scope repository comes from one aliased GraphQL query per batch
+of repositories rather than a round-trip per repository. GitHub bounds how
+much work a single GraphQL query may do — roughly ten seconds of execution —
+and reports a breach in one of three shapes, each after the query has already
+run for the full limit: an **HTTP 502** (or 504) gateway timeout from the
+edge, an HTTP 200 carrying no `data`, or an HTTP 200 whose `errors` say
+`Resource limits for this query exceeded` with the unresolved fields nulled.
+Each aliased repository adds a few hundred milliseconds, so the batch
+size is bounded by *latency*, not by rate-limit cost: 25 repositories measured
+at ~9 s against `lfreleng-actions` and failed intermittently; 10 measured at
+~3.5 s.
+
+`report.graph_batch` (default `10`; minimum `1`) sets the starting batch size,
+globally or per organisation. It is a lever for a slow day rather than a hard
+limit: a batch that still fails on size after the transport's own retries is
+re-issued at **half** the size, and the smaller size is kept for the rest of
+that organisation's collection (each organisation in a multi-org run starts
+from its own configured size, since organisations differ in how much data a
+repository carries). Only a single-repository query GitHub still cannot
+answer, or a failure that a smaller query could not fix (a `403`, an
+exhausted rate limit, or a `500`/`503` outage rather than a `502`/`504`
+timeout), aborts the run with exit code `3`.
+
+The value can be set three ways, in this order of precedence:
+
+1. `--graph-batch N` on the command line (the action's `graph_batch` input).
+2. A **repository or organisation variable** passed to that input. The
+   bundled `reporting.yaml` passes `vars.GSR_GRAPH_BATCH`, so the size can be
+   adjusted from repository settings without a workflow edit or a release;
+   a caller's own workflow can do the same. (The `vars` context is not
+   available inside a composite action, so the action cannot read it itself.)
+3. `report.graph_batch` in the configuration; otherwise the built-in `10`.
+
+Batching changes how many requests carry the data, not how many GraphQL nodes
+are resolved, so the node work is the same whatever the batch size. A smaller
+batch does cost slightly more of the rate-limit budget — each query is charged
+at least one point, and per-query cost rounding adds a little when one query
+becomes several — and a few more requests (a 124-repository organisation is 13
+queries at 10, 5 at 25). Against the 5,000-point hourly budget that is noise;
+a larger value buys nothing except a longer first failure.
+
+> The flag and input are recent additions. An empty input is not passed to
+> the tool, so an unset variable cannot break a run; but a pinned tool version
+> predating the flag will reject a set one with `No such option`, so set
+> `GSR_GRAPH_BATCH` only once the pinned release supports it.
+
 ### Pass/fail severity cutoff
 
 The severity-ranked signals (CodeQL, Scorecard, Zizmor, aislop, Dependabot
@@ -1204,6 +1259,7 @@ and the Slack **bot token** is consumed by the workflow, not the CLI.
 | `fail_threshold` | No | `none` | `none`/`low`/`medium`/`high`/`critical`/`any` (repo mode) |
 | `force_notify` | No | `false` | Post to Slack regardless of `report_day` |
 | `hide` | No | `""` | Category keys to suppress on every output (space- or comma-separated). Overrides config, and is one-way: it cannot re-enable a disabled category |
+| `graph_batch` | No | `""` | Repositories per batched GraphQL prefetch query (minimum `1`; default: config, else `10`). Pass a repository or organisation variable such as `${{ vars.GSR_GRAPH_BATCH }}` to adjust it from settings. A batch that still fails is halved automatically, so this sets the starting size (see [GraphQL prefetch batch size](#graphql-prefetch-batch-size)) |
 | `tool_version` | No | `""` | Published PyPI version to install. Empty (the default) uses the Dependabot-managed pin in `.github/runtime-pin/requirements.txt`; set a specific version to override. Ignored on pull requests or when `use_local_source` is `true` (both run from source) |
 | `use_local_source` | No | `false` | Run from the checked-out source instead of PyPI (for testing unreleased code from any event) |
 
@@ -1247,7 +1303,7 @@ API in the same way.
 | `0` | The report ran. |
 | `1` | Repo mode only: findings met or exceeded `--fail-threshold`. |
 | `2` | Usage or configuration error (bad flag, unreadable config). |
-| `3` | The GitHub API was unreachable after the retry budget. |
+| `3` | The GitHub API was unusable after the retry budget (see below). |
 | `4` | GitHub rejected the credentials (HTTP 401). |
 
 Codes `3` and `4` are **aborts, not reports**: nothing is written and no Pages
@@ -1258,8 +1314,19 @@ section `No data` or `All Clean` — and a scheduled job would then publish it
 over the last good one. Reporting false data is worse than reporting none, so
 the run stops at the first rejected request.
 
+Code `3` covers a GitHub API that could not be reached at all, and a GraphQL
+prefetch that failed as a whole: either a size-shaped failure (`502`/`504`
+timeout, `200` with no data, resource limits exceeded) that persisted even
+after the batch had been halved down to one repository, or a failure that
+halving could not have fixed (`403`, an exhausted GraphQL rate limit, a
+`500`/`503` outage), which aborts at once. The message reports the status or
+cause and ends with the matching next step — a smaller `--graph-batch`, retry
+later, wait for the rate-limit budget to reset, or check the token's
+permissions — so read it rather than assuming "retry later" (see
+[GraphQL prefetch batch size](#graphql-prefetch-batch-size)).
+
 The two are separate codes because the remedy differs: `4` means rotate or fix
-the token, `3` means retry later.
+the token, `3` usually means retry later.
 
 ## Remediation
 

{github_security_report-0.14.0 → github_security_report-0.14.2}/README.md

@@ -258,6 +258,7 @@ environment-variable name, never embedded.
     "include_test": false,
     "repo_min_age_days": 28,
     "release_max_age_days": 60,
+    "graph_batch": 10,
     "order": { "style": "auto" }
   },
   "organizations": [
@@ -744,10 +745,15 @@ pull request that cannot progress until somebody responds. The count also feeds
 the row ranking, so a backlog awaiting review outranks an untouched one of the
 same size.
 
-The scan reads up to **20 review threads per pull request**, which is the single
-most expensive part of the prefetch (see the cost figures above). Where a pull
-request carries more threads than that and none of the collected ones qualifies,
-the pull request is treated as **indeterminate** rather than clear — see below.
+The scan reads the **newest 20 review threads per pull request**, which is the
+single most expensive part of the prefetch (see the cost figures above). The
+newest end is deliberate: GitHub returns review threads oldest-first and offers
+no way to order them, so reading from the front would window the threads a
+review has already worked through — the resolved ones — and report a pull
+request still waiting on its latest round as clear. Unanswered feedback is the
+recent kind. Where a pull request carries more threads than the window and none
+of the collected ones qualifies, it is treated as **indeterminate** rather than
+clear — see below.
 
 #### Automation backlog thresholds
 
@@ -1083,6 +1089,55 @@ Gating decides **collection**; the per-category render toggles above decide
 **presentation**. A skipped section still renders (as the one-line notice)
 unless its category is also disabled.
 
+### GraphQL prefetch batch size
+
+The releases/tags, Dependabot-enablement, open-issues and pull-request data
+for every in-scope repository comes from one aliased GraphQL query per batch
+of repositories rather than a round-trip per repository. GitHub bounds how
+much work a single GraphQL query may do — roughly ten seconds of execution —
+and reports a breach in one of three shapes, each after the query has already
+run for the full limit: an **HTTP 502** (or 504) gateway timeout from the
+edge, an HTTP 200 carrying no `data`, or an HTTP 200 whose `errors` say
+`Resource limits for this query exceeded` with the unresolved fields nulled.
+Each aliased repository adds a few hundred milliseconds, so the batch
+size is bounded by *latency*, not by rate-limit cost: 25 repositories measured
+at ~9 s against `lfreleng-actions` and failed intermittently; 10 measured at
+~3.5 s.
+
+`report.graph_batch` (default `10`; minimum `1`) sets the starting batch size,
+globally or per organisation. It is a lever for a slow day rather than a hard
+limit: a batch that still fails on size after the transport's own retries is
+re-issued at **half** the size, and the smaller size is kept for the rest of
+that organisation's collection (each organisation in a multi-org run starts
+from its own configured size, since organisations differ in how much data a
+repository carries). Only a single-repository query GitHub still cannot
+answer, or a failure that a smaller query could not fix (a `403`, an
+exhausted rate limit, or a `500`/`503` outage rather than a `502`/`504`
+timeout), aborts the run with exit code `3`.
+
+The value can be set three ways, in this order of precedence:
+
+1. `--graph-batch N` on the command line (the action's `graph_batch` input).
+2. A **repository or organisation variable** passed to that input. The
+   bundled `reporting.yaml` passes `vars.GSR_GRAPH_BATCH`, so the size can be
+   adjusted from repository settings without a workflow edit or a release;
+   a caller's own workflow can do the same. (The `vars` context is not
+   available inside a composite action, so the action cannot read it itself.)
+3. `report.graph_batch` in the configuration; otherwise the built-in `10`.
+
+Batching changes how many requests carry the data, not how many GraphQL nodes
+are resolved, so the node work is the same whatever the batch size. A smaller
+batch does cost slightly more of the rate-limit budget — each query is charged
+at least one point, and per-query cost rounding adds a little when one query
+becomes several — and a few more requests (a 124-repository organisation is 13
+queries at 10, 5 at 25). Against the 5,000-point hourly budget that is noise;
+a larger value buys nothing except a longer first failure.
+
+> The flag and input are recent additions. An empty input is not passed to
+> the tool, so an unset variable cannot break a run; but a pinned tool version
+> predating the flag will reject a set one with `No such option`, so set
+> `GSR_GRAPH_BATCH` only once the pinned release supports it.
+
 ### Pass/fail severity cutoff
 
 The severity-ranked signals (CodeQL, Scorecard, Zizmor, aislop, Dependabot
@@ -1160,6 +1215,7 @@ and the Slack **bot token** is consumed by the workflow, not the CLI.
 | `fail_threshold` | No | `none` | `none`/`low`/`medium`/`high`/`critical`/`any` (repo mode) |
 | `force_notify` | No | `false` | Post to Slack regardless of `report_day` |
 | `hide` | No | `""` | Category keys to suppress on every output (space- or comma-separated). Overrides config, and is one-way: it cannot re-enable a disabled category |
+| `graph_batch` | No | `""` | Repositories per batched GraphQL prefetch query (minimum `1`; default: config, else `10`). Pass a repository or organisation variable such as `${{ vars.GSR_GRAPH_BATCH }}` to adjust it from settings. A batch that still fails is halved automatically, so this sets the starting size (see [GraphQL prefetch batch size](#graphql-prefetch-batch-size)) |
 | `tool_version` | No | `""` | Published PyPI version to install. Empty (the default) uses the Dependabot-managed pin in `.github/runtime-pin/requirements.txt`; set a specific version to override. Ignored on pull requests or when `use_local_source` is `true` (both run from source) |
 | `use_local_source` | No | `false` | Run from the checked-out source instead of PyPI (for testing unreleased code from any event) |
 
@@ -1203,7 +1259,7 @@ API in the same way.
 | `0` | The report ran. |
 | `1` | Repo mode only: findings met or exceeded `--fail-threshold`. |
 | `2` | Usage or configuration error (bad flag, unreadable config). |
-| `3` | The GitHub API was unreachable after the retry budget. |
+| `3` | The GitHub API was unusable after the retry budget (see below). |
 | `4` | GitHub rejected the credentials (HTTP 401). |
 
 Codes `3` and `4` are **aborts, not reports**: nothing is written and no Pages
@@ -1214,8 +1270,19 @@ section `No data` or `All Clean` — and a scheduled job would then publish it
 over the last good one. Reporting false data is worse than reporting none, so
 the run stops at the first rejected request.
 
+Code `3` covers a GitHub API that could not be reached at all, and a GraphQL
+prefetch that failed as a whole: either a size-shaped failure (`502`/`504`
+timeout, `200` with no data, resource limits exceeded) that persisted even
+after the batch had been halved down to one repository, or a failure that
+halving could not have fixed (`403`, an exhausted GraphQL rate limit, a
+`500`/`503` outage), which aborts at once. The message reports the status or
+cause and ends with the matching next step — a smaller `--graph-batch`, retry
+later, wait for the rate-limit budget to reset, or check the token's
+permissions — so read it rather than assuming "retry later" (see
+[GraphQL prefetch batch size](#graphql-prefetch-batch-size)).
+
 The two are separate codes because the remedy differs: `4` means rotate or fix
-the token, `3` means retry later.
+the token, `3` usually means retry later.
 
 ## Remediation
 

{github_security_report-0.14.0 → github_security_report-0.14.2}/pyproject.toml

@@ -45,7 +45,7 @@ keywords = [
 ]
 dependencies = [
   "httpx[http2]==0.28.1",
-  "typer==0.27.1",
+  "typer==0.27.2",
   "rich==15.0.0",
   "jinja2==3.1.6",
   "jsonschema==4.26.0",
@@ -129,6 +129,15 @@ markers = [
 [tool.coverage.run]
 source = ["github_security_report"]
 omit = ["tests/*"]
+# Keep the transient coverage database out of the working tree.
+# coverage.py resolves a relative data_file against the current
+# directory, so the default '.coverage' lands wherever pytest runs and
+# can be picked up by tests that inspect on-disk content. TEMP is
+# set on Windows, where /tmp does not exist; elsewhere it is normally
+# unset and the /tmp fallback applies. An explicit value also
+# satisfies python-test-action's data_file configuration check.
+# COVERAGE_FILE (env) still takes precedence when set.
+data_file = "${TEMP-/tmp}/.coverage.github-security-report"
 
 [tool.coverage.report]
 show_missing = true

{github_security_report-0.14.0 → github_security_report-0.14.2}/src/github_security_report/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.14.0'
-__version_tuple__ = version_tuple = (0, 14, 0)
+__version__ = version = '0.14.2'
+__version_tuple__ = version_tuple = (0, 14, 2)
 
 __commit_id__ = commit_id = None

{github_security_report-0.14.0 → github_security_report-0.14.2}/src/github_security_report/cli/app.py

@@ -166,6 +166,11 @@ def report(
         "--include-test",
         help="Analyse test repositories, which are excluded by default.",
     ),
+    graph_batch: int | None = typer.Option(
+        None,
+        "--graph-batch",
+        help="Repositories per batched GraphQL prefetch query (minimum 1; default: config, else 10). GitHub bounds how long one query may run, so a batch that still fails is halved automatically; this sets the starting size.",
+    ),
     no_color: bool = typer.Option(False, "--no-color", help="Disable coloured output."),
 ) -> None:
     """Generate a security and quality report."""
@@ -181,6 +186,7 @@ def report(
     )
     boundary.check_non_negative(console, "--repo-min-age-days", repo_min_age_days)
     boundary.check_non_negative(console, "--release-max-age-days", release_max_age_days)
+    boundary.check_positive(console, "--graph-batch", graph_batch)
     hidden = boundary.resolve_hidden(console, hide)
 
     cfg = _load_config(config_file, config_data, org, token_env, console=console)
@@ -210,6 +216,7 @@ def report(
                 gating=False if no_gating else None,
                 include_archived=True if include_archived else None,
                 include_test=True if include_test else None,
+                graph_batch=graph_batch,
             ),
             hidden=hidden,
         )
@@ -236,6 +243,7 @@ def report(
                     ("--no-gating", no_gating),
                     ("--include-archived", include_archived),
                     ("--include-test", include_test),
+                    ("--graph-batch", graph_batch is not None),
                 )
                 if supplied
             ],

{github_security_report-0.14.0 → github_security_report-0.14.2}/src/github_security_report/cli/boundary.py

@@ -41,6 +41,18 @@ def check_non_negative(console: Console, name: str, value: int | None) -> None:
         raise typer.Exit(2)
 
 
+def check_positive(console: Console, name: str, value: int | None) -> None:
+    """Reject a zero or negative override where 0 has no meaning.
+
+    For a batch size, unlike a row limit, there is no "unlimited" reading of
+    0 -- a query carrying no repositories is not a query -- so the floor is 1,
+    matching the config schema's minimum for the same control.
+    """
+    if value is not None and value < 1:
+        console.print(f"[red]{name} must be 1 or greater[/red]")
+        raise typer.Exit(2)
+
+
 def check_limits(console: Console, limits: Sequence[tuple[str, int | None]]) -> None:
     """Reject a negative row limit, naming the flag and the 0 convention."""
     for name, value in limits:

{github_security_report-0.14.0 → github_security_report-0.14.2}/src/github_security_report/cli/options.py

@@ -34,6 +34,10 @@ class ReportOverrides:
     not on, and ``--include-archived`` / ``--include-test`` can widen the scope
     but not narrow it, so a flag can loosen what the configuration asked for
     without being able to tighten it behind the operator's back.
+
+    ``graph_batch`` is an operational lever rather than report policy: it
+    changes how the GraphQL prefetch is issued, never what the report says, so
+    it may move in either direction.
     """
 
     repo_min_age_days: int | None = None
@@ -42,13 +46,15 @@ class ReportOverrides:
     gating: bool | None = None
     include_archived: bool | None = None
     include_test: bool | None = None
+    graph_batch: int | None = None
 
     def apply(self, org_cfg: OrgConfig) -> tuple[OrgConfig, ReportConfig]:
         """The org and report configs to collect with, overrides applied.
 
-        The two age thresholds and the three booleans are scalar policy, so
-        applying one uniformly across every configured organisation is what a
-        reader of the flag expects, and matches how ``--top-n`` already behaves.
+        The two age thresholds, the three booleans and the batch size are
+        scalar policy, so applying one uniformly across every configured
+        organisation is what a reader of the flag expects, and matches how
+        ``--top-n`` already behaves.
 
         ``releases_exclude`` is not scalar: it is a curated per-organisation
         list, and one flag replacing all of them loses data the config
@@ -63,6 +69,7 @@ class ReportOverrides:
             "gating",
             "include_archived",
             "include_test",
+            "graph_batch",
         ):
             value = getattr(self, name)
             if value is not None:

{github_security_report-0.14.0 → github_security_report-0.14.2}/src/github_security_report/client/__init__.py

@@ -35,6 +35,11 @@ from github_security_report.client.endpoints import (
     SCORECARD_API,
     _https_endpoint,
 )
+from github_security_report.client.errors import (
+    AuthError,
+    GraphBatchError,
+    NetworkError,
+)
 from github_security_report.client.org_reads import OrgReadClient
 from github_security_report.client.parsers import (
     _last_published,
@@ -52,8 +57,6 @@ from github_security_report.client.queries import (
 )
 from github_security_report.client.reads import ReadClient
 from github_security_report.client.transport import (
-    AuthError,
-    NetworkError,
     Transport,
     _endpoint_diagnostics,
 )
@@ -72,6 +75,7 @@ __all__ = [
     "AlertReads",
     "AuthError",
     "GitHubClient",
+    "GraphBatchError",
     "NetworkError",
     "OrgReadClient",
     "ReadClient",

{github_security_report-0.14.0 → github_security_report-0.14.2}/src/github_security_report/client/alerts.py

@@ -27,7 +27,8 @@ import logging
 from typing import NamedTuple
 
 from github_security_report.client.endpoints import BULK_KINDS
-from github_security_report.client.transport import AuthError, NetworkError, Transport
+from github_security_report.client.errors import AuthError, NetworkError
+from github_security_report.client.transport import Transport
 from github_security_report.secret_patterns import (
     GENERIC_SECRET_TYPES,
     PATTERN_CONFIG_PATH,