github-security-report 0.14.2__tar.gz → 0.16.0__tar.gz

{github_security_report-0.14.2 → github_security_report-0.16.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: github-security-report
-Version: 0.14.2
+Version: 0.16.0
 Summary: Security and quality reporting across GitHub organisations
 Project-URL: Homepage, https://github.com/lfreleng-actions/github-security-report-action
 Project-URL: Repository, https://github.com/lfreleng-actions/github-security-report-action
@@ -36,8 +36,8 @@ Requires-Dist: pytest-asyncio==1.4.0; extra == 'dev'
 Requires-Dist: pytest-cov==7.1.0; extra == 'dev'
 Requires-Dist: pytest==9.1.1; extra == 'dev'
 Requires-Dist: respx==0.23.1; extra == 'dev'
-Requires-Dist: ruff==0.16.5; extra == 'dev'
-Requires-Dist: syrupy==6.0.0; extra == 'dev'
+Requires-Dist: ruff==0.16.8; extra == 'dev'
+Requires-Dist: syrupy==6.1.1; extra == 'dev'
 Requires-Dist: types-jsonschema==4.26.0.20260518; extra == 'dev'
 Requires-Dist: types-pyyaml==6.0.12.20250915; extra == 'dev'
 Description-Content-Type: text/markdown
@@ -101,6 +101,33 @@ if you want to probe everything regardless.
 Further sections report **configuration posture** and **freshness** as plain
 tables (org mode):
 
+- **CodeQL scan health** — two tables beneath **CodeQL: Results/Findings** (the
+  CodeQL alerts table), whose "Clean" says nothing about whether CodeQL is
+  still running:
+  - **Stale Configurations** lists every CodeQL configuration whose last scan
+    trails the default branch's newest commit by more than
+    `codeql_stale_days` (default 30) — the condition GitHub's tool status page
+    flags as *"Code Scanning results may be out of date"*. Each row names the
+    setup type (**Default**, GitHub-managed; or **Advanced**, a workflow in the
+    repository), the language, the last *successful* scan, and the cause:
+    analyses failing (a run that errors still uploads, but its results do not
+    count as a scan), default setup switched off, a workflow removed or
+    disabled, or an advanced workflow
+    **superseded** by default setup (GitHub rejects advanced CodeQL uploads
+    while default setup is on). An *orphaned* configuration will never scan
+    again; delete it from the tool status page once the live setup covers its
+    language.
+  - **Language Coverage** lists repositories where GitHub detects a
+    CodeQL-supported language that no current configuration scans — typically
+    an advanced workflow whose language matrix omits one the repository
+    contains, such as `actions` for its own workflows. On default setup GitHub
+    reports only the languages it is set to scan, so a detected language
+    deliberately left out of default setup is not visible to this check.
+
+  Both consider only repositories where CodeQL has run at least once; the rest
+  already appear in the Results/Findings not-enabled list. Finding a stale
+  configuration means reading each repository's whole CodeQL analysis history,
+  since GitHub cannot filter it by configuration.
 - **Dependabot** — three tables: repositories with vulnerability **alerts not
   enabled**, repositories with **security updates not enabled**, and ecosystems
   with no update `cooldown` configured (mandatory; any value passes).
@@ -118,6 +145,20 @@ tables (org mode):
   exposes no org-wide or GraphQL equivalent) and, like every other category,
   always collected; hide it with the `private_vulnerability_reporting` render
   toggle.
+- **Auto-merge** — repositories where the **Allow auto-merge** setting is off,
+  so a pull request cannot be queued to merge itself once its requirements are
+  met. The setting only offers the option: an auto-merging pull request still
+  waits for the required checks, reviews and branch protections the repository
+  already enforces, so enabling it relaxes nothing. What it removes is the
+  interval between a change becoming mergeable and somebody noticing — the
+  window a reviewed dependency update sits in while the vulnerability it fixes
+  stays unpatched. Read from the batched GraphQL prefetch, so it costs no extra
+  request; hide it with the `auto_merge` render toggle.
+
+The four enablement categories count the repositories with and without the
+feature (plus any whose state could not be read), and by default name whichever
+known list is shorter — see
+[Enabled or disabled repository lists](#enabled-or-disabled-repository-lists).
 
 ## Operating modes
 
@@ -182,6 +223,7 @@ organisation and **Repository access** set to *All repositories*, then grant:
 | Secret scanning alerts | Open secret-scanning alerts, across every GitHub pattern category |
 | Issues | Open issues and their labels (GitHub Issues table) |
 | Administration | Dependabot enablement + security-updates status, and effective branch rules |
+| Actions | Workflow state behind a stale CodeQL configuration |
 
 **Organization permissions:**
 
@@ -302,6 +344,7 @@ environment-variable name, never embedded.
     "include_test": false,
     "repo_min_age_days": 28,
     "release_max_age_days": 60,
+    "codeql_stale_days": 30,
     "graph_batch": 10,
     "order": { "style": "auto" }
   },
@@ -326,6 +369,17 @@ individual output. Set a value to `0` to remove the limit entirely and show
 every offender. Each can also be set at the CLI with `--top-n`,
 `--top-n-report`, `--top-n-cli`, and `--top-n-slack`.
 
+`report.codeql_stale_days` (default `30`, minimum `1`) is the CodeQL
+stale-configuration threshold: a configuration is stale once its last scan
+trails the default branch's newest commit by more than that many days. It is
+measured against the branch head rather than the clock, so a repository nobody
+has pushed to is not flagged just for being quiet. GitHub does not publish the
+threshold behind its own "may be out of date" warning; across the
+`lfreleng-actions` estate healthy configurations trailed their head by at most
+five days and abandoned ones by more than eighty, so the default sits well
+clear of both. The same threshold decides which configurations count as current
+for Language Coverage.
+
 The Releases / Tagging section has two independent freshness levers:
 
 - `report.repo_min_age_days` (default `28`, `0` = include all) is a grace
@@ -355,6 +409,23 @@ The per-org `exclude` list removes repositories from analysis entirely; they are
 reported as **excluded** (distinct from "not enabled"), so an intentional
 exclusion is visible rather than silently dropped.
 
+Every category reports those exclusions beneath its counts, so an
+organisation-wide list repeats under each one. `report.excluded_display`
+governs that line on every surface:
+
+| Value | Excluded line |
+| ----- | ------------- |
+| `always-show` (default) | Shown under every category |
+| `always-hide` | Never shown |
+| `conditional-hide` | Shown only for a category whose exclusions differ from the organisation's `exclude` list, and then in full |
+
+Hiding the line changes nothing else: every count is unaffected, and the report
+header's repository count already leaves excluded repositories out. So a clean
+category reads `All Clean` once no Excluded line qualifies it. Today every
+category reports the organisation's own list, so `conditional-hide` currently
+hides the same lines `always-hide` does. It differs once a category excludes
+repositories of its own. The `report.json` artifact always lists the exclusions.
+
 Archived and test repositories are excluded from analysis by default. Opt them
 back in with `report.include_archived` / `report.include_test` in the config, or
 for a single run with `--include-archived` / `--include-test`.
@@ -384,16 +455,17 @@ both true.
 
 The example above hides Zizmor on every surface, and keeps Releases / Tagging
 out of the terminal and Slack while still publishing it to the Markdown and HTML
-Pages output. The valid category keys are: `codeql`, `scorecard`, `zizmor`,
-`aislop`, `dependabot_alerts`, `secret_scanning`, `dependabot_alerts_enabled`,
-`dependabot_updates_enabled`, `dependabot_cooldown`, `releases`,
-`mutable_releases`, `private_vulnerability_reporting`, `github_issues`. Like the
-other `report`
-settings, `categories` can be set
-globally and overridden per organisation (overrides merge key-by-key, so
-flipping one output leaves the rest untouched). The machine-readable
-`report.json` artifact always contains the complete dataset, regardless of these
-toggles.
+Pages output. The valid category keys are: `codeql`,
+`codeql_stale_configurations`, `codeql_language_coverage`, `scorecard`,
+`zizmor`, `aislop`, `dependabot_alerts`, `secret_scanning`,
+`dependabot_alerts_enabled`, `dependabot_updates_enabled`,
+`dependabot_cooldown`, `releases`, `mutable_releases`,
+`private_vulnerability_reporting`, `auto_merge`, `github_issues`,
+`pull_requests`, `pull_requests_assigned`. Like the other `report` settings,
+`categories` can be set globally and overridden per organisation (overrides
+merge key-by-key, so flipping one output leaves the rest untouched). The
+machine-readable `report.json` artifact always contains the complete dataset,
+regardless of these toggles.
 
 When several organisations share one Slack channel they render into a single
 combined digest, so the per-org Slack toggles are unioned for that channel: a
@@ -454,6 +526,52 @@ Slack-style ceiling, but they still apply their own row limits — only the
 GitHub Pages report whenever `pages_url` is set and short enough to render as a
 link.
 
+### Enabled or disabled repository lists
+
+The four boolean feature categories — `dependabot_alerts_enabled`,
+`dependabot_updates_enabled`, `private_vulnerability_reporting` and `auto_merge`
+— sort each repository into one of three buckets: enabled, not enabled, or
+**unknown** when the feature's state could not be read. The unknown bucket is
+counted but never named, and never treated as either side, so the enabled and
+not-enabled counts need not sum to the repositories analysed. Beneath the counts,
+one of the two known sides is named; `report.repo_list` chooses which, on every
+surface:
+
+| Value | Names |
+| ----- | ----- |
+| `auto` (default) | Whichever list is shorter |
+| `enabled` | The repositories with the feature on |
+| `disabled` | The repositories with the feature off |
+
+`auto` keeps a footer short whichever way an organisation leans: a feature
+nearly every repository has lists its few holdouts, and one almost none have
+lists its few adopters. Two cases resolve towards the actionable side: a tie
+names the repositories without the feature, and so does a category where
+*nothing* is enabled, since naming an empty list would print nothing where a
+reader wants the repositories to fix. Both sides are always counted, whichever is
+named.
+
+A category can override the global value:
+
+```json
+{
+  "report": {
+    "repo_list": "auto",
+    "categories": {
+      "dependabot_alerts_enabled": { "repo_list": "disabled" }
+    }
+  },
+  "organizations": [{ "name": "lfreleng-actions" }]
+}
+```
+
+Here Dependabot alerts always name the repositories to fix, and the other three
+feature categories name whichever side is shorter. `repo_list` applies only to
+those four categories; setting it on any other is a configuration error, since a
+table with qualitative columns has no enabled list to name. These categories
+render as a name list rather than a one-column table on every surface. The
+`report.json` artifact is unaffected.
+
 ### Per-category row ordering
 
 Each table ships a sensible default ordering — largest backlog first, stalest
@@ -736,17 +854,17 @@ organisation](#inside-or-outside-the-organisation):
 
 ```text
 Pull Requests
-┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━┳━━━━━━━━━┳━━━━━━━┳━━━━━━━┓
-┃ Repository           ┃ Human ┃ Ext ┃ Auto ┃ Conflict ┃ Fail ┃ Copilot ┃ Draft ┃ Total ┃
-┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━╇━━━━━━━━━╇━━━━━━━╇━━━━━━━┩
-│ lftools-uv           │     8 │   0 │    0 │        0 │    0 │       0 │     0 │     8 │
-│ dependamerge         │     3 │   0 │    0 │        0 │    0 │       2 │     0 │     3 │
-│ gha-workflow-linter  │     2 │   0 │    0 │        0 │    1 │       0 │     1 │     2 │
-│ harden-runner-block- │     1 │   0 │    0 │        1 │    0 │       0 │     1 │     1 │
-│ action               │       │     │      │          │      │         │       │       │
-├─────────────────────┼───────┼─────┼──────┼──────────┼──────┼─────────┼───────┼───────┤
-│ Total                │    14 │   0 │    0 │        1 │    1 │       2 │     2 │    14 │
-└─────────────────────┴───────┴─────┴──────┴──────────┴──────┴─────────┴───────┴───────┘
+┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━┳━━━━━━━┓
+┃ Repository           ┃ Human ┃ Ext ┃ Auto ┃ Conflict ┃ Fail ┃ Review ┃ Copilot ┃ Draft ┃ Total ┃
+┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━╇━━━━━━━┩
+│ lftools-uv           │     8 │   0 │    0 │        0 │    0 │      0 │       0 │     0 │     8 │
+│ dependamerge         │     3 │   0 │    0 │        0 │    0 │      1 │       2 │     0 │     3 │
+│ gha-workflow-linter  │     2 │   0 │    0 │        0 │    1 │      0 │       0 │     1 │     2 │
+│ harden-runner-block- │     1 │   0 │    0 │        1 │    0 │      0 │       0 │     1 │     1 │
+│ action               │       │     │      │          │      │        │         │       │       │
+├──────────────────────┼───────┼─────┼──────┼──────────┼──────┼────────┼─────────┼───────┼───────┤
+│ Total                │    14 │   0 │    0 │        1 │    1 │      1 │       2 │     2 │    14 │
+└──────────────────────┴───────┴─────┴──────┴──────────┴──────┴────────┴─────────┴───────┴───────┘
   … and 9 more
   ❌ 13 With open pull requests
   ✅ 104 No open pull requests
@@ -760,34 +878,73 @@ The columns form **two independent groupings**:
 | `Ext` | Human pull requests raised from outside the organisation — a **subset of Human**, which is why it sits beside it. |
 | `Conflict` | Blocked on a merge conflict. |
 | `Fail` | Latest checks did not pass. The rollup includes optional checks, so this is not by itself proof that the merge is blocked. |
+| `Review` | A reviewer has **requested changes** — a person waiting on the author. |
 | `Copilot` | Carries at least one **unresolved review thread** opened by GitHub's automated code reviewer. |
 | `Draft` | Marked as a draft. |
 
-`Conflict`, `Fail`, `Copilot` and `Draft` **overlap** each other and the author
-split, so they do not sum to `Total` and are not meant to: one pull request that
-is conflicting, failing *and* a draft is counted once in each of those three
-columns, and once under `Human` or `Auto`. Only `Human` + `Auto` reconciles with
-the collected total. They are ordered worst-first — a conflict needs a human to
-rebase, a failing check may only need a re-run, unresolved review feedback needs
-a human but does not hold the merge button down, and a draft is not blocked at
-all.
+`Conflict`, `Fail`, `Review`, `Copilot` and `Draft` **overlap** each other and
+the author split, so they do not sum to `Total` and are not meant to: one pull
+request that is conflicting, failing *and* a draft is counted once in each of
+those three columns, and once under `Human` or `Auto`. Only `Human` + `Auto`
+reconciles with the collected total. They are ordered worst-first — a conflict
+needs a human to rebase, a failing check may only need a re-run, requested
+changes need the author to act, unresolved automated feedback needs somebody but
+does not hold the merge button down, and a draft is not blocked at all.
+
+`Review` and `Copilot` sit together inside the blocker group, and deliberately
+not beside `Human`: that column names who **raised** a pull request, whereas
+these two name who is **waiting on** it. The grouping is what keeps those two
+readings of "human" apart.
+
+#### Requested changes
+
+`Review` counts pull requests where **a person** has asked for changes and not
+withdrawn it.
+
+Two fields answer that, each covering the other's blind spot. `reviewDecision` is
+GitHub's own verdict, computed over every review and accounting for dismissals,
+so it is **exact at any review count** — but it names nobody, and a GitHub App
+holding pull-request write permission can request changes exactly as a person
+can. `latestOpinionatedReviews` names the reviewers, but is a bounded window of
+five (one review per reviewer, so five reviewers rather than five reviews).
+
+So the decision **gates** and the window **attributes**. Anything other than
+`CHANGES_REQUESTED` is a definite zero whatever the window holds, and only once
+GitHub says changes are outstanding does the reviewer's identity matter — which
+keeps the cheap exact answer for almost every pull request and pays the window's
+uncertainty on the few where it changes the reading. Automated reviewers are
+recognised by the same rule as the `Auto` column.
+
+Only `CHANGES_REQUESTED` gates. **`REVIEW_REQUIRED` is deliberately excluded**:
+it reports that a branch rule demands a review, not that anybody objected, so on
+an organisation that requires review by default it would mark nearly every human
+pull request and distinguish none of them. The rule is written as an allow-list
+of the one blocking state rather than as "anything that is not `APPROVED`", so a
+state GitHub adds later arrives uncounted rather than pre-counted as a blocker.
+
+Where a request for changes cannot be attributed — the reviews were unreadable,
+the author is gone, or the window held only automated requests without covering
+every reviewer — the pull request is left **uncounted rather than credited to a
+person**, and the table says so in its description.
 
 #### Unresolved Copilot feedback
 
-`Copilot` counts the pull requests **waiting on a human to answer a review**. A
-review thread counts when it was opened by GitHub's automated code reviewer (the
-`copilot-pull-request-reviewer` bot) and nobody has resolved it. The reviewer is
-identified from each thread's **opening** comment, since later replies are
-usually the human answering the review.
+`Copilot` counts the pull requests **waiting on somebody to answer an automated
+review**. A review thread counts when it was opened by GitHub's automated code
+reviewer (the `copilot-pull-request-reviewer` bot) and nobody has resolved it.
+The reviewer is identified from each thread's **opening** comment, since later
+replies are usually the human answering the review.
 
 An **outdated** thread still counts: GitHub marks a thread outdated when the
 code beneath it changes, but that does not resolve it, so the feedback remains
 unanswered.
 
-Any non-zero count is coloured **red**, like `Conflict` and `Fail` — it marks a
-pull request that cannot progress until somebody responds. The count also feeds
-the row ranking, so a backlog awaiting review outranks an untouched one of the
-same size.
+Any non-zero count is coloured **amber**, one step below the red of `Conflict`,
+`Fail` and `Review`. Automated feedback is worth answering, but nobody is
+waiting on it, and there is far more of it — colouring the two alike would let
+the bot's threads mask the rarer case that actually holds a person up. The count
+still feeds the row ranking, so a backlog awaiting review outranks an untouched
+one of the same size.
 
 The scan reads the **newest 20 review threads per pull request**, which is the
 single most expensive part of the prefetch (see the cost figures above). The
@@ -802,8 +959,8 @@ clear — see below.
 #### Automation backlog thresholds
 
 On the terminal the counts are coloured so the table reads at a glance: `Human`
-and `Ext` **green**, `Conflict`, `Fail` and `Copilot` **red**, and `Auto`
-coloured against two configured thresholds.
+and `Ext` **green**, `Conflict`, `Fail` and `Review` **red**, `Copilot`
+**amber**, and `Auto` coloured against two configured thresholds.
 
 Dependabot stops raising pull requests once a repository reaches its
 `open-pull-requests-limit`, so the repository quietly stops receiving dependency
@@ -1025,7 +1182,7 @@ reports as an App and any unrecognised login carrying the `[bot]` marker — so
 future bot is classified as automation rather than mistaken for an outside
 contributor.
 
-`Fail`, `Conflict` and `Copilot` count only **established** states. GitHub
+`Fail`, `Conflict`, `Review` and `Copilot` count only **established** states. GitHub
 computes mergeability lazily and answers `UNKNOWN` until it settles, and reports
 no check rollup at all when no checks have run; neither absence is evidence that
 a pull request is ready, so neither is counted either way. `Copilot` follows the
@@ -1039,10 +1196,15 @@ holding any such reading says so in its description: `Copilot` then
 **undercounts and should be read as a lower bound**, exactly as `Ext` does when
 membership cannot be read.
 
+`Review` carries a weaker version of the same hedge, and earns it far less
+often: `reviewDecision` is exact, so only a pull request GitHub *does* report
+changes requested on can leave the question open, and then only when the
+opinionated-review window fails to attribute it to anybody.
+
 Repositories with no open pull requests are counted in the footer rather than
 listed. Rows rank by total open pull requests, then by those failing,
-conflicting or awaiting Copilot feedback (counted once each, since the columns
-overlap), so two repositories with equal backlogs surface the more
+conflicting or awaiting review — automated or human (counted once each, since
+the columns overlap) — so two repositories with equal backlogs surface the more
 stuck one first.
 
 > **Accuracy note.** As with the issues table, `Total` is exact at any size,
@@ -1369,6 +1531,7 @@ enablement endpoint:
 | `dependabot_alerts_enabled` | Dependabot vulnerability alerts |
 | `dependabot_updates_enabled` | Dependabot security updates (plus alerts) |
 | `private_vulnerability_reporting` | Private vulnerability reporting |
+| `auto_merge` | The repository's "Allow auto-merge" setting |
 
 Qualitative findings (Scorecard, zizmor, open Dependabot alerts, cooldown,
 release freshness/mutability) are reported but not auto-remediated. Remediation