mcpprobe-scanner 0.3.1__tar.gz → 0.3.2__tar.gz

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/CHANGELOG.md

@@ -6,6 +6,33 @@ All notable changes to this project are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.3.2] - 2026-11-02
+
+Two counting fixes. Nothing about what the checks detect has changed.
+
+### Fixed
+
+- Attack chains were left out of the summary counts and the pass/fail decision. A scan
+  could list a CRITICAL `CHAIN-01` while the summary line said "critical 0", and
+  `--fail-on critical` passed (exit 0) because no single finding was critical. The
+  terminal now prints a second count line for attack chains under the findings line, for
+  example `Attack chains  critical 1  high 1 ... (2 chains, built from the findings counted
+  above)`. Chains are counted on their own line, not added to the findings counts, because
+  each chain is made of findings that are already counted. The exit code and the
+  `Result:` line now take chains into account, and the `Result:` line says whether
+  findings, attack chains or both reached the threshold.
+- The JSON summary has a new `chain_counts` field and a `findings_max_severity` field;
+  `max_severity` is now the worst of findings and attack chains (the value the exit code
+  uses). SARIF run properties have a new `chainCounts`. `counts` is unchanged: findings
+  only, one per SARIF result.
+- The GitHub Action's annotation and job summary mention attack chains and show them in
+  their own column.
+- Registry survey funnel: a remote URL listed more than once (by a second entry, or as
+  both streamable-http and SSE) was dropped without being counted, so `remote_urls` minus
+  the exclusions did not equal `unique_hosts` (36,687 - 9,976 = 26,711, not 26,617). These
+  are now counted as `duplicate_url`, the summary has `selected_urls`, and the funnel adds
+  up exactly. Re-run `reaggregate` to correct an existing `aggregates.json`.
+
 ## [0.3.1] - 2026-11-01
 
 Fewer false tool-poisoning alarms. META-01 (tool poisoning) was flagging ordinary

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/CHECKS.md

@@ -1,6 +1,6 @@
 # mcpprobe checks catalogue
 
-Generated from the check metadata of mcpprobe 0.3.1 (`mcpprobe checks --format markdown`). Do not edit by hand.
+Generated from the check metadata of mcpprobe 0.3.2 (`mcpprobe checks --format markdown`). Do not edit by hand.
 
 Standards: OWASP MCP Top 10 (IDs `MCP0x:2025`, still a beta list, so shown with `(beta)`), OWASP Top 10 for Agentic Applications (`ASI0x`), OWASP Top 10 for LLM Applications 2026 (`LLM0x:2026`, primary) and 2025 (`LLM0x:2025`, kept for one release cycle). Severities are defaults; a check can raise or lower the severity of an individual finding (for example loopback-only servers are reported one level lower), and heuristic checks never rate a finding Critical on their own.
 

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: mcpprobe-scanner
-Version: 0.3.1
+Version: 0.3.2
 Summary: Security scanner for Model Context Protocol (MCP) servers: auth, transport, tool-metadata and poisoning checks with JSON and SARIF output.
 Project-URL: Homepage, https://mcpprobe.tech
 Project-URL: Scanning policy, https://mcpprobe.tech/scanning-policy
@@ -182,8 +182,8 @@ process are in [docs/registry-report.md](docs/registry-report.md).
 
 | Code | Meaning |
 |---|---|
-| `0` | No finding at or above `--fail-on` (default `high`) |
-| `1` | At least one finding at or above `--fail-on` |
+| `0` | No finding or attack chain at or above `--fail-on` (default `high`) |
+| `1` | At least one finding or attack chain at or above `--fail-on` |
 | `2` | Error: target unreachable, bad arguments, ownership verification failed, authorisation refused, or the target opted out |
 | `3` | The target answered but is not an MCP server (for example an ordinary web site); every check is reported not applicable and nothing passes |
 
@@ -202,7 +202,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: mcpprobe-dev/mcpprobe@v0.3.1
+      - uses: mcpprobe-dev/mcpprobe@v0.3.2
         id: scan
         with:
           target: https://staging.your-server.example/mcp

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/README.md

@@ -139,8 +139,8 @@ process are in [docs/registry-report.md](docs/registry-report.md).
 
 | Code | Meaning |
 |---|---|
-| `0` | No finding at or above `--fail-on` (default `high`) |
-| `1` | At least one finding at or above `--fail-on` |
+| `0` | No finding or attack chain at or above `--fail-on` (default `high`) |
+| `1` | At least one finding or attack chain at or above `--fail-on` |
 | `2` | Error: target unreachable, bad arguments, ownership verification failed, authorisation refused, or the target opted out |
 | `3` | The target answered but is not an MCP server (for example an ordinary web site); every check is reported not applicable and nothing passes |
 
@@ -159,7 +159,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: mcpprobe-dev/mcpprobe@v0.3.1
+      - uses: mcpprobe-dev/mcpprobe@v0.3.2
         id: scan
         with:
           target: https://staging.your-server.example/mcp

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/docs/github-action.md

@@ -32,7 +32,7 @@ jobs:
 
       - name: Scan the MCP server
         id: scan
-        uses: mcpprobe-dev/mcpprobe@v0.3.1
+        uses: mcpprobe-dev/mcpprobe@v0.3.2
         with:
           target: https://staging.your-server.example/mcp
           fail-on: high
@@ -61,7 +61,7 @@ A manually triggered version with a URL input lives in
 | Input | Required | Default | Meaning |
 |---|---|---|---|
 | `target` | yes | | MCP endpoint URL (Streamable HTTP), for example `https://staging.example.com/mcp`. |
-| `fail-on` | no | `high` | Fail the step when any finding is at or above this severity: `info`, `low`, `medium`, `high`, `critical`. |
+| `fail-on` | no | `high` | Fail the step when any finding or attack chain is at or above this severity: `info`, `low`, `medium`, `high`, `critical`. |
 | `rate` | no | `5` | Maximum requests per second, 0.1 to 10 (10 is a hard ceiling). |
 | `sarif-output` | no | `mcpprobe.sarif` | Where to write SARIF 2.1.0, relative to the workspace. |
 | `sarif-location` | no | target URL | Repository file to anchor results to. Without it, results point at the endpoint URL, which code scanning shows without a file link. |
@@ -82,8 +82,8 @@ A manually triggered version with a URL input lives in
 
 | Code | `result` | Meaning | What to do |
 |---|---|---|---|
-| `0` | `clean` | No finding at or above `fail-on`. Lower findings are still in the SARIF. | Upload SARIF. |
-| `1` | `findings` | At least one finding at or above `fail-on`. | Upload SARIF; fix or triage the alerts. |
+| `0` | `clean` | No finding or attack chain at or above `fail-on`. Lower findings are still in the SARIF. | Upload SARIF. |
+| `1` | `findings` | At least one finding or attack chain at or above `fail-on`. | Upload SARIF; fix or triage the alerts. |
 | `2` | `error` | Target unreachable, bad input, the target opted out, or authorisation refused. | Read the step log. Do not upload. |
 | `3` | `not-mcp` | The URL answered, but not as an MCP server (for example an HTML page or the wrong path). Nothing was checked and nothing passed. | Fix the URL or the deployment. Do not upload. |
 
@@ -109,7 +109,7 @@ notifications on the analysis, never as passes.
 - **Reachability.** The container runs on the runner's Docker network, so
   `127.0.0.1` inside it is not the runner. To scan a server started earlier in the
   same job, use the CLI instead of the Action:
-  `pipx run --spec mcpprobe-scanner==0.3.1 mcpprobe http://127.0.0.1:8000/mcp --sarif mcpprobe.sarif`.
+  `pipx run --spec mcpprobe-scanner==0.3.2 mcpprobe http://127.0.0.1:8000/mcp --sarif mcpprobe.sarif`.
   For a remote target, it must be reachable from GitHub-hosted runners.
 - **Private repositories** need GitHub Code Security (Advanced Security) for code
   scanning uploads. Without it, keep the scan step as the gate and save
@@ -121,6 +121,6 @@ notifications on the analysis, never as passes.
   plans to deprecate v3; `upload-sarif@v4` takes the same inputs. Switch when your
   organisation does.
 - **Active mode** (`extra-args: --active ...`) also needs `--authorised-by`,
-  `--scope` and `--verify dns-txt`; see the README. In 0.3.1 the active checks are
+  `--scope` and `--verify dns-txt`; see the README. In 0.3.2 the active checks are
   stubs.
-- **Pin a version.** Use `@v0.3.1` (or a full commit SHA), not `@main`.
+- **Pin a version.** Use `@v0.3.2` (or a full commit SHA), not `@main`.

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/docs/registry-report.md

@@ -47,6 +47,10 @@ These rules are in code, not in this document only. The tests in
   loopback, private, CGNAT, link-local or reserved address. The connection is pinned to
   the vetted address; TLS still verifies the hostname through SNI. IP-literal URLs,
   templated URLs (`{tenant}`) and local names are dropped before scanning.
+- **Funnel arithmetic.** Every listed remote URL is either selected or counted under exactly
+  one reason in `funnel.excluded` (`duplicate_host`: another URL on a host already selected;
+  `duplicate_url` *(0.3.2)*: the same URL listed again by another entry or transport), so
+  `remote_urls - sum(excluded) == selected_urls == unique_hosts` with one URL per host.
 - **Same origin only.** A request to any host other than the target is refused.
 - **Stop on refusal.** The first HTTP 429 or 403 stops all traffic to that rate key for
   the rest of the run and for resumed runs. `Retry-After` is not waited out.

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/docs/report-schema.md

@@ -24,7 +24,7 @@ or removed.
 | `readiness` *(0.3)* | array | Directory listing readiness, see below. Always present (for `not_mcp`/`unreachable` every item is `not_assessable`). |
 | `attestations` | array | Governance items a scan cannot assess. |
 | `notes` | array of string | |
-| `summary` | object | `counts` per severity, `max_severity`, `target_kind`, `checks_passed`, `checks_with_findings`, `checks_not_applicable`, `checks_applicable`, `checks_ran`, `checks_skipped`, `checks_not_implemented`, `checks_error`. |
+| `summary` | object | `counts` per severity (findings only), `chain_counts` *(0.3.2, attack chains per severity; each chain is built from findings already in `counts`)*, `max_severity` (from 0.3.2 the worst of findings and chains, the value the exit code uses), `findings_max_severity` *(0.3.2)*, `target_kind`, `checks_passed`, `checks_with_findings`, `checks_not_applicable`, `checks_applicable`, `checks_ran`, `checks_skipped`, `checks_not_implemented`, `checks_error`. |
 | `evidence` | array | Every request/response pair. |
 
 ## `checks[]`

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/mcpprobe/__init__.py

@@ -13,7 +13,7 @@ The package is organised as follows:
 
 from __future__ import annotations
 
-__version__ = "0.3.1"
+__version__ = "0.3.2"
 
 POLICY_URL = "https://mcpprobe.tech/scanning-policy"
 USER_AGENT = f"mcpprobe/{__version__} (+{POLICY_URL})"

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/mcpprobe/action.py

@@ -6,8 +6,8 @@ always writes SARIF, records outputs in ``$GITHUB_OUTPUT``, writes a job
 summary to ``$GITHUB_STEP_SUMMARY``, emits one workflow annotation that names
 the outcome, and exits with the scanner's exit code:
 
-- 0 clean (no finding at or above ``fail-on``)
-- 1 findings at or above ``fail-on``
+- 0 clean (no finding or attack chain at or above ``fail-on``)
+- 1 a finding or attack chain at or above ``fail-on``
 - 2 error (unreachable target, bad input, opt-out, authorisation refused)
 - 3 the target answered but is not an MCP server (wrong URL or path)
 
@@ -89,19 +89,49 @@ def sarif_counts(path: Path) -> dict[str, int]:
     return counts
 
 
-def describe(code: int, target: str, fail_on: str, counts: Mapping[str, int]) -> tuple[str, str, str]:
+def sarif_chain_counts(path: Path) -> dict[str, int]:
+    """Count attack chains (``runs[].properties.chains``) per severity; empty when unreadable.
+
+    Chains are not SARIF results: each one is built from findings that are already
+    results, so they are counted on their own.
+    """
+    counts = dict.fromkeys(_SEVERITIES, 0)
+    try:
+        data: Any = json.loads(path.read_text(encoding="utf-8"))
+        for run in data.get("runs", []):
+            for chain in run.get("properties", {}).get("chains", []):
+                sev = str(chain.get("severity", "info"))
+                counts[sev if sev in counts else "info"] += 1
+    except (OSError, ValueError, AttributeError):
+        return {}
+    return counts
+
+
+def describe(
+    code: int,
+    target: str,
+    fail_on: str,
+    counts: Mapping[str, int],
+    chain_counts: Mapping[str, int] | None = None,
+) -> tuple[str, str, str]:
     """Return (annotation level, title, message) for an exit code."""
     total = sum(counts.values())
     breakdown = ", ".join(f"{counts[s]} {s}" for s in _SEVERITIES if counts.get(s))
+    chain_counts = chain_counts or {}
+    chains = sum(chain_counts.values())
+    chain_breakdown = ", ".join(f"{chain_counts[s]} {s}" for s in _SEVERITIES if chain_counts.get(s))
+    chain_text = f" and {chains} attack chain(s) built from them ({chain_breakdown})" if chains else ""
     if code == EXIT_CLEAN:
-        found = f" ({breakdown} below the threshold)" if breakdown else ""
-        return "notice", "mcpprobe: passed", f"No finding at or above '{fail_on}' on {target}{found}."
+        below = ", ".join(x for x in (breakdown, f"attack chains: {chain_breakdown}" if chains else "") if x)
+        found = f" ({below} below the threshold)" if below else ""
+        what = "finding or attack chain" if chains else "finding"
+        return "notice", "mcpprobe: passed", f"No {what} at or above '{fail_on}' on {target}{found}."
     if code == EXIT_FINDINGS:
         return (
             "error",
             "mcpprobe: findings",
-            f"{total} finding(s) on {target} ({breakdown}); at least one is at or above '{fail_on}'. "
-            "See the job summary or the code scanning alerts.",
+            f"{total} finding(s) on {target} ({breakdown}){chain_text}; at least one is at or above "
+            f"'{fail_on}'. See the job summary or the code scanning alerts.",
         )
     if code == EXIT_NOT_MCP:
         return (
@@ -119,7 +149,14 @@ def describe(code: int, target: str, fail_on: str, counts: Mapping[str, int]) ->
     )
 
 
-def _summary(code: int, target: str, fail_on: str, counts: Mapping[str, int], message: str) -> str:
+def _summary(
+    code: int,
+    target: str,
+    fail_on: str,
+    counts: Mapping[str, int],
+    message: str,
+    chain_counts: Mapping[str, int] | None = None,
+) -> str:
     lines = [
         "## mcpprobe MCP security scan",
         "",
@@ -130,8 +167,13 @@ def _summary(code: int, target: str, fail_on: str, counts: Mapping[str, int], me
         "",
     ]
     if code in (EXIT_CLEAN, EXIT_FINDINGS) and counts:
-        lines += ["| Severity | Findings |", "|---|---|"]
-        lines += [f"| {s} | {counts[s]} |" for s in _SEVERITIES]
+        if chain_counts and sum(chain_counts.values()):
+            lines += ["| Severity | Findings | Attack chains |", "|---|---|---|"]
+            lines += [f"| {s} | {counts[s]} | {chain_counts.get(s, 0)} |" for s in _SEVERITIES]
+            lines += ["", "Each attack chain is built from findings counted in the Findings column."]
+        else:
+            lines += ["| Severity | Findings |", "|---|---|"]
+            lines += [f"| {s} | {counts[s]} |" for s in _SEVERITIES]
         lines.append("")
     lines.append(
         "Docs: https://mcpprobe.tech · checks: https://github.com/mcpprobe-dev/mcpprobe/blob/main/CHECKS.md"
@@ -157,7 +199,8 @@ def run(env: Mapping[str, str] | None = None) -> int:
     fail_on = argv[argv.index("--fail-on") + 1]
     sarif = argv[argv.index("--sarif") + 1]
     counts = sarif_counts(Path(sarif))
-    level, title, message = describe(code, target, fail_on, counts)
+    chain_counts = sarif_chain_counts(Path(sarif))
+    level, title, message = describe(code, target, fail_on, counts, chain_counts)
     print(f"::{level} title={_escape_property(title)}::{_escape(message)}", file=sys.stderr)
     out = env.get("GITHUB_OUTPUT")
     if out:
@@ -168,7 +211,7 @@ def run(env: Mapping[str, str] | None = None) -> int:
     summary = env.get("GITHUB_STEP_SUMMARY")
     if summary:
         with Path(summary).open("a", encoding="utf-8") as fh:
-            fh.write(_summary(code, target, fail_on, counts, message))
+            fh.write(_summary(code, target, fail_on, counts, message, chain_counts))
     return code
 
 

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/mcpprobe/cli.py

@@ -2,8 +2,8 @@
 
 ``mcpprobe <target>`` is shorthand for ``mcpprobe scan <target>``.
 
-Exit codes: 0 = no findings at or above ``--fail-on``; 1 = findings at or
-above ``--fail-on``; 2 = error (unreachable target, opt-out, authorisation
+Exit codes: 0 = no finding or attack chain at or above ``--fail-on``; 1 = a
+finding or attack chain at or above ``--fail-on``; 2 = error (unreachable target, opt-out, authorisation
 refused, bad arguments).
 """
 
@@ -112,7 +112,7 @@ def _parse_kv(values: Sequence[str], what: str, sep: str) -> dict[str, str]:
     default="high",
     type=click.Choice(SEVERITIES),
     show_default=True,
-    help="Exit 1 if any finding is at or above this severity.",
+    help="Exit 1 if any finding or attack chain is at or above this severity.",
 )
 @click.option(
     "--json", "json_path", type=click.Path(dir_okay=False), help="Write the JSON report (with evidence) here."

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/mcpprobe/models.py

@@ -346,12 +346,23 @@ class ScanReport(BaseModel):
     notes: list[str] = Field(default_factory=list)
 
     def counts(self) -> dict[str, int]:
-        """Number of findings per severity (all five keys always present)."""
+        """Number of findings per severity (all five keys always present).
+
+        Attack chains are not findings: each one is built from findings already
+        counted here, so they are counted separately by :meth:`chain_counts`.
+        """
         out = {s.value: 0 for s in _SEVERITY_ORDER}
         for f in self.findings:
             out[f.severity.value] += 1
         return out
 
+    def chain_counts(self) -> dict[str, int]:
+        """Number of attack chains per severity (all five keys always present)."""
+        out = {s.value: 0 for s in _SEVERITY_ORDER}
+        for c in self.chains:
+            out[c.severity.value] += 1
+        return out
+
     def coverage(self) -> dict[str, int]:
         """Check outcomes. ``passed`` counts only checks that ran and found nothing."""
         out = {
@@ -370,7 +381,18 @@ class ScanReport(BaseModel):
         return out
 
     def max_severity(self) -> Severity | None:
-        """Highest severity present, or ``None`` when there are no findings."""
+        """Highest severity of any finding or attack chain, or ``None`` when there is neither.
+
+        A chain can be rated above every finding it is built from (CHAIN-01 is critical
+        while AUTH-01 and META-03 are high), so it takes part in the pass/fail decision.
+        """
+        sevs = [f.severity for f in self.findings] + [c.severity for c in self.chains]
+        if not sevs:
+            return None
+        return max(sevs, key=lambda s: s.rank)
+
+    def findings_max_severity(self) -> Severity | None:
+        """Highest severity among findings only, or ``None`` when there are no findings."""
         if not self.findings:
             return None
         return max((f.severity for f in self.findings), key=lambda s: s.rank)

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/mcpprobe/registry/select.py

@@ -89,7 +89,12 @@ def select_targets(
     *,
     max_urls_per_host: int = 1,
 ) -> Selection:
-    """One target per host (streamable-http preferred over SSE); every decision counted."""
+    """One target per host (streamable-http preferred over SSE); every decision counted.
+
+    Every remote URL ends up either selected or under exactly one reason in ``excluded``, so
+    ``remote_urls - sum(excluded) == selected_urls``, and with ``max_urls_per_host=1``
+    ``selected_urls == unique_hosts``.
+    """
     exclusions = exclusions or Exclusions()
     sel = Selection()
     by_host: dict[str, dict[str, Any]] = {}
@@ -130,6 +135,9 @@ def select_targets(
             if entry["name"] not in target["entries"]:
                 target["entries"].append(entry["name"])
             if url in target["urls"]:
+                # The same URL listed again (another entry, version or transport): count it,
+                # or the funnel would not add up (remote_urls - excluded != selected URLs).
+                sel.excluded["duplicate_url"] += 1
                 continue
             if len(target["urls"]) >= max_urls_per_host:
                 sel.excluded["duplicate_host"] += 1
@@ -146,6 +154,7 @@ def select_targets(
         "no_remote_no_package_entries": kinds["no_remote_no_package"],
         "remote_urls": remote_urls,
         "excluded": dict(sorted(sel.excluded.items())),
+        "selected_urls": sum(len(t["urls"]) for t in sel.targets),
         "unique_hosts": len(sel.targets),
         "rate_keys": len({t["rate_key"] for t in sel.targets}),
         "max_urls_per_host": max_urls_per_host,

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/mcpprobe/report/json_report.py

@@ -13,10 +13,15 @@ from mcpprobe.transport import EvidenceLog
 def report_to_dict(report: ScanReport, evidence: EvidenceLog | None = None) -> dict[str, Any]:
     data: dict[str, Any] = report.model_dump(mode="json")
     worst = report.max_severity()
+    worst_finding = report.findings_max_severity()
     coverage = report.coverage()
     data["summary"] = {
+        # Findings only; chains are built from these findings and are counted separately.
         "counts": report.counts(),
+        "chain_counts": report.chain_counts(),
+        # Worst of findings and attack chains: the value the exit code is decided on.
         "max_severity": worst.value if worst is not None else None,
+        "findings_max_severity": worst_finding.value if worst_finding is not None else None,
         "target_kind": report.target_kind,
         # "passed" = ran and found nothing; not_applicable never counts as passed.
         "checks_passed": coverage["passed"],

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/mcpprobe/report/sarif.py

@@ -194,6 +194,7 @@ def to_sarif(report: ScanReport, *, location_uri: str | None = None) -> dict[str
                 "mode": report.mode.value,
                 "fingerprint": report.fingerprint.model_dump(mode="json") if report.fingerprint else None,
                 "counts": report.counts(),
+                "chainCounts": report.chain_counts(),
                 "targetKind": report.target_kind,
                 # Only checks that ran and found nothing are passes; not_applicable is not a pass.
                 "coverage": report.coverage(),

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/mcpprobe/report/terminal.py

@@ -93,6 +93,49 @@ def _finding_block(f: Finding, verbose: bool) -> Group:
     return Group(*lines)
 
 
+def _count_line(label: str, counts: dict[str, int], tail: str) -> Text:
+    line = Text(label)
+    for sev in _ORDER:
+        style = severity_style(sev, badge=True) if counts[sev.value] else "dim"
+        line.append(f" {sev.value} {counts[sev.value]} ", style=style)
+        line.append(" ")
+    line.append(tail, style="dim")
+    return line
+
+
+def summary_lines(report: ScanReport) -> list[Text]:
+    """The per-severity count lines: findings, then attack chains when there are any.
+
+    Chains are counted on their own line because each one is built from findings that
+    are already counted; adding them to the findings counts would count those twice.
+    """
+    counts = report.counts()
+    total = sum(counts.values())
+    if not report.chains:
+        return [_count_line("Findings ", counts, f"({total} {'finding' if total == 1 else 'findings'})")]
+    chain_counts = report.chain_counts()
+    n = sum(chain_counts.values())
+    return [
+        _count_line("Findings      ", counts, f"({total} {'finding' if total == 1 else 'findings'})"),
+        _count_line(
+            "Attack chains ",
+            chain_counts,
+            f"({n} {'chain' if n == 1 else 'chains'}, built from the findings counted above)",
+        ),
+    ]
+
+
+def fail_reason(report: ScanReport, fail_on: Severity) -> str:
+    """Why the result is FAIL: findings, attack chains, or both, at or above ``fail_on``."""
+    findings = any(f.severity >= fail_on for f in report.findings)
+    chains = any(c.severity >= fail_on for c in report.chains)
+    if findings and chains:
+        return f"findings and attack chains at or above {fail_on.value}"
+    if chains:
+        return f"attack chain at or above {fail_on.value}"
+    return f"findings at or above {fail_on.value}"
+
+
 def render(
     report: ScanReport,
     console: Console | None = None,
@@ -136,13 +179,8 @@ def render(
         console.print(Text("No check was run against this target; nothing passed.", style="dim"))
         return
 
-    counts = report.counts()
-    summary = Text("Findings ")
-    for sev in _ORDER:
-        style = severity_style(sev, badge=True) if counts[sev.value] else "dim"
-        summary.append(f" {sev.value} {counts[sev.value]} ", style=style)
-        summary.append(" ")
-    console.print(summary)
+    for line in summary_lines(report):
+        console.print(line)
     console.print()
 
     for chain in report.chains:
@@ -212,9 +250,7 @@ def render(
     if fail_on is not None:
         top = report.max_severity()
         if top is not None and top >= fail_on:
-            console.print(
-                Text(f"Result: FAIL (findings at or above {fail_on.value})", style=severity_style(top))
-            )
+            console.print(Text(f"Result: FAIL ({fail_reason(report, fail_on)})", style=severity_style(top)))
         else:
             console.print(
                 Text(f"Result: PASS (nothing at or above {fail_on.value})", style=f"bold {PASS_HEX}")

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/tests/test_docs_and_action.py

@@ -2,11 +2,12 @@
 
 from __future__ import annotations
 
+import json
 from pathlib import Path
 
 import pytest
 
-from mcpprobe.action import build_argv, describe, run
+from mcpprobe.action import _summary, build_argv, describe, run, sarif_chain_counts, sarif_counts
 from mcpprobe.catalogue import checks_markdown
 from mcpprobe.checks import ALL_CHECKS
 from mcpprobe.cli import EXIT_CLEAN, EXIT_ERROR, EXIT_FINDINGS, EXIT_NOT_MCP, main
@@ -110,6 +111,40 @@ def test_action_describe_every_exit_code():
     assert describe(EXIT_ERROR, "u", "high", {})[1] == "mcpprobe: scan error"
 
 
+def test_action_counts_attack_chains_separately(tmp_path):
+    sarif = tmp_path / "r.sarif"
+    sarif.write_text(
+        json.dumps(
+            {
+                "runs": [
+                    {
+                        "results": [
+                            {"properties": {"severity": "high"}},
+                            {"properties": {"severity": "high"}},
+                        ],
+                        "properties": {"chains": [{"id": "CHAIN-01", "severity": "critical"}]},
+                    }
+                ]
+            }
+        )
+    )
+    counts, chains = sarif_counts(sarif), sarif_chain_counts(sarif)
+    assert counts["high"] == 2 and sum(counts.values()) == 2
+    assert chains["critical"] == 1 and sum(chains.values()) == 1
+    level, _, message = describe(EXIT_FINDINGS, "u", "critical", counts, chains)
+    assert level == "error"
+    assert "2 finding(s) on u (2 high) and 1 attack chain(s) built from them (1 critical)" in message
+    md = _summary(EXIT_FINDINGS, "u", "critical", counts, message, chains)
+    assert "| Severity | Findings | Attack chains |" in md and "| critical | 0 | 1 |" in md
+    assert "| high | 2 | 0 |" in md
+    _, _, clean = describe(EXIT_CLEAN, "u", "critical", {"high": 1}, {"high": 1})
+    assert (
+        clean
+        == "No finding or attack chain at or above 'critical' on u (1 high, attack chains: 1 high below the threshold)."
+    )
+    assert sarif_chain_counts(tmp_path / "missing.sarif") == {}
+
+
 def test_main_accepts_options_before_target():
     with pytest.raises(SystemExit) as exc:
         main(["--stdio", " ".join(stdio_command()), "--only", "META-01", "-q"])

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/tests/test_registry_report.py

@@ -253,6 +253,41 @@ def test_select_exclusion_list_and_local_names():
     assert sel.summary["package_only_entries"] == 1
 
 
+def test_select_funnel_adds_up_with_repeated_urls():
+    """remote_urls - sum(excluded) == unique_hosts: a repeated URL is counted, not dropped silently."""
+    same = "https://same.example.com/mcp"
+    entries = [
+        {"name": "b/1", "remotes": [{"type": "streamable-http", "url": same}]},
+        # The same URL listed by a second entry, and again as SSE by a third.
+        {"name": "b/2", "remotes": [{"type": "streamable-http", "url": same}]},
+        {"name": "b/3", "remotes": [{"type": "streamable-http", "url": same}, {"type": "sse", "url": same}]},
+        # Another URL on a host already selected.
+        {"name": "b/4", "remotes": [{"type": "sse", "url": "https://same.example.com/sse"}]},
+        {"name": "b/5", "remotes": [{"type": "streamable-http", "url": "https://other.example.net/mcp"}]},
+        {"name": "b/6", "remotes": [{"type": "streamable-http", "url": "https://{tenant}.example.com/mcp"}]},
+        {"name": "b/7", "remotes": [{"type": "streamable-http", "url": "http://10.0.0.1/mcp"}]},
+        {"name": "b/8", "remotes": [{"type": "streamable-http", "url": "https://svc.local/mcp"}]},
+    ]
+    s = select_targets(entries).summary
+    assert s["remote_urls"] == 9
+    assert s["excluded"] == {
+        "duplicate_url": 3,
+        "duplicate_host": 1,
+        "templated_url": 1,
+        "ip_literal": 1,
+        "local_name": 1,
+    }
+    assert s["unique_hosts"] == s["selected_urls"] == 2
+    assert s["remote_urls"] - sum(s["excluded"].values()) == s["unique_hosts"]
+    same_target = next(t for t in select_targets(entries).targets if t["host"] == "same.example.com")
+    assert same_target["entries"] == ["b/1", "b/2", "b/3", "b/4"]
+
+
+def test_select_funnel_adds_up_on_fixture_registry(pipeline):
+    s = pipeline["res"].selection.summary
+    assert s["remote_urls"] - sum(s["excluded"].values()) == s["unique_hosts"] == s["selected_urls"]
+
+
 def test_static_checks_on_registry_tool_definitions(pipeline):
     static = {r["name"]: r for r in read_jsonl(pipeline["wd"].static)}
     assert len(static) == 10

{mcpprobe_scanner-0.3.1 → mcpprobe_scanner-0.3.2}/tests/test_reports_and_cli.py

@@ -113,8 +113,11 @@ async def test_json_report_shape(make_server):
     async with make_server(preset="vulnerable") as srv:
         report = await run(srv.url)
     data = report_to_dict(report)
-    assert data["summary"]["max_severity"] == "high"
+    # CHAIN-01 is critical while every finding is at most high: the overall worst includes chains.
+    assert data["summary"]["max_severity"] == "critical"
+    assert data["summary"]["findings_max_severity"] == "high"
     assert sum(data["summary"]["counts"].values()) == len(data["findings"]) > 10
+    assert sum(data["summary"]["chain_counts"].values()) == len(data["chains"]) >= 1
     assert [a["id"] for a in data["attestations"]] == ["GOV-01", "GOV-02", "GOV-03", "GOV-04", "GOV-05"]
     assert all(a["status"] == "attestation-only" for a in data["attestations"])
     for f in data["findings"]:
@@ -214,9 +217,15 @@ def test_cli_exit_1_on_findings_and_writes_reports(live_server, tmp_path):
 
 
 def test_cli_fail_on_threshold(live_server):
+    # Every finding on loopback is at most high, so only the attack chains decide here.
     with live_server("vulnerable") as srv:
+        res = invoke("scan", srv.url, "--rate", "10", "--fail-on", "critical")
+    # 0.3.1 exited 0 here while listing a CRITICAL CHAIN-01; the exit code now counts chains.
+    assert res.exit_code == EXIT_FINDINGS, res.output
+    assert "Result: FAIL (attack chain at or above critical)" in res.output
+    with live_server("vulnerable", require_auth=True) as srv:
         res = invoke("scan", srv.url, "--rate", "10", "--fail-on", "critical", "-q")
-    assert res.exit_code == EXIT_CLEAN  # worst finding on loopback is high
+    assert res.exit_code == EXIT_CLEAN, res.output  # no AUTH-01, so no chain and nothing critical
 
 
 def test_cli_exit_0_clean(live_server):
@@ -354,3 +363,48 @@ def test_cli_token_command():
     res = invoke("token", "mcp.example.com")
     assert res.exit_code == 0
     assert "_mcpprobe.mcp.example.com.  TXT" in res.output and "mcpprobe-verification=" in res.output
+
+
+# ----------------------------------------------------------------------------- counts and chains
+async def test_counts_and_chains_consistent_across_outputs(make_server):
+    """Every severity the report lists is counted somewhere, the same way in every output."""
+    async with make_server(preset="vulnerable") as srv:
+        report = await run(srv.url)
+    assert any(c.id == "CHAIN-01" and c.severity == Severity.CRITICAL for c in report.chains)
+    assert all(f.severity < Severity.CRITICAL for f in report.findings)
+    counts, chain_counts = report.counts(), report.chain_counts()
+    for sev in Severity:
+        assert counts[sev.value] == sum(1 for f in report.findings if f.severity == sev)
+        assert chain_counts[sev.value] == sum(1 for c in report.chains if c.severity == sev)
+    assert chain_counts["critical"] >= 1
+    assert report.max_severity() == Severity.CRITICAL
+    assert report.findings_max_severity() == Severity.HIGH
+
+    data = report_to_dict(report)
+    assert data["summary"]["counts"] == counts and data["summary"]["chain_counts"] == chain_counts
+    sarif = to_sarif(report)
+    props = sarif["runs"][0]["properties"]
+    assert props["counts"] == counts and props["chainCounts"] == chain_counts
+    assert len(sarif["runs"][0]["results"]) == sum(counts.values())
+
+    console = Console(record=True, width=200)
+    render(report, console, fail_on=Severity.CRITICAL)
+    text = console.export_text()
+    findings_line = next(line for line in text.splitlines() if line.startswith("Findings"))
+    chains_line = next(line for line in text.splitlines() if line.startswith("Attack chains"))
+    for sev in Severity:
+        assert f" {sev.value} {counts[sev.value]} " in findings_line
+        assert f" {sev.value} {chain_counts[sev.value]} " in chains_line
+    assert f"({sum(counts.values())} findings)" in findings_line
+    # The chain is the only thing at critical, so the verdict must name it.
+    assert "Result: FAIL (attack chain at or above critical)" in text
+
+
+async def test_terminal_has_no_chain_line_without_chains(make_server):
+    async with make_server() as srv:
+        report = await run(srv.url, only=["META-04"])
+    assert report.chains == []
+    console = Console(record=True, width=200)
+    render(report, console, fail_on=Severity.HIGH)
+    text = console.export_text()
+    assert "Attack chains" not in text and "Result: PASS (nothing at or above high)" in text