agenteval-toolkit 0.1.0__tar.gz → 0.2.0__tar.gz

{agenteval_toolkit-0.1.0 → agenteval_toolkit-0.2.0}/CHANGELOG.md

@@ -6,6 +6,25 @@ All notable changes to this project are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.2.0] - 2026-10-12
+
+### Added
+- **A2A (Agent2Agent) evaluation adapter** (`adapter: a2a`): discovers the Agent Card, prefers a v1.0 JSON-RPC
+  interface and falls back to v0.3; one A2A conversation per exam case (`contextId`), `taskId` continuation for
+  `input-required` / `auth-required` tasks, `GetTask` polling for long-running tasks, tool calls read from
+  `metadata.tool_calls` or DataParts. Verified against the official `a2a-sdk` 1.2 server (v1.0 and v0.3 compat).
+- **A2A Agent Card scanning**: `agenteval a2a card` and `agenteval scan --a2a-card` (new `a2a_agent` asset type).
+- **MCP client-config scanning**: `agenteval scan --mcp-config` for `mcpServers` / `servers` JSON (new `mcp_server` asset type).
+- **`agent-protocols` rule pack** (new `baseline` tier): 6 A2A and 6 MCP rules, with Chinese translations.
+- **`agenteval mcp serve`**: dependency-free MCP stdio server with 8 tools (scan, inspect_a2a_card, run_eval,
+  check_certification, verify_certificate, …), confined to `--root`, network-gated by `--allow-network`.
+  Verified with the official MCP Python SDK 2.3 client.
+- `agent.schema.json` for agent-under-test configs; editor modelines on the example agents.
+
+### Changed
+- `scan` no longer requires `--snapshot` when `--a2a-card` or `--mcp-config` is given.
+- Adapter error messages default to English.
+
 ## [0.1.0] - 2026-10-11
 
 ### Added

{agenteval_toolkit-0.1.0 → agenteval_toolkit-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agenteval-toolkit
-Version: 0.1.0
+Version: 0.2.0
 Summary: Scan AI infrastructure against policy-as-code rule packs, evaluate AI agents with standard exams, and issue signed certificates when both pass.
 Author: The AgentEval Authors
 License-Expression: Apache-2.0
@@ -47,13 +47,13 @@ Dynamic: license-file
 
 **Scan, evaluate, certify.** AgentEval is an Apache-2.0 command-line toolkit with three jobs:
 
-- it **scans** self-hosted AI infrastructure against policy-as-code rule packs,
+- it **scans** self-hosted AI infrastructure, A2A Agent Cards and MCP configurations against policy-as-code rule packs,
 - it **evaluates** AI agents with standard exam papers, and
 - it **issues signed certificates** when a system passes both.
 
 [English](https://github.com/pony360/AgentEval/blob/main/README.md) · [简体中文](https://github.com/pony360/AgentEval/blob/main/README.zh-CN.md)
 
-> **Status: alpha (0.1.0).** The rule packs are starter mappings, and every one is marked `verified: false`.
+> **Status: alpha (0.2.0).** The rule packs are starter mappings, and every one is marked `verified: false`.
 > They are a starting point for your own legal and compliance review. They are not legal advice.
 
 ---
@@ -68,6 +68,7 @@ Dynamic: license-file
 - [Rule packs](#rule-packs)
 - [Writing rules](#writing-rules)
 - [Evaluating agents](#evaluating-agents)
+- [Agent protocols: A2A and MCP](#agent-protocols-a2a-and-mcp)
 - [Certification](#certification)
 - [Configuration](#configuration)
 - [Security model](#security-model)
@@ -95,7 +96,9 @@ combined result into a certificate that anyone can verify offline.
 | **Tiered rule packs** | National (US federal guidance), regional (California) and industry (PCI DSS) starter packs |
 | **Reports** | JSON, SARIF (GitHub code scanning) and HTML, in English or Simplified Chinese |
 | **Agent evaluation** | Scripted multi-turn exams with tool mocks, deterministic assertions plus rubric judging, and critical-failure vetoes |
-| **Pluggable adapters and judges** | OpenAI-compatible endpoints, generic webhooks and demo agents. Heuristic (offline), Anthropic or OpenAI-compatible judges |
+| **Pluggable adapters and judges** | OpenAI-compatible endpoints, A2A agents (v0.3 and v1.0), generic webhooks and demo agents. Heuristic (offline), Anthropic or OpenAI-compatible judges |
+| **Agent protocols** | Checks A2A Agent Cards (TLS, auth, OAuth flows, signing) and MCP client configs (plaintext secrets, unpinned packages, download-and-execute, privileged containers) |
+| **MCP server** | `agenteval mcp serve` exposes scanning, evaluation and certificate checks as MCP tools, confined to a root directory and offline by default |
 | **Certification** | Strict gates, Ed25519-signed JSON, HTML/PDF certificates, optional QR verification URL and revocation status check |
 | **CI-friendly** | Stable exit codes. No database or server required |
 
@@ -172,6 +175,8 @@ Every command accepts `--lang zh` for Chinese output.
 | `agenteval schemas -o DIR` | Export the JSON Schemas for editors |
 | `agenteval eval papers \| validate \| run` | List or validate exam papers. Run an exam against an agent |
 | `agenteval cert check \| issue \| verify \| keygen` | Certification |
+| `agenteval a2a card TARGET` | Fetch and check an A2A Agent Card |
+| `agenteval mcp serve` | Run AgentEval as an MCP server over stdio |
 
 | Exit code | Meaning |
 |---|---|
@@ -185,6 +190,7 @@ Every command accepts `--lang zh` for Chinese output.
 |---|---|---|---|
 | `us-federal` | national | 16 | NIST AI RMF 1.0, AI 600-1, SP 800-53, SP 800-190, SP 800-218A and OMB M-25-21: authentication, TLS, network policy, image pinning, secrets, provenance, governance attestations |
 | `us-ca` | regional | 3 | Bot disclosure (B&P §17940), CCPA data minimization, AB 2013 training-data transparency |
+| `agent-protocols` | baseline | 12 | A2A Agent Cards and MCP client configs: TLS, authentication, OAuth flows, card signing, plaintext secrets, version pinning, download-and-execute, container isolation |
 | `pci-dss` | industry | 4 | PCI DSS v4.0.1 (Req. 3, 4, 8, 10): no full card numbers in logs, strong cryptography in transit, authentication, and audit logging for AI that handles payment conversations |
 
 Rule text is in English, and Simplified Chinese translations are in `rules/*/i18n/zh.yaml`.
@@ -228,6 +234,22 @@ agenteval eval run -p cs-ecommerce-v1 -a my-agent.yaml --judge anthropic --judge
 
 See **[docs/exams.md](https://github.com/pony360/AgentEval/blob/main/docs/exams.md)** for the paper format, scoring, adapters and judges.
 
+## Agent protocols: A2A and MCP
+
+```bash
+# Evaluate an A2A agent (v1.0 or v0.3): each exam case runs as one A2A conversation
+agenteval eval run -p cs-ecommerce-v1 -a examples/agents/a2a.example.yaml
+
+# Check an Agent Card, and the MCP servers your tools launch
+agenteval a2a card https://agents.example.com
+agenteval scan --a2a-card ./agent-card.json --mcp-config ~/.cursor/mcp.json -f html -o protocols.html
+
+# Use AgentEval from Claude, an IDE or any MCP client
+agenteval mcp serve --root .
+```
+
+See **[docs/a2a.md](https://github.com/pony360/AgentEval/blob/main/docs/a2a.md)** and **[docs/mcp.md](https://github.com/pony360/AgentEval/blob/main/docs/mcp.md)**.
+
 ## Certification
 
 `cert issue` signs a certificate only if all of the following hold:
@@ -271,17 +293,19 @@ To report a vulnerability, see [SECURITY.md](https://github.com/pony360/AgentEva
 
 ```
 agenteval/            Python package (CLI, engine, collectors, evaluation, certification, schemas)
-rules/                Rule packs (national / regional / industry)
+rules/                Rule packs (national / regional / industry / baseline)
 exams/                Exam papers
 certification/        Certification policy and registered systems
-examples/             Sample snapshot, attestations and agent configs
+examples/             Sample snapshot, attestations, agent configs, A2A cards and MCP configs
 docs/                 Reference documentation
 tests/                Test suite (pytest)
 ```
 
 ## Roadmap
 
-- More collectors (cloud-managed endpoints, model registries, API gateways)
+- More collectors (cloud-managed endpoints, model registries, API gateways, live MCP server introspection)
+- A2A streaming (`SendStreamingMessage`) and gRPC / HTTP+JSON bindings; Agent Card signature verification
+- An MCP mock-tool server so agents that consume MCP can be scored on their tool calls
 - More rule packs (EU AI Act, ISO/IEC 42001, HIPAA), contributed with reviewer sign-off
 - More exam papers (IT helpdesk, sales assistant, RAG question answering)
 - A hosted verification and status service reference implementation

{agenteval_toolkit-0.1.0 → agenteval_toolkit-0.2.0}/README.md

@@ -2,13 +2,13 @@
 
 **Scan, evaluate, certify.** AgentEval is an Apache-2.0 command-line toolkit with three jobs:
 
-- it **scans** self-hosted AI infrastructure against policy-as-code rule packs,
+- it **scans** self-hosted AI infrastructure, A2A Agent Cards and MCP configurations against policy-as-code rule packs,
 - it **evaluates** AI agents with standard exam papers, and
 - it **issues signed certificates** when a system passes both.
 
 [English](https://github.com/pony360/AgentEval/blob/main/README.md) · [简体中文](https://github.com/pony360/AgentEval/blob/main/README.zh-CN.md)
 
-> **Status: alpha (0.1.0).** The rule packs are starter mappings, and every one is marked `verified: false`.
+> **Status: alpha (0.2.0).** The rule packs are starter mappings, and every one is marked `verified: false`.
 > They are a starting point for your own legal and compliance review. They are not legal advice.
 
 ---
@@ -23,6 +23,7 @@
 - [Rule packs](#rule-packs)
 - [Writing rules](#writing-rules)
 - [Evaluating agents](#evaluating-agents)
+- [Agent protocols: A2A and MCP](#agent-protocols-a2a-and-mcp)
 - [Certification](#certification)
 - [Configuration](#configuration)
 - [Security model](#security-model)
@@ -50,7 +51,9 @@ combined result into a certificate that anyone can verify offline.
 | **Tiered rule packs** | National (US federal guidance), regional (California) and industry (PCI DSS) starter packs |
 | **Reports** | JSON, SARIF (GitHub code scanning) and HTML, in English or Simplified Chinese |
 | **Agent evaluation** | Scripted multi-turn exams with tool mocks, deterministic assertions plus rubric judging, and critical-failure vetoes |
-| **Pluggable adapters and judges** | OpenAI-compatible endpoints, generic webhooks and demo agents. Heuristic (offline), Anthropic or OpenAI-compatible judges |
+| **Pluggable adapters and judges** | OpenAI-compatible endpoints, A2A agents (v0.3 and v1.0), generic webhooks and demo agents. Heuristic (offline), Anthropic or OpenAI-compatible judges |
+| **Agent protocols** | Checks A2A Agent Cards (TLS, auth, OAuth flows, signing) and MCP client configs (plaintext secrets, unpinned packages, download-and-execute, privileged containers) |
+| **MCP server** | `agenteval mcp serve` exposes scanning, evaluation and certificate checks as MCP tools, confined to a root directory and offline by default |
 | **Certification** | Strict gates, Ed25519-signed JSON, HTML/PDF certificates, optional QR verification URL and revocation status check |
 | **CI-friendly** | Stable exit codes. No database or server required |
 
@@ -127,6 +130,8 @@ Every command accepts `--lang zh` for Chinese output.
 | `agenteval schemas -o DIR` | Export the JSON Schemas for editors |
 | `agenteval eval papers \| validate \| run` | List or validate exam papers. Run an exam against an agent |
 | `agenteval cert check \| issue \| verify \| keygen` | Certification |
+| `agenteval a2a card TARGET` | Fetch and check an A2A Agent Card |
+| `agenteval mcp serve` | Run AgentEval as an MCP server over stdio |
 
 | Exit code | Meaning |
 |---|---|
@@ -140,6 +145,7 @@ Every command accepts `--lang zh` for Chinese output.
 |---|---|---|---|
 | `us-federal` | national | 16 | NIST AI RMF 1.0, AI 600-1, SP 800-53, SP 800-190, SP 800-218A and OMB M-25-21: authentication, TLS, network policy, image pinning, secrets, provenance, governance attestations |
 | `us-ca` | regional | 3 | Bot disclosure (B&P §17940), CCPA data minimization, AB 2013 training-data transparency |
+| `agent-protocols` | baseline | 12 | A2A Agent Cards and MCP client configs: TLS, authentication, OAuth flows, card signing, plaintext secrets, version pinning, download-and-execute, container isolation |
 | `pci-dss` | industry | 4 | PCI DSS v4.0.1 (Req. 3, 4, 8, 10): no full card numbers in logs, strong cryptography in transit, authentication, and audit logging for AI that handles payment conversations |
 
 Rule text is in English, and Simplified Chinese translations are in `rules/*/i18n/zh.yaml`.
@@ -183,6 +189,22 @@ agenteval eval run -p cs-ecommerce-v1 -a my-agent.yaml --judge anthropic --judge
 
 See **[docs/exams.md](https://github.com/pony360/AgentEval/blob/main/docs/exams.md)** for the paper format, scoring, adapters and judges.
 
+## Agent protocols: A2A and MCP
+
+```bash
+# Evaluate an A2A agent (v1.0 or v0.3): each exam case runs as one A2A conversation
+agenteval eval run -p cs-ecommerce-v1 -a examples/agents/a2a.example.yaml
+
+# Check an Agent Card, and the MCP servers your tools launch
+agenteval a2a card https://agents.example.com
+agenteval scan --a2a-card ./agent-card.json --mcp-config ~/.cursor/mcp.json -f html -o protocols.html
+
+# Use AgentEval from Claude, an IDE or any MCP client
+agenteval mcp serve --root .
+```
+
+See **[docs/a2a.md](https://github.com/pony360/AgentEval/blob/main/docs/a2a.md)** and **[docs/mcp.md](https://github.com/pony360/AgentEval/blob/main/docs/mcp.md)**.
+
 ## Certification
 
 `cert issue` signs a certificate only if all of the following hold:
@@ -226,17 +248,19 @@ To report a vulnerability, see [SECURITY.md](https://github.com/pony360/AgentEva
 
 ```
 agenteval/            Python package (CLI, engine, collectors, evaluation, certification, schemas)
-rules/                Rule packs (national / regional / industry)
+rules/                Rule packs (national / regional / industry / baseline)
 exams/                Exam papers
 certification/        Certification policy and registered systems
-examples/             Sample snapshot, attestations and agent configs
+examples/             Sample snapshot, attestations, agent configs, A2A cards and MCP configs
 docs/                 Reference documentation
 tests/                Test suite (pytest)
 ```
 
 ## Roadmap
 
-- More collectors (cloud-managed endpoints, model registries, API gateways)
+- More collectors (cloud-managed endpoints, model registries, API gateways, live MCP server introspection)
+- A2A streaming (`SendStreamingMessage`) and gRPC / HTTP+JSON bindings; Agent Card signature verification
+- An MCP mock-tool server so agents that consume MCP can be scored on their tool calls
 - More rule packs (EU AI Act, ISO/IEC 42001, HIPAA), contributed with reviewer sign-off
 - More exam papers (IT helpdesk, sales assistant, RAG question answering)
 - A hosted verification and status service reference implementation

{agenteval_toolkit-0.1.0 → agenteval_toolkit-0.2.0}/README.zh-CN.md

@@ -1,10 +1,10 @@
 # AgentEval
 
-**扫描 · 评测 · 认证。** AgentEval 是一个 Apache-2.0 开源命令行工具：按策略即代码（policy-as-code）规则包**扫描**自托管 AI 基础设施，用标准试卷**评测** AI Agent，两项都通过时**签发数字签名证书**。
+**扫描 · 评测 · 认证。** AgentEval 是一个 Apache-2.0 开源命令行工具：按策略即代码（policy-as-code）规则包**扫描**自托管 AI 基础设施、A2A Agent Card 与 MCP 配置，用标准试卷**评测** AI Agent，两项都通过时**签发数字签名证书**。
 
 [English](https://github.com/pony360/AgentEval/blob/main/README.md) · [简体中文](https://github.com/pony360/AgentEval/blob/main/README.zh-CN.md)
 
-> **状态：Alpha（0.1.0）。** 内置规则包为入门级映射，全部标记为 `verified: false`，供贵司法务/合规团队审核的起点，不构成法律意见。
+> **状态：Alpha（0.2.0）。** 内置规则包为入门级映射，全部标记为 `verified: false`，供贵司法务/合规团队审核的起点，不构成法律意见。
 
 ## 目录
 
@@ -29,7 +29,9 @@
 | **分层规则包** | 国家（美国联邦指引）、区域（加州）、行业（PCI DSS）入门规则包 |
 | **报告** | JSON、SARIF（GitHub 代码扫描）、HTML；中英文输出 |
 | **Agent 评测** | 脚本化多轮对话 + 工具模拟；确定性断言 + 评分细则（rubric）裁判；关键项一票否决 |
-| **可插拔** | OpenAI 兼容接口、通用 Webhook、演示 Agent；离线启发式 / Anthropic / OpenAI 兼容裁判 |
+| **可插拔** | OpenAI 兼容接口、A2A Agent（v0.3 与 v1.0）、通用 Webhook、演示 Agent；离线启发式 / Anthropic / OpenAI 兼容裁判 |
+| **智能体协议** | 检查 A2A Agent Card（TLS、认证、OAuth 模式、签名）与 MCP 客户端配置（明文密钥、未固定版本、下载即执行、特权容器） |
+| **MCP 服务器** | `agenteval mcp serve` 把扫描、评测与证书校验作为 MCP 工具提供；文件访问限定在根目录，默认不联网 |
 | **认证** | 严格门槛、Ed25519 签名 JSON、HTML/PDF 证书、可选二维码验证地址与吊销状态查询 |
 | **适合 CI** | 稳定退出码；无需数据库或服务端 |
 
@@ -84,6 +86,7 @@ agenteval cert verify certificates/AEV-*.json --public-key issuer.pem --lang zh
 | `us-federal` | 国家 | 16 | NIST AI RMF 1.0、AI 600-1、SP 800-53、SP 800-190、SP 800-218A、OMB M-25-21 |
 | `us-ca` | 区域 | 3 | 机器人身份披露（B&P §17940）、CCPA 数据最小化、AB 2013 训练数据透明度 |
 | `pci-dss` | 行业 | 4 | PCI DSS v4.0.1（要求 3、4、8、10） |
+| `agent-protocols` | 基线 | 12 | A2A Agent Card 与 MCP 客户端配置：TLS、认证、OAuth 模式、卡片签名、明文密钥、版本固定、下载即执行、容器隔离 |
 
 规则以英文为源语言，简体中文译文位于 `rules/*/i18n/zh.yaml`。
 
@@ -95,6 +98,15 @@ agenteval cert verify certificates/AEV-*.json --public-key issuer.pem --lang zh
 
 - 试卷格式、评分与适配器：[docs/exams.md](https://github.com/pony360/AgentEval/blob/main/docs/exams.md)
 - 认证门槛、密钥、验证与吊销：[docs/certification.md](https://github.com/pony360/AgentEval/blob/main/docs/certification.md)
+- A2A 评测适配器与 Agent Card 检查：[docs/a2a.md](https://github.com/pony360/AgentEval/blob/main/docs/a2a.md)
+- MCP 服务器与 MCP 配置扫描：[docs/mcp.md](https://github.com/pony360/AgentEval/blob/main/docs/mcp.md)
+
+```bash
+agenteval eval run -p cs-ecommerce-v1 -a examples/agents/a2a.example.yaml   # 评测 A2A Agent
+agenteval a2a card https://agents.example.com                               # 检查 Agent Card
+agenteval scan --mcp-config ~/.cursor/mcp.json --lang zh                    # 扫描 MCP 配置
+agenteval mcp serve --root .                                                # 作为 MCP 服务器运行
+```
 
 颁证条件：安全得分 **> 90** 且质量得分 **> 90**；无未修复的高危/严重项；无待证明项；评测无关键项失败与会话错误；证据不超过 30 天。离线启发式裁判只能获得 **DEMO** 证书。证书有效期 6 个月。
 

{agenteval_toolkit-0.1.0 → agenteval_toolkit-0.2.0}/agenteval/__init__.py

@@ -2,4 +2,4 @@
 # Copyright 2026 The AgentEval Authors
 """AgentEval: scan AI infrastructure, evaluate AI agents, and certify systems that pass both."""
 
-__version__ = "0.1.0"
+__version__ = "0.2.0"

{agenteval_toolkit-0.1.0 → agenteval_toolkit-0.2.0}/agenteval/certification/render.py

@@ -17,7 +17,7 @@ UI = {
         "certifies": "兹证明", "statement": "已通过 AgentEval 工具的安全合规扫描与标准化质量评测，满足本认证全部颁证条件。",
         "security": "安全", "security_sub": "国家、区域及行业安全规范", "quality": "质量", "quality_sub": "标准化评测指标",
         "score": "得分", "threshold": "阈值", "tier": "层级", "pack": "规则包", "checks": "通过 / 适用", "status": "结论",
-        "national": "国家", "regional": "区域", "industry": "行业", "unknown": "其他",
+        "national": "国家", "regional": "区域", "industry": "行业", "baseline": "基线", "unknown": "其他",
         "st_passed": "通过", "st_failed": "未通过", "st_not_applicable": "不适用", "st_not_assessed": "未评估",
         "exam": "试卷", "grade": "等级", "judge": "裁判", "cases": "样本通过", "dimension": "维度",
         "issued": "颁发日期", "expires": "有效期至", "evidence": "证据", "scan": "扫描", "rules": "规则集", "run": "评测",
@@ -38,7 +38,7 @@ UI = {
         "security": "Security", "security_sub": "National, regional and industry security requirements",
         "quality": "Quality", "quality_sub": "Standardized evaluation metrics",
         "score": "Score", "threshold": "Threshold", "tier": "Tier", "pack": "Rule pack", "checks": "Passed / applicable",
-        "status": "Result", "national": "National", "regional": "Regional", "industry": "Industry", "unknown": "Other",
+        "status": "Result", "national": "National", "regional": "Regional", "industry": "Industry", "baseline": "Baseline", "unknown": "Other",
         "st_passed": "Passed", "st_failed": "Failed", "st_not_applicable": "Not applicable", "st_not_assessed": "Not assessed",
         "exam": "Exam", "grade": "Grade", "judge": "Judge", "cases": "Cases passed", "dimension": "Dimension",
         "issued": "Issued", "expires": "Valid until", "evidence": "Evidence", "scan": "Scan", "rules": "Rule set", "run": "Eval run",

{agenteval_toolkit-0.1.0 → agenteval_toolkit-0.2.0}/agenteval/cli.py

@@ -79,8 +79,8 @@ CLI_MESSAGES = {
     "attest_skipped": {"en": "Evaluation did not pass; no pre-deployment testing attestation written",
                        "zh": "评测未通过，未生成部署前测试证明"},
     "snapshot_written": {"en": "Exported {n} workloads to {path}", "zh": "已导出 {n} 个工作负载到 {path}"},
-    "snapshot_required": {"en": "--snapshot is required when --source=snapshot",
-                          "zh": "source=snapshot 时必须提供 --snapshot"},
+    "snapshot_required": {"en": "Nothing to scan: pass --snapshot, --source kubernetes, --a2a-card or --mcp-config",
+                          "zh": "没有可扫描的对象：请提供 --snapshot、--source kubernetes、--a2a-card 或 --mcp-config"},
     "collect_failed": {"en": "Collection failed: {e}", "zh": "采集失败: {e}"},
     "unknown_system": {"en": "Unknown AI system: {ref} (registered: {ids})", "zh": "未登记的 AI 系统: {ref}（可用: {ids}）"},
     "bad_inputs": {"en": "Cannot read scan/eval JSON (produce them with -f json): {e}",
@@ -141,6 +141,8 @@ def scan(
     context: Optional[str] = typer.Option(None, help="kubeconfig context."),
     namespace: list[str] = typer.Option([], "--namespace", "-n", help="Limit to a namespace; repeatable."),
     in_cluster: bool = typer.Option(False, help="Use the in-cluster ServiceAccount."),
+    a2a_card: list[str] = typer.Option([], "--a2a-card", help="A2A Agent Card URL, agent base URL or JSON file; repeatable."),
+    mcp_config: list[Path] = typer.Option([], "--mcp-config", help="MCP client config (mcp.json / claude_desktop_config.json); repeatable."),
     rules: Path = typer.Option(None, help="Rules directory or file (default ./rules or $AGENTEVAL_RULES_DIR)."),
     attestations: list[Path] = typer.Option([], help="Attestation YAML; repeatable, files are merged."),
     fmt: OutFormat = typer.Option(OutFormat.json, "--format", "-f", help="Report format."),
@@ -148,19 +150,32 @@ def scan(
     lang: Lang = LANG_OPT,
     fail_on: FailOn = typer.Option(FailOn.high, help="Exit 1 if any failed check is at this severity or above."),
 ) -> None:
-    """Scan a cluster (or snapshot) and evaluate it against the rule packs."""
+    """Scan a cluster (or snapshot), A2A Agent Cards and MCP configs against the rule packs."""
+    from agenteval.collectors.a2a import collect_a2a_cards
+    from agenteval.collectors.mcp import collect_mcp_configs
+
     rule_set = _load_rules_or_exit(rules or default_rules_dir())
     try:
         atts = load_attestations(list(attestations))
+        labels: list[str] = []
+        assets = []
         if source == SourceKind.snapshot:
-            if not snapshot:
+            if snapshot:
+                assets, _ = collect_from_snapshot_file(snapshot)
+                labels.append(f"snapshot:{snapshot.name}")
+            elif not (a2a_card or mcp_config):
                 _err(_m("snapshot_required", lang.value))
                 raise typer.Exit(2)
-            assets, _ = collect_from_snapshot_file(snapshot)
-            source_label = f"snapshot:{snapshot.name}"
         else:
             assets, _ = collect_from_kubernetes(context=context, namespaces=namespace or None, in_cluster=in_cluster)
-            source_label = f"kubernetes:{context or ('in-cluster' if in_cluster else 'current-context')}"
+            labels.append(f"kubernetes:{context or ('in-cluster' if in_cluster else 'current-context')}")
+        if a2a_card:
+            assets += collect_a2a_cards(list(a2a_card))
+            labels.append(f"a2a:{len(a2a_card)}")
+        if mcp_config:
+            assets += collect_mcp_configs(list(mcp_config))
+            labels.append("mcp:" + ",".join(p.name for p in mcp_config))
+        source_label = " + ".join(labels)
     except typer.Exit:
         raise
     except Exception as e:
@@ -529,5 +544,60 @@ def cert_keygen(
     typer.echo(public_pem(key) + f"key_id: {key_id(key)}")
 
 
+# ======================= Agent protocols (A2A / MCP) =======================
+
+a2a_app = typer.Typer(help="Agent2Agent (A2A) protocol helpers.", no_args_is_help=True)
+mcp_app = typer.Typer(help="Model Context Protocol (MCP): run AgentEval as an MCP server.", no_args_is_help=True)
+app.add_typer(a2a_app, name="a2a")
+app.add_typer(mcp_app, name="mcp")
+
+
+@a2a_app.command("card")
+def a2a_card_cmd(
+    target: str = typer.Argument(..., help="Agent Card file, card URL, or agent base URL."),
+    rules: Path = typer.Option(None, help="Rules directory (default ./rules or $AGENTEVAL_RULES_DIR)."),
+    lang: Lang = LANG_OPT,
+) -> None:
+    """Fetch and check an A2A Agent Card: endpoints, protocol versions, auth and signing. Exit 1 on high findings."""
+    from agenteval.collectors.a2a import CardError, fetch_card, normalize_card
+    from agenteval.i18n import localize_result
+
+    try:
+        asset = normalize_card(*fetch_card(target))
+    except CardError as e:
+        _err(str(e))
+        raise typer.Exit(2)
+    rule_set = [r for r in _load_rules_or_exit(rules or default_rules_dir()) if "a2a_agent" in r.applies_to.asset_types]
+    result = run_scan(rule_set, [asset], source=f"a2a:{target}")
+    f = asset.facts
+    typer.echo(f"{asset.name}  ({asset.id})")
+    typer.echo(f"  protocol: {', '.join(f['protocol_versions']) or '-'}  bindings: {', '.join(f['bindings']) or '-'}")
+    typer.echo(f"  endpoints: {', '.join(f['endpoint']['urls']) or '-'}")
+    typer.echo(f"  auth: {', '.join(f['auth']['schemes']) or 'none'} (required: {f['auth']['required']})  "
+               f"signed: {f['card']['signed']}  skills: {f['skills']['count']}")
+    localized = localize_result(result, lang.value)
+    worst = 0
+    for x in localized.findings:
+        if x.status == Status.not_applicable:
+            continue
+        mark = "✓" if x.status == Status.passed else "✕"
+        typer.secho(f"  {mark} {x.rule_id:<14} {x.severity.value:<8} {x.rule_title}",
+                    fg="green" if x.status == Status.passed else "red")
+        if x.status == Status.failed:
+            worst = max(worst, x.severity.rank)
+    raise typer.Exit(1 if worst >= Severity.high.rank else 0)
+
+
+@mcp_app.command("serve")
+def mcp_serve_cmd(
+    root: Path = typer.Option(Path("."), help="Only files under this directory can be read by tools."),
+    allow_network: bool = typer.Option(False, help="Allow tools that fetch remote Agent Cards or call remote agents."),
+) -> None:
+    """Run AgentEval as an MCP server over stdio (for Claude, IDEs and other MCP clients)."""
+    from agenteval.mcp_server import serve
+
+    serve(root, allow_network)
+
+
 if __name__ == "__main__":  # pragma: no cover
     app()

agenteval_toolkit-0.2.0/agenteval/collectors/a2a.py

@@ -0,0 +1,176 @@
+# SPDX-License-Identifier: Apache-2.0
+# Copyright 2026 The AgentEval Authors
+"""Normalize A2A (Agent2Agent) Agent Cards into `a2a_agent` assets.
+
+Supports both the v0.3 card shape (top-level `url`, `preferredTransport`,
+`additionalInterfaces`, `security`) and the v1.0 shape (`supportedInterfaces[]`
+with `protocolBinding` / `protocolVersion`, `capabilities.extendedAgentCard`).
+Only metadata is recorded; nothing is sent to the agent.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+from typing import Any
+from urllib.parse import parse_qs, urlparse
+
+import httpx
+
+from agenteval.models import Asset
+
+WELL_KNOWN = "/.well-known/agent-card.json"
+LOCAL_HOSTS = {"localhost", "127.0.0.1", "::1"}
+
+# v1.0 proto-JSON wraps each scheme in a oneof; map wrapper names to the classic `type` values.
+_SCHEME_WRAPPERS = {
+    "apiKeySecurityScheme": "apiKey",
+    "httpAuthSecurityScheme": "http",
+    "oauth2SecurityScheme": "oauth2",
+    "openIdConnectSecurityScheme": "openIdConnect",
+    "mtlsSecurityScheme": "mutualTLS",
+}
+
+
+class CardError(ValueError):
+    pass
+
+
+def _slug(text: str) -> str:
+    return re.sub(r"[^a-z0-9]+", "-", text.lower()).strip("-") or "agent"
+
+
+def _is_local(url: str) -> bool:
+    return (urlparse(url).hostname or "") in LOCAL_HOSTS
+
+
+def card_url_for(target: str) -> str:
+    """Accept a card URL or an agent base URL; return the URL to fetch."""
+    if target.endswith(".json"):
+        return target
+    return target.rstrip("/") + WELL_KNOWN
+
+
+def fetch_card(target: str, client: httpx.Client | None = None, timeout: float = 15.0,
+               headers: dict[str, str] | None = None) -> tuple[dict, str]:
+    """Load a card from a local file or over HTTPS (plain HTTP only for localhost)."""
+    p = Path(target)
+    if not target.startswith(("http://", "https://")):
+        if not p.exists():
+            raise CardError(f"Agent Card not found: {target}")
+        return json.loads(p.read_text(encoding="utf-8")), str(p)
+    url = card_url_for(target)
+    if url.startswith("http://") and not _is_local(url):
+        raise CardError(f"Refusing to fetch an Agent Card over plain HTTP: {url}")
+    own = client is None
+    client = client or httpx.Client(follow_redirects=False)
+    try:
+        r = client.get(url, headers=headers or {}, timeout=timeout)
+        r.raise_for_status()
+        return r.json(), url
+    except (httpx.HTTPError, ValueError) as e:
+        raise CardError(f"Could not fetch Agent Card {url}: {e}") from e
+    finally:
+        if own:
+            client.close()
+
+
+def interfaces(card: dict) -> list[dict[str, str]]:
+    """All declared endpoints as {url, binding, version}, newest card shape first."""
+    out: list[dict[str, str]] = []
+    for i in card.get("supportedInterfaces") or []:
+        if isinstance(i, dict) and i.get("url"):
+            out.append({"url": i["url"], "binding": str(i.get("protocolBinding") or "JSONRPC").upper(),
+                        "version": str(i.get("protocolVersion") or "1.0")})
+    if card.get("url"):
+        out.append({"url": card["url"], "binding": str(card.get("preferredTransport") or "JSONRPC").upper(),
+                    "version": str(card.get("protocolVersion") or "0.3")})
+    for i in card.get("additionalInterfaces") or []:
+        if isinstance(i, dict) and i.get("url"):
+            out.append({"url": i["url"], "binding": str(i.get("transport") or "JSONRPC").upper(),
+                        "version": str(card.get("protocolVersion") or "0.3")})
+    seen, uniq = set(), []
+    for i in out:
+        key = (i["url"], i["binding"], i["version"])
+        if key not in seen:
+            seen.add(key)
+            uniq.append(i)
+    return uniq
+
+
+def _schemes(card: dict) -> dict[str, dict]:
+    out: dict[str, dict] = {}
+    for name, raw in (card.get("securitySchemes") or {}).items():
+        if not isinstance(raw, dict):
+            continue
+        if "type" in raw:
+            out[name] = raw
+            continue
+        for wrapper, typ in _SCHEME_WRAPPERS.items():
+            if isinstance(raw.get(wrapper), dict):
+                out[name] = {"type": typ, **raw[wrapper]}
+                break
+    return out
+
+
+def _requirements(card: dict) -> list | None:
+    for key in ("security", "securityRequirements"):
+        if key in card:
+            return card.get(key) or []
+    return None
+
+
+def normalize_card(card: dict, source: str = "") -> Asset:
+    if not isinstance(card, dict) or not card.get("name"):
+        raise CardError(f"Not an A2A Agent Card (missing name): {source}")
+    ifaces = interfaces(card)
+    urls = [i["url"] for i in ifaces]
+    schemes = _schemes(card)
+    types = sorted({str(s.get("type")) for s in schemes.values()})
+    reqs = _requirements(card)
+    flows = {f for s in schemes.values() if s.get("type") == "oauth2" for f in (s.get("flows") or {})}
+    api_key_in_query = any(s.get("type") == "apiKey" and str(s.get("in", s.get("location", ""))).lower() == "query"
+                           for s in schemes.values())
+    query_secrets = [u for u in urls if any(k.lower() in ("key", "api_key", "apikey", "token", "access_token")
+                                            for k in parse_qs(urlparse(u).query))]
+    caps = card.get("capabilities") or {}
+    provider = card.get("provider") or {}
+    host = (urlparse(urls[0]).hostname if urls else None) or "local"
+    facts: dict[str, Any] = {
+        "name": card.get("name"),
+        "version": card.get("version"),
+        "protocol_versions": sorted({i["version"] for i in ifaces}),
+        "bindings": sorted({i["binding"] for i in ifaces}),
+        "endpoint": {
+            "urls": urls,
+            "all_https": bool(urls) and all(u.startswith("https://") or _is_local(u) for u in urls),
+            "any_plain_http": any(u.startswith("http://") and not _is_local(u) for u in urls),
+            "secrets_in_url": bool(query_secrets),
+        },
+        "auth": {
+            "schemes": types,
+            "declared": bool(schemes),
+            # No explicit requirement list: treat declared schemes as required (common in practice).
+            "required": bool(reqs) if reqs is not None else bool(schemes),
+            "api_key_in_query": api_key_in_query,
+            "mutual_tls": "mutualTLS" in types,
+        },
+        "oauth": {"flows": sorted(flows), "deprecated_flows": bool(flows & {"implicit", "password"})},
+        "card": {"signed": bool(card.get("signatures")), "source": source},
+        "capabilities": {
+            "streaming": bool(caps.get("streaming")),
+            "push_notifications": bool(caps.get("pushNotifications")),
+            "extended_card": bool(caps.get("extendedAgentCard") or card.get("supportsAuthenticatedExtendedCard")),
+        },
+        "provider": {"organization": provider.get("organization"), "url": provider.get("url")},
+        "skills": {"count": len(card.get("skills") or []),
+                   "ids": [s.get("id") for s in card.get("skills") or [] if isinstance(s, dict)]},
+        "documentation_url": card.get("documentationUrl"),
+    }
+    return Asset(id=f"a2a/{host}/{_slug(card['name'])}", type="a2a_agent", name=card["name"],
+                 namespace=None, facts=facts)
+
+
+def collect_a2a_cards(targets: list[str], client: httpx.Client | None = None) -> list[Asset]:
+    return [normalize_card(*fetch_card(t, client=client)) for t in targets]