openapi-agent-ready 0.4.0__tar.gz → 0.4.1__tar.gz

{openapi_agent_ready-0.4.0 → openapi_agent_ready-0.4.1}/CHANGELOG.md

@@ -5,6 +5,16 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+## [0.4.1] - 2026-10-24
+
+### Fixed
+- **Swagger 2.0 specs are now rejected (exit 2)** instead of being audited.
+  They have `paths`, so they were scored, but auth lives in
+  `securityDefinitions`, so a spec with scoped OAuth2 was reported as having no
+  security schemes at all.
+- **Progress bar counts operations, not paths.** A spec with 100 paths × 3
+  methods (300 operations) previously never showed progress.
+
 ## [0.4.0] - 2026-08-08
 
 ### Added

{openapi_agent_ready-0.4.0 → openapi_agent_ready-0.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: openapi-agent-ready
-Version: 0.4.0
+Version: 0.4.1
 Summary: Audit any OpenAPI spec for AI-agent and MCP readiness, and generate an MCP server scaffold.
 Project-URL: Homepage, https://github.com/sfaisal/agent-ready
 Project-URL: Issues, https://github.com/sfaisal/agent-ready/issues
@@ -64,7 +64,7 @@ Description-Content-Type: text/markdown
 **Audit any OpenAPI spec for AI-agent readiness — then generate an MCP server scaffold for the endpoints that pass.**
 
 Your API works fine for human developers. That doesn't mean an AI agent can use
-it reliably. `agent-ready` scores a spec against seven categories of
+it reliably. `agent-ready` scores a spec against eight categories of
 agent-specific failure, tells you exactly which endpoints will cause trouble and
 why, and can gate a CI pipeline so specs don't regress.
 
@@ -75,17 +75,18 @@ agent-ready https://example.com/openapi.yaml
 
 ```
 Generic Booking Platform API (v1.0)
-AI-readiness score: 54.4/100
-Endpoints: 4  Fails: 6  Warnings: 12
+AI-readiness score: 58.4/100
+Endpoints: 4  Fails: 7  Warnings: 14
 
 Category scores:
-  Description Clarity            37.5/100
-  Side Effects                   70.0/100
-  Error Responses                62.5/100
-  Auth Clarity                   50.0/100
-  Ambiguity                     100.0/100
-  Parameter Explanation          16.7/100
-  Usage Guidelines               50.0/100
+  Description Clarity           37.5/100
+  Side Effects                  70.0/100
+  Error Responses               62.5/100
+  Auth Clarity                  50.0/100
+  Ambiguity                    100.0/100
+  Parameter Explanation         10.0/100
+  Usage Guidelines              50.0/100
+  Tool Surface                 100.0/100
 ```
 
 ## Why this exists
@@ -111,9 +112,8 @@ directly onto the rubric:
 | **Frictionless authentication** | 4 (auth/scope clarity) |
 | **Structured error handling** | 3 (actionable errors) |
 
-Categories 2 (side effects) and 5 (ambiguity) were added during implementation
-— they turned out to matter as much as the original three, which is the usual
-reward for actually building the thing you wrote about.
+Categories 2 (side effects) and 5 (ambiguity) weren't in the article. They came
+up once I started building, and they matter as much as the original three.
 
 The gap is measurable, and enormous. A 2026 study of 856 tools across 103 MCP
 servers found that **97.1% of tool descriptions contained at least one quality
@@ -219,6 +219,19 @@ future release does not invalidate your baseline.
 
 As the team fixes the backlog, regenerate the baseline to lock in the progress.
 
+### Python API
+
+```python
+from agent_ready import load_spec, run_audit
+
+spec = load_spec("openapi.yaml")
+result = run_audit(spec)
+
+print(result["overall_score"])
+for finding in result["gaps"]:
+    print(finding.severity, finding.endpoint, finding.message)
+```
+
 ## Real-world results
 
 Running it against Stripe's production spec (587 endpoints), as of v0.1.0.
@@ -244,8 +257,8 @@ Two things worth drawing out.
 in this tool, not in Stripe.** Stripe documents errors via OpenAPI's `default`
 response rather than enumerating explicit 4xx/5xx codes. Perfectly valid, and
 arguably cleaner. The check only looked for `4`/`5` prefixes, so it failed all
-587 endpoints. Fixed, regression-tested, and the reason the README says to run
-this against a real spec before trusting it.
+587 endpoints. Fixed and regression-tested — and a good reason to run this
+against your own real spec before trusting the score.
 
 **Stripe is widely and correctly regarded as agent-ready, yet its raw OpenAPI
 spec scores 62.9.** That isn't a contradiction — it's the whole point. Stripe's
@@ -260,19 +273,6 @@ Genuine findings that survived scrutiny: `GET /v1/account` and
 uses bearer/basic API keys rather than scoped OAuth2, so there's no way to
 express "this agent may only read."
 
-### Python API
-
-```python
-from agent_ready import load_spec, run_audit
-
-spec = load_spec("openapi.yaml")
-result = run_audit(spec)
-
-print(result["overall_score"])
-for finding in result["gaps"]:
-    print(finding.severity, finding.endpoint, finding.message)
-```
-
 ## The eight categories
 
 | # | Category | What it catches | Published anchor |
@@ -336,7 +336,7 @@ doesn't:
 
 - **The ambiguity check uses string similarity**, so it misses endpoints that
   are semantically similar but differently worded. Embedding-based similarity
-  would be stronger. ([#1](https://github.com/sfaisal/agent-ready/issues))
+  would be stronger. ([#1](https://github.com/sfaisal/agent-ready/issues/1))
 - **Category weights are judgement calls, not measured effect sizes.**
   Description clarity is weighted highest on the prior that wrong tool selection
   is the costliest failure — that prior is untested here.
@@ -359,7 +359,7 @@ doesn't:
 - [ ] LLM-assisted rewriting of flagged descriptions (`--fix`)
 - [ ] Live tool-selection testing against a generated MCP server
 - [ ] GitHub Action wrapper
-- [ ] Baseline mode (`--baseline`) so CI only fails on *new* regressions
+- [x] Baseline mode (`--baseline`) so CI only fails on *new* regressions
 
 ## Contributing
 

{openapi_agent_ready-0.4.0 → openapi_agent_ready-0.4.1}/README.md

@@ -9,7 +9,7 @@
 **Audit any OpenAPI spec for AI-agent readiness — then generate an MCP server scaffold for the endpoints that pass.**
 
 Your API works fine for human developers. That doesn't mean an AI agent can use
-it reliably. `agent-ready` scores a spec against seven categories of
+it reliably. `agent-ready` scores a spec against eight categories of
 agent-specific failure, tells you exactly which endpoints will cause trouble and
 why, and can gate a CI pipeline so specs don't regress.
 
@@ -20,17 +20,18 @@ agent-ready https://example.com/openapi.yaml
 
 ```
 Generic Booking Platform API (v1.0)
-AI-readiness score: 54.4/100
-Endpoints: 4  Fails: 6  Warnings: 12
+AI-readiness score: 58.4/100
+Endpoints: 4  Fails: 7  Warnings: 14
 
 Category scores:
-  Description Clarity            37.5/100
-  Side Effects                   70.0/100
-  Error Responses                62.5/100
-  Auth Clarity                   50.0/100
-  Ambiguity                     100.0/100
-  Parameter Explanation          16.7/100
-  Usage Guidelines               50.0/100
+  Description Clarity           37.5/100
+  Side Effects                  70.0/100
+  Error Responses               62.5/100
+  Auth Clarity                  50.0/100
+  Ambiguity                    100.0/100
+  Parameter Explanation         10.0/100
+  Usage Guidelines              50.0/100
+  Tool Surface                 100.0/100
 ```
 
 ## Why this exists
@@ -56,9 +57,8 @@ directly onto the rubric:
 | **Frictionless authentication** | 4 (auth/scope clarity) |
 | **Structured error handling** | 3 (actionable errors) |
 
-Categories 2 (side effects) and 5 (ambiguity) were added during implementation
-— they turned out to matter as much as the original three, which is the usual
-reward for actually building the thing you wrote about.
+Categories 2 (side effects) and 5 (ambiguity) weren't in the article. They came
+up once I started building, and they matter as much as the original three.
 
 The gap is measurable, and enormous. A 2026 study of 856 tools across 103 MCP
 servers found that **97.1% of tool descriptions contained at least one quality
@@ -164,6 +164,19 @@ future release does not invalidate your baseline.
 
 As the team fixes the backlog, regenerate the baseline to lock in the progress.
 
+### Python API
+
+```python
+from agent_ready import load_spec, run_audit
+
+spec = load_spec("openapi.yaml")
+result = run_audit(spec)
+
+print(result["overall_score"])
+for finding in result["gaps"]:
+    print(finding.severity, finding.endpoint, finding.message)
+```
+
 ## Real-world results
 
 Running it against Stripe's production spec (587 endpoints), as of v0.1.0.
@@ -189,8 +202,8 @@ Two things worth drawing out.
 in this tool, not in Stripe.** Stripe documents errors via OpenAPI's `default`
 response rather than enumerating explicit 4xx/5xx codes. Perfectly valid, and
 arguably cleaner. The check only looked for `4`/`5` prefixes, so it failed all
-587 endpoints. Fixed, regression-tested, and the reason the README says to run
-this against a real spec before trusting it.
+587 endpoints. Fixed and regression-tested — and a good reason to run this
+against your own real spec before trusting the score.
 
 **Stripe is widely and correctly regarded as agent-ready, yet its raw OpenAPI
 spec scores 62.9.** That isn't a contradiction — it's the whole point. Stripe's
@@ -205,19 +218,6 @@ Genuine findings that survived scrutiny: `GET /v1/account` and
 uses bearer/basic API keys rather than scoped OAuth2, so there's no way to
 express "this agent may only read."
 
-### Python API
-
-```python
-from agent_ready import load_spec, run_audit
-
-spec = load_spec("openapi.yaml")
-result = run_audit(spec)
-
-print(result["overall_score"])
-for finding in result["gaps"]:
-    print(finding.severity, finding.endpoint, finding.message)
-```
-
 ## The eight categories
 
 | # | Category | What it catches | Published anchor |
@@ -281,7 +281,7 @@ doesn't:
 
 - **The ambiguity check uses string similarity**, so it misses endpoints that
   are semantically similar but differently worded. Embedding-based similarity
-  would be stronger. ([#1](https://github.com/sfaisal/agent-ready/issues))
+  would be stronger. ([#1](https://github.com/sfaisal/agent-ready/issues/1))
 - **Category weights are judgement calls, not measured effect sizes.**
   Description clarity is weighted highest on the prior that wrong tool selection
   is the costliest failure — that prior is untested here.
@@ -304,7 +304,7 @@ doesn't:
 - [ ] LLM-assisted rewriting of flagged descriptions (`--fix`)
 - [ ] Live tool-selection testing against a generated MCP server
 - [ ] GitHub Action wrapper
-- [ ] Baseline mode (`--baseline`) so CI only fails on *new* regressions
+- [x] Baseline mode (`--baseline`) so CI only fails on *new* regressions
 
 ## Contributing
 

{openapi_agent_ready-0.4.0 → openapi_agent_ready-0.4.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "openapi-agent-ready"
-version = "0.4.0"
+version = "0.4.1"
 description = "Audit any OpenAPI spec for AI-agent and MCP readiness, and generate an MCP server scaffold."
 readme = "README.md"
 requires-python = ">=3.10"

{openapi_agent_ready-0.4.0 → openapi_agent_ready-0.4.1}/src/agent_ready/cli.py

@@ -3,8 +3,8 @@ Command-line interface for agent-ready.
 
 Exit codes (designed for CI use):
     0  audit passed all configured gates
-    1  audit ran, but a gate failed (score below --min-score, or
-       --max-fails exceeded)
+    1  audit ran, but a gate failed (score below --min-score,
+       --max-fails exceeded, or --max-new exceeded against --baseline)
     2  the spec could not be loaded or parsed
 """
 
@@ -17,6 +17,7 @@ from . import __version__
 from .auditor import (
     SpecLoadError,
     endpoints_ready_for_mcp,
+    extract_endpoints,
     load_spec,
     result_to_dict,
     run_audit,
@@ -183,6 +184,16 @@ def _run(argv=None) -> int:
         print(f"error: {exc}", file=sys.stderr)
         return EXIT_SPEC_ERROR
 
+    # Swagger 2.0 has `paths` too, but keeps auth in `securityDefinitions` and
+    # bodies in `in: body` params, so auditing it yields confidently wrong scores.
+    if isinstance(spec, dict) and "swagger" in spec:
+        print(
+            f"error: Swagger {spec['swagger']} is not supported — only OpenAPI 3.x. "
+            "Convert it first (e.g. with swagger2openapi).",
+            file=sys.stderr,
+        )
+        return EXIT_SPEC_ERROR
+
     if not isinstance(spec, dict) or not spec.get("paths"):
         print(
             "error: no 'paths' found — this does not look like an OpenAPI spec.",
@@ -190,7 +201,8 @@ def _run(argv=None) -> int:
         )
         return EXIT_SPEC_ERROR
 
-    endpoint_count = len(spec.get("paths") or {})
+    # Operations, not paths — the audit (and the ambiguity check) is per operation.
+    endpoint_count = len(extract_endpoints(spec))
     result = run_audit(spec, progress=_make_progress(endpoint_count, args.quiet))
 
     if result["endpoint_count"] == 0:

{openapi_agent_ready-0.4.0 → openapi_agent_ready-0.4.1}/src/agent_ready/rubric.py

@@ -1,7 +1,7 @@
 """
 AI-Readiness Rubric
 -------------------
-Seven categories used to score how ready an API (from an OpenAPI spec) is
+Eight categories used to score how ready an API (from an OpenAPI spec) is
 for reliable use by AI agents / MCP servers. Each endpoint gets checked
 against every category; failures become concrete, cite-able gaps.
 
@@ -13,8 +13,9 @@ Categories:
 5. Ambiguity between endpoints      -- Wang et al. (2026) found 73% repeated tool names
 6. Parameter explanation            -- "Opaque Parameters" smell: 84.3% prevalence
 7. Usage guidelines (when to call)  -- "Missing Usage Guidelines" smell: 89.3% prevalence
+8. Tool surface & schema strictness -- OpenAI function-calling guidance (keep tool count small)
 
-Provenance note (be honest about this in interviews):
+Provenance:
 Categories 1-5 were derived independently from how MCP tool selection works
 plus standard API design principles. Categories 6-7 were added after checking
 the rubric against published empirical work:
@@ -29,7 +30,7 @@ the rubric against published empirical work:
     Server Descriptions," arXiv:2602.18914. 10,831 servers, 18 smell
     categories across accuracy / functionality / completeness / conciseness.
 
-IMPORTANT TRADE-OFF (cite this, it's the senior-level point):
+IMPORTANT TRADE-OFF:
 Hasan et al. found that augmenting ALL description components improved task
 success by a median 5.85pp but increased execution steps by 67.46% and
 REGRESSED performance in 16.67% of cases. More completeness is not strictly

{openapi_agent_ready-0.4.0 → openapi_agent_ready-0.4.1}/tests/test_audit.py

@@ -477,6 +477,13 @@ def test_cli_bad_spec_exits_two(tmp_path, capsys):
     assert main([str(p), "--quiet"]) == 2
 
 
+def test_cli_rejects_swagger_2(tmp_path, capsys):
+    """Swagger 2.0 has `paths`, so it used to be scored (wrongly) instead of rejected."""
+    spec = {"swagger": "2.0", "info": {"title": "Old", "version": "1"}, **GOOD_SPEC}
+    assert main([_write(tmp_path, spec), "--quiet"]) == 2
+    assert "Swagger 2.0 is not supported" in capsys.readouterr().err
+
+
 def test_cli_json_output_is_valid(tmp_path, capsys):
     main([_write(tmp_path, GOOD_SPEC), "--format", "json"])
     payload = json.loads(capsys.readouterr().out)