pytest-wirecheck 0.1.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- pytest_wirecheck-0.1.0.dist-info/METADATA +175 -0
- pytest_wirecheck-0.1.0.dist-info/RECORD +16 -0
- pytest_wirecheck-0.1.0.dist-info/WHEEL +5 -0
- pytest_wirecheck-0.1.0.dist-info/entry_points.txt +5 -0
- pytest_wirecheck-0.1.0.dist-info/licenses/LICENSE +21 -0
- pytest_wirecheck-0.1.0.dist-info/top_level.txt +1 -0
- wirecheck/__init__.py +3 -0
- wirecheck/__main__.py +3 -0
- wirecheck/capture.py +231 -0
- wirecheck/cli.py +93 -0
- wirecheck/clock.py +17 -0
- wirecheck/plugin.py +314 -0
- wirecheck/report.py +183 -0
- wirecheck/rules.py +239 -0
- wirecheck/scan.py +320 -0
- wirecheck/versions.py +65 -0
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: pytest-wirecheck
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Record what your code really sends to the Claude API in tests, and fail the build if a request would break under API version 2026-10-21.
|
|
5
|
+
Author: Ishan Yash
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/ishanyash/wirecheck
|
|
8
|
+
Project-URL: Source, https://github.com/ishanyash/wirecheck
|
|
9
|
+
Project-URL: Issues, https://github.com/ishanyash/wirecheck/issues
|
|
10
|
+
Keywords: pytest,anthropic,claude,claude-api,migration,ci,lint
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Framework :: Pytest
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
22
|
+
Classifier: Topic :: Software Development :: Testing
|
|
23
|
+
Requires-Python: >=3.10
|
|
24
|
+
Description-Content-Type: text/markdown
|
|
25
|
+
License-File: LICENSE
|
|
26
|
+
Requires-Dist: pytest>=8.0
|
|
27
|
+
Provides-Extra: test
|
|
28
|
+
Requires-Dist: anthropic>=2.0.0; extra == "test"
|
|
29
|
+
Dynamic: license-file
|
|
30
|
+
|
|
31
|
+
# WireCheck
|
|
32
|
+
|
|
33
|
+
**Your tests pass. Your Claude API calls return 400. WireCheck checks what your code really sends to the Claude API in tests, and fails the build if a request would break under API version 2026-10-21.**
|
|
34
|
+
|
|
35
|
+

|
|
36
|
+
|
|
37
|
+
[PyPI: pytest-wirecheck](https://pypi.org/project/pytest-wirecheck/) · [GitHub Marketplace: WireCheck for Claude API](https://github.com/marketplace/actions/wirecheck-for-claude-api) · [Demo](demo/)
|
|
38
|
+
|
|
39
|
+
## Why
|
|
40
|
+
|
|
41
|
+
On 21 Oct 2026 Anthropic released Claude API version `2026-10-21`. The new SDK majors (Python `anthropic` 2.0.0 and later) send it by default. Under it, a Messages request that sends `temperature`, `top_p` or `top_k` returns **400, even at the default value** ([migration guide](https://platform.claude.com/docs/en/api/migration-2026-10-21)).
|
|
42
|
+
|
|
43
|
+
Most test suites mock the API, so they never see that 400. And these parameters are hard to grep for. Since `anthropic` 1.0.0, `temperature=` is no longer a keyword of `messages.create` (passing it is a `TypeError`, so tests catch it), and the SDK 1.0 migration guide said to pass it through `extra_body` instead. `extra_body` is merged into the JSON at runtime, so no type checker sees it, and it often sits in a shared wrapper or a defaults dict, far from the call. That is exactly how [Nojoin broke](https://github.com/Valtora/Nojoin/issues/554): a loose version floor pulled in `anthropic` 2.x, a wrapper still sent `temperature: 0.2` through `extra_body`, and the tests mocked the client, so they all passed.
|
|
44
|
+
|
|
45
|
+
WireCheck looks at the final request as it leaves the HTTP client, after the SDK has merged every wrapper, default and `extra_body`.
|
|
46
|
+
|
|
47
|
+
## Quick start
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
pip install pytest-wirecheck
|
|
51
|
+
pytest --wirecheck
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
A test that sends a breaking request now fails, and the run ends with a short report. This is the [demo](demo/) on `anthropic` 2.3.0:
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
3 requests to the Claude API checked: 3 errors, 0 warnings
|
|
58
|
+
anthropic SDK: 2.3.0; API versions seen: 2026-10-21 x3
|
|
59
|
+
|
|
60
|
+
RULE LEVEL COUNT TEST CALL SITE WHAT
|
|
61
|
+
WC001 error 1 broken/test_broken.py::test_direct_call broken/app_direct.py:12 temperature
|
|
62
|
+
WC001 error 1 broken/test_broken.py::test_extra_body broken/app_extra_body.py:18 temperature
|
|
63
|
+
WC001 error 1 broken/test_broken.py::test_wrapper_factory broken/app_factory.py:21 top_p
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Each row shows the rule, the test that sent the request, and the file and line in your code that made the call. Works with `pytest-xdist` too.
|
|
67
|
+
|
|
68
|
+
### GitHub Action
|
|
69
|
+
|
|
70
|
+
```yaml
|
|
71
|
+
name: WireCheck
|
|
72
|
+
on: pull_request
|
|
73
|
+
permissions:
|
|
74
|
+
contents: read
|
|
75
|
+
pull-requests: write # to post the findings comment
|
|
76
|
+
jobs:
|
|
77
|
+
wirecheck:
|
|
78
|
+
runs-on: ubuntu-latest
|
|
79
|
+
steps:
|
|
80
|
+
- uses: actions/checkout@v7
|
|
81
|
+
- uses: actions/setup-python@v7
|
|
82
|
+
with:
|
|
83
|
+
python-version: "3.12"
|
|
84
|
+
- run: pip install -e ".[test]" # your project and its test deps
|
|
85
|
+
- uses: ishanyash/wirecheck@v0.1.0
|
|
86
|
+
with:
|
|
87
|
+
pytest-args: tests/
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
The Action runs the static scan and your tests with `--wirecheck`, writes the findings to the job summary, and posts **one** pull request comment with a findings table (it updates the same comment on later pushes).
|
|
91
|
+
|
|
92
|
+
| Input | Default | Meaning |
|
|
93
|
+
|---|---|---|
|
|
94
|
+
| `path` | `.` | Folders for the static scan |
|
|
95
|
+
| `run-pytest` | `true` | Run `pytest --wirecheck` |
|
|
96
|
+
| `pytest-args` | | Extra pytest arguments |
|
|
97
|
+
| `working-directory` | `.` | Where to run |
|
|
98
|
+
| `fail-on` | `error` | `error`, `warn` or `never` |
|
|
99
|
+
| `comment` | `true` | Post the PR comment |
|
|
100
|
+
| `install` | `action` | `action` installs WireCheck from the tagged Action source; `skip` if you install it yourself |
|
|
101
|
+
|
|
102
|
+
## What it checks
|
|
103
|
+
|
|
104
|
+
Rules come from the [migration guide](https://platform.claude.com/docs/en/api/migration-2026-10-21) and the [release notes](https://platform.claude.com/docs/en/release-notes/overview).
|
|
105
|
+
|
|
106
|
+
**In the real requests your tests send (`pytest --wirecheck`):**
|
|
107
|
+
|
|
108
|
+
| Rule | What | Level |
|
|
109
|
+
|---|---|---|
|
|
110
|
+
| WC001 | `temperature`, `top_p` or `top_k` in a Messages, token count or batch request | error on 2026-10-21 |
|
|
111
|
+
| WC002 | `document` block directly in an `assistant` message | error on 2026-10-21 |
|
|
112
|
+
| WC003 | `text` block with non-empty `citations` in a `user` message, including inside `search_result` | error on 2026-10-21 |
|
|
113
|
+
| WC004 | `X-Api-Key` header set by your code. SDK 2.x sends `Authorization: Bearer`, so your key no longer replaces the client key (both are sent) | warning |
|
|
114
|
+
| WC005 | `claude-opus-4-8` with `speed: "fast"` | warning now, error from 2026-11-30 |
|
|
115
|
+
|
|
116
|
+
**In source (`wirecheck scan`):**
|
|
117
|
+
|
|
118
|
+
| Rule | What | Level |
|
|
119
|
+
|---|---|---|
|
|
120
|
+
| WCS001 | `MessageParam(...)` / `BetaMessageParam(...)` called or subclassed (TypeError in SDK 2.x) | error |
|
|
121
|
+
| WCS002 | `X-Api-Key` in `extra_headers` / `default_headers` | warning |
|
|
122
|
+
| WCS003 | `ant --api-key` / `ant --auth-token` (removed in the `ant` CLI 2.0.0) | error |
|
|
123
|
+
| WCS004 | Text Completions calls (`client.completions.create`, `/v1/complete`) | error |
|
|
124
|
+
| WCS005 | `temperature` / `top_p` / `top_k` visible in a Messages call (a hint; the runtime check is the real one) | warning |
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
wirecheck scan . # text
|
|
128
|
+
wirecheck scan src --format json # or markdown
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
## Which API version applies
|
|
132
|
+
|
|
133
|
+
For each request WireCheck reads the `anthropic-version` header. If there is none, it uses the installed SDK (`anthropic` 2.0.0 and later means `2026-10-21`).
|
|
134
|
+
|
|
135
|
+
- On `2026-10-21`, WC001 to WC003 are **errors** and fail the test that sent the request.
|
|
136
|
+
- On `2023-06-01` (SDK 1.x and older), the same findings are **warnings**. Your code works today and breaks when you upgrade. Use `--wirecheck-fail-on=warn` to fail on them now.
|
|
137
|
+
|
|
138
|
+
## How it works
|
|
139
|
+
|
|
140
|
+
When enabled, the plugin wraps `Client.send` and `AsyncClient.send` of `httpx2` (used by `anthropic` 1.x and 2.x) and `httpx` (older SDKs). Every request to `api.anthropic.com`, `$ANTHROPIC_BASE_URL` or a `--wirecheck-base-url` is recorded before it reaches the transport, so it is seen even when the test mocks the response with a mock transport, `respx`, or the `wirecheck_stub` fixture, and even when there is no network. API keys are redacted before anything is stored. The hook is removed when the run ends.
|
|
141
|
+
|
|
142
|
+
## Options
|
|
143
|
+
|
|
144
|
+
| Option | Meaning |
|
|
145
|
+
|---|---|
|
|
146
|
+
| `--wirecheck` | Turn it on (or `wirecheck = true` in your pytest ini, or `WIRECHECK=1`) |
|
|
147
|
+
| `--wirecheck-fail-on` | `error` (default), `warn`, or `never` |
|
|
148
|
+
| `--wirecheck-base-url URL` | Also capture a gateway or proxy URL (repeatable) |
|
|
149
|
+
| `--wirecheck-report PATH` | Write findings as JSON |
|
|
150
|
+
| `--wirecheck-markdown PATH` | Write findings as Markdown |
|
|
151
|
+
| `WIRECHECK_TODAY=YYYY-MM-DD` | Pin the date used by date-based rules |
|
|
152
|
+
|
|
153
|
+
Fixtures and markers:
|
|
154
|
+
|
|
155
|
+
- `wirecheck_stub` answers every Claude API request in a test with a canned reply, with no network. It mocks the response, not the client, so WireCheck still sees what you send. Set `wirecheck_stub["text"]` to change the reply.
|
|
156
|
+
- `wirecheck_requests` lists the requests the current test sent.
|
|
157
|
+
- `@pytest.mark.wirecheck_skip` ignores requests from one test.
|
|
158
|
+
|
|
159
|
+
## Known limits
|
|
160
|
+
|
|
161
|
+
- **It only sees requests that are really built.** If a test replaces `client.messages.create` (or the whole client) with a `MagicMock`, no request exists and WireCheck sees nothing. It prints a hint when a run captured no requests. Mock the HTTP response instead.
|
|
162
|
+
- Python only, and only the `httpx` / `httpx2` HTTP clients (the SDK default). The optional `aiohttp` backend is not hooked.
|
|
163
|
+
- Bedrock and Vertex requests go to other hosts and carry the version differently. Add their URL with `--wirecheck-base-url` if you want them recorded; version detection there falls back to the SDK version.
|
|
164
|
+
- `wirecheck_stub` answers non-streaming calls only.
|
|
165
|
+
- WC003 checks the places the guide names (user `text` blocks and `search_result` blocks). It does not flag bare `text` blocks inside `tool_result`, because the guide does not say.
|
|
166
|
+
- The static scanner is a set of simple patterns. It misses anything assembled at runtime, which is why the pytest plugin exists.
|
|
167
|
+
- No auto-fix and no TypeScript or other SDKs in v0.1.
|
|
168
|
+
|
|
169
|
+
## Licence
|
|
170
|
+
|
|
171
|
+
MIT. See [LICENSE](LICENSE).
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
Built by [Ishan Yash](https://ishanyash.com)
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
pytest_wirecheck-0.1.0.dist-info/licenses/LICENSE,sha256=MuK8B3MG-z13clfuY50YCK1V2vcVKfjb5Jn1-IpM5XA,1067
|
|
2
|
+
wirecheck/__init__.py,sha256=VO7vlDU6YC0km-9pW1oMQdEE0wK3wb-HUQ4xrI23N0c,93
|
|
3
|
+
wirecheck/__main__.py,sha256=k1ocEWawweo1qCJWNFAAvyxz3tcY13dzvCenHszij30,48
|
|
4
|
+
wirecheck/capture.py,sha256=ps1xUgsp2gD3rBDN8fHje6fNvNK1RMxcEGs5Vy8PyAY,8408
|
|
5
|
+
wirecheck/cli.py,sha256=ocDEU6G1hUrhNPGiWlmC9j9abSAriDTVeYJ5sLG15Tc,3537
|
|
6
|
+
wirecheck/clock.py,sha256=Lkx13akupxZE9IHjZsXHzNR2mi8YIZI4c6h3g85ll40,449
|
|
7
|
+
wirecheck/plugin.py,sha256=3K5gwrSx_g-mRk_68HY5eEE0QUCUjPu952L248wHXr4,12292
|
|
8
|
+
wirecheck/report.py,sha256=sP0oRWcsVLGWhBOw8KnQsT5Mg06-H1VZNMWi6D5p6jk,7588
|
|
9
|
+
wirecheck/rules.py,sha256=H95XEuBlJsczdq8cQmYqVNs0kVDHifry-Bb87h1LFyE,8809
|
|
10
|
+
wirecheck/scan.py,sha256=QeGyrtgQhGqPuyX6OS1-pR9595SUs7dAQjotUhJ2Nxs,12096
|
|
11
|
+
wirecheck/versions.py,sha256=1436fYEgvr78aTZeol7MuSTjQsT3A2DxS7uez9mT1q4,1787
|
|
12
|
+
pytest_wirecheck-0.1.0.dist-info/METADATA,sha256=o4wVKGC7T0dc86X9dCIiI7-maKE5zGiQ7ag6OgUh6ps,9642
|
|
13
|
+
pytest_wirecheck-0.1.0.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
|
|
14
|
+
pytest_wirecheck-0.1.0.dist-info/entry_points.txt,sha256=R9WSrpOsfRyRPWBfLVtuWn4UGpLyww2kWZeAAFWCOQU,90
|
|
15
|
+
pytest_wirecheck-0.1.0.dist-info/top_level.txt,sha256=ch9iUWiBVYpV5H6owQAPrfDLrX05pqZEx1b_U_A06wY,10
|
|
16
|
+
pytest_wirecheck-0.1.0.dist-info/RECORD,,
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Ishan Yash
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
wirecheck
|
wirecheck/__init__.py
ADDED
wirecheck/__main__.py
ADDED
wirecheck/capture.py
ADDED
|
@@ -0,0 +1,231 @@
|
|
|
1
|
+
"""Capture every request that leaves httpx / httpx2 for the Claude API.
|
|
2
|
+
|
|
3
|
+
The hook wraps ``Client.send`` and ``AsyncClient.send`` at class level, so it sees
|
|
4
|
+
the final request after the SDK has merged ``extra_body``, ``extra_headers`` and
|
|
5
|
+
client defaults, and it sees it even when the test swaps in a mock transport.
|
|
6
|
+
|
|
7
|
+
anthropic>=1.x uses httpx2; older SDKs use httpx. Both are hooked when importable.
|
|
8
|
+
"""
|
|
9
|
+
|
|
10
|
+
from __future__ import annotations
|
|
11
|
+
|
|
12
|
+
import functools
|
|
13
|
+
import importlib
|
|
14
|
+
import json
|
|
15
|
+
import os
|
|
16
|
+
import sys
|
|
17
|
+
import sysconfig
|
|
18
|
+
from typing import Any, Callable
|
|
19
|
+
|
|
20
|
+
from .rules import Captured
|
|
21
|
+
|
|
22
|
+
DEFAULT_HOSTS = ("api.anthropic.com",)
|
|
23
|
+
LIBRARIES = ("httpx2", "httpx")
|
|
24
|
+
REDACT = {"x-api-key", "authorization", "proxy-authorization", "cookie"}
|
|
25
|
+
|
|
26
|
+
def _norm(path: str) -> str:
|
|
27
|
+
"""Pure string normalising, no file system calls.
|
|
28
|
+
|
|
29
|
+
The hook runs inside the user's event loop. Tools such as blockbuster raise on
|
|
30
|
+
blocking calls there (os.path.abspath, os.getcwd, ...), so nothing on the hot
|
|
31
|
+
path may touch the file system.
|
|
32
|
+
"""
|
|
33
|
+
if os.name == "nt":
|
|
34
|
+
return path.replace("/", "\\").lower()
|
|
35
|
+
return path
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
# Computed once at import time, outside any event loop.
|
|
39
|
+
_THIS_DIR = _norm(os.path.dirname(os.path.abspath(__file__)))
|
|
40
|
+
_LIB_DIRS = tuple(
|
|
41
|
+
_norm(os.path.abspath(p))
|
|
42
|
+
for p in {
|
|
43
|
+
sysconfig.get_paths().get("stdlib"),
|
|
44
|
+
sysconfig.get_paths().get("platstdlib"),
|
|
45
|
+
sysconfig.get_paths().get("purelib"),
|
|
46
|
+
sysconfig.get_paths().get("platlib"),
|
|
47
|
+
}
|
|
48
|
+
if p
|
|
49
|
+
)
|
|
50
|
+
_SEP = "\\" if os.name == "nt" else "/"
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
def _is_library_file(filename: str) -> bool:
|
|
54
|
+
if not filename or filename.startswith("<"):
|
|
55
|
+
return True
|
|
56
|
+
path = _norm(filename)
|
|
57
|
+
if path.startswith(_THIS_DIR + _SEP):
|
|
58
|
+
return True
|
|
59
|
+
if f"{_SEP}site-packages{_SEP}" in path or f"{_SEP}dist-packages{_SEP}" in path:
|
|
60
|
+
return True
|
|
61
|
+
return path.startswith(_LIB_DIRS)
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def find_call_site(root: str | None = None, skip: int = 2) -> str | None:
|
|
65
|
+
"""File and line of the innermost frame that belongs to the user's code."""
|
|
66
|
+
try:
|
|
67
|
+
frame = sys._getframe(skip)
|
|
68
|
+
except ValueError:
|
|
69
|
+
return None
|
|
70
|
+
prefix = _norm(root).rstrip(_SEP) + _SEP if root else None
|
|
71
|
+
while frame is not None:
|
|
72
|
+
filename = frame.f_code.co_filename
|
|
73
|
+
if not _is_library_file(filename):
|
|
74
|
+
shown = filename
|
|
75
|
+
if prefix and _norm(filename).startswith(prefix):
|
|
76
|
+
shown = filename[len(prefix):]
|
|
77
|
+
return f"{shown.replace(_SEP, '/')}:{frame.f_lineno}"
|
|
78
|
+
frame = frame.f_back
|
|
79
|
+
return None
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
def _normalise_base(url: str) -> str:
|
|
83
|
+
url = url.strip().rstrip("/")
|
|
84
|
+
if "://" not in url:
|
|
85
|
+
url = "https://" + url
|
|
86
|
+
return url
|
|
87
|
+
|
|
88
|
+
|
|
89
|
+
class Recorder:
|
|
90
|
+
"""Holds the hook state. One active recorder at a time."""
|
|
91
|
+
|
|
92
|
+
def __init__(
|
|
93
|
+
self,
|
|
94
|
+
on_capture: Callable[[Captured], None],
|
|
95
|
+
base_urls: list[str] | None = None,
|
|
96
|
+
root: str | None = None,
|
|
97
|
+
) -> None:
|
|
98
|
+
self.on_capture = on_capture
|
|
99
|
+
self.base_urls = [_normalise_base(u) for u in (base_urls or []) if u and u.strip()]
|
|
100
|
+
self.hosts = set(DEFAULT_HOSTS)
|
|
101
|
+
self.root = root
|
|
102
|
+
self.current_test: str | None = None
|
|
103
|
+
self.stub: dict[str, Any] | None = None # set by the wirecheck_stub fixture
|
|
104
|
+
self._originals: list[tuple[type, str, Any]] = []
|
|
105
|
+
self.libraries: list[str] = []
|
|
106
|
+
self.errors: list[str] = [] # internal failures, shown in the report instead of hidden
|
|
107
|
+
|
|
108
|
+
# -- matching -----------------------------------------------------------
|
|
109
|
+
def matches(self, url: Any) -> bool:
|
|
110
|
+
host = getattr(url, "host", None)
|
|
111
|
+
if host in self.hosts:
|
|
112
|
+
return True
|
|
113
|
+
text = str(url)
|
|
114
|
+
return any(text == base or text.startswith(base + "/") for base in self.base_urls)
|
|
115
|
+
|
|
116
|
+
# -- observing ----------------------------------------------------------
|
|
117
|
+
def observe(self, request: Any, library: str) -> None:
|
|
118
|
+
if not self.matches(request.url):
|
|
119
|
+
return
|
|
120
|
+
headers: dict[str, str] = {}
|
|
121
|
+
for name, value in request.headers.items():
|
|
122
|
+
key = name.lower()
|
|
123
|
+
headers[key] = "<redacted>" if key in REDACT else value
|
|
124
|
+
body: Any = None
|
|
125
|
+
try:
|
|
126
|
+
raw = request.content # raises if the body is an unread stream
|
|
127
|
+
except Exception:
|
|
128
|
+
raw = None
|
|
129
|
+
if raw:
|
|
130
|
+
try:
|
|
131
|
+
body = json.loads(raw)
|
|
132
|
+
except (ValueError, UnicodeDecodeError):
|
|
133
|
+
body = None
|
|
134
|
+
try:
|
|
135
|
+
call_site = find_call_site(self.root, skip=3)
|
|
136
|
+
except Exception as exc: # a missing call site must not lose the request
|
|
137
|
+
self.errors.append(f"call site: {exc!r}")
|
|
138
|
+
call_site = None
|
|
139
|
+
captured = Captured(
|
|
140
|
+
method=request.method,
|
|
141
|
+
url=str(request.url),
|
|
142
|
+
path=request.url.path,
|
|
143
|
+
headers=headers,
|
|
144
|
+
body=body,
|
|
145
|
+
test=self.current_test,
|
|
146
|
+
call_site=call_site,
|
|
147
|
+
library=library,
|
|
148
|
+
)
|
|
149
|
+
# No lock: list.append is atomic, and lock waits count as blocking calls in async checkers.
|
|
150
|
+
self.on_capture(captured)
|
|
151
|
+
|
|
152
|
+
def stub_response(self, request: Any, module: Any) -> Any | None:
|
|
153
|
+
if self.stub is None or not self.matches(request.url):
|
|
154
|
+
return None
|
|
155
|
+
path = request.url.path.rstrip("/")
|
|
156
|
+
if path.endswith("/v1/messages/count_tokens"):
|
|
157
|
+
payload: Any = {"input_tokens": 1}
|
|
158
|
+
elif path.endswith("/v1/messages"):
|
|
159
|
+
model = "stub"
|
|
160
|
+
try:
|
|
161
|
+
model = json.loads(request.content).get("model", model)
|
|
162
|
+
except Exception:
|
|
163
|
+
pass
|
|
164
|
+
payload = {
|
|
165
|
+
"id": "msg_wirecheck_stub",
|
|
166
|
+
"type": "message",
|
|
167
|
+
"role": "assistant",
|
|
168
|
+
"model": model,
|
|
169
|
+
"content": [{"type": "text", "text": self.stub.get("text", "stub reply")}],
|
|
170
|
+
"stop_reason": "end_turn",
|
|
171
|
+
"stop_sequence": None,
|
|
172
|
+
"usage": {"input_tokens": 1, "output_tokens": 1},
|
|
173
|
+
}
|
|
174
|
+
else:
|
|
175
|
+
payload = {}
|
|
176
|
+
return module.Response(200, json=payload, request=request)
|
|
177
|
+
|
|
178
|
+
# -- install / uninstall -------------------------------------------------
|
|
179
|
+
def install(self) -> None:
|
|
180
|
+
for name in LIBRARIES:
|
|
181
|
+
try:
|
|
182
|
+
module = importlib.import_module(name)
|
|
183
|
+
except Exception:
|
|
184
|
+
continue
|
|
185
|
+
self._patch(module, name)
|
|
186
|
+
self.libraries.append(name)
|
|
187
|
+
|
|
188
|
+
def _patch(self, module: Any, name: str) -> None:
|
|
189
|
+
recorder = self
|
|
190
|
+
sync_orig = module.Client.send
|
|
191
|
+
async_orig = module.AsyncClient.send
|
|
192
|
+
|
|
193
|
+
def hook(request: Any) -> Any:
|
|
194
|
+
# Never break the user's test because of us, but never hide a failure either:
|
|
195
|
+
# it is counted and shown in the report.
|
|
196
|
+
try:
|
|
197
|
+
recorder.observe(request, name)
|
|
198
|
+
except Exception as exc:
|
|
199
|
+
recorder.errors.append(f"{request.method} {request.url}: {exc!r}")
|
|
200
|
+
try:
|
|
201
|
+
return recorder.stub_response(request, module)
|
|
202
|
+
except Exception as exc:
|
|
203
|
+
recorder.errors.append(f"stub: {exc!r}")
|
|
204
|
+
return None
|
|
205
|
+
|
|
206
|
+
@functools.wraps(sync_orig)
|
|
207
|
+
def send(client: Any, request: Any, *args: Any, **kwargs: Any) -> Any:
|
|
208
|
+
stubbed = hook(request)
|
|
209
|
+
if stubbed is not None:
|
|
210
|
+
return stubbed
|
|
211
|
+
return sync_orig(client, request, *args, **kwargs)
|
|
212
|
+
|
|
213
|
+
@functools.wraps(async_orig)
|
|
214
|
+
async def asend(client: Any, request: Any, *args: Any, **kwargs: Any) -> Any:
|
|
215
|
+
stubbed = hook(request)
|
|
216
|
+
if stubbed is not None:
|
|
217
|
+
return stubbed
|
|
218
|
+
return await async_orig(client, request, *args, **kwargs)
|
|
219
|
+
|
|
220
|
+
send.__wirecheck__ = True # type: ignore[attr-defined]
|
|
221
|
+
asend.__wirecheck__ = True # type: ignore[attr-defined]
|
|
222
|
+
self._originals.append((module.Client, "send", sync_orig))
|
|
223
|
+
self._originals.append((module.AsyncClient, "send", async_orig))
|
|
224
|
+
module.Client.send = send
|
|
225
|
+
module.AsyncClient.send = asend
|
|
226
|
+
|
|
227
|
+
def uninstall(self) -> None:
|
|
228
|
+
while self._originals:
|
|
229
|
+
cls, attr, orig = self._originals.pop()
|
|
230
|
+
setattr(cls, attr, orig)
|
|
231
|
+
self.libraries.clear()
|
wirecheck/cli.py
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
"""Command line: ``wirecheck scan`` and ``wirecheck comment``."""
|
|
2
|
+
|
|
3
|
+
from __future__ import annotations
|
|
4
|
+
|
|
5
|
+
import argparse
|
|
6
|
+
import json
|
|
7
|
+
import os
|
|
8
|
+
import sys
|
|
9
|
+
from typing import Any, Sequence
|
|
10
|
+
|
|
11
|
+
from . import __version__, report, scan
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
def _load(path: str | None) -> dict[str, Any] | None:
|
|
15
|
+
if not path or not os.path.exists(path):
|
|
16
|
+
return None
|
|
17
|
+
with open(path, encoding="utf-8") as fh:
|
|
18
|
+
return json.load(fh)
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
def _exit_code(findings: list[dict[str, Any]], fail_on: str) -> int:
|
|
22
|
+
if fail_on == "never":
|
|
23
|
+
return 0
|
|
24
|
+
if fail_on == "warn":
|
|
25
|
+
return 1 if findings else 0
|
|
26
|
+
return 1 if any(f["severity"] == "error" for f in findings) else 0
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
def cmd_scan(args: argparse.Namespace) -> int:
|
|
30
|
+
data = scan.scan(args.paths or ["."])
|
|
31
|
+
if args.format == "json":
|
|
32
|
+
text = json.dumps(data, indent=2) + "\n"
|
|
33
|
+
elif args.format == "markdown":
|
|
34
|
+
text = report.render_markdown(None, data)
|
|
35
|
+
else:
|
|
36
|
+
text = scan.render_text(data) + "\n"
|
|
37
|
+
if args.output:
|
|
38
|
+
with open(args.output, "w", encoding="utf-8") as fh:
|
|
39
|
+
fh.write(text)
|
|
40
|
+
else:
|
|
41
|
+
sys.stdout.write(text)
|
|
42
|
+
return _exit_code(data["findings"], args.fail_on)
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
def cmd_comment(args: argparse.Namespace) -> int:
|
|
46
|
+
pytest_data, scan_data = _load(args.pytest), _load(args.scan)
|
|
47
|
+
text = report.render_markdown(pytest_data, scan_data, pytest_exit=args.pytest_exit)
|
|
48
|
+
if args.output:
|
|
49
|
+
with open(args.output, "w", encoding="utf-8") as fh:
|
|
50
|
+
fh.write(text)
|
|
51
|
+
elif not args.counts:
|
|
52
|
+
sys.stdout.write(text)
|
|
53
|
+
if args.counts:
|
|
54
|
+
findings = (pytest_data or {}).get("findings", []) + (scan_data or {}).get("findings", [])
|
|
55
|
+
errors = sum(1 for f in findings if f["severity"] == "error")
|
|
56
|
+
sys.stdout.write(f"errors={errors}\nwarnings={len(findings) - errors}\n")
|
|
57
|
+
return 0
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def build_parser() -> argparse.ArgumentParser:
|
|
61
|
+
parser = argparse.ArgumentParser(
|
|
62
|
+
prog="wirecheck",
|
|
63
|
+
description="Check code for Claude API version 2026-10-21 problems. "
|
|
64
|
+
"For the runtime check, run: pytest --wirecheck",
|
|
65
|
+
)
|
|
66
|
+
parser.add_argument("--version", action="version", version=f"wirecheck {__version__}")
|
|
67
|
+
sub = parser.add_subparsers(dest="command", required=True)
|
|
68
|
+
|
|
69
|
+
p = sub.add_parser("scan", help="Scan source files for things that break under 2026-10-21.")
|
|
70
|
+
p.add_argument("paths", nargs="*", help="Files or folders (default: .)")
|
|
71
|
+
p.add_argument("--format", choices=("text", "json", "markdown"), default="text")
|
|
72
|
+
p.add_argument("--output", "-o", help="Write to this file instead of stdout.")
|
|
73
|
+
p.add_argument("--fail-on", choices=("error", "warn", "never"), default="error",
|
|
74
|
+
help="Exit 1 on errors (default), on any finding, or never.")
|
|
75
|
+
p.set_defaults(func=cmd_scan)
|
|
76
|
+
|
|
77
|
+
c = sub.add_parser("comment", help="Build the Markdown PR comment from JSON reports.")
|
|
78
|
+
c.add_argument("--pytest", help="JSON from pytest --wirecheck-report")
|
|
79
|
+
c.add_argument("--scan", help="JSON from wirecheck scan --format json")
|
|
80
|
+
c.add_argument("--pytest-exit", type=int, default=None, help="pytest exit code, noted in the comment if unusual")
|
|
81
|
+
c.add_argument("--output", "-o")
|
|
82
|
+
c.add_argument("--counts", action="store_true", help="Print errors=N and warnings=N (for GitHub outputs).")
|
|
83
|
+
c.set_defaults(func=cmd_comment)
|
|
84
|
+
return parser
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
def main(argv: Sequence[str] | None = None) -> int:
|
|
88
|
+
args = build_parser().parse_args(argv)
|
|
89
|
+
return args.func(args)
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
if __name__ == "__main__":
|
|
93
|
+
raise SystemExit(main())
|
wirecheck/clock.py
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
"""One place to read "today", so tests can freeze it.
|
|
2
|
+
|
|
3
|
+
Set WIRECHECK_TODAY=YYYY-MM-DD to pin the date (handy in CI), or monkeypatch
|
|
4
|
+
``wirecheck.clock.today`` in a test.
|
|
5
|
+
"""
|
|
6
|
+
|
|
7
|
+
from __future__ import annotations
|
|
8
|
+
|
|
9
|
+
import datetime as _dt
|
|
10
|
+
import os
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
def today() -> _dt.date:
|
|
14
|
+
override = os.environ.get("WIRECHECK_TODAY", "").strip()
|
|
15
|
+
if override:
|
|
16
|
+
return _dt.date.fromisoformat(override)
|
|
17
|
+
return _dt.datetime.now(_dt.timezone.utc).date()
|