idemcheck 0.2.0__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: idemcheck
3
- Version: 0.2.0
3
+ Version: 0.3.0
4
4
  Summary: Find the HTTP handlers that will double-charge when a client retries.
5
5
  Author: mgkvibez
6
6
  License: MIT
@@ -109,6 +109,54 @@ atomic insert, so two workers racing on one key produce one winner and one confl
109
109
  `on_conflict="raise"` is the default. `"wait"` blocks for the in-flight call and replays it;
110
110
  `"run"` lets the work through, which is only right when repeating it is already harmless.
111
111
 
112
+ ## Middleware: the header, not a changed signature
113
+
114
+ The runtime needs the key passed to the function. A real app receives it as a header and will
115
+ not rewrite every handler, so `idemcheck.adapters` does the mechanics for you.
116
+
117
+ ```python
118
+ from idemcheck.adapters import IdempotencyMiddleware
119
+ from idemcheck.runtime import SQLiteStore
120
+
121
+ app.add_middleware(IdempotencyMiddleware, store=SQLiteStore("idempotency.db"))
122
+ ```
123
+
124
+ ```python
125
+ # Flask, or anything WSGI
126
+ from idemcheck.adapters import WSGIMiddleware
127
+
128
+ app.wsgi_app = WSGIMiddleware(app.wsgi_app, store=store)
129
+ ```
130
+
131
+ ```
132
+ $ python examples/adapter_demo.py
133
+ delivery 1: 201 {"charged": 2500} replayed=False
134
+ delivery 2: 201 {"charged": 2500} replayed=True
135
+ times the gateway was charged: 1
136
+ ```
137
+
138
+ What the middleware does with each delivery of an `Idempotency-Key`:
139
+
140
+ | Situation | Response |
141
+ |---|---|
142
+ | no header, `require_key=False` | passes through untouched, nothing stored |
143
+ | first delivery | runs the app, stores the status, headers and body |
144
+ | duplicate of a completed call | the stored response, plus `Idempotency-Replayed: true` |
145
+ | another call is still in flight | `409 idempotency_conflict`, the handler never runs |
146
+ | same key, different body | `422 idempotency_mismatch` |
147
+ | the app answered 5xx or 429 | key released, nothing stored, so the retry really retries |
148
+
149
+ Neither middleware imports FastAPI, Starlette, Flask or Django, so installing this cannot pin a
150
+ version of one. Three behaviours are deliberate rather than incidental:
151
+
152
+ * only `POST` is intercepted by default (`methods=(...)` to change it), and anything unhandled
153
+ is passed straight to the app without its response being buffered, so SSE and streaming
154
+ endpoints behave exactly as before
155
+ * a body over `max_body` (1 MB) is passed through **intact but unprotected**, rather than
156
+ silently swallowed, because a request the app never sees is worse than an unprotected one
157
+ * a 5xx or 429 releases the key instead of storing it. Caching a server error turns a transient
158
+ failure into a permanent one, which is the opposite of what a retry is for
159
+
112
160
  ## What it looks for
113
161
 
114
162
  A handler is a finding when it answers a mutating method **and** calls something that
@@ -148,7 +196,7 @@ The clean line is exactly what it says: no unguarded write was **recognised**.
148
196
  ## Tests
149
197
 
150
198
  ```bash
151
- python -m pytest -q # 33 tests
199
+ python -m pytest -q # 51 tests
152
200
  bash scripts/verify.sh # the release gate
153
201
  ```
154
202
 
@@ -89,6 +89,54 @@ atomic insert, so two workers racing on one key produce one winner and one confl
89
89
  `on_conflict="raise"` is the default. `"wait"` blocks for the in-flight call and replays it;
90
90
  `"run"` lets the work through, which is only right when repeating it is already harmless.
91
91
 
92
+ ## Middleware: the header, not a changed signature
93
+
94
+ The runtime needs the key passed to the function. A real app receives it as a header and will
95
+ not rewrite every handler, so `idemcheck.adapters` does the mechanics for you.
96
+
97
+ ```python
98
+ from idemcheck.adapters import IdempotencyMiddleware
99
+ from idemcheck.runtime import SQLiteStore
100
+
101
+ app.add_middleware(IdempotencyMiddleware, store=SQLiteStore("idempotency.db"))
102
+ ```
103
+
104
+ ```python
105
+ # Flask, or anything WSGI
106
+ from idemcheck.adapters import WSGIMiddleware
107
+
108
+ app.wsgi_app = WSGIMiddleware(app.wsgi_app, store=store)
109
+ ```
110
+
111
+ ```
112
+ $ python examples/adapter_demo.py
113
+ delivery 1: 201 {"charged": 2500} replayed=False
114
+ delivery 2: 201 {"charged": 2500} replayed=True
115
+ times the gateway was charged: 1
116
+ ```
117
+
118
+ What the middleware does with each delivery of an `Idempotency-Key`:
119
+
120
+ | Situation | Response |
121
+ |---|---|
122
+ | no header, `require_key=False` | passes through untouched, nothing stored |
123
+ | first delivery | runs the app, stores the status, headers and body |
124
+ | duplicate of a completed call | the stored response, plus `Idempotency-Replayed: true` |
125
+ | another call is still in flight | `409 idempotency_conflict`, the handler never runs |
126
+ | same key, different body | `422 idempotency_mismatch` |
127
+ | the app answered 5xx or 429 | key released, nothing stored, so the retry really retries |
128
+
129
+ Neither middleware imports FastAPI, Starlette, Flask or Django, so installing this cannot pin a
130
+ version of one. Three behaviours are deliberate rather than incidental:
131
+
132
+ * only `POST` is intercepted by default (`methods=(...)` to change it), and anything unhandled
133
+ is passed straight to the app without its response being buffered, so SSE and streaming
134
+ endpoints behave exactly as before
135
+ * a body over `max_body` (1 MB) is passed through **intact but unprotected**, rather than
136
+ silently swallowed, because a request the app never sees is worse than an unprotected one
137
+ * a 5xx or 429 releases the key instead of storing it. Caching a server error turns a transient
138
+ failure into a permanent one, which is the opposite of what a retry is for
139
+
92
140
  ## What it looks for
93
141
 
94
142
  A handler is a finding when it answers a mutating method **and** calls something that
@@ -128,7 +176,7 @@ The clean line is exactly what it says: no unguarded write was **recognised**.
128
176
  ## Tests
129
177
 
130
178
  ```bash
131
- python -m pytest -q # 33 tests
179
+ python -m pytest -q # 51 tests
132
180
  bash scripts/verify.sh # the release gate
133
181
  ```
134
182
 
@@ -1,3 +1,3 @@
1
1
  """idemcheck: find HTTP handlers that will double-charge on a retry."""
2
2
 
3
- __version__ = "0.2.0"
3
+ __version__ = "0.3.0"
@@ -0,0 +1,382 @@
1
+ """Framework adapters: the runtime behind a header, not behind a changed signature.
2
+
3
+ `idemcheck.runtime` needs the key passed to the function. A real app receives it as an HTTP
4
+ header and will not rewrite every handler, so these middlewares do the mechanics: read
5
+ `Idempotency-Key`, claim it, run the app once, store the response, and replay it for every
6
+ later delivery of the same key.
7
+
8
+ Both are dependency-free on purpose: an ASGI middleware for FastAPI and Starlette, and a WSGI
9
+ middleware for Flask and Django. Nothing imports those frameworks, so installing this cannot
10
+ pin a version of one.
11
+
12
+ What is deliberately not done here:
13
+
14
+ * Requests whose method is not in ``methods`` (POST by default) pass straight through, and
15
+ their response is never buffered, so streaming and SSE endpoints are untouched.
16
+ * A body larger than ``max_body`` passes through unprotected rather than being buffered, since
17
+ a middleware that silently eats a 200 MB upload is worse than a gap the linter will flag.
18
+ * A 5xx (and 429) releases the key instead of storing it: a server error is exactly the case a
19
+ client should be allowed to retry.
20
+ """
21
+
22
+ from __future__ import annotations
23
+
24
+ import base64
25
+ import hashlib
26
+ import json
27
+ from typing import Any, Dict, Iterable, List, Optional, Sequence, Tuple
28
+
29
+ from .runtime import MemoryStore, Outcome, _current_key
30
+
31
+ DEFAULT_HEADER = "idempotency-key"
32
+ REPLAYED_HEADER = "idempotency-replayed"
33
+
34
+ #: Statuses that must not be remembered, because they invite a retry rather than describe a
35
+ #: final answer.
36
+ TRANSIENT = frozenset({429, 500, 502, 503, 504})
37
+
38
+
39
+ def _body_fingerprint(method: str, path: str, query: bytes, body: bytes) -> str:
40
+ """Everything that makes this a different request, hashed."""
41
+ digest = hashlib.sha256()
42
+ for part in (method.encode(), path.encode(), query or b"", body or b""):
43
+ digest.update(part)
44
+ digest.update(b"\x00")
45
+ return digest.hexdigest()[:32]
46
+
47
+
48
+ def _header_value(headers: Iterable[Tuple[bytes, bytes]], name: str) -> Optional[str]:
49
+ wanted = name.lower().encode()
50
+ for key, value in headers:
51
+ if key.lower() == wanted:
52
+ return value.decode("latin-1")
53
+ return None
54
+
55
+
56
+ def _encode_response(status: int, headers: Sequence[Tuple[bytes, bytes]],
57
+ body: bytes) -> Dict[str, Any]:
58
+ return {
59
+ "status": status,
60
+ "headers": [[k.decode("latin-1"), v.decode("latin-1")] for k, v in headers],
61
+ "body": base64.b64encode(body).decode("ascii"),
62
+ }
63
+
64
+
65
+ def _decode_body(payload: Dict[str, Any]) -> bytes:
66
+ return base64.b64decode(payload["body"].encode("ascii"))
67
+
68
+
69
+ def _json_response(status: int, payload: Dict[str, Any]):
70
+ body = json.dumps(payload).encode()
71
+ return status, [(b"content-type", b"application/json"),
72
+ (b"content-length", str(len(body)).encode())], body
73
+
74
+
75
+ def _conflict_body(key: str) -> Tuple[int, List[Tuple[bytes, bytes]], bytes]:
76
+ return _json_response(409, {
77
+ "error": "idempotency_conflict",
78
+ "message": "another request with this Idempotency-Key is still being processed; "
79
+ "retry after it completes",
80
+ "key": key,
81
+ })
82
+
83
+
84
+ def _mismatch_body(key: str) -> Tuple[int, List[Tuple[bytes, bytes]], bytes]:
85
+ return _json_response(422, {
86
+ "error": "idempotency_mismatch",
87
+ "message": "this Idempotency-Key was already used with a different request body",
88
+ "key": key,
89
+ })
90
+
91
+
92
+ def _missing_body(header: str) -> Tuple[int, List[Tuple[bytes, bytes]], bytes]:
93
+ return _json_response(400, {
94
+ "error": "idempotency_key_required",
95
+ "message": "this endpoint requires the %s header" % header,
96
+ })
97
+
98
+
99
+ class IdempotencyMiddleware:
100
+ """ASGI middleware. Works with FastAPI and Starlette without importing either.
101
+
102
+ .. code-block:: python
103
+
104
+ from idemcheck.adapters import IdempotencyMiddleware
105
+ from idemcheck.runtime import SQLiteStore
106
+
107
+ app.add_middleware(IdempotencyMiddleware, store=SQLiteStore("idem.db"))
108
+ """
109
+
110
+ def __init__(self, app, store=None, header: str = DEFAULT_HEADER,
111
+ methods: Sequence[str] = ("POST",), require_key: bool = False,
112
+ skip_paths: Sequence[str] = (), max_body: int = 1_000_000,
113
+ key_scope=None) -> None:
114
+ self.app = app
115
+ self.store = store if store is not None else MemoryStore()
116
+ self.header = header
117
+ self.methods = tuple(m.upper() for m in methods)
118
+ self.require_key = require_key
119
+ self.skip_paths = tuple(skip_paths)
120
+ self.max_body = max_body
121
+ self.key_scope = key_scope
122
+
123
+ async def __call__(self, scope, receive, send):
124
+ if scope["type"] != "http" or not self._applies(scope):
125
+ await self.app(scope, receive, send)
126
+ return
127
+
128
+ headers = scope.get("headers") or []
129
+ key = _header_value(headers, self.header)
130
+ if not key:
131
+ if not self.require_key:
132
+ await self.app(scope, receive, send)
133
+ return
134
+ status, out_headers, body = _missing_body(self.header)
135
+ await _send_response(send, status, out_headers, body)
136
+ return
137
+
138
+ body, oversized = await _read_body(receive, self.max_body)
139
+ if oversized: # too large to buffer: intact, just unprotected
140
+ await self.app(scope, _chain_body(body, receive), send)
141
+ return
142
+
143
+ path = scope.get("path", "")
144
+ query = scope.get("query_string", b"")
145
+ fingerprint = _body_fingerprint(scope.get("method", ""), path, query, body)
146
+ scoped_key = self.key_scope(path, key, scope) if self.key_scope else key
147
+ claim = self.store.claim(scoped_key, fingerprint)
148
+
149
+ if claim.outcome is Outcome.MISMATCH:
150
+ status, out_headers, payload = _mismatch_body(scoped_key)
151
+ await _send_response(send, status, out_headers, payload)
152
+ return
153
+ if claim.outcome is Outcome.CONFLICT:
154
+ status, out_headers, payload = _conflict_body(scoped_key)
155
+ await _send_response(send, status, out_headers, payload)
156
+ return
157
+ if claim.outcome is Outcome.REPLAY:
158
+ stored = claim.record.response
159
+ await _send_response(send, stored["status"],
160
+ _with_replay_header(stored["headers"]),
161
+ _decode_body(stored))
162
+ return
163
+
164
+ captured = {"status": 0, "headers": [], "chunks": []}
165
+
166
+ async def capturing_send(message):
167
+ if message["type"] == "http.response.start":
168
+ captured["status"] = message["status"]
169
+ captured["headers"] = message.get("headers", [])
170
+ elif message["type"] == "http.response.body":
171
+ captured["chunks"].append(message.get("body", b""))
172
+ await send(message)
173
+
174
+ token = _current_key.set(scoped_key)
175
+ try:
176
+ await self.app(scope, _replay_body(body), capturing_send)
177
+ except BaseException:
178
+ self.store.release(scoped_key)
179
+ raise
180
+ finally:
181
+ _current_key.reset(token)
182
+
183
+ if captured["status"] in TRANSIENT:
184
+ self.store.release(scoped_key) # a retry is the right response to this
185
+ return
186
+ self.store.complete(scoped_key, _encode_response(
187
+ captured["status"], captured["headers"], b"".join(captured["chunks"])))
188
+
189
+ def _applies(self, scope) -> bool:
190
+ if scope.get("method", "").upper() not in self.methods:
191
+ return False
192
+ path = scope.get("path", "")
193
+ return not any(path.startswith(prefix) for prefix in self.skip_paths)
194
+
195
+
196
+ async def _read_body(receive, max_body: int):
197
+ """Buffer the request body. Returns (bytes, oversized)."""
198
+ chunks: List[bytes] = []
199
+ size = 0
200
+ while True:
201
+ message = await receive()
202
+ if message.get("type") != "http.request":
203
+ break
204
+ chunk = message.get("body", b"")
205
+ size += len(chunk)
206
+ chunks.append(chunk)
207
+ if size > max_body or not message.get("more_body"):
208
+ break
209
+ return b"".join(chunks), size > max_body
210
+
211
+
212
+ def _replay_body(body: bytes):
213
+ """A receive that hands the buffered body downstream exactly once."""
214
+ sent = {"done": False}
215
+
216
+ async def receive():
217
+ if not sent["done"]:
218
+ sent["done"] = True
219
+ return {"type": "http.request", "body": body, "more_body": False}
220
+ return {"type": "http.request", "body": b"", "more_body": False}
221
+
222
+ return receive
223
+
224
+
225
+ def _chain_body(prefix: bytes, receive):
226
+ """Buffered bytes first, then whatever the real client is still sending.
227
+
228
+ Used when a body had no content-length and outgrew the buffer: the request must reach the
229
+ app intact even though it is not being protected.
230
+ """
231
+ state = {"sent": False}
232
+
233
+ async def chained():
234
+ if not state["sent"]:
235
+ state["sent"] = True
236
+ return {"type": "http.request", "body": prefix, "more_body": True}
237
+ return await receive()
238
+
239
+ return chained
240
+
241
+
242
+ async def _send_response(send, status: int, headers: Sequence, body: bytes) -> None:
243
+ pairs = headers if headers and isinstance(headers[0], tuple) else [
244
+ (k.encode() if isinstance(k, str) else k, v.encode() if isinstance(v, str) else v)
245
+ for k, v in headers]
246
+ await send({"type": "http.response.start", "status": status, "headers": list(pairs)})
247
+ await send({"type": "http.response.body", "body": body})
248
+
249
+
250
+ def _with_replay_header(headers: Sequence[Sequence[str]]) -> List[Tuple[bytes, bytes]]:
251
+ pairs = [(k.encode("latin-1"), v.encode("latin-1")) for k, v in headers]
252
+ pairs = [(k, v) for k, v in pairs if k.lower() != REPLAYED_HEADER.encode()]
253
+ pairs.append((REPLAYED_HEADER.encode(), b"true"))
254
+ return pairs
255
+
256
+
257
+ class WSGIMiddleware:
258
+ """WSGI middleware for Flask and Django. No framework import, so no version pin.
259
+
260
+ .. code-block:: python
261
+
262
+ from idemcheck.adapters import WSGIMiddleware
263
+ app.wsgi_app = WSGIMiddleware(app.wsgi_app, store=store)
264
+ """
265
+
266
+ def __init__(self, app, store=None, header: str = DEFAULT_HEADER,
267
+ methods: Sequence[str] = ("POST",), require_key: bool = False,
268
+ skip_paths: Sequence[str] = (), max_body: int = 1_000_000,
269
+ key_scope=None) -> None:
270
+ self.app = app
271
+ self.store = store if store is not None else MemoryStore()
272
+ self.header = header
273
+ self.methods = tuple(m.upper() for m in methods)
274
+ self.require_key = require_key
275
+ self.skip_paths = tuple(skip_paths)
276
+ self.max_body = max_body
277
+ self.key_scope = key_scope
278
+
279
+ def _header(self, environ) -> Optional[str]:
280
+ return environ.get("HTTP_" + self.header.upper().replace("-", "_"))
281
+
282
+ def __call__(self, environ, start_response):
283
+ method = environ.get("REQUEST_METHOD", "").upper()
284
+ path = environ.get("PATH_INFO", "")
285
+ if method not in self.methods or any(path.startswith(p) for p in self.skip_paths):
286
+ return self.app(environ, start_response)
287
+
288
+ key = self._header(environ)
289
+ if not key:
290
+ if not self.require_key:
291
+ return self.app(environ, start_response)
292
+ status, headers, body = _missing_body(self.header)
293
+ return _wsgi_send(start_response, status, headers, body)
294
+
295
+ try:
296
+ length = int(environ.get("CONTENT_LENGTH") or 0)
297
+ except ValueError:
298
+ length = 0
299
+ if length > self.max_body:
300
+ return self.app(environ, start_response) # too large: intact, unprotected
301
+
302
+ stream = environ.get("wsgi.input")
303
+ body = stream.read(self.max_body + 1) if stream is not None else b""
304
+ if len(body) > self.max_body:
305
+ environ["wsgi.input"] = _PrefixedBody(body, stream)
306
+ return self.app(environ, start_response)
307
+ environ["wsgi.input"] = _PrefixedBody(body, None)
308
+
309
+ query = environ.get("QUERY_STRING", "").encode()
310
+ fingerprint = _body_fingerprint(method, path, query, body)
311
+ scoped_key = self.key_scope(path, key, environ) if self.key_scope else key
312
+ claim = self.store.claim(scoped_key, fingerprint)
313
+
314
+ if claim.outcome is Outcome.MISMATCH:
315
+ status, headers, payload = _mismatch_body(scoped_key)
316
+ return _wsgi_send(start_response, status, headers, payload)
317
+ if claim.outcome is Outcome.CONFLICT:
318
+ status, headers, payload = _conflict_body(scoped_key)
319
+ return _wsgi_send(start_response, status, headers, payload)
320
+ if claim.outcome is Outcome.REPLAY:
321
+ stored = claim.record.response
322
+ pairs = _with_replay_header(stored["headers"])
323
+ start_response(str(stored["status"]),
324
+ [(k.decode("latin-1"), v.decode("latin-1")) for k, v in pairs])
325
+ return [_decode_body(stored)]
326
+
327
+ captured: Dict[str, Any] = {"status": "200 OK", "headers": []}
328
+
329
+ def capture_start(status, headers, exc_info=None):
330
+ captured["status"] = status
331
+ captured["headers"] = headers
332
+ return lambda chunk: None
333
+
334
+ token = _current_key.set(scoped_key)
335
+ try:
336
+ iterable = self.app(environ, capture_start)
337
+ payload = b"".join(iterable)
338
+ close = getattr(iterable, "close", None)
339
+ if callable(close):
340
+ close()
341
+ except BaseException:
342
+ self.store.release(scoped_key)
343
+ raise
344
+ finally:
345
+ _current_key.reset(token)
346
+
347
+ code = int(str(captured["status"]).split()[0])
348
+ body_out = b"" if isinstance(payload, dict) else payload
349
+ if code in TRANSIENT:
350
+ self.store.release(scoped_key)
351
+ start_response(captured["status"], captured["headers"])
352
+ return [body_out]
353
+
354
+ self.store.complete(scoped_key, _encode_response(
355
+ code, [(k.encode("latin-1"), v.encode("latin-1"))
356
+ for k, v in captured["headers"]], body_out))
357
+ start_response(captured["status"], captured["headers"])
358
+ return [body_out]
359
+
360
+
361
+ class _PrefixedBody:
362
+ """A wsgi.input whose first read returns the bytes we already consumed."""
363
+
364
+ def __init__(self, prefix: bytes, stream) -> None:
365
+ self._prefix = prefix
366
+ self._stream = stream
367
+
368
+ def read(self, size: int = -1) -> bytes:
369
+ if self._prefix:
370
+ prefix, self._prefix = self._prefix, b""
371
+ return prefix
372
+ return self._stream.read(size) if self._stream is not None else b""
373
+
374
+ def readline(self, size: int = -1) -> bytes:
375
+ return self.read(size)
376
+
377
+
378
+ def _wsgi_send(start_response, status: int, headers: Sequence[Tuple[bytes, bytes]],
379
+ body: bytes):
380
+ start_response(str(status), [(k.decode("latin-1"), v.decode("latin-1"))
381
+ for k, v in headers])
382
+ return [body]
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: idemcheck
3
- Version: 0.2.0
3
+ Version: 0.3.0
4
4
  Summary: Find the HTTP handlers that will double-charge when a client retries.
5
5
  Author: mgkvibez
6
6
  License: MIT
@@ -109,6 +109,54 @@ atomic insert, so two workers racing on one key produce one winner and one confl
109
109
  `on_conflict="raise"` is the default. `"wait"` blocks for the in-flight call and replays it;
110
110
  `"run"` lets the work through, which is only right when repeating it is already harmless.
111
111
 
112
+ ## Middleware: the header, not a changed signature
113
+
114
+ The runtime needs the key passed to the function. A real app receives it as a header and will
115
+ not rewrite every handler, so `idemcheck.adapters` does the mechanics for you.
116
+
117
+ ```python
118
+ from idemcheck.adapters import IdempotencyMiddleware
119
+ from idemcheck.runtime import SQLiteStore
120
+
121
+ app.add_middleware(IdempotencyMiddleware, store=SQLiteStore("idempotency.db"))
122
+ ```
123
+
124
+ ```python
125
+ # Flask, or anything WSGI
126
+ from idemcheck.adapters import WSGIMiddleware
127
+
128
+ app.wsgi_app = WSGIMiddleware(app.wsgi_app, store=store)
129
+ ```
130
+
131
+ ```
132
+ $ python examples/adapter_demo.py
133
+ delivery 1: 201 {"charged": 2500} replayed=False
134
+ delivery 2: 201 {"charged": 2500} replayed=True
135
+ times the gateway was charged: 1
136
+ ```
137
+
138
+ What the middleware does with each delivery of an `Idempotency-Key`:
139
+
140
+ | Situation | Response |
141
+ |---|---|
142
+ | no header, `require_key=False` | passes through untouched, nothing stored |
143
+ | first delivery | runs the app, stores the status, headers and body |
144
+ | duplicate of a completed call | the stored response, plus `Idempotency-Replayed: true` |
145
+ | another call is still in flight | `409 idempotency_conflict`, the handler never runs |
146
+ | same key, different body | `422 idempotency_mismatch` |
147
+ | the app answered 5xx or 429 | key released, nothing stored, so the retry really retries |
148
+
149
+ Neither middleware imports FastAPI, Starlette, Flask or Django, so installing this cannot pin a
150
+ version of one. Three behaviours are deliberate rather than incidental:
151
+
152
+ * only `POST` is intercepted by default (`methods=(...)` to change it), and anything unhandled
153
+ is passed straight to the app without its response being buffered, so SSE and streaming
154
+ endpoints behave exactly as before
155
+ * a body over `max_body` (1 MB) is passed through **intact but unprotected**, rather than
156
+ silently swallowed, because a request the app never sees is worse than an unprotected one
157
+ * a 5xx or 429 releases the key instead of storing it. Caching a server error turns a transient
158
+ failure into a permanent one, which is the opposite of what a retry is for
159
+
112
160
  ## What it looks for
113
161
 
114
162
  A handler is a finding when it answers a mutating method **and** calls something that
@@ -148,7 +196,7 @@ The clean line is exactly what it says: no unguarded write was **recognised**.
148
196
  ## Tests
149
197
 
150
198
  ```bash
151
- python -m pytest -q # 33 tests
199
+ python -m pytest -q # 51 tests
152
200
  bash scripts/verify.sh # the release gate
153
201
  ```
154
202
 
@@ -2,6 +2,7 @@ LICENSE
2
2
  README.md
3
3
  pyproject.toml
4
4
  idemcheck/__init__.py
5
+ idemcheck/adapters.py
5
6
  idemcheck/analyze.py
6
7
  idemcheck/cli.py
7
8
  idemcheck/runtime.py
@@ -11,5 +12,6 @@ idemcheck.egg-info/dependency_links.txt
11
12
  idemcheck.egg-info/entry_points.txt
12
13
  idemcheck.egg-info/requires.txt
13
14
  idemcheck.egg-info/top_level.txt
15
+ tests/test_adapters.py
14
16
  tests/test_idemcheck.py
15
17
  tests/test_runtime.py
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "idemcheck"
7
- version = "0.2.0"
7
+ version = "0.3.0"
8
8
  description = "Find the HTTP handlers that will double-charge when a client retries."
9
9
  readme = "README.md"
10
10
  requires-python = ">=3.9"
@@ -0,0 +1,259 @@
1
+ """Adapters tested against bare ASGI and WSGI apps, so no framework has to be installed."""
2
+
3
+ import asyncio
4
+ import io
5
+
6
+ from idemcheck.adapters import (
7
+ IdempotencyMiddleware,
8
+ WSGIMiddleware,
9
+ _body_fingerprint,
10
+ )
11
+ from idemcheck.runtime import MemoryStore
12
+
13
+
14
+ class FakeASGI:
15
+ """Minimal ASGI app that records how many times it was actually reached."""
16
+
17
+ def __init__(self, status=200, body=b'{"ok":true}'):
18
+ self.status = status
19
+ self.body = body
20
+ self.calls = 0
21
+ self.seen_body = b""
22
+
23
+ async def __call__(self, scope, receive, send):
24
+ self.calls += 1
25
+ while True:
26
+ message = await receive()
27
+ self.seen_body += message.get("body", b"")
28
+ if not message.get("more_body"):
29
+ break
30
+ await send({"type": "http.response.start", "status": self.status,
31
+ "headers": [(b"content-type", b"application/json")]})
32
+ await send({"type": "http.response.body", "body": self.body})
33
+
34
+
35
+ def call_asgi(app, method="POST", path="/charge", headers=(), body=b"{}"):
36
+ scope = {"type": "http", "method": method, "path": path, "query_string": b"",
37
+ "headers": list(headers)}
38
+ sent = []
39
+ chunks = [body]
40
+
41
+ async def receive():
42
+ if chunks:
43
+ return {"type": "http.request", "body": chunks.pop(0), "more_body": False}
44
+ return {"type": "http.request", "body": b"", "more_body": False}
45
+
46
+ async def send(message):
47
+ sent.append(message)
48
+
49
+ asyncio.run(app(scope, receive, send))
50
+ status = next((m["status"] for m in sent if m["type"] == "http.response.start"), None)
51
+ payload = b"".join(m.get("body", b"") for m in sent if m["type"] == "http.response.body")
52
+ out_headers = dict(next((m.get("headers", []) for m in sent
53
+ if m["type"] == "http.response.start"), []))
54
+ return status, payload, out_headers
55
+
56
+
57
+ def _key(k):
58
+ return [(b"idempotency-key", k.encode())]
59
+
60
+
61
+ def test_a_request_without_the_header_is_untouched():
62
+ inner = FakeASGI()
63
+ mw = IdempotencyMiddleware(inner)
64
+ status, _, _ = call_asgi(mw)
65
+ assert status == 200 and inner.calls == 1
66
+
67
+
68
+ def test_the_second_delivery_replays_without_reaching_the_app():
69
+ inner = FakeASGI()
70
+ store = MemoryStore()
71
+ mw = IdempotencyMiddleware(inner, store=store)
72
+
73
+ first = call_asgi(mw, headers=_key("k1"))
74
+ second = call_asgi(mw, headers=_key("k1"))
75
+
76
+ assert inner.calls == 1, "the handler ran twice"
77
+ assert first[1] == second[1]
78
+ assert first[2].get(b"idempotency-replayed") is None
79
+ assert second[2][b"idempotency-replayed"] == b"true"
80
+
81
+
82
+ def test_a_non_idempotent_method_passes_through():
83
+ inner = FakeASGI()
84
+ mw = IdempotencyMiddleware(inner, store=MemoryStore())
85
+ call_asgi(mw, method="GET", headers=_key("k1"))
86
+ call_asgi(mw, method="GET", headers=_key("k1"))
87
+ assert inner.calls == 2
88
+
89
+
90
+ def test_a_request_still_in_flight_gets_409_and_does_not_run():
91
+ inner = FakeASGI()
92
+ store = MemoryStore()
93
+ store.claim("k1", _body_fingerprint("POST", "/charge", b"", b"{}"))
94
+ mw = IdempotencyMiddleware(inner, store=store)
95
+ status, payload, _ = call_asgi(mw, headers=_key("k1"))
96
+ assert status == 409
97
+ assert b"idempotency_conflict" in payload
98
+ assert inner.calls == 0
99
+
100
+
101
+ def test_the_same_key_with_a_different_body_gets_422():
102
+ inner = FakeASGI()
103
+ store = MemoryStore()
104
+ store.claim("k1", _body_fingerprint("POST", "/charge", b"", b'{"amount":1}'))
105
+ mw = IdempotencyMiddleware(inner, store=store)
106
+ status, payload, _ = call_asgi(mw, headers=_key("k1"), body=b'{"amount":999}')
107
+ assert status == 422
108
+ assert b"idempotency_mismatch" in payload
109
+ assert inner.calls == 0
110
+
111
+
112
+ def test_a_server_error_is_not_remembered_so_the_retry_can_work():
113
+ inner = FakeASGI(status=503)
114
+ store = MemoryStore()
115
+ mw = IdempotencyMiddleware(inner, store=store)
116
+ call_asgi(mw, headers=_key("k1"))
117
+ call_asgi(mw, headers=_key("k1"))
118
+ assert inner.calls == 2, "a 503 was cached, so the retry got a stale failure"
119
+ assert len(store) == 0
120
+
121
+
122
+ def test_a_429_is_not_remembered_either():
123
+ inner = FakeASGI(status=429)
124
+ store = MemoryStore()
125
+ mw = IdempotencyMiddleware(inner, store=store)
126
+ call_asgi(mw, headers=_key("k1"))
127
+ call_asgi(mw, headers=_key("k1"))
128
+ assert inner.calls == 2
129
+
130
+
131
+ def test_a_client_error_is_remembered():
132
+ """A 400 is a final answer for that payload, so replaying it is correct."""
133
+ inner = FakeASGI(status=400)
134
+ store = MemoryStore()
135
+ mw = IdempotencyMiddleware(inner, store=store)
136
+ call_asgi(mw, headers=_key("k1"))
137
+ status, _, _ = call_asgi(mw, headers=_key("k1"))
138
+ assert inner.calls == 1 and status == 400
139
+
140
+
141
+ def test_require_key_refuses_a_missing_header():
142
+ inner = FakeASGI()
143
+ mw = IdempotencyMiddleware(inner, store=MemoryStore(), require_key=True)
144
+ status, payload, _ = call_asgi(mw)
145
+ assert status == 400 and b"idempotency_key_required" in payload
146
+ assert inner.calls == 0
147
+
148
+
149
+ def test_skip_paths_is_honoured():
150
+ inner = FakeASGI()
151
+ mw = IdempotencyMiddleware(inner, store=MemoryStore(), skip_paths=("/webhooks",))
152
+ call_asgi(mw, path="/webhooks/stripe", headers=_key("k1"))
153
+ call_asgi(mw, path="/webhooks/stripe", headers=_key("k1"))
154
+ assert inner.calls == 2
155
+
156
+
157
+ def test_the_body_still_reaches_the_app_intact():
158
+ inner = FakeASGI()
159
+ mw = IdempotencyMiddleware(inner, store=MemoryStore())
160
+ call_asgi(mw, headers=_key("k1"), body=b'{"amount":2500}')
161
+ assert inner.seen_body == b'{"amount":2500}'
162
+
163
+
164
+ def test_an_oversized_body_passes_through_unbroken_and_unprotected():
165
+ """Buffering is skipped, but the app must still receive every byte."""
166
+ inner = FakeASGI()
167
+ store = MemoryStore()
168
+ mw = IdempotencyMiddleware(inner, store=store, max_body=4)
169
+ status, _, _ = call_asgi(mw, headers=_key("k1"), body=b"0123456789")
170
+ assert status == 200
171
+ assert inner.seen_body == b"0123456789"
172
+ assert len(store) == 0
173
+
174
+
175
+ def test_the_stored_response_keeps_its_status_and_content_type():
176
+ inner = FakeASGI(status=201, body=b'{"id":7}')
177
+ mw = IdempotencyMiddleware(inner, store=MemoryStore())
178
+ call_asgi(mw, headers=_key("k1"))
179
+ status, payload, headers = call_asgi(mw, headers=_key("k1"))
180
+ assert status == 201
181
+ assert payload == b'{"id":7}'
182
+ assert headers[b"content-type"] == b"application/json"
183
+
184
+
185
+ # --- WSGI ---------------------------------------------------------------------
186
+
187
+
188
+ class FakeWSGI:
189
+ def __init__(self, status="200 OK", body=b'{"ok":true}'):
190
+ self.status = status
191
+ self.body = body
192
+ self.calls = 0
193
+ self.seen_body = b""
194
+
195
+ def __call__(self, environ, start_response):
196
+ self.calls += 1
197
+ self.seen_body = environ["wsgi.input"].read()
198
+ start_response(self.status, [("content-type", "application/json")])
199
+ return [self.body]
200
+
201
+
202
+ def call_wsgi(app, method="POST", path="/charge", headers=None, body=b"{}"):
203
+ environ = {"REQUEST_METHOD": method, "PATH_INFO": path, "QUERY_STRING": "",
204
+ "CONTENT_LENGTH": str(len(body)), "wsgi.input": io.BytesIO(body)}
205
+ for name, value in (headers or {}).items():
206
+ environ["HTTP_" + name.upper().replace("-", "_")] = value
207
+ captured = {}
208
+
209
+ def start_response(status, pairs, exc_info=None):
210
+ captured["status"] = int(status.split()[0])
211
+ captured["headers"] = dict(pairs)
212
+
213
+ payload = b"".join(app(environ, start_response))
214
+ return captured["status"], payload, captured["headers"]
215
+
216
+
217
+ def test_wsgi_replays_the_second_delivery():
218
+ inner = FakeWSGI()
219
+ mw = WSGIMiddleware(inner, store=MemoryStore())
220
+ first = call_wsgi(mw, headers={"Idempotency-Key": "k1"})
221
+ second = call_wsgi(mw, headers={"Idempotency-Key": "k1"})
222
+ assert inner.calls == 1
223
+ assert first[1] == second[1]
224
+ assert second[2].get("idempotency-replayed") == "true"
225
+
226
+
227
+ def test_wsgi_leaves_other_methods_alone():
228
+ inner = FakeWSGI()
229
+ mw = WSGIMiddleware(inner, store=MemoryStore())
230
+ call_wsgi(mw, method="GET", headers={"Idempotency-Key": "k1"})
231
+ call_wsgi(mw, method="GET", headers={"Idempotency-Key": "k1"})
232
+ assert inner.calls == 2
233
+
234
+
235
+ def test_wsgi_conflict_returns_409_without_running():
236
+ inner = FakeWSGI()
237
+ store = MemoryStore()
238
+ store.claim("k1", _body_fingerprint("POST", "/charge", b"", b"{}"))
239
+ mw = WSGIMiddleware(inner, store=store)
240
+ status, payload, _ = call_wsgi(mw, headers={"Idempotency-Key": "k1"})
241
+ assert status == 409 and b"idempotency_conflict" in payload
242
+ assert inner.calls == 0
243
+
244
+
245
+ def test_wsgi_body_reaches_the_app_intact():
246
+ inner = FakeWSGI()
247
+ mw = WSGIMiddleware(inner, store=MemoryStore())
248
+ call_wsgi(mw, headers={"Idempotency-Key": "k1"}, body=b'{"amount":2500}')
249
+ assert inner.seen_body == b'{"amount":2500}'
250
+
251
+
252
+ def test_wsgi_server_error_is_released():
253
+ inner = FakeWSGI(status="503 Service Unavailable")
254
+ store = MemoryStore()
255
+ mw = WSGIMiddleware(inner, store=store)
256
+ call_wsgi(mw, headers={"Idempotency-Key": "k1"})
257
+ call_wsgi(mw, headers={"Idempotency-Key": "k1"})
258
+ assert inner.calls == 2
259
+ assert len(store) == 0
File without changes
File without changes
File without changes