traffic-diff 0.1.0__tar.gz

traffic_diff-0.1.0/.gitignore

@@ -0,0 +1,18 @@
+__pycache__/
+*.py[cod]
+.venv/
+venv/
+dist/
+build/
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+*.db
+*.db-journal
+*.db-wal
+*.db-shm
+report.md
+report.html
+.env
+.pypirc
+data/

traffic_diff-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ayaan Ali
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

traffic_diff-0.1.0/PKG-INFO

@@ -0,0 +1,207 @@
+Metadata-Version: 2.4
+Name: traffic-diff
+Version: 0.1.0
+Summary: Record real API traffic, replay it against a new version, and flag only the response changes that matter.
+Project-URL: Homepage, https://github.com/jengaWiz/traffic-diff
+Project-URL: Repository, https://github.com/jengaWiz/traffic-diff
+Project-URL: Issues, https://github.com/jengaWiz/traffic-diff/issues
+Author: Ayaan Ali
+License-Expression: MIT
+License-File: LICENSE
+Keywords: api,ci,diff,regression,testing,traffic-replay
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Framework :: FastAPI
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.11
+Requires-Dist: fastapi>=0.110
+Requires-Dist: httpx>=0.27
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: typer>=0.12
+Requires-Dist: uvicorn[standard]>=0.29
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff>=0.5; extra == 'dev'
+Requires-Dist: twine>=5; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# shadowdiff
+
+Record real API traffic, replay it against a new version before you ship, and flag only the
+response changes that matter.
+
+Unit and integration tests only cover the cases engineers think of, so regressions like a renamed
+field, a changed type or a flipped status code slip through to production. shadowdiff replays the
+requests your users actually sent and reports structural differences, ignoring noise like
+timestamps, generated IDs and reordered lists.
+
+> On PyPI the package is named **`traffic-diff`** (`shadowdiff` is too close to an existing
+> project there). The command and the Python package are still `shadowdiff`.
+
+> Status: MVP (v0.1.0). `GET` requests only. See [Limitations](#limitations).
+
+## How it works
+
+```
+             record                         replay
+ clients ──► shadowdiff proxy ──► API v1    shadowdiff replay ──► API v2
+              (:8080)    │                        │
+                         ▼                        ▼
+                    shadowdiff.db  ─────►  smart diff ──► report.md, exit code
+```
+
+1. **Record.** `shadowdiff record` runs a reverse proxy in front of the current version. It forwards
+   every request unchanged and records `GET` request/response pairs (2xx-4xx, bodies up to 1 MB) to
+   SQLite. Recording happens off the response path: exchanges go onto an in-memory queue that a
+   background task flushes in batches.
+2. **Replay.** `shadowdiff replay` sends the recorded requests to the new version with async httpx,
+   a concurrency limit and a 10 s timeout. Timeouts and connection errors are recorded as failures,
+   not crashes.
+3. **Smart diff.** Each JSON response is walked against the recording. Every difference gets a
+   JSON path and a kind: `type_changed`, `field_removed` and `status_changed` are **breaking**;
+   `field_added` and `value_changed` are warnings. Ignore rules in `shadowdiff.yaml` drop the noise.
+4. **Report.** Identical diffs are grouped across requests ("`$.price` type changed number → string
+   on 844 requests to `GET /products/{id}`") and written to `report.md`. The CLI exits **1** if
+   there is any breaking diff or request error, so it works as a CI gate.
+
+## Quickstart
+
+```bash
+git clone <this repo> && cd shadowdiff
+pip install -e ".[dev]"             # once published: pip install traffic-diff
+```
+
+Run the demo shop: v1 on `:8001`, a buggy v2 on `:8002`, and the recorder on `:8080` in front of v1.
+
+```bash
+docker compose up -d --build
+python demo/traffic.py --requests 2000      # ~2,000 realistic GETs through the recorder
+docker compose stop recorder                # flush and close the recording database
+shadowdiff replay --session demo --target http://localhost:8002 --db data/shadowdiff.db
+echo $?                                     # 1: breaking changes found
+shadowdiff report --db data/shadowdiff.db --format html   # re-render the latest run as HTML
+docker compose down
+```
+
+Without Docker, run the same three processes locally:
+
+```bash
+VERSION=v1 uvicorn demo.app:app --port 8001 &
+VERSION=v2 uvicorn demo.app:app --port 8002 &
+shadowdiff record --target http://127.0.0.1:8001 --port 8080 --session demo &
+python demo/traffic.py --base-url http://127.0.0.1:8080
+kill %3                                     # stop the recorder
+shadowdiff replay --session demo --target http://127.0.0.1:8002 --concurrency 50
+```
+
+### Commands
+
+| Command | Does |
+| --- | --- |
+| `shadowdiff record --target URL [--port 8080] [--session NAME]` | Run the recording proxy |
+| `shadowdiff replay --session NAME --target URL [--concurrency 20] [--config shadowdiff.yaml]` | Replay, diff, write `report.md`, exit 1 on breaking diffs |
+| `shadowdiff report [--run RUN_ID] [--format md\|html]` | Re-render a run's report (default: latest run) |
+
+All commands take `--db` (default `shadowdiff.db`). Run any command with `--help` for every option.
+
+### Ignore rules (`shadowdiff.yaml`)
+
+```yaml
+ignore_fields: ["*_at", "id", "request_id"]   # field-name globs; "$.meta.*" matches a full path
+ignore_values: [uuid, iso_timestamp]          # a UUID -> other UUID change is noise
+unordered_lists:
+  "$.items": "id"                             # match list items by key, not position
+```
+
+Unordered-list paths can use `[*]` for nested lists (`"$.orders[*].lines": "sku"`), and a `null`
+key compares the list as a multiset.
+
+## Sample report
+
+From the demo: v1 traffic replayed against v2, which returns `price` as a string and renames
+`in_stock` to `available` on `/products/{id}`. Each response also carries a fresh `request_id`
+UUID and `served_at` timestamp, and `/products` returns its items in random order. None of that
+noise shows up.
+
+> **FAIL**: 2 breaking change(s) across 844 request(s)
+>
+> | Replayed | Passed | Failed | Breaking changes | Warnings | Duration |
+> | --- | --- | --- | --- | --- | --- |
+> | 2001 | 1157 | 844 | 2 | 1 | 1.93s |
+>
+> **Breaking changes**
+>
+> | Endpoint | Path | Change | Requests | Example |
+> | --- | --- | --- | --- | --- |
+> | `GET /products/{id}` | `$.in_stock` | field removed | 844 | `/products/20`: was `false` |
+> | `GET /products/{id}` | `$.price` | type changed number → string | 844 | `/products/20`: `128.85` → `"128.85"` |
+>
+> **Warnings**
+>
+> | Endpoint | Path | Change | Requests | Example |
+> | --- | --- | --- | --- | --- |
+> | `GET /products/{id}` | `$.available` | field added | 844 | `/products/20`: added `false` |
+>
+> **By endpoint**
+>
+> | Endpoint | Requests | Passed | Failed |
+> | --- | --- | --- | --- |
+> | `GET /health` | 93 | 93 | 0 |
+> | `GET /orders/{id}` | 388 | 388 | 0 |
+> | `GET /products` | 605 | 605 | 0 |
+> | `GET /products/{id}` | 914 | 70 | 844 |
+
+Replaying the same recording against **v1** reports 0 breaking changes and 0 warnings, and exits 0.
+That run is the check that noise filtering works.
+
+## Measured numbers
+
+Measured on an Apple M3 laptop (8 cores) with everything on one machine, using
+`demo/traffic.py` (2,000 requests) and `demo/bench.py` (6,000 requests per side, direct and proxied
+rounds alternated). Other desktop apps were running, so treat these as indicative.
+
+| Metric | Result | Target |
+| --- | --- | --- |
+| Planted bugs caught | **2 / 2** | 2 / 2 |
+| False positives from noise (v1 vs v1) | **0** | 0 |
+| Replay throughput, 2,000 requests at concurrency 50 | **~1,050-1,300 req/s** | ≥ 500 req/s |
+| Proxy overhead, sequential requests | **+1.0 ms p50, +1.2 ms p95** | < 5 ms p95 |
+| Proxy overhead, 10 concurrent clients | +3.4-4.6 ms p50, +5.7-13.8 ms p95 | |
+
+The single-process demo API tops out around 1,100 req/s, so replay throughput is bounded by the
+target here, not by shadowdiff. At 10 concurrent clients the single-process proxy is close to
+saturation on one core, which is where the higher, noisier p95 comes from.
+
+Reproduce:
+
+```bash
+python -m demo.bench --direct http://127.0.0.1:8001 --proxy http://127.0.0.1:8080 --concurrency 1
+shadowdiff replay --session demo --target http://127.0.0.1:8001 --concurrency 50
+```
+
+## Limitations
+
+- Only `GET` requests are recorded and replayed. Safe replay of writes is the next big piece.
+- Recorded headers and bodies are stored as-is in SQLite, including `Authorization` headers and
+  cookies. Don't record production traffic with credentials or PII until redaction lands.
+- `value_changed` is a warning, not breaking (per the PRD), so a changed value alone won't fail CI.
+- The proxy is a single process. Under heavy concurrent load its added latency grows.
+- Bodies are compared fully in memory, and diffs for a run are computed after all requests return.
+
+## Development
+
+```bash
+python -m venv .venv && . .venv/bin/activate
+pip install -e ".[dev]"
+ruff check . && ruff format --check . && pytest
+```
+
+## License
+
+MIT

traffic_diff-0.1.0/README.md

@@ -0,0 +1,174 @@
+# shadowdiff
+
+Record real API traffic, replay it against a new version before you ship, and flag only the
+response changes that matter.
+
+Unit and integration tests only cover the cases engineers think of, so regressions like a renamed
+field, a changed type or a flipped status code slip through to production. shadowdiff replays the
+requests your users actually sent and reports structural differences, ignoring noise like
+timestamps, generated IDs and reordered lists.
+
+> On PyPI the package is named **`traffic-diff`** (`shadowdiff` is too close to an existing
+> project there). The command and the Python package are still `shadowdiff`.
+
+> Status: MVP (v0.1.0). `GET` requests only. See [Limitations](#limitations).
+
+## How it works
+
+```
+             record                         replay
+ clients ──► shadowdiff proxy ──► API v1    shadowdiff replay ──► API v2
+              (:8080)    │                        │
+                         ▼                        ▼
+                    shadowdiff.db  ─────►  smart diff ──► report.md, exit code
+```
+
+1. **Record.** `shadowdiff record` runs a reverse proxy in front of the current version. It forwards
+   every request unchanged and records `GET` request/response pairs (2xx-4xx, bodies up to 1 MB) to
+   SQLite. Recording happens off the response path: exchanges go onto an in-memory queue that a
+   background task flushes in batches.
+2. **Replay.** `shadowdiff replay` sends the recorded requests to the new version with async httpx,
+   a concurrency limit and a 10 s timeout. Timeouts and connection errors are recorded as failures,
+   not crashes.
+3. **Smart diff.** Each JSON response is walked against the recording. Every difference gets a
+   JSON path and a kind: `type_changed`, `field_removed` and `status_changed` are **breaking**;
+   `field_added` and `value_changed` are warnings. Ignore rules in `shadowdiff.yaml` drop the noise.
+4. **Report.** Identical diffs are grouped across requests ("`$.price` type changed number → string
+   on 844 requests to `GET /products/{id}`") and written to `report.md`. The CLI exits **1** if
+   there is any breaking diff or request error, so it works as a CI gate.
+
+## Quickstart
+
+```bash
+git clone <this repo> && cd shadowdiff
+pip install -e ".[dev]"             # once published: pip install traffic-diff
+```
+
+Run the demo shop: v1 on `:8001`, a buggy v2 on `:8002`, and the recorder on `:8080` in front of v1.
+
+```bash
+docker compose up -d --build
+python demo/traffic.py --requests 2000      # ~2,000 realistic GETs through the recorder
+docker compose stop recorder                # flush and close the recording database
+shadowdiff replay --session demo --target http://localhost:8002 --db data/shadowdiff.db
+echo $?                                     # 1: breaking changes found
+shadowdiff report --db data/shadowdiff.db --format html   # re-render the latest run as HTML
+docker compose down
+```
+
+Without Docker, run the same three processes locally:
+
+```bash
+VERSION=v1 uvicorn demo.app:app --port 8001 &
+VERSION=v2 uvicorn demo.app:app --port 8002 &
+shadowdiff record --target http://127.0.0.1:8001 --port 8080 --session demo &
+python demo/traffic.py --base-url http://127.0.0.1:8080
+kill %3                                     # stop the recorder
+shadowdiff replay --session demo --target http://127.0.0.1:8002 --concurrency 50
+```
+
+### Commands
+
+| Command | Does |
+| --- | --- |
+| `shadowdiff record --target URL [--port 8080] [--session NAME]` | Run the recording proxy |
+| `shadowdiff replay --session NAME --target URL [--concurrency 20] [--config shadowdiff.yaml]` | Replay, diff, write `report.md`, exit 1 on breaking diffs |
+| `shadowdiff report [--run RUN_ID] [--format md\|html]` | Re-render a run's report (default: latest run) |
+
+All commands take `--db` (default `shadowdiff.db`). Run any command with `--help` for every option.
+
+### Ignore rules (`shadowdiff.yaml`)
+
+```yaml
+ignore_fields: ["*_at", "id", "request_id"]   # field-name globs; "$.meta.*" matches a full path
+ignore_values: [uuid, iso_timestamp]          # a UUID -> other UUID change is noise
+unordered_lists:
+  "$.items": "id"                             # match list items by key, not position
+```
+
+Unordered-list paths can use `[*]` for nested lists (`"$.orders[*].lines": "sku"`), and a `null`
+key compares the list as a multiset.
+
+## Sample report
+
+From the demo: v1 traffic replayed against v2, which returns `price` as a string and renames
+`in_stock` to `available` on `/products/{id}`. Each response also carries a fresh `request_id`
+UUID and `served_at` timestamp, and `/products` returns its items in random order. None of that
+noise shows up.
+
+> **FAIL**: 2 breaking change(s) across 844 request(s)
+>
+> | Replayed | Passed | Failed | Breaking changes | Warnings | Duration |
+> | --- | --- | --- | --- | --- | --- |
+> | 2001 | 1157 | 844 | 2 | 1 | 1.93s |
+>
+> **Breaking changes**
+>
+> | Endpoint | Path | Change | Requests | Example |
+> | --- | --- | --- | --- | --- |
+> | `GET /products/{id}` | `$.in_stock` | field removed | 844 | `/products/20`: was `false` |
+> | `GET /products/{id}` | `$.price` | type changed number → string | 844 | `/products/20`: `128.85` → `"128.85"` |
+>
+> **Warnings**
+>
+> | Endpoint | Path | Change | Requests | Example |
+> | --- | --- | --- | --- | --- |
+> | `GET /products/{id}` | `$.available` | field added | 844 | `/products/20`: added `false` |
+>
+> **By endpoint**
+>
+> | Endpoint | Requests | Passed | Failed |
+> | --- | --- | --- | --- |
+> | `GET /health` | 93 | 93 | 0 |
+> | `GET /orders/{id}` | 388 | 388 | 0 |
+> | `GET /products` | 605 | 605 | 0 |
+> | `GET /products/{id}` | 914 | 70 | 844 |
+
+Replaying the same recording against **v1** reports 0 breaking changes and 0 warnings, and exits 0.
+That run is the check that noise filtering works.
+
+## Measured numbers
+
+Measured on an Apple M3 laptop (8 cores) with everything on one machine, using
+`demo/traffic.py` (2,000 requests) and `demo/bench.py` (6,000 requests per side, direct and proxied
+rounds alternated). Other desktop apps were running, so treat these as indicative.
+
+| Metric | Result | Target |
+| --- | --- | --- |
+| Planted bugs caught | **2 / 2** | 2 / 2 |
+| False positives from noise (v1 vs v1) | **0** | 0 |
+| Replay throughput, 2,000 requests at concurrency 50 | **~1,050-1,300 req/s** | ≥ 500 req/s |
+| Proxy overhead, sequential requests | **+1.0 ms p50, +1.2 ms p95** | < 5 ms p95 |
+| Proxy overhead, 10 concurrent clients | +3.4-4.6 ms p50, +5.7-13.8 ms p95 | |
+
+The single-process demo API tops out around 1,100 req/s, so replay throughput is bounded by the
+target here, not by shadowdiff. At 10 concurrent clients the single-process proxy is close to
+saturation on one core, which is where the higher, noisier p95 comes from.
+
+Reproduce:
+
+```bash
+python -m demo.bench --direct http://127.0.0.1:8001 --proxy http://127.0.0.1:8080 --concurrency 1
+shadowdiff replay --session demo --target http://127.0.0.1:8001 --concurrency 50
+```
+
+## Limitations
+
+- Only `GET` requests are recorded and replayed. Safe replay of writes is the next big piece.
+- Recorded headers and bodies are stored as-is in SQLite, including `Authorization` headers and
+  cookies. Don't record production traffic with credentials or PII until redaction lands.
+- `value_changed` is a warning, not breaking (per the PRD), so a changed value alone won't fail CI.
+- The proxy is a single process. Under heavy concurrent load its added latency grows.
+- Bodies are compared fully in memory, and diffs for a run are computed after all requests return.
+
+## Development
+
+```bash
+python -m venv .venv && . .venv/bin/activate
+pip install -e ".[dev]"
+ruff check . && ruff format --check . && pytest
+```
+
+## License
+
+MIT

traffic_diff-0.1.0/demo/app.py

@@ -0,0 +1,105 @@
+"""Demo e-commerce API used to show shadowdiff end to end.
+
+Run v1:  VERSION=v1 uvicorn demo.app:app --port 8001
+Run v2:  VERSION=v2 uvicorn demo.app:app --port 8002
+
+Every response carries noise (a fresh ``request_id`` UUID and a ``served_at``
+timestamp), and ``GET /products`` returns its items in random order. v2 ships
+two planted bugs on ``GET /products/{id}``: ``price`` is a string, and
+``in_stock`` is renamed to ``available``.
+"""
+
+import os
+import random
+import uuid
+from datetime import UTC, datetime
+
+from fastapi import FastAPI
+from fastapi.responses import JSONResponse
+
+CATEGORIES = ["books", "games", "kitchen", "garden", "toys"]
+
+_rng = random.Random(42)
+PRODUCTS = {
+    i: {
+        "id": i,
+        "name": f"Product {i}",
+        "category": _rng.choice(CATEGORIES),
+        "price": round(_rng.uniform(3, 250), 2),
+        "in_stock": _rng.random() > 0.2,
+        "tags": sorted(_rng.sample(["new", "sale", "popular", "gift", "eco"], k=2)),
+        "updated_at": "2026-10-01T12:00:00Z",
+    }
+    for i in range(1, 101)
+}
+ORDERS = {
+    i: {
+        "id": i,
+        "status": _rng.choice(["pending", "shipped", "delivered"]),
+        "lines": [
+            {"product_id": pid, "qty": _rng.randint(1, 4)}
+            for pid in _rng.sample(list(PRODUCTS), k=_rng.randint(1, 4))
+        ],
+        "created_at": "2026-10-02T08:30:00Z",
+    }
+    for i in range(1, 51)
+}
+for order in ORDERS.values():
+    order["total"] = round(
+        sum(PRODUCTS[line["product_id"]]["price"] * line["qty"] for line in order["lines"]), 2
+    )
+
+
+def _noisy(body: dict, version: str, status_code: int = 200) -> JSONResponse:
+    body = {
+        **body,
+        "request_id": str(uuid.uuid4()),
+        "served_at": datetime.now(UTC).isoformat(),
+    }
+    return JSONResponse(body, status_code=status_code, headers={"x-demo-version": version})
+
+
+def create_app(version: str = "v1") -> FastAPI:
+    app = FastAPI(title=f"shadowdiff demo shop ({version})")
+
+    def not_found(kind: str, item_id: int) -> JSONResponse:
+        return _noisy({"error": f"{kind} {item_id} not found"}, version, status_code=404)
+
+    @app.get("/health")
+    async def health():
+        return _noisy({"ok": True}, version)
+
+    @app.get("/products")
+    async def list_products(category: str | None = None, limit: int = 20, offset: int = 0):
+        items = [p for p in PRODUCTS.values() if category is None or p["category"] == category]
+        page = items[offset : offset + limit]
+        # Simulate a backend that doesn't guarantee ordering (e.g. a hash-partitioned store).
+        page = random.sample(page, k=len(page))
+        return _noisy(
+            {"items": page, "total": len(items), "limit": limit, "offset": offset}, version
+        )
+
+    @app.get("/products/{product_id}")
+    async def get_product(product_id: int):
+        product = PRODUCTS.get(product_id)
+        if product is None:
+            return not_found("product", product_id)
+        product = dict(product)
+        if version == "v2":
+            # Planted bug 1: price serialized as a string.
+            product["price"] = f"{product['price']:.2f}"
+            # Planted bug 2: in_stock renamed to available.
+            product["available"] = product.pop("in_stock")
+        return _noisy(product, version)
+
+    @app.get("/orders/{order_id}")
+    async def get_order(order_id: int):
+        order = ORDERS.get(order_id)
+        if order is None:
+            return not_found("order", order_id)
+        return _noisy(order, version)
+
+    return app
+
+
+app = create_app(os.environ.get("VERSION", "v1"))

traffic_diff-0.1.0/demo/bench.py

@@ -0,0 +1,85 @@
+"""Measure the latency the recording proxy adds.
+
+Sends the same GET requests to the demo API directly and through the proxy,
+then prints p50/p95/p99 for each and the difference.
+
+    python demo/bench.py --direct http://localhost:8001 --proxy http://localhost:8080
+"""
+
+import argparse
+import asyncio
+import statistics
+import time
+
+import httpx
+
+from demo.traffic import make_paths
+
+
+def percentile(values: list[float], p: float) -> float:
+    values = sorted(values)
+    k = max(0, min(len(values) - 1, round(p / 100 * (len(values) - 1))))
+    return values[k]
+
+
+async def measure(base_url: str, paths: list[str], concurrency: int) -> list[float]:
+    sem = asyncio.Semaphore(concurrency)
+    latencies: list[float] = []
+    # One single-connection client per slot, like the replayer, so client-side
+    # pool overhead doesn't dominate the measurement.
+    limits = httpx.Limits(max_connections=1, max_keepalive_connections=1)
+    free = [
+        httpx.AsyncClient(base_url=base_url, timeout=10, limits=limits) for _ in range(concurrency)
+    ]
+
+    async def one(path: str) -> None:
+        async with sem:
+            client = free.pop()
+            try:
+                start = time.perf_counter()
+                await client.get(path)
+                latencies.append((time.perf_counter() - start) * 1000)
+            finally:
+                free.append(client)
+
+    # Warm up connections and code paths before measuring.
+    await asyncio.gather(*(one(p) for p in paths[:100]))
+    latencies.clear()
+    await asyncio.gather(*(one(p) for p in paths))
+    for client in free:
+        await client.aclose()
+    return latencies
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--direct", default="http://localhost:8001")
+    parser.add_argument("--proxy", default="http://localhost:8080")
+    parser.add_argument("--requests", type=int, default=2000)
+    parser.add_argument("--concurrency", type=int, default=1)
+    parser.add_argument("--rounds", type=int, default=3)
+    args = parser.parse_args()
+
+    paths = make_paths(args.requests, seed=11)
+    direct: list[float] = []
+    proxied: list[float] = []
+    # Alternate rounds so drift (thermal, background load) hits both sides equally.
+    for _ in range(args.rounds):
+        direct += asyncio.run(measure(args.direct, paths, args.concurrency))
+        proxied += asyncio.run(measure(args.proxy, paths, args.concurrency))
+
+    print(f"{len(direct)} requests per side, concurrency {args.concurrency}")
+    print(f"{'':8}{'p50':>9}{'p95':>9}{'p99':>9}{'mean':>9}  (ms)")
+    for name, xs in [("direct", direct), ("proxied", proxied)]:
+        row = [percentile(xs, 50), percentile(xs, 95), percentile(xs, 99), statistics.mean(xs)]
+        print(f"{name:8}" + "".join(f"{v:9.2f}" for v in row))
+    print(
+        f"added   {percentile(proxied, 50) - percentile(direct, 50):9.2f}"
+        f"{percentile(proxied, 95) - percentile(direct, 95):9.2f}"
+        f"{percentile(proxied, 99) - percentile(direct, 99):9.2f}"
+        f"{statistics.mean(proxied) - statistics.mean(direct):9.2f}"
+    )
+
+
+if __name__ == "__main__":
+    main()

traffic_diff-0.1.0/demo/traffic.py

@@ -0,0 +1,72 @@
+"""Send realistic GET traffic to the demo shop (normally through the recorder).
+
+python demo/traffic.py --base-url http://localhost:8080 --requests 2000
+"""
+
+import argparse
+import asyncio
+import random
+import time
+from collections import Counter
+
+import httpx
+
+CATEGORIES = ["books", "games", "kitchen", "garden", "toys"]
+
+
+def make_paths(n: int, seed: int) -> list[str]:
+    rng = random.Random(seed)
+    paths = []
+    for _ in range(n):
+        r = rng.random()
+        if r < 0.45:
+            # A few IDs past the catalogue end produce realistic 404s.
+            paths.append(f"/products/{rng.randint(1, 110)}")
+        elif r < 0.75:
+            params = {"limit": rng.choice([5, 10, 20])}
+            if rng.random() < 0.6:
+                params["category"] = rng.choice(CATEGORIES)
+            if rng.random() < 0.3:
+                params["offset"] = rng.choice([0, 5, 10])
+            paths.append("/products?" + "&".join(f"{k}={v}" for k, v in params.items()))
+        elif r < 0.95:
+            paths.append(f"/orders/{rng.randint(1, 55)}")
+        else:
+            paths.append("/health")
+    return paths
+
+
+async def send(base_url: str, paths: list[str], concurrency: int) -> Counter:
+    sem = asyncio.Semaphore(concurrency)
+    statuses: Counter = Counter()
+    async with httpx.AsyncClient(base_url=base_url, timeout=10) as client:
+
+        async def one(path: str) -> None:
+            async with sem:
+                try:
+                    statuses[(await client.get(path)).status_code] += 1
+                except httpx.HTTPError as exc:
+                    statuses[type(exc).__name__] += 1
+
+        await asyncio.gather(*(one(p) for p in paths))
+    return statuses
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--base-url", default="http://localhost:8080")
+    parser.add_argument("--requests", type=int, default=2000)
+    parser.add_argument("--concurrency", type=int, default=20)
+    parser.add_argument("--seed", type=int, default=7)
+    args = parser.parse_args()
+
+    paths = make_paths(args.requests, args.seed)
+    start = time.perf_counter()
+    statuses = asyncio.run(send(args.base_url, paths, args.concurrency))
+    elapsed = time.perf_counter() - start
+    summary = ", ".join(f"{k}: {v}" for k, v in sorted(statuses.items(), key=str))
+    print(f"Sent {len(paths)} requests to {args.base_url} in {elapsed:.2f}s ({summary})")
+
+
+if __name__ == "__main__":
+    main()