traffic-diff 0.1.0__py3-none-any.whl

traffic_diff-0.1.0.dist-info/METADATA

@@ -0,0 +1,207 @@
+Metadata-Version: 2.4
+Name: traffic-diff
+Version: 0.1.0
+Summary: Record real API traffic, replay it against a new version, and flag only the response changes that matter.
+Project-URL: Homepage, https://github.com/jengaWiz/traffic-diff
+Project-URL: Repository, https://github.com/jengaWiz/traffic-diff
+Project-URL: Issues, https://github.com/jengaWiz/traffic-diff/issues
+Author: Ayaan Ali
+License-Expression: MIT
+License-File: LICENSE
+Keywords: api,ci,diff,regression,testing,traffic-replay
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Framework :: FastAPI
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.11
+Requires-Dist: fastapi>=0.110
+Requires-Dist: httpx>=0.27
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: typer>=0.12
+Requires-Dist: uvicorn[standard]>=0.29
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff>=0.5; extra == 'dev'
+Requires-Dist: twine>=5; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# shadowdiff
+
+Record real API traffic, replay it against a new version before you ship, and flag only the
+response changes that matter.
+
+Unit and integration tests only cover the cases engineers think of, so regressions like a renamed
+field, a changed type or a flipped status code slip through to production. shadowdiff replays the
+requests your users actually sent and reports structural differences, ignoring noise like
+timestamps, generated IDs and reordered lists.
+
+> On PyPI the package is named **`traffic-diff`** (`shadowdiff` is too close to an existing
+> project there). The command and the Python package are still `shadowdiff`.
+
+> Status: MVP (v0.1.0). `GET` requests only. See [Limitations](#limitations).
+
+## How it works
+
+```
+             record                         replay
+ clients ──► shadowdiff proxy ──► API v1    shadowdiff replay ──► API v2
+              (:8080)    │                        │
+                         ▼                        ▼
+                    shadowdiff.db  ─────►  smart diff ──► report.md, exit code
+```
+
+1. **Record.** `shadowdiff record` runs a reverse proxy in front of the current version. It forwards
+   every request unchanged and records `GET` request/response pairs (2xx-4xx, bodies up to 1 MB) to
+   SQLite. Recording happens off the response path: exchanges go onto an in-memory queue that a
+   background task flushes in batches.
+2. **Replay.** `shadowdiff replay` sends the recorded requests to the new version with async httpx,
+   a concurrency limit and a 10 s timeout. Timeouts and connection errors are recorded as failures,
+   not crashes.
+3. **Smart diff.** Each JSON response is walked against the recording. Every difference gets a
+   JSON path and a kind: `type_changed`, `field_removed` and `status_changed` are **breaking**;
+   `field_added` and `value_changed` are warnings. Ignore rules in `shadowdiff.yaml` drop the noise.
+4. **Report.** Identical diffs are grouped across requests ("`$.price` type changed number → string
+   on 844 requests to `GET /products/{id}`") and written to `report.md`. The CLI exits **1** if
+   there is any breaking diff or request error, so it works as a CI gate.
+
+## Quickstart
+
+```bash
+git clone <this repo> && cd shadowdiff
+pip install -e ".[dev]"             # once published: pip install traffic-diff
+```
+
+Run the demo shop: v1 on `:8001`, a buggy v2 on `:8002`, and the recorder on `:8080` in front of v1.
+
+```bash
+docker compose up -d --build
+python demo/traffic.py --requests 2000      # ~2,000 realistic GETs through the recorder
+docker compose stop recorder                # flush and close the recording database
+shadowdiff replay --session demo --target http://localhost:8002 --db data/shadowdiff.db
+echo $?                                     # 1: breaking changes found
+shadowdiff report --db data/shadowdiff.db --format html   # re-render the latest run as HTML
+docker compose down
+```
+
+Without Docker, run the same three processes locally:
+
+```bash
+VERSION=v1 uvicorn demo.app:app --port 8001 &
+VERSION=v2 uvicorn demo.app:app --port 8002 &
+shadowdiff record --target http://127.0.0.1:8001 --port 8080 --session demo &
+python demo/traffic.py --base-url http://127.0.0.1:8080
+kill %3                                     # stop the recorder
+shadowdiff replay --session demo --target http://127.0.0.1:8002 --concurrency 50
+```
+
+### Commands
+
+| Command | Does |
+| --- | --- |
+| `shadowdiff record --target URL [--port 8080] [--session NAME]` | Run the recording proxy |
+| `shadowdiff replay --session NAME --target URL [--concurrency 20] [--config shadowdiff.yaml]` | Replay, diff, write `report.md`, exit 1 on breaking diffs |
+| `shadowdiff report [--run RUN_ID] [--format md\|html]` | Re-render a run's report (default: latest run) |
+
+All commands take `--db` (default `shadowdiff.db`). Run any command with `--help` for every option.
+
+### Ignore rules (`shadowdiff.yaml`)
+
+```yaml
+ignore_fields: ["*_at", "id", "request_id"]   # field-name globs; "$.meta.*" matches a full path
+ignore_values: [uuid, iso_timestamp]          # a UUID -> other UUID change is noise
+unordered_lists:
+  "$.items": "id"                             # match list items by key, not position
+```
+
+Unordered-list paths can use `[*]` for nested lists (`"$.orders[*].lines": "sku"`), and a `null`
+key compares the list as a multiset.
+
+## Sample report
+
+From the demo: v1 traffic replayed against v2, which returns `price` as a string and renames
+`in_stock` to `available` on `/products/{id}`. Each response also carries a fresh `request_id`
+UUID and `served_at` timestamp, and `/products` returns its items in random order. None of that
+noise shows up.
+
+> **FAIL**: 2 breaking change(s) across 844 request(s)
+>
+> | Replayed | Passed | Failed | Breaking changes | Warnings | Duration |
+> | --- | --- | --- | --- | --- | --- |
+> | 2001 | 1157 | 844 | 2 | 1 | 1.93s |
+>
+> **Breaking changes**
+>
+> | Endpoint | Path | Change | Requests | Example |
+> | --- | --- | --- | --- | --- |
+> | `GET /products/{id}` | `$.in_stock` | field removed | 844 | `/products/20`: was `false` |
+> | `GET /products/{id}` | `$.price` | type changed number → string | 844 | `/products/20`: `128.85` → `"128.85"` |
+>
+> **Warnings**
+>
+> | Endpoint | Path | Change | Requests | Example |
+> | --- | --- | --- | --- | --- |
+> | `GET /products/{id}` | `$.available` | field added | 844 | `/products/20`: added `false` |
+>
+> **By endpoint**
+>
+> | Endpoint | Requests | Passed | Failed |
+> | --- | --- | --- | --- |
+> | `GET /health` | 93 | 93 | 0 |
+> | `GET /orders/{id}` | 388 | 388 | 0 |
+> | `GET /products` | 605 | 605 | 0 |
+> | `GET /products/{id}` | 914 | 70 | 844 |
+
+Replaying the same recording against **v1** reports 0 breaking changes and 0 warnings, and exits 0.
+That run is the check that noise filtering works.
+
+## Measured numbers
+
+Measured on an Apple M3 laptop (8 cores) with everything on one machine, using
+`demo/traffic.py` (2,000 requests) and `demo/bench.py` (6,000 requests per side, direct and proxied
+rounds alternated). Other desktop apps were running, so treat these as indicative.
+
+| Metric | Result | Target |
+| --- | --- | --- |
+| Planted bugs caught | **2 / 2** | 2 / 2 |
+| False positives from noise (v1 vs v1) | **0** | 0 |
+| Replay throughput, 2,000 requests at concurrency 50 | **~1,050-1,300 req/s** | ≥ 500 req/s |
+| Proxy overhead, sequential requests | **+1.0 ms p50, +1.2 ms p95** | < 5 ms p95 |
+| Proxy overhead, 10 concurrent clients | +3.4-4.6 ms p50, +5.7-13.8 ms p95 | |
+
+The single-process demo API tops out around 1,100 req/s, so replay throughput is bounded by the
+target here, not by shadowdiff. At 10 concurrent clients the single-process proxy is close to
+saturation on one core, which is where the higher, noisier p95 comes from.
+
+Reproduce:
+
+```bash
+python -m demo.bench --direct http://127.0.0.1:8001 --proxy http://127.0.0.1:8080 --concurrency 1
+shadowdiff replay --session demo --target http://127.0.0.1:8001 --concurrency 50
+```
+
+## Limitations
+
+- Only `GET` requests are recorded and replayed. Safe replay of writes is the next big piece.
+- Recorded headers and bodies are stored as-is in SQLite, including `Authorization` headers and
+  cookies. Don't record production traffic with credentials or PII until redaction lands.
+- `value_changed` is a warning, not breaking (per the PRD), so a changed value alone won't fail CI.
+- The proxy is a single process. Under heavy concurrent load its added latency grows.
+- Bodies are compared fully in memory, and diffs for a run are computed after all requests return.
+
+## Development
+
+```bash
+python -m venv .venv && . .venv/bin/activate
+pip install -e ".[dev]"
+ruff check . && ruff format --check . && pytest
+```
+
+## License
+
+MIT

traffic_diff-0.1.0.dist-info/RECORD

@@ -0,0 +1,13 @@
+shadowdiff/__init__.py,sha256=vq5fsY6Cti-mJOvqQQIkzAaoY-CGFNhCU2Mblx-kau8,113
+shadowdiff/cli.py,sha256=cbGGRNxNxw4JHHMGEU1kMkwKp4232TwZ1jQUryekdSY,4494
+shadowdiff/config.py,sha256=hauc1m0XxkGOuM8smJRtzAxWpoE9xqaalQx4AfcLyCI,2002
+shadowdiff/diff.py,sha256=DpZyCnLGzQFpJNzvwwW3nmyxrlFDVZ4SSaYXVRGoE1I,8253
+shadowdiff/proxy.py,sha256=P2LjIZpYzgQeDPKmRnw39EB1uQ-iHo-xjDNqWEPGL-M,6313
+shadowdiff/replay.py,sha256=8Yp7hu-ekC-u0tWa0BCjDhgWwAv-BjcSl1vGvijLY8o,4900
+shadowdiff/report.py,sha256=Pj6l_UhqYNVCXP3ZjISsr1c8vOFeu3_oHGG45VZyeK8,9918
+shadowdiff/store.py,sha256=0MXcwigds0UPgt5yG6czJ1QVqmY0xZlmzukPL4nYtq4,7738
+traffic_diff-0.1.0.dist-info/METADATA,sha256=ylFC9U0YxEzPsSirfh98Bn8tIODvWguaSyqiM5_2S1Y,9032
+traffic_diff-0.1.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
+traffic_diff-0.1.0.dist-info/entry_points.txt,sha256=IzjEcucEBhwbPgHiL07sO8HKsn33Ur1ekRoCY3pjlHY,50
+traffic_diff-0.1.0.dist-info/licenses/LICENSE,sha256=K2kxC7Laj4G5BICABtfiO7k92tB4biPBSAxVkQu4LTQ,1066
+traffic_diff-0.1.0.dist-info/RECORD,,

traffic_diff-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.32.4
+Root-Is-Purelib: true
+Tag: py3-none-any

traffic_diff-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+shadowdiff = shadowdiff.cli:app

traffic_diff-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ayaan Ali
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.